Feedback (Beta)

We would like to know how our model performs in production.

We can only do this if we know what the actual value of the property predicted by the model is. We can then compare then it to the model's prediction.

The Qwak platform allows you to store the actual values obtained from your production application as feedback on the model's production performance.

from qwak.inference.clients import FeedbackClient

client = FeedbackClient(model_id='titanic_survival_classification')
response = client.actual(
    entity_name='PassengerId',
    entity_value='1234-5678', 
    tag='survived',
    actuals=['0']
)

import com.qwak.ai.feedback.api.Feedback;
import com.qwak.ai.inference.client.FeedbackClient;

import java.text.ParseException;
import java.time.Instant;
import java.util.List;

public class Main {
    public static void main(String[] args) throws ParseException {
        String myKey = "<FILL_ME>";
        String environment = "<FILL_ME>";
        String modelId = "titanic_model";
        FeedbackClient client = new FeedbackClient.Builder()
                .apiKey(myKey)
                .modelId(modelId)
                .environment(environment)
                .build();
        String entityName = "PassengerId";
        String entityValue = "1234";
        String tag = "survived";
        Instant timestamp = null; // When did the feedback occurred?, Default is "now".
        List<String> actuals = List.of("1.0");
        Feedback.ActualValuesResponse response = client.actual(entityName, entityValue, tag, actuals, timestamp);
        System.out.println(response);
    }
}

The entity name and value refer to a feature sent to the deployed model. It doesn't need to be a property used by the model to get a prediction, but it should identify a unique request.

The tag describes the predicted value. In the actuals array, you should store the feedback (real-world values of the model's prediction).

We can analyze the model's responses and the actual values in the Feedback view:

In order to configure the Feedback capabilities please contact Qwak support.