Prediction Timers

JFrog ML prediction timers help troubleshoot model prediction latency, measure prediction times, find bottlenecks, and optimize performance.

Configure multiple timers with custom names for clear visibility using either a context manager or a decorator.
After building, deploying and sending inference requests to your model, you can view these timers on the Latency breakdown graph.

📘

Note: The prediction timer graph may display up to 5 timers including the default overall and predict timers. This leave room for additional 3 custom timers.

Configure timers via decorators

JFrog ML timers may be configured via a function decorates manager, to easily wrap different methods that are called during the prediction process.

from qwak import qwak_timer
from qwak.model.base import QwakModel

class MyModel(QwakModel):
    def build():
        pass

    @qwak_timer("my_custom_timer_name")
    def pre_process(self, df):
        # Perform custom processing
        return df

    def predict(self, df):
        self.pre_process(df)
        return df

Configure timers via context managers

JFrog ML timers may be configured via a context manager, to easily wrap parts of your code that need measurements.

from qwak import qwak_timer
from qwak.model.base import QwakModel

class MyModel(QwakModel):
    def build():
        pass

    def predict(self, df):
        with qwak_timer("my_custom_timer_name"):
            # Perform custom processing
            df = df + 1

        return df

Default timers

Overall timer

Measure the end to end inference time, starting at the moment a request arrived, until the inference output.

Queue timer

Measures the the time it takes requests to leave the input queue before reaching the model.