Building a Pre-Trained Model

The fastest way to start using Qwak is to deploy a model you have trained in the past as a Qwak service.

Let's assume that you have already trained the model and stored it in S3. The storage mechanism doesn't matter as long as you can download it using Python code.

Generating an Empty Model

First, we have to create a new Qwak project (or reuse an existing one if you want), add a new model to the project, and generate an empty Qwak service using the qwak-sdk:

qwak projects create --project-name "Build pre-trained" --project-description "Deploying a pre-trained model"

# copy the UUID returned by qwak projects create!
qwak models create --project-id {project_id} --model-name "pre_trained_model"  --model-description "here you put a description"
qwak models init .

Now, we can start editing the files.

Adding Dependencies

Qwak-compatible models use conda for dependency management. In addition to conda, Qwak supports creating virtual environments using requirements.txt files and Poetry dependency manager.

In this example, we will use conda so we have to edit the conda.yml file and put the required libraries into the dependency list. In this example, we need pandas, scikit-learn, and catboost. Additionally, we have to add boto3 because we will use it to download the pre-trained model.

- pandas=1.1.5
- scikit-learn=0.24.1
- catboost=0.26.1
- boto3=1.20.46

Loading the Model

We will show two different methods of loading the pre-trained model. In the first method, we will download the model from S3. In the second, we store the model as a pickle file in the build directory and load it from the file.

Why do we have two ways to load a pre-trained model?

Qwak offers two ways of loading an existing model. We can use the build function or the initialize_model function. Which one should you choose?

The build function is more flexible. You can load not only the model but also run additional fine-tuning training. You can preprocess the training data for fine-tuning.

In general, you can do whatever you want. The only difference between full training and loading a pre-training model is loading a model from a file instead of creating the model in the code.

When the build method finishes running, all model class fields will be pickled. Those fields are loaded at the model serving stage and are available in the predict function.

If you use the initialize_model function, we will load the model while starting the inference service!

The deployment will take longer if you load a model from an S3 file here!

On the other hand, using the initialize_model function lets you skip additional serialization/deserialization between the build and inference stages. If you already have the ready-to-use model in a pickle file, you can use the initialize_model function.

In the following sections, we will show you how to load the model from a file stored S3 in the build function and load a model from a pickle file included in the project directory.

Loading the Model from S3

In the build function, instead of training the model, we will download it and load the Python object from a file.

First, we have to download the file. We use the boto3 client to get the file from S3. Note that, we used the Qwak secrets manager to pass the credentials. You can learn more about the secrets manager in our tutorial about credentials management.

import boto3
from qwak.secret_service.client import SecretServiceGrpcClient


secret_service = SecretServiceGrpcClient()
aws_api_key = secret_service.get_secret('aws_api_key')
aws_secret_key = secret_service.get_secret('aws_secret_key')
aws_region = secret_service.get_secret('aws_region')

s3_client = boto3.client(
  's3',
  aws_access_key_id=aws_api_key,
  aws_secret_access_key=aws_secret_key,
  region_name=aws_region
)
s3_client.download_file('bucket_id', 'object_key', 'model_file.cbm')

After downloading the file, we can load it in memory and start using it. The model loading code depends on the library you use.

In the case of catboost it looks like this:

def __init__(self):
   self.model = CatBoostClassifier()

def build(self):
   self.__download_the_model()
   self.model.load_model('model_file.cbm')

If we were using Tensorflow, we would have to download all model files to a new directory and load it like this:

model_path = 'the directory with the pb file and the variables'
self.model = keras.models.load_model(model_path)

Right now, our entire class should look like this:

class TitanicSurvivalPrediction(QwakModelInterface):
    def __init__(self):
        self.model = CatBoostClassifier()
        print(os.getenv('TEST_VAR'), 'no TEST_VAR')
        

    def __download_the_model(self):
        secret_service = SecretServiceGrpcClient()
        aws_api_key = secret_service.get_secret('aws_api_key')
        aws_secret_key = secret_service.get_secret('aws_secret_key')
        aws_region = secret_service.get_secret('aws_region')

        s3_client = boto3.client(
            's3',
            aws_access_key_id=aws_api_key,
            aws_secret_access_key=aws_secret_key,
            region_name=aws_region
        )
        s3_client.download_file('bucket_id', 'object_key', 'model_file.cbm')

    def build(self):
        self.__download_the_model()
        self.model.load_model('model_file.cbm')

Loading the Model from a Pickle File

If you have your model in a pickle file, you can put it in the main directory and use the initialize_model method to load it.

We don't implement the build method in this case, but we still must include it in the class!

We can have an empty implementation of the build method:

def build(self):
    pass

Now, we can define the initialize_model method and load the model from a pickle file:

import pickle
...

def initialize_model(self):
    with open('model.pkl', 'rb') as infile:
        self.model = pickle.load(infile)

Adding the Preprocessing Code

Every machine learning model running in production requires some preprocessing code converting the data from the business domain into model-compatible values. In Qwak models, we put such code in the predict function. It is also the place where we call the model to obtain the predictions:

@qwak.api()
def predict(self, df: pd.DataFrame) -> pd.DataFrame:
    df = df.drop(['PassengerId'], axis=1)
    return pd.DataFrame(self.model.predict_proba(df)[:, 1], columns=['Survived_Probability'])

Building the Model

Now you have everything you need to deploy your pre-trained model as a Qwak model. In this example, we will run the build locally:

qwak models build --model-id "pre_trained_model" .

The build command prints the build id. We will need it during deployment!

When the build finishes, we can run the deploy command to start a real-time endpoint:

qwak models deploy realtime --model-id pre_trained_model --build-id the_id_from_previous_step