Programmatically Creating a Build

Some of our clients want to run a Qwak build in a CI/CD script or as a task in Airflow.

You can do it either by creating a bash script and using our CLI in the script or by using the Python API.

In this tutorial, we will show how to create a project and model using Python API. We will also run the model build using Python.

Creating a Project and a Model

First, we have to define a new project and model in the Qwak platform.

Before you run the code shown below, remember to install and configure the Qwak SDK. If you need assistance, take a look at our Getting Started guide.

We must import the ProjectsManagementClient and ModelsManagementClient:

from qwak.projects.client import ProjectsManagementClient
from qwak.models.client import ModelsManagementClient

Now, we create instances of both clients.

projects_client = ProjectsManagementClient()
models_client = ModelsManagementClient()

Using the ProjectsManagementClient, we create a new project. The function will return a Python dictionary containing the project identifier. We are going to need it later.

response = projects_client.create_project('programatic-build', 'this is a description')
project_id = response.project.project_id
models_client.create_model(project_id, 'model-name', 'model description')

Now, we can create the model files and start the build.

Creating Model Configuration

In the next step, we have to create a yaml configuration of the build. You can generate the configuration by adding the --out-conf argument to the qwak models build command.

After generating the configuration, we modify the uri parameter, so it points to the location of your model project. The result may look like this:

build_env:
  docker:
    assumed_iam_role_arn: null
    base_image: qwakai/qwak:0.0.13-cpu-py39
    env_vars: []
    no_cache: false
    params: []
  local:
    aws_profile: null
    no_push: false
  python_env:
    conda:
      conda_file: conda.yml
    git_credentials: null
    git_credentials_secret: null
    poetry: null
    qwak_sdk_extra_index_url: null
    virtualenv: null
  remote:
    is_remote: false
    resources:
      cpus: 2.0
      gpu_amount: 1
      gpu_type: NVIDIA_K80
      memory: 20Gi
build_properties:
  branch: main
  build_id: null
  model_id: model-name
  model_uri:
    git_branch: master
    git_credentials: null
    git_credentials_secret: null
    main_dir: main
    dependency_required_folders:
      - additional_directory_to_copy_before_build
      - dir_2 
    uri: /home/ml_engineer/the_best_model_ever
  tags: []
post_build: null
pre_build: null

Starting the Build

Finally, we will start the build.

We need two information, the model name you specified earlier and the location of the yaml build configuration.

Let's import the build_cli function:

from qwak.models.build import build_cli

and run the build_cli command:

build_cli([
"-vv",
"--remote",
"--model-id", "model-name",
"-f", "absolute_path_to_build_configuration_file"],
standalone_mode=False
])

📘

Note

Note that we had to specify the model-id (model-name) a few times. If you copy-paste the code from this example, remember to change the identifier everywhere!