Programmatically Creating a Build
Some of our clients want to run a Qwak build in a CI/CD script or as a task in Airflow.
You can do it either by creating a bash script and using our CLI in the script or by using the Python API.
In this tutorial, we will show how to create a project and model using Python API. We will also run the model build using Python.
Creating a Project and a Model
First, we have to define a new project and model in the Qwak platform.
Before you run the code shown below, remember to install and configure the Qwak SDK. If you need assistance, take a look at our Getting Started guide.
We must import the ProjectsManagementClient and ModelsManagementClient:
from qwak.projects.client import ProjectsManagementClient
from qwak.models.client import ModelsManagementClient
Now, we create instances of both clients.
projects_client = ProjectsManagementClient()
models_client = ModelsManagementClient()
Using the ProjectsManagementClient
, we create a new project. The function will return a Python dictionary containing the project identifier. We are going to need it later.
response = projects_client.create_project('programatic-build', 'this is a description')
project_id = response.project.project_id
models_client.create_model(project_id, 'model-name', 'model description')
Now, we can create the model files and start the build.
Creating Model Configuration
In the next step, we have to create a yaml configuration of the build. You can generate the configuration by adding the --out-conf
argument to the qwak models build
command.
After generating the configuration, we modify the uri parameter, so it points to the location of your model project. The result may look like this:
build_env:
docker:
assumed_iam_role_arn: null
base_image: qwakai/qwak:0.0.13-cpu-py39
env_vars: []
no_cache: false
params: []
local:
aws_profile: null
no_push: false
python_env:
conda:
conda_file: conda.yml
git_credentials: null
git_credentials_secret: null
poetry: null
qwak_sdk_extra_index_url: null
virtualenv: null
remote:
is_remote: false
resources:
cpus: 2.0
gpu_amount: 1
gpu_type: NVIDIA_K80
memory: 20Gi
build_properties:
branch: main
build_id: null
model_id: model-name
model_uri:
git_branch: master
git_credentials: null
git_credentials_secret: null
main_dir: main
dependency_required_folders:
- additional_directory_to_copy_before_build
- dir_2
uri: /home/ml_engineer/the_best_model_ever
tags: []
post_build: null
pre_build: null
Starting the Build
Finally, we will start the build.
We need two information, the model name you specified earlier and the location of the yaml build configuration.
Let's import the build_cli function:
from qwak.models.build import build_cli
and run the build_cli
command:
build_cli([
"-vv",
"--remote",
"--model-id", "model-name",
"-f", "absolute_path_to_build_configuration_file"],
standalone_mode=False
])
Note
Note that we had to specify the
model-id
(model-name
) a few times. If you copy-paste the code from this example, remember to change the identifier everywhere!
Updated 9 months ago