Home

Building a Qwak Model

Suggest Edits

At Qwak, a build is an ML equivalent of a software build — a trained and serialized model instance. Builds are the actual deployable entities in the Qwak ecosystem.

The Build Lifecycle

A Qwak build process is composed mainly of the following steps:

  • Creating the model's virtual environment
  • Executing the build function
  • Running unit and integration tests
  • Serializing the model
  • Building a serving docker image
  • Pushing it to Qwak's model registry

In the following sections we will see how to create a customize the Qwak build process.

Creating a Basic Build

Create a Qwak-based model in your local environment. You can use an example of a Qwak-based model by running the following command:

qwak models init \
    --example titanic \
    <dest>

<dest> - Qwak based model local directory path.

Build the Qwak-based model using the following command: 

qwak models build \
    --model-id <model-id> \
    <uri>

<model-id> - Qwak model id, Which exists in one of you projects.
<uri> - Qwak based model URI.

Creating a Remote Build

If you build your model on your local machine, available resources may be limited.

By using a remote build you will gain the following benefits:

  • A stable network (within the cluster).
  • Configurable RAM size (via the CLI)
  • Configurable CPU / GPU usage (via the CLI)
  • Fewer local dependencies. There will be no need for Docker in the localhost.

Right now, all builds are remote by default. If you want to switch to a local build add the --no-remote option to the build command.

Controlling the Available Resources

While using a remote build, you can control the number of CPUs, amount of available RAM, or get access to a GPU using the following options:

  • CPU limit - Configure the CPU limit by using the following flag --cpus
  • RAM limit - Configure the memory limit values by using the following flag --memory

The configuration units are defined by k8s docs.

  • GPU type - Configure the GPU type by using the following flag --gpu-type (available types: NVIDIA_K80, NVIDIA_V100)
  • GPU amount - Configure the amount of GPU's by using the following flag --gpu-amount

The configuration units are defined by AWS docs (EC2 Accelerated Computing - P2, P3).
(available amounts per type: NVIDIA_K80 - 1 / 8 / 16, NVIDIA_V100 - 1 / 4 / 8).

Remote Build Logs

The remote build will run asynchronously and therefore you will not see the logs on the screen while the build is being performed. There are two options for viewing the logs:

  1. Qwak UI (Build page):
1815
  1. Follow the logs by using the following command: 
qwak models builds logs \
    -b <build-id> \
    --follow

<build-id> - The build ID that you received when executing the build remotely.

Using Multiple AWS Profiles

The AWS profile that you wish to use while interacting with the Docker registry, can be explicitly configured by using the --aws-profile flag. For example:

qwak models build \
    --model-id <model-id> \
    --aws-profile <profile-name> \
    <uri>

<model-id> - Model ID that the build should be related to.
<profile-name> - The AWS profile name, as shown in ~/.aws/config.
<uri> - Qwak-based model URI.

If you choose not to specify which AWS profile you wish to use, the default AWS profile will be used.

Using Custom AWS IAM Role

In some cases, a model needs to access external services during its build process.
If a build needs to access external AWS resources, a custom AWS IAM role can be passed to the Qwak build process.

The IAM role should be created with the following trust policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-id>:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "ArnLike": {
          "aws:PrincipalArn": "arn:aws:iam::<account-id>:role/eksctl-Qwak-EKS-Cluster-nodegroup-*"
        }
      }
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-id>:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "ArnLike": {
          "aws:PrincipalArn": "arn:aws:iam::<account-id>:role/qwak-eks-base*"
        }
      }
    }
  ]
}

The IAM role ARN can be passed directly to a build using the --iam-role-arn flag. For example:

qwak models build \
    --model-id <model-id> \
    --remote \
    --iam-role-arn arn:aws:iam::<account-id>:role/<role-name> \
    <uri>

The Supplied IAM role ARN role can be accessed from the model code using the following code example:

from boto3

session = boto3.Session(profile_name='user-provided-role')
# Example for an S3 client:
s3 = session.client('s3')

Tagging a Build

Tags can be attached to specific builds for later identification:

qwak models build \
    --model-id <model-id> \
    -T <tag_1> -T <tag_2> \
    <uri>

<model-id> - model id which the build should be related to.
<tag> - Tag name as string.
<uri> - Qwak based model URI.

Build config is a yaml file that describes the build command configuration. It is often useful when integrating a build command into a CI/CD pipeline.

Generating a Config File from a Command

To generate a config file from a command that you already use, specify the --outconf flag, which will print to the console a ready config based on the options you specified. For example assume you run the following command:

qwak models build \
    --model-id test_model \
    -T tag_1 -T tag_2 \
    -E OS=WINDOWS \
    -E VERSION=3.9 \
    ~/model

Here is an example of the output:

build_env:
  docker:
    base_image: qwak-base:0.0.18-cpu
    build_args: {}
    env_vars: ["OS=WINDOWS", "VERSION=3.9"]
    no_cache: true
    params: []
  local:
    aws_profile: null
    no_push: true
  python_env:
    conda:
      conda_file: conda.yml
    git_credentials: null
    git_credentials_secret: null
    poetry: null
    qwak_sdk_extra_index_url: null
    virtualenv: null
  remote:
    is_remote: false
    resources:
      cpus: 2.0
      memory: 4Gi
build_properties:
  branch: main
  build_id: f139b71a-533a-11ec-a934-367dda8b746f
  model_id: test_model
  model_uri:
    git_branch: master
    git_credentials: null
    git_credentials_secret: null
    main_dir: main
    uri: ~/model
  tags:
  - tag1
  - tag2

To save the configuration into redirection , For example: qwak models build --model-id test_model ~/model --out-conf > config.yml

Customizing Your Build

Custom Docker Image

You can use a custom Docker image for the build process. The custom base image will also be used for the serving image.

When executing a build, the image should be specified using the --base-image flag:

qwak models build \
    --model-id <model> \
    --base-image <image-id> \
    <dest>

<model-id> - The model ID associated with this build.
<image-id> - Docker image ID.
<uri> - Qwak-based model URI.

The custom docker image should be based on: public.ecr.aws/w8k8y6b6/qwak-base:0.0.18-cpu or public.ecr.aws/w8k8y6b6/qwak-base:0.0.6-gpu (in case of models requiring a GPU).

The Docker image entry point shouldn't be changed.

Environment Variables

Configuring environment variable in build and serving environment, The environment variables should specified with the flag -E when executing a build:

qwak models build \
    --model-id <model> \
    -E <key>=<value> -E <key>=<value> \
    <dest>

<model-id> - model id which the build should be related to.
<key> - Environment variable key.
<value> - Environment variable value.
<uri> - Qwak based model URI.

Customizing the Main Directory

Given the following Qwak-based model directory structure:

qwak_based_model/
├── main/
├── tests/

You can change the main/ directorie’s name. For example, changing its name to iris_classifier/:

qwak_based_model/
├── iris_classifier/
├── tests/

To configure the name in the main directory, use a build config as follows:

build_properties:
  model_uri:
    main_dir: <new-directory-name>

So for the above example, the build config would be:

build_properties:
  model_uri:
    main_dir: iris-classifier

Installing packages using a pypi repositories

During the build process you can download and use packages from private repositories.

Create an environment variable PIP_EXTRA_INDEX_URL .

qwak models build --env-vars PIP_EXTRA_INDEX_URL=https://USERNAME:PASSWORD@JFROG_ARTIFACTORY_URL

You can also send this data not in clear text, by creating local environment variables:

export JFROG_USERNAME=<USERNAME>
export JFROG_PASSWORD=<PASSWORD>

qwak models build --env-vars PIP_EXTRA_INDEX_URL=https://${JFROG_USERNAME}:${JFROG_PASSWORD}@JFROG_ARTIFACTORY_URL

Updated about 1 year ago