Advanced CI/CD Patterns

Introduction

This guide continues Build & Deploy Actions and it is specifically tailored for teams who have more specific pipeline needs.

This article is currently a stub but will be regularly updated with new patterns and workflows as the automation requirements of our customers and users continue to evolve.

JFrog ML Configuration-as-Code

In a GitOps workflow, all configurations for builds and deployments are version-controlled and stored within your Git repository. This approach allows you to apply versioning, peer review, and automated testing to your infrastructure, just like you do with your code.

For that principle, let's consider the following JFrog ML model directory example:

qwak_model/
├── main/
│   ├── __init__.py                 # Required for exporting model from main
│   ├── model.py                    # Qwak model definition
│   ├── conda.yml                   # Conda environment file
├── configuration/
    ├── build_conf.yml              # Build configuration
    ├── deployment_conf.yml         # Deployment configuration

Within the configuration directory, you have the option to centralize your JFrog ML Build and Deploy settings. By doing so, you can reference these configurations in both the build-action and deploy-action. This approach offers an advantage over embedding configurations directly in your GitHub workflow files, as it allows for easier management and version control.

Build Configuration

For instance, your build_conf.yml might be structured as follows:

buildProperties:
  modelId: "credit_risk_with_feature_store"
  modelUri:
    uri: "."
    mainDir: "main"
  tagsList:
    - "build-with-conf-file"
  environmentName: ""
  gpuCompatible: false
buildEnv:
  docker:
    envVarsList: []
    paramsList: []
    assumedIamRoleArn: ""
  pythonEnv:
    conda:
      condaFile: "conda.yml"
  remote:
    isRemote: false
    resources:
      instance: "small"

This would be the equivalent of running the following JFrog ML CLI command:

qwak models build --model-id credit_risk_with_feature_store \
                  --tags "build-with-conf-file" \
                  --instance "small" \
                  .

Of course the build configuration file can store more complex build configs, however, the primary advantage of this approach is the decoupling of JFrog ML configurations from your GitHub workflow files. This separation allows you to modify JFrog ML settings without altering the workflow code, thereby simplifying maintenance and enhancing version control.

Deploy Configuration

As opposed to a Build config, the deployment configuration is dependent on the deployment type; that being realtime, batch and streaming.

Let's consider the following realtime example:

resources:
  instance_size: "small"
  pods: 2                                 # number of pods per instance
realtime:
  timeout: 6000                           # REST endpoint timeout in ms
  variation_name: "default"               
  audiences:                              # define the audiences for this deployment
    - id: audience_1
      routes:
        - variation_name: default
          weight: 100											# 100% of the live traffic goes to the default endpoint
          shadow: false
        - variation_name: shadow
          weight: 20											# replicate 20% of the live traffic to a shadow endpoint
          shadow: true
  fallback_variation: shadow

In this deployment configuration we defined a realtime deployment with 2 Audiences, a default one which should serve all the production traffic and a Shadow Variation that replicates 20% of the live requests to a shadow endpoint for testing purposes.

📘

As opposed to the build configuration example, this deployment config file can only be called by a qwak models deploy command, but cannot be substituted by one as it contains the Audience configuration.

In the next section you'll learn how to leverage these 2 configuration files in a real Github Workflow example.

Build and Deploy Shadow Variations on Pull-Request

In this workflow, you will learn how to leverage the build_config.yml and deploy_config.yml configuration files defined earlier in a real-life Github workflow scenario.

This GitHub Actions workflow automates the process of building and deploying a JFrog ML model whenever a pull request is made against the main branch. The workflow consists of two main jobs: build and deploy.

name: Build and Deploy JFrog ML Model as a Shadow Variation on PR

on:
  pull_request:
    types: [opened, reopened, synchronize]
    branches:
        - 'main'                    # replace with your `main` or `master` branch
    paths-ignore:
        - "*.md"                    # Ignore modifications to Readme files
  workflow_dispatch:                # Enables manual trigger
  
jobs:
  build:
    runs-on: ubuntu-latest        # change with your required instance for example [ubuntu-latest]
    outputs:
      build_id: ${{ env.build-id }}
      build_status: ${{ env.build-status }}

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Build Qwak Model
      id: qwak-build
      uses: qwak-ai/build-action@v1
      with:
        qwak-api-key: ${{ secrets.QWAK_API_KEY }}
        model-id: ${{ vars.QWAK_MODEL_ID }}                 # stored as a Repository variable
        tags: ${{ github.head_ref }}                        # tag the build with the branch name
        from-file: 'configuration/build_conf.yml'
        # other config inputs as needed

    # [Optional] Additional step to notify e.g. Slack

    # [Optional] Additional step to post Qwak Build link on the Pull Request page
    
  deploy:
    if: needs.build.outputs.build_status == 'SUCCESSFUL'   # Only call deploy on successful builds
    runs-on: ubuntu-latest
    needs: build
    steps:

    # Deploy model to a shadow/canary endpoint  
    - name: Deploy Qwak Build
      uses: qwak-ai/deploy-action@v1
      with:
        qwak-api-key: ${{ secrets.QWAK_API_KEY }}
        deploy-type: 'realtime'                           # Replace with the deployment type you need
        model-id: ${{ vars.QWAK_MODEL_ID }}               # Using the repository variable for model ID
        build-id: ${{ needs.build.outputs.build_id }}     # Using the build ID from the previous job
        param-list: |                                     # This will enforce the `shadow` variation over the default from the config file
          variation-name=shadow,from-file=configuration/deploy_conf.yaml
        
    # [Optional] Additional step to post the Qwak deployment URL on the Pull Request page.
    
    # [Optional] Additional step to notify with the deployment status and details on Slack.
        
# [Optional] Additional job/separate workflow to run integration tests on the live model endpoint

Trigger Conditions

The workflow is triggered on pull request events such as opening, reopening, or synchronizing.
It focuses on changes made to the main branch and ignores any modifications to Markdown files (.md).
Manual triggering is also enabled through workflow_dispatch.

Build Job

• Checkout Code: Checks out the repository code.
• Build JFrog ML Model: Initiates the JFrog ML model build process using JFrog ML build-action . The build is tagged with the branch name and uses the build_conf.yml configuration file for additional build settings.

📘

Build Tagging

In this example we are tagging the Qwak Builds with the Git branch to make it easier to find the latest successful build when merging the code to the main or master branch.

Alternative tagging can be the PR Title with ${{ github.event.pull_request.title }}.

Deploy Job

• Conditional Execution: This job runs only if the build job is successful.
• Deploy JFrog ML Build: Deploys the successfully built JFrog ML model to a realtime shadow/canary endpoint as defined in the deploy_conf.yml file.

📘

Deployment Type

If your use case requires batch or streaming deployments, instead of the Shadow Variation you can use a test or staging Qwak Environment for the test deployment.

Troubleshooting

Validating Configuration Files

The easiest way to create JFrog ML build and deploy configuration files is to start from the JFrog ML CLI command using the --out-conf flag as follows:

qwak models build --model-id <your-model-id> --out-conf

qwak models deploy realtime --model-id <your-model-id> --out-conf

Starting from these commands output you can strip down to a configuration that's reflecting your needs without redundancy.