Secret Management
Secrets store sensitive data such as passwords, tokens, or keys.
Securely managing sensitive data and credentials in complex data projects is crucial.
Use secrets to avoid confidential information in model deployments and data source connections.
In traditional approaches, using credentials in an ML build involved including them in the Python code or using environment variables. Both options pose significant security risks.
Using the Qwak Secret Service, you can easily and securely store your credentials and pass them to your Python code with full confidentiality.
Secret naming conventions
Secret names follow the valid DNS format, which should be all lowercase, a minimum of 3 characters and up to 100 characters without underscores.
Creating Secrets via UI
Secrets may be created via the application by visiting the Secrets page under Account Settings to view, create or delete credentials.

Access Secrets from Account Settings
Creating Secrets via CLI
Secret names may be up to 36 characters and only contain letters, numbers and dash ("-"), and should start with a letter.
Create secrets via the Qwak CLI with the following commands:
qwak secrets set --name <aws-api-key> --value <the_value_of_the_key>
qwak secrets set --name <aws-api-secret> --value <the_value_of_the_secret>
Retrieving Credentials in a build
You may need the credentials during a build process. For example, retrieving a pre-trained model or data that was not stored in Qwak's Feature Store. To retrieve the credentials, import the SecretServiceGrpcClient
and use it to retrieve the secret:
from qwak import QwakModel
from qwak.clients.secret_service import SecretServiceClient
class TestModel(QwakModel)
# ...
def build():
secret_service = SecretServiceClient()
aws_api_key = secret_service.get_secret('aws_api_key')
aws_secret_key = secret_service.get_secret('aws_secret_key')
Protecting Secrets
Avoid printing or logging secret values as the model
stdout
is visible on the build logs.
Retrieving Credentials in the Feature Store
Qwak's Feature Store integrates with the Secret Service to enable secure access to data sources. Once you've created a new secret, use it in the data source definition to ensure secure authorized access.
Connecting to Snowflake Using Secrets
First we need to create new secrets with the user name and password.
qwak secrets set --name qwak_secret_snowflake_user --value <snowflake_user>
qwak secrets set --name qwak_secret_snowflake_password --value <secured_password_1234>
Define the Snowflake data source using the secret names:
from qwak.feature_store.sources.data_sources import SnowflakeSource
snowflake_source = SnowflakeSource(
name='my-snowflake-datasource',
description='An example snowflake data source',
date_created_column='DATE_COLUMN',
host='<SnowflakeAddress/DNS:port>',
username_secret_name='qwak_secret_snowflake_user', # secret name, stored in the Secret Service
password_secret_name='qwak_secret_snowflake_password', # secret name, stored in the Secret Service
database='snowflake_db_name',
schema='snowflake_schema_name',
warehouse='snowflake_data_warehouse_name'
)
Updated 7 months ago