Data Sources

Qwak's data sources are used to configure connections to your data. Data sources are used in order to create create feature sets.

There are two main types of data sources:

• Batch: Data-at-rest sources of data, such as Athena, Snowflake, and Redshift.
• Streaming: Data in motion sources, such as Kafka and Kinesis.

To connect to a data source:

1. Enable network connectivity between the data sources and Qwak's cluster if they are not publicly accessible.
2. Grant Qwak access to your data lake components by creating read-only service accounts and/or IAM roles.

Data Source Creation in the UI

1. Select Data Sources.

2. Click Create New Data Source.

3. Select the required data source type from the list.

4. Fill in the form (all required fields are marked with an asterisk).

5. Typically at this point you would want to test connection with the provided details, but it's not yet supported via the UI. Currently you can only do it via SDK.

6. Click Save.

7. The data source is created. 👍