Qwak's data sources are used to configure connections to your data. Data sources are used in order to create create feature sets.
There are two main types of data sources:
- Batch: Data-at-rest sources of data, such as Athena, Snowflake, and Redshift.
- Streaming: Data in motion sources, such as Kafka and Kinesis.
To connect to a data source:
- Enable network connectivity between the data sources and Qwak's cluster if they are not publicly accessible.
- Grant Qwak access to your data lake components by creating read-only service accounts and/or IAM roles.
- Select Data Sources.
- Click Create New Data Source.
- Select the required data source type from the list.
- Fill in the form (all required fields are marked with an asterisk).
- Typically at this point you would want to test connection with the provided details, but it's not yet supported via the UI. Currently you can only do it via SDK.
- Click Save.
- The data source is created.
Updated over 1 year ago