Run make test-integration to run all the integration tests. Tests require Docker to be installed and running. The command will handle starting and stopping docker containers for you.

If you want to run the integration tests against your AWS DynamoDB instead of docker, You must set the environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION) before running make test, or before running the tests manually.

Source

A source connector that pulls data from a DynamoDB table to downstream resources via Conduit.

The connector starts with a snapshot of the data currently existent in the table, sends these records to the destination, then starts the CDC (Change Data Capture) mode which will listen to events happening on the table in real-time, and sends these events' records to the destination (these events include: updates, deletes, and inserts). You can opt out from taking the snapshot by setting the parameter skipSnapshot to true, meaning that only the CDC events will be captured.

The source connector uses DynamoDB Streams to get CDC events, so you need to enable the stream before running the connector. Check out the documentation for how to enable a DynamoDB Stream.

Configuration

name	description	required	default	example
`table`	Table is the DynamoDB table name to pull data from.	true		Employees
`aws.region`	AWS region.	true		us-east-1
`aws.accessKeyId`	AWS access key id.	true		MY_ACCESS_KEY_ID
`aws.secretAccessKey`	AWS secret access key.	true		MY_SECRET_ACCESS_KEY
`aws.url`	The URL for AWS (useful when testing the connector with localstack).	false		http://localhost:4566
`discoveryPollingPeriod`	Polling period for the CDC mode of how often to check for new shards in the DynamoDB Stream, formatted as a time.Duration string.	false	10s	100ms, 1m, 10m, 1h
`recordsPollingPeriod`	Polling period for the CDC mode of how often to get new records from a shard, formatted as a time.Duration string.	false	1s	100ms, 1m, 10m, 1h
`skipSnapshot`	Determines weather to skip the snapshot or not.	false	false	true