# README
MongoDB adaptor
The MongoDB adaptor is capable of reading/tailing collections and receiving data for inserts.
NOTE You may want to check your collections to ensure the proper index(es) are in place or performance may suffer.
Example
m = mongodb({
"uri": "mongodb://127.0.0.1:27017/test"
// "timeout": "30s",
// "tail": false,
// "ssl": false,
// "cacerts": ["/path/to/cert.pem"],
// "wc": 1,
// "fsync": false,
// "bulk": false,
// "collection_filters": "{\"foo\": {\"i\": {\"$gt\": 10}}}"
})
Options
| Parameter | Description | Default |
|---|---|---|
| uri | Defines the full connection string of the MongoDB database. | mongodb://127.0.0.1:27017/test |
| timeout | Overrides the default session timeout and should be parseable by time.ParseDuration | 10s |
| tail | Whether the source connection will listen for updates after the initial sync (requires oplog access) | false |
| ssl | Configures the database connection to connect via TLS | false |
| cacerts | Configures the RootCAs for the underlying TLS connection | [] |
| wc | Configures the write concern option for the session | 0 |
| fsync | Whether the server will wait for Fsync to complete before returning a response | false |
| bulk | Whether the sink connection will use bulk inserts rather than writing one record at a time. | false |
| collection_filters | A JSON string where the top level key is the collection name and its value is a query that will be used when iterating the collection. The commented out example above would only include documents where the i field had a value greater than 10 | {} |
Run adaptor test
Spin up required containers
You'll need those ports on your local machine: 10000, 10001, 27017, 27018, 11112, 29017, 15000, 20000
So make sure to kill anything that might use them (like a local mongo instance)
# From transporter's root folder
version=3.2.11
# Pay attention to a WARNING telling you to add a line to /etc/hosts in the following command
scripts/run_db_in_docker.sh mongodb $version
Run the tests
# From transporter's root folder
go test -v ./adaptor/mongodb/
Tear down containers
Once you're done
TESTDIR=adaptor/mongodb scripts/teardown_db_in_docker.sh
# Functions
NewClient creates a new client to work with MongoDB.
WithCACerts configures the RootCAs for the underlying TLS connection.
WithFsync configures whether the server will wait for Fsync to complete before returning a response (Default: false).
WithReadPreference sets the MongoDB read preference based on the provided string.
WithSSL configures the database connection to connect via TLS.
WithTail set the flag to tell the Client whether or not access to the oplog will be needed (Default: false).
WithTimeout overrides the DefaultSessionTimeout and should be parseable by time.ParseDuration.
WithURI defines the full connection string of the MongoDB database.
WithWriteConcern configures the write concern option for the session (Default: 0).
# Constants
DefaultMaxWriteBatchSize when using the bulk interface.
DefaultReadPreference when connecting to a mongo replica set.
DefaultSessionTimeout is the default timeout after which the session times out when unable to connect to the provided URI.
DefaultURI is the default endpoint of MongoDB on the local machine.
# Variables
DefaultCollectionFilter is an empty map of empty maps.
DefaultSafety is the default saftey mode used for the underlying session.
ErrCollectionFilter is returned when an error occurs attempting to Unmarshal the string.
# Structs
Bulk implements client.Writer for use with MongoDB and takes advantage of the Bulk API for performance improvements.
Client represents a client to the underlying MongoDB source.
InvalidReadPreferenceError represents the error when an incorrect mongo read preference has been set.
OplogAccessError wraps the underlying error when access to the oplog fails.
Reader implements the behavior defined by client.Reader for interfacing with MongoDB.
Session serves as a wrapper for the underlying mgo.Session.
Writer implements client.Writer for use with MongoDB.
# Type aliases
ClientOptionFunc is a function that configures a Client.
CollectionFilter is just a typed map of strings of map[string]interface{}.