# README
Conduit Connector for HTTP
The HTTP connector is a Conduit plugin. It provides both, a source and a destination HTTP connectors.
How to build?
Run make build to build the connector's binary.
Testing
Run make test to run all the unit tests.
Source
The HTTP source connector pulls data from the HTTP URL every pollingPeriod, the source adds the params and headers
to the request, and sends it to the URL with the specified method from the Configuration. The returned data is
used to create an openCDC record and return it.
Note: when using the OPTIONS method, the resulted options will be added to the record's metadata.
Configuration
| name | description | required | default value | example |
|---|---|---|---|---|
url | HTTP URL to send requests to. | true | https://example.com/api/v1 | |
method | HTTP method to use in the request, supported methods are (GET,HEAD,OPTIONS). | false | GET | POST |
headers | HTTP headers to use in the request, comma separated list of : separated pairs. | false | Authorization:Bearer TOKEN_VALUE,Content-Type:application/xml | |
params.* | parameters to use in the request, use params.* as the config key and specify its value, ex: set "params.id" as "1". | false | params.query="foobar" | |
pollingperiod | how often the connector will get data from the url, formatted as a time.Duration. | false | "5m" | "5m" |
script.parseResponse |
The path to a .js file containing the code to parse the response. The signature of the function needs to be: where The function needs to return a | false | /path/to/get_request_data.js An example script can be found in test/get_request_data.js | |
script.getRequestData |
The path to a .js file containing the code to prepare the request data. The signature of the function needs to be: where:
The function needs to return a | false | /path/to/parse_response.js An example script can be found in test/parse_response.js
|
Destination
The HTTP destination connector pushes data from upstream resources to an HTTP URL via Conduit. the destination adds the
params and headers to the request, and sends it to the URL with the specified method from the Configuration.
Note: The request Body that will be sent is the value under record.Payload.After, if you want to change the format
of that or manipulate the field in any way, please check our Builtin Processors Docs
, or check Standalone Processors Docs if you'd like to build your own processor .
Configuration
| name | description | required | default value |
|---|---|---|---|
url | Is a Go template expression for the URL used in the HTTP request, using Go templates. The value provided to the template is opencdc.Record, so the template has access to all its fields (e.g. .Position, .Key, .Metadata, and so on). We also inject all template functions provided by sprig to make it easier to write templates. | true | |
method | Http method to use in the request, supported methods are (POST,PUT,DELETE,PATCH). | false | POST |
headers | Http headers to use in the request, comma separated list of : separated pairs. | false | |
params.* | parameters to use in the request, use params.* as the config key and specify its value, ex: set "params.id" as "1". | false |