Categorygithub.com/ONSdigital/dp-import-api
module
1.27.0
Repository: https://github.com/onsdigital/dp-import-api.git
Documentation: pkg.go.dev

# README

DP Import API

Getting started

MongoDB

  • Run brew install mongodb
  • Run brew services start mongodb

kafka

  • Run brew install java
  • Run brew install kafka
  • Run brew services start zookeeper
  • Run brew services start kafka

Scripts for updating and debugging Kafka can be found here(dp-data-tools)

Configuration

Environment variableDefaultDescription
BIND_ADDR:21800The host and port to bind to
HOSThttp://localhost:21800The host name used to build URLs
KAFKA_ADDRlocalhost:9092A list of kafka brokers (TLS-ready)
KAFKA_VERSION1.0.2The version of (TLS-ready) Kafka being used
KAFKA_MAX_BYTES200000The max message size for kafka producer
KAFKA_SEC_PROTOunsetif set to TLS, kafka connections will use TLS (ref-1)
KAFKA_SEC_CLIENT_KEYunsetPEM for the client key (ref-1)
KAFKA_SEC_CLIENT_CERTunsetPEM for the client certificate (ref-1)
KAFKA_SEC_CA_CERTSunsetCA cert chain for the server cert (ref-1)
KAFKA_SEC_SKIP_VERIFYfalseignores server certificate issues if true (ref-1)
DATABAKER_IMPORT_TOPICdata-bake-job-availableThe topic to place messages to data-baker
INPUT_FILE_AVAILABLE_TOPICinput-file-availableThe topic to place V4 files
MONGODB_BIND_ADDRlocalhost:27017The MongoDB bind address
MONGODB_USERNAMEThe MongoDB Username
MONGODB_PASSWORDThe MongoDB Password
MONGODB_DATABASEimportsThe MongoDB database
MONGODB_COLLECTIONSImportsCollection:imports, ImportsLockCollection:imports_locksThe MongoDB collections
MONGODB_REPLICA_SETThe name of the MongoDB replica set
MONGODB_ENABLE_READ_CONCERNfalseSwitch to use (or not) majority read concern
MONGODB_ENABLE_WRITE_CONCERNtrueSwitch to use (or not) majority write concern
MONGODB_CONNECT_TIMEOUT5sThe timeout when connecting to MongoDB (time.Duration format)
MONGODB_QUERY_TIMEOUT15sThe timeout for querying MongoDB (time.Duration format)
MONGODB_IS_SSLfalseSwitch to use (or not) TLS when connecting to mongodb
DATASET_API_URLhttp://localhost:22000The URL for the DatasetAPI
RECIPE_API_URLhttp://localhost:22300The URL for the RecipeAPI
ZEBEDEE_URLhttp://localhost:8082The URL Zebedee
SERVICE_AUTH_TOKEN0C30662F-6CF6-43B0-A96A-954772267FF5The token used to identify this service when authenticating
HEALTHCHECK_INTERVAL30sThe time between calling healthcheck endpoints for check subsystems
HEALTHCHECK_CRITICAL_TIMEOUT90sThe time taken for the health changes from warning state to critical due to subsystem check failures
DEFAULT_MAXIMUM_LIMIT1000Default maximum limit for pagination
DEFAULT_LIMIT20Default limit for pagination
DEFAULT_OFFSET0Default offset for pagination

Contributing

See CONTRIBUTING for details.

License

Copyright © 2016-2022, Office for National Statistics

Released under MIT license, see LICENSE for details.

# Packages

No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author
No description provided by the author