Categorygithub.com/triggermesh/brokers
module
1.5.0
Repository: https://github.com/triggermesh/brokers.git
Documentation: pkg.go.dev
Overview
Versions
2
Dependencies
0
Dependents
0
Files
0

# README

Release Slack

TriggerMesh Brokers

TriggerMesh supported brokers.

Configuration

Configuration informs about the Triggers that send events to targets. Durations follow ISO 8601 format

triggers:
  trigger1:
    filters:
    - exact:
        type: example.type
    target:
      url: http://localhost:8888
      deliveryOptions:
        retry: 2
        backoffDelay: PT2S
        backoffPolicy: linear
  trigger2:
    target:
      url: http://localhost:9999
      deliveryOptions:
        retry: 5
        backoffDelay: PT5S
        backoffPolicy: constant
        deadLetterURL: http://localhost:9000

Usage

Produce CloudEvents by sending then using an HTTP client.

curl -v  http://localhost:8080/ \
  -H "Ce-Specversion: 1.0" \
  -H "Ce-Type: example.type" \
  -H "Ce-Source: example.source" \
  -H "Ce-Id: 1234-abcd-x" \
  -H "Content-Type: application/json" \
  -d '{"hello":"broker"}'

Redis

Redis Broker needs a Redis backing server to perform pub/sub operations and storage.

The broker uses a single Redis stream named triggermesh by default, that can be customized using redis.stream argument. The Redis user must be configured to use the stream group of commands on the stream key, plus using the client command with id subcomand for probes.

When using a single Redis backend, it is important to use a unique stream per broker to isolate messages.

# In this example the broker will be configured with user triggermesh1
# and stream name triggermeshstream

ACL SETUSER triggermesh1 on >7r!663R +@stream +client|id ~triggermeshstream

Non Authenticated Redis

# Create storage folder
mkdir -p .local/data

# Run Redis alternative
docker run -d -v $PWD/.local/data:/data \
    -e REDIS_ARGS="--appendonly yes --appendfsync always --rdbcompression yes" \
    --name redis-stack-server \
    -p 6379:6379 \
    redis/redis-stack-server:latest

Launch the broker providing parameters for the backing server.

go run ./cmd/redis-broker start \
  --redis.address "0.0.0.0:6379" \
  --broker-config-path ".local/broker-config.yaml"

Authenticated Redis

When using an authenticated Redis instance, user and password can be informed via redis.username and redis.password arguments.

go run ./cmd/redis-broker start \
  --redis.username triggermesh1 \
  --redis.password "7r\!663R" \
  --redis.address "some.redis.server:25101" \
  --broker-config-path .local/broker-config.yaml

TLS Enabled Redis

If the Redis instance is exposed using TLS, it must enabled at the broker config via redis.tls-enabled flag. For self-signed certificates you can inform them with redis.tls-ca-certificate or skip verification (not recommended) with redis.tls-skip-verify.

go run ./cmd/redis-broker start \
  --redis.username triggermesh1 \
  --redis.password "7r\!663R" \
  --redis.tls-enabled  \
  --redis.tls-ca-certificate="-----BEGIN CERTIFICATE-----abc123-----END CERTIFICATE-----" \
  --redis.address "tls.self.signed.redis.server:25102" \
  --broker-config-path .local/broker-config.yaml

When configuring TLS certificates for Redis authentication, make use of redis.tls-certificate and redis.tls-key.

go run ./cmd/redis-broker start \
  --redis.tls-enabled  \
  --redis.tls-certificate='-----BEGIN CERTIFICATE-----
deadbeef..
-----END CERTIFICATE-----' \
  --redis.tls-key='-----BEGIN PRIVATE KEY-----
c0ff33...
-----END PRIVATE KEY-----' \
  --redis.address "tls.redis.server:25102" \
  --broker-config-path .local/broker-config.yaml

Using Environment Variables

Parameters for the broker can be set as environment variables.

BROKER_CONFIG_PATH=.local/broker-config.yaml \
REDIS_ADDRESS=tls.self.signed.redis.server:25102 \
REDIS_USERNAME=triggermesh1 \
REDIS_PASSWORD=7r\!663R \
REDIS_TLS_ENABLED=true \
REDIS_TLS_SKIP_VERIFY=true \
go run ./cmd/redis-broker start

Note: when using a Redis cluster provide a comma separated list of nodes at REDIS_CLUSTER_ADDRESSES instead of the REDIS_ADDRESS parameter.

Memory

go run ./cmd/memory-broker start --memory.buffer-size 100 --memory.produce-timeout 1s --broker-config-path ".local/config.yaml"

Alternatively environment variables could be used.

CONFIG_PATH=.local/config.yaml MEMORY_BUFFER_SIZE=100 MEMORY_PRODUCE_TIMEOUT=1s go run ./cmd/memory-broker start

Container Images

docker build -t my-repo/redis-broker:my-version .
docker push my-repo/redis-broker:my-version

docker build -t my-repo/memory-broker:my-version .
docker push my-repo/memory-broker:my-version

Observability

The observability-config-path flag allows you to customize observability settings.

go run ./cmd/redis-broker start --redis.address "0.0.0.0:6379" \
  --broker-config-path .local/broker-config.yaml \
  --observability-config-path .local/observability-config.yaml

The file contains a zap-logger-config element where a zap configuration should be located. Updating the file will update the logging level.

zap-logger-config: |
  {
    "level": "info",
    "development": false,
    "outputPaths": ["stdout"],
    "errorOutputPaths": ["stderr"],
    "encoding": "json",
    "encoderConfig": {
      "timeKey": "timestamp",
      "levelKey": "severity",
      "nameKey": "logger",
      "callerKey": "caller",
      "messageKey": "message",
      "stacktraceKey": "stacktrace",
      "lineEnding": "",
      "levelEncoder": "",
      "timeEncoder": "iso8601",
      "durationEncoder": "",
      "callerEncoder": ""
    }
  }

Broker Parameters

Prefixes redis. and memory. apply only to their respective broker binaries.

NameEnvironmentDefaultInformation
broker-config-pathBROKER_CONFIG_PATH/etc/triggermesh/broker.confPath to broker configuration file.
observability-config-pathOBSERVABILITY_CONFIG_PATHPath to observability configuration file.
portPORT8080HTTP Port to listen for CloudEvents.
broker-nameBROKER_NAME{hostname}Instance name. When running at Kubernetes should be set to the pod name.
kubernetes-namespaceKUBERNETES_NAMESPACENamespace where the broker is running.
kubernetes-broker-config-secret-nameKUBERNETES_BROKER_CONFIG_SECRET_NAMESecret object name that contains the broker configuration.
kubernetes-broker-config-secret-keyKUBERNETES_BROKER_CONFIG_SECRET_KEYSecret object key that contains the broker configuration.
kubernetes-observability-configmap-nameKUBERNETES_OBSERVABILITY_CONFIGMAP_NAMEConfigMap object name that contains the observability configuration.
kubernetes-status-configmap-nameKUBERNETES_STATUS_CONFIGMAP_NAMEConfigMap object name where the broker instance should write its status.
kubernetes-status-configmap-keyKUBERNETES_STATUS_CONFIGMAP_KEYstatusConfigMap object key where the broker instance should write its status.
status-reporter-resync-check-periodSTATUS_REPORTER_RESYNC_CHECK_PERIODPT10SPeriod for running status checks for pending changes, using ISO8601.
status-reporter-resync-force-periodSTATUS_REPORTER_RESYNC_FORCE_PERIODPT1MPeriod for running status resync cycles that force status writes, using ISO8601.
config-polling-periodCONFIG_POLLING_PERIODPT0SISO8601 duration for config polling. Disabled if PT0S. Enabling it will disable other configuration methods.
broker-configBROKER_CONFIGJSON representation of broker configuration. Enabling it will disable other configuration methods.
observability-configBROKER_CONFIGJSON representation of observability configuration. Enabling it will disable other configuration methods.
observability-metrics-domainOBSERVABILITY_CONFIGtriggermesh.io/eventingDomain to be used for some metrics reporters.
redis.addressREDIS_ADDRESS0.0.0.0:6379Redis address for standalone instances.
redis.cluster-addressesREDIS_CLUSTER_ADDRESSESComma separated list of redis addresses for clustered instances.
redis.usernameREDIS_USERNAMERedis username.
redis.passwordREDIS_PASSWORDRedis password.
redis.databaseREDIS_DATABASE0Database ordinal at Redis.
redis.tls-enabledREDIS_TLS_ENABLEDfalseTLS enablement for Redis connection.
redis.tls-skip-verifyREDIS_TLS_SKIP_VERIFYfalseTLS skipping certificate verification.
redis.tls-ca-certificateREDIS_TLS_CA_CERTIFICATETLS CA certificate used to connect to Redis.
redis.tls-certificateREDIS_TLS_CERTIFICATETLS certificate used to authenticate with Redis.
redis.tls-keyREDIS_TLS_KEYTLS key used to authenticate with Redis.
redis.tracking-id-enabledREDIS_TRACKING_ID_ENABLEDfalseAdds the Redis ID for the event as triggermeshbackendid CloudEvents attribute.
redis.streamREDIS_STREAMtriggermeshStream name that stores the broker's CloudEvents.
redis.groupREDIS_GROUPdefaultRedis stream consumer group name.
redis.stream-max-lenREDIS_STREAM_MAX_LEN1000Limit the number of items in a stream by trimming it. Set to 0 for unlimited.
memory.buffer-sizeMEMORY_BUFFER_SIZE10000Number of events that can be hosted in the backend.
memory.produce-timeoutMEMORY_PRODUCE_TIMEOUTPT5SMaximum wait time for producing an event to the backend. Formatted as ISO8601 duration.

Generate License

Install addlicense:

go install github.com/google/[email protected]

Make sure all files contain a license

addlicense -c "TriggerMesh Inc." -y $(date +"%Y") -l apache -s=only ./**/*.go

# Packages

No description provided by the author
No description provided by the author
No description provided by the author