Cloudwatch Receiver

Status
Stability	alpha: logs
Distributions	contrib
Issues
Code Owners	@schmikei | Seeking more code owners!
Emeritus	@djaglowski

Receives Cloudwatch events from AWS Cloudwatch via the AWS SDK for Cloudwatch Logs

Getting Started

This receiver uses the AWS SDK as mode of authentication, which includes Credentials File and IMDS authentication for EC2 instances.

Configuration

Top Level Parameters

Parameter	Notes	type	Description
region	required	string	The AWS recognized region string
profile	optional	string	The AWS profile used to authenticate, if none is specified the default is chosen from the list of profiles
imds_endpoint	optional	string	A way of specifying a custom URL to be used by the EC2 IMDS client to validate the session. If unset, and the environment variable AWS_EC2_METADATA_SERVICE_ENDPOINT has a value the client will use the value of the environment variable as the endpoint for operation calls.
logs	optional	Logs	Configuration for Logs ingestion of this receiver

Logs Parameters

Parameter	Notes	type	Description
poll_interval	default=1m	duration	The duration waiting in between requests.
max_events_per_request	default=1000	int	The maximum number of events to process per request to Cloudwatch
groups	optional	See Group Parameters	Configuration for Log Groups, by default all Log Groups and Log Streams will be collected.

Group Parameters

autodiscover and named are ways to control and filter which log groups and log streams which are collected from. They are mutually exclusive and are incompatible to be configured at the same time.

• autodiscover
  • limit: (optional; default = 50) Limits the number of discovered log groups. This does not limit how large each API call to discover the log groups will be.
  • prefix: (optional) A prefix for log groups to limit the number of log groups discovered.
    • if omitted, all log streams up to the limit are collected from
  • streams: (optional) If streams is omitted, then all streams will be attempted to retrieve events from.
    • names: A list of full log stream names to filter the discovered log groups to collect from.
    • prefixes: A list of prefixes to filter the discovered log groups to collect from.
• named
  • This is a map of log group name to stream filtering options
    • streams: (optional)
      • names: A list of full log stream names to filter the discovered log groups to collect from.
      • prefixes: A list of prefixes to filter the discovered log groups to collect from.

Autodiscovery Example Configuration

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 1m
    groups:
      autodiscover:
        limit: 100
        prefix: /aws/eks/
        streams:
          prefixes: [kube-api-controller]

Named Example

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 5m
    groups:
      named:
        /aws/eks/dev-0/cluster: 
          names: [kube-apiserver-ea9c831555adca1815ae04b87661klasdj]

Sample Configs

This receiver has a number of sample configs for reference.

1. Default

   • Minimal configuration of the receiver
   • Performs autodiscovery
   • Collects all log groups and log streams

2. Autodiscover Filtering Log Groups

   • Performs autodiscovery
   • Only collects log groups matching a prefix
   • Limits the number of discovered Log Groups

3. Autodiscover Filtering Log Streams

   • Performs autodiscovery for all Log Groups
   • Filters log streams

4. Named Groups

   • Specifies and only collects from the desired Log Groups
   • Does not attempt autodiscovery

5. Named Groups Filter Log Streams

   • Specifies the names of the log groups to collect
   • Does not attempt autodiscovery
   • Only collects from log streams matching a prefix