Filter Processor

Status
Stability	alpha: traces, metrics, logs
Distributions	core, contrib, k8s
Warnings	Orphaned Telemetry, Other
Issues
Code Owners	@TylerHelmuth, @boostchicken

The filterprocessor allows dropping spans, span events, metrics, datapoints, and logs from the collector.

Configuration

The filterprocessor utilizes the OpenTelemetry Transformation Language to create conditions that determine when telemetry should be dropped. If any condition is met, the telemetry is dropped (each condition is ORed together). Each configuration option corresponds with a different type of telemetry and OTTL Context. See the table below for details on each context and the fields it exposes.

Config	OTTL Context
traces.span	Span
traces.spanevent	SpanEvent
metrics.metric	Metric
metrics.datapoint	DataPoint
logs.log_record	Log

The OTTL allows the use of and, or, and () in conditions. See OTTL Boolean Expressions for more details.

For conditions that apply to the same signal, such as spans and span events, if the "higher" level telemetry matches a condition and is dropped, the "lower" level condition will not be checked. This means that if a span is dropped but a span event condition was defined, the span event condition will not be checked for that span. The same relationship applies to metrics and datapoints.

If all span events for a span are dropped, the span will be left intact. If all datapoints for a metric are dropped, the metric will also be dropped.

The filter processor also allows configuring an optional field, error_mode, which will determine how the processor reacts to errors that occur while processing an OTTL condition.

error_mode	description
ignore	The processor ignores errors returned by conditions, logs them, and continues on to the next condition. This is the recommended mode.
silent	The processor ignores errors returned by conditions, does not log them, and continues on to the next condition.
propagate	The processor returns the error up the pipeline. This will result in the payload being dropped from the collector.

If not specified, propagate will be used.

Examples

processors:
  filter/ottl:
    error_mode: ignore
    traces:
      span:
        - 'attributes["container.name"] == "app_container_1"'
        - 'resource.attributes["host.name"] == "localhost"'
        - 'name == "app_3"'
      spanevent:
        - 'attributes["grpc"] == true'
        - 'IsMatch(name, ".*grpc.*")'
    metrics:
      metric:
          - 'name == "my.metric" and resource.attributes["my_label"] == "abc123"'
          - 'type == METRIC_DATA_TYPE_HISTOGRAM'
      datapoint:
          - 'metric.type == METRIC_DATA_TYPE_SUMMARY'
          - 'resource.attributes["service.name"] == "my_service_name"'
    logs:
      log_record:
        - 'IsMatch(body, ".*password.*")'
        - 'severity_number < SEVERITY_NUMBER_WARN'

Dropping data based on a resource attribute

processors:
  filter:
    error_mode: ignore
    traces:
      span:
        - IsMatch(resource.attributes["k8s.pod.name"], "my-pod-name.*")

Dropping metrics with invalid type

processors:
  filter:
    error_mode: ignore
    metrics:
      metric:
        - type == METRIC_DATA_TYPE_NONE

Dropping specific metric and value

processors:
  filter:
    error_mode: ignore
    metrics:
      datapoint:
        - metric.name == "k8s.pod.phase" and value_int == 4

Dropping non-HTTP spans

processors:
  filter:
    error_mode: ignore
    traces:
      span:
        - attributes["http.request.method"] == nil

Dropping HTTP spans

processors:
  filter:
    error_mode: ignore
    traces:
      span:
        - attributes["http.request.method"] != nil

OTTL Functions

The filter processor has access to all OTTL Converter functions

In addition, the processor defines a few of its own functions:

Metrics only functions

• HasAttrKeyOnDatapoint
• HasAttrOnDatapoint

HasAttrKeyOnDatapoint

HasAttrKeyOnDatapoint(key)

Returns true if the given key appears in the attribute map of any datapoint on a metric. key must be a string. You must use the metrics.metric context.

Examples:

• HasAttrKeyOnDatapoint("http.method")

# Drops metrics containing the 'bad.metric' attribute key
filter/keep_good_metrics:
  error_mode: ignore
  metrics:
    metric:
      - 'HasAttrKeyOnDatapoint("bad.metric")'

HasAttrOnDatapoint

HasAttrOnDatapoint(key, value)

Returns true if the given key and value appears in the attribute map of any datapoint on a metric. key and value must both be strings. If the value of the attribute on the datapoint is not a string, value will be compared to "". You must use the metrics.metric context.

Examples:

• HasAttrOnDatapoint("http.method", "GET")

# Drops metrics containing the 'bad.metric' attribute key and 'true' value
filter/keep_good_metrics:
  error_mode: ignore
  metrics:
    metric:
      - 'HasAttrOnDatapoint("bad.metric", "true")'

Troubleshooting

When using OTTL you can enable debug logging in the collector to print out useful information, such as if the condition matched and the TransformContext used in the condition, to help you troubleshoot why a condition is not behaving as you expect. This feature is very verbose, but provides you an accurate view into how OTTL views the underlying data.

receivers:
  filelog:
    start_at: beginning
    include: [ /Users/tylerhelmuth/projects/opentelemetry-collector-contrib/local/test.log ]


processors:
  filter:
    error_mode: ignore
    logs:
      log_record:
        - body == "test"

exporters:
  debug:

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    logs:
      receivers:
        - filelog
      processors:
        - filter
      exporters:
        - debug

2024-05-29T16:47:04.362-0600    debug   [email protected]/parser.go:338     condition evaluation result     {"kind": "processor", "name": "filter", "pipeline": "logs", "condition": "body == \"test\"", "match": true, "TransformContext": {"resource": {"attributes": {}, "dropped_attribute_count": 0}, "scope": {"attributes": {}, "dropped_attribute_count": 0, "name": "", "version": ""}, "log_record": {"attributes": {"log.file.name": "test.log"}, "body": "test", "dropped_attribute_count": 0, "flags": 0, "observed_time_unix_nano": 1717022824262063000, "severity_number": 0, "severity_text": "", "span_id": "", "time_unix_nano": 0, "trace_id": ""}, "cache": {}}}

Warnings

In general, understand your data before using the filter processor.

• When using the filterprocessor make sure you understand the look of your incoming data and test the configuration thoroughly. In general, use as specific a configuration as possible to lower the risk of the wrong data being dropped.
• Orphaned Telemetry: The processor allows dropping spans. Dropping a span may lead to orphaned spans if the dropped span is a parent. Dropping a span may lead to orphaned logs if the log references the dropped span.