# README
Netflow Receiver
The netflow receiver is capable of listening for netflow, sflow or IPFIX UDP traffic and generating log entries based on the flow content.
This gives Opentelemetry users the capability of monitoring network traffic, and answer questions like:
- Which protocols are passing through the network?
- Which servers and clients are producing the highest amount of traffic?
- What ports are involved in these network calls?
- How many bytes and packets are being sent and received?
The receiver listens for flows and decodes them using the templates that are sent by the flow producers. The data then is converted to JSON and produces structured log records.
Using the receiver
Docker
A collector with the receiver can be run with docker:
# windows
docker run --name otelcol -v ${PWD}/config.yaml:/etc/otel/config.yaml -p 2055:2055/udp dlopes7/otelcol-netflow-receiver
# linux
docker run --name otelcol -v $(pwd)/config.yaml:/etc/otel/config.yaml -p 2055:2055/udp dlopes7/otelcol-netflow-receiver
This repo contains an example of a config.yaml
:
receivers:
netflow:
hostname: "0.0.0.0"
scheme: netflow
port: 2055
sockets: 16
workers: 32
processors:
batch:
send_batch_size: 30
timeout: 30s
exporters:
debug:
otlphttp:
endpoint: https://<environment>.live.dynatrace.com/api/v2/otlp
headers:
Authorization: "Api-Token <dynatrace_ingest_logs_token>"
service:
pipelines:
logs:
receivers: [netflow]
processors: [batch]
exporters: [debug, otlphttp]
telemetry:
logs:
level: debug
Will start a collector listening on the port 2055 UDP of the host for netflow data. You can send demo netflow data to that container with:
docker run --net="host" dlopes7/netflow-generator -r 10 -t localhost -p 2055
Building a collector
Build a collector with the receiver by using:
Example builder-config.yaml
:
dist:
name: otelcol-dev
description: Basic OTel Collector distribution for Developers
output_path: ./otelcol-dev
exporters:
- gomod:
go.opentelemetry.io/collector/exporter/debugexporter v0.116.0
- gomod:
go.opentelemetry.io/collector/exporter/otlphttpexporter v0.116.0
processors:
- gomod:
go.opentelemetry.io/collector/processor/batchprocessor v0.116.0
receivers:
- gomod: github.com/dynatrace-extensions/netflowreceiver v1.3.0
path: .
The collector can be built with:
go get go.opentelemetry.io/collector/cmd/builder
go install go.opentelemetry.io/collector/cmd/builder
builder --config builder-config.yaml
Netflow receiver
Status | |
---|---|
Stability | development: logs |
Distributions | [] |
Issues | |
Code Owners | @evan-bradley, @dlopes7 |
The netflow receiver can listen for netflow, sflow, and ipfix data and convert it to OpenTelemetry logs. The receiver is based on the goflow2 project.
This gives OpenTelemetry users the capability of monitoring network traffic, and answer questions like:
- Which protocols are passing through the network?
- Which servers and clients are producing the highest amount of traffic?
- What ports are involved in these network calls?
- How many bytes and packets are being sent and received?
Getting started
By default the receiver will listen for ipfix and netflow on port 2055
. The receiver can be configured to listen on different ports and protocols.
Example configuration:
receivers:
netflow:
- scheme: netflow
port: 2055
sockets: 16
workers: 32
processors:
batch:
send_batch_size: 2000
timeout: 30s
exporters:
debug:
verbosity: detailed
service:
pipelines:
logs:
receivers: [netflow]
processors: [batch]
exporters: [debug]
telemetry:
logs:
level: debug
We recommend using the batch processor to reduce the number of log requests being sent to the exporter. The batch processor will batch log records together and send them in a single request to the exporter.
You would then configure your network devices to send netflow, sflow, or ipfix data to the Collector on the specified ports.
Configuration
Field | Description | Examples | Default |
---|---|---|---|
scheme | The type of flow data that to receive | sflow , netflow , flow | netflow |
hostname | The hostname or IP address to bind to | localhost | 0.0.0.0 |
port | The port to bind to | 2055 or 6343 | 2055 |
sockets | The number of sockets to use | 1 | 1 |
workers | The number of workers used to decode incoming flow messages | 2 | 2 |
queue_size | The size of the incoming netflow packets queue | 1000 | 1000000 |
Data format
The netflow data is standardized for the different schemas and is converted to OpenTelemetry logs following the semantic conventions
The output will adhere the format:
{
"destination": {
"address": "192.168.0.1",
"port": 22
},
"flow": {
"end": 1731073104662487000,
"sampler_address": "192.168.0.2",
"sequence_num": 49,
"start": 1731073077662487000,
"time_received": 1731073138662487000,
"type": "NETFLOW_V5"
},
"io": {
"bytes": 529,
"packets": 378
},
"source": {
"address": "192.168.0.3",
"port": 40
},
"transport": "TCP",
"type": "IPv4"
}