Package bconfig provides configuration interfaces for log processing units and factory mechanism.

Package bmatch provides log matchers used for filtering and transforms.

Package bsupport provides helpers for log processing and abstract types for implementations, bsupport+base should be sufficient to construct and run a log pipeline.

Package btest provides test utilities and stubs of interfaces inside the base package.

MustNewLogSchema creates a new LogSchema or panic.

NewFieldSetExtractor creates a FieldSetExtractor.

NewLogAllocator creates LogAllocator linked to the given schema.

NewLogInputCounter creates a LogInputCounter.

NewLogProcessCounter creates a LogProcessCounter.

NewLogSchema creates a new LogSchema with field names and environment field names.

NewSinkLogger creates a derived logger for sinks created from MultiSinkMessageReceiver or MultiSinkBufferReceiver.

DROP means transform aborts and the record is to be dropped.

MaxClientNumber represents the max value of client number passed to MultiSinkMessageReceiver.NewSink The value affects how many clients we can have at the same time.

MissingFieldLocator represents non-existing index to a log field.

PASS means transform succeeds (whether there is change or not).

ChunkConsumerArgs is the parameters to create a ChunkConsumer For any chunk, either OnChunkConsumed or OnChunkSkipped must be called.

FieldSetExtractor extracts values from log records by pre-defined set of fields.

LogAllocator allocates empty log records and backing buffers Local-cache with buffers of recycled logs has been tried and made minimal improvement.

LogChunk represents a chunk of log records serialized and ready for storage or transport as its own unit.

LogChunkInfo contains information that can be extracted from a chunk binary for test purposes.

LogInputCounterSet tracks metrics for incoming logs LogInputCounterSet must be accessed through pointer.

LogProcessCounterSet tracks metrics for log transform, serialization and chunk making.

LogRecord defines the structure of log record before it's finalized for forwarding.

LogSchema defines the field names and mark fields which should belong to "environment".

BufferReceiverSink under MultiSinkBufferReceiver receives buffered logs from a single source, e.g.

ChunkBufferer is a worker to buffer completed chunks in memory and/or persistent storage Accept can be called concurrently by different goroutines It is to the bufferer to process any chunks left in input or output channels during shutdown.

ChunkConsumer is a worker to consume buffered chunks for forwarding or else.

ChunkDecoder provides an interface to verify and decode resulting chunks.

LogChunkMaker packages serialized log streams into certain chunk format for storage or transport, e.g.

LogCustomCounterRegistry allows registration of custom record counters by label RegisterCustomCounter returns a function to be called to count record length.

LogInput represents an input source in the beginning of pipeline, e.g.

LogListener represents an input endpoint for logs, e.g.

LogParser parses incoming logs from LogListener to structured records one by one.

LogRewriter rewrites the value of specific field during log serialization.

LogSerializer serializes log records into a stream of certain format, e.g.

LogTransform filters and/or transforms logs one by one May have persistent states.

MessageReceiverSink receives raw log messages from a single source, e.g.

MultiSinkBufferReceiver receives buffered logs from a multi-source input, e.g.

MultiSinkMessageReceiver receives raw log messages from a multi-source input, e.g.

Orchestrator takes log records and distribute them to internal pipelines.

PipelineWorker represents a background worker in a stage of the processing pipeline, e.g.

ChunkConsumerOverrideCreator defines a function to construct a ChunkConsumer override for testing purposes.

ClientNumber uniquely identifies a currently connected client from one of the inputs It's currently set to the FD of incoming connections TODO: use something other than FD in future for inputs not based on FD.

FilterResult defines the result of filtering, pass (true) or drop (false).

LogChunkAccepter is a function which accepts completed and loaded chunks for buffering or saving.

LogFieldLocator is used to locate a named field in LogRecord, bound to a LogSchema.

LogFields represents named fields in LogRecord, to be used with LogSchema.

LogStream represents serialized log record.

LogTransformFunc defines a function to perform transformation on a single log record.