Batch

Overview

The Batch processor accepts spans, metrics, or logs and places them into batches. Batching helps better compress the data and reduce the number of outgoing connections required to transmit the data. This processor supports both size and time based batching.

The batch processor should be defined in the pipeline after the memory_limiter as well as any sampling processors. This is because batching should happen after any data drops such as sampling.

Supported types: Logs · Metrics · Traces

Configuration

Parameter	Type	Default Value	Description
send_batch_size	int	8192	Number of spans, metric data points, or log records after which a batch will be sent regardless of the timeout. This acts as a trigger and does not affect the size of the batch.
timeout	duration	200ms	Time duration after which a batch will be sent regardless of size. If set to 0, send_batch_size is ignored as data will be sent immediately, subject to only send_batch_max_size.
send_batch_max_size	int	0	The upper limit of the batch size. 0 means no upper limit of the batch size. This property ensures that larger batches are split into smaller units. It must be greater than or equal to send_batch_size.

Example Configuration

{
  "send_batch_size": 8192, // required, default: 8192
  "timeout": "200ms", // required, default: "200ms"
  "send_batch_max_size": 0, // required, default: 0
}