Memory Limiter

Overview

The Memory Limiter processor is a circuit breaker that prevents the collector from running out of memory by rejecting incoming data when memory usage approaches a configured threshold. Two thresholds:

• Soft limit (limit_* minus spike_limit_*): when crossed, the processor returns OK to forwarding receivers but tells push-based receivers to back off and triggers a forced GC.
• Hard limit (limit_*): when crossed, the processor returns errors back to receivers, which translate that into "drop" or "retry" depending on receiver type.

This processor is strongly recommended as the first processor in every pipeline — without it, a sustained ingest spike will OOM-kill the collector pod.

Supported types: Logs · Metrics · Traces

Configuration

Specify limits using either absolute MiB or percentage of available memory — not both.

Parameter	Type	Default	Required	Description
check_interval	duration	—	Yes	How often the processor samples the runtime memory stats. Must be > 0. Typical: 1s.
limit_mib	int	—	Yes (or limit_percentage)	Absolute hard limit in MiB. Above this, the processor refuses data. Mutually exclusive with limit_percentage.
spike_limit_mib	int	20% of limit_mib	No	Distance below limit_mib to set the soft limit. Must be smaller than limit_mib.
limit_percentage	int	—	Yes (or limit_mib)	Hard limit as a percentage of available memory (1–100). Mutually exclusive with limit_mib.
spike_limit_percentage	int	—	No	Distance below limit_percentage for the soft limit. Must be smaller.
min_gc_interval_when_soft_limited	duration	upstream default	No	Minimum interval between forced GCs when above the soft limit. Prevents GC thrashing.
min_gc_interval_when_hard_limited	duration	upstream default	No	Minimum interval between forced GCs when above the hard limit. Should be ≤ min_gc_interval_when_soft_limited.

Example Configuration

Absolute limits

{
  "check_interval": "1s",
  "limit_mib": 512,
  "spike_limit_mib": 128,
  "min_gc_interval_when_soft_limited": "10s",
  "min_gc_interval_when_hard_limited": "1s",
}

Percentage of available memory

{
  "check_interval": "1s",
  "limit_percentage": 80,
  "spike_limit_percentage": 20,
}

Notes

• Always first. Place this processor before any other processor in every pipeline. Receivers respect its backpressure signal; downstream processors don't get the chance to refuse data once it's already through.
• Pair with K8s resource limits. Set the collector pod's resources.limits.memory to roughly 1.5× limit_mib so the kubelet doesn't OOM-kill the pod before the limiter has a chance to apply backpressure.
• Percentage is calculated against the cgroup limit when running in a container, not host memory — safe in K8s.
• Sampling cost: check_interval of 1s costs ~negligible CPU; don't push it below 100ms or runtime stats sampling becomes the dominant cost.
• Bursty workloads: widen the spike margin. A small spike margin will cause the limiter to oscillate between soft/hard limits, dropping more than necessary.