Lookup Processor

Overview

Advanced CSV-based lookup processor supporting multiple match types, duplicate handling, fallback fields, caching, and enrichment across logs, metrics, and traces.

Supported types: Logs · Metrics · Traces

General Settings

Parameter	Type	Default Value	Required	Description
csv	string	none	Yes	Path to the CSV file containing lookup data.
cache_size	integer	0	No	Maximum number of lookup results to cache for performance.
refresh_interval	string	1m	No	How often to reload the CSV file.

Lookup Mappings

Array of field mapping configurations.

Parameter	Type	Default Value	Required	Description
source_field	string	none	Yes	Field name to use for lookup matching.
source_context	string	none	Yes	Location of the source field in the data (body, attributes, resource.attributes).
target_fields	object	none	Yes	Mapping of CSV columns to target fields.
target_context	string	none	Yes	Location where enriched fields will be added (body, attributes, resource.attributes).
match_type	string	exact	No	Type of matching to perform against source field (supported values: exact, regex, cidr).
case_sensitive	boolean	false	No	Whether to perform case-sensitive matching.
duplicate_handling	string	overwrite	No	How to handle duplicate matches (supported values: error, overwrite, skip, append_unique).
fallback_fields	array	[]	No	Additional source fields to try if primary lookup fails.
default_values	object	{}	No	Default values to use when no match is found.

Example Configuration

{
  "csv": "/path/to/lookup.csv", // required
  "cache_size": 1000,
  "refresh_interval": "1m", // default: "1m"

  "mappings": [
    // required, min 1
    {
      "source_field": "ip",
      "source_context": "attributes", // required
      "target_fields": {
        "owner": "owner",
        "department": "dept",
      },
      "target_context": "attributes", // required
      "match_type": "cidr", // default: "exact"
      "case_sensitive": false,
      "duplicate_handling": "overwrite", // default: "overwrite"
      "fallback_fields": ["fallback_ip"],
      "default_values": {
        "owner": "unknown",
      },
    },
  ],
}