Azure Blob Storage

Overview

Azure Blob Storage reads log blobs out of a container in an Azure storage account. That container is usually one of two things: an archive an Azure diagnostic setting writes to, or the output of a Praxis Azure Blob Storage destination — this source reads both, so a container can be an archive and a replay point at the same time.

Supported types: Logs

Supported platforms

Linux: Logs
Windows: Logs
macOS: Logs

Typical uses:

Azure diagnostic archives — a diagnostic setting pointed at a storage account rather than an Event Hub. The blobs land under y=/m=/d=/h=/m=/ paths, one JSON record per line.
Replay — re-read what an Azure Blob Storage destination already archived, without touching the original producer.
Third-party drop box — a container some other system writes log files into.

Requires collector v0.5.54 or later. Earlier releases do not build the Azure Blob receiver, and the node will not be offered for older agents.

Blobs are left in place. Reading does not delete. Turn on Delete Blobs After Reading only when the container is a dedicated drop box nothing else consumes.

Authentication Configuration

The node holds no secret. Attach one or two credentials — which ones you attach also decides how the source discovers new blobs:

Credential	Required	What it does
Azure Blob Storage (`azure_blob`)	Yes	The storage account identity. This is the same credential type the Azure Blob Storage destination uses, so one credential can serve both.
Connection String (`connection_string`)	No	An Event Hub receiving Event Grid `BlobCreated` notifications. Attaching it switches the node from polling to notification mode.

Both slots live inside the Credentials group on the node form. Use Add a credential to fill a slot; the picker only offers the two types above.

azure_blob offers connection_string (a storage account connection string), service_principal, system_managed_identity, user_managed_identity and workload_identity. Only connection_string and service_principal store a secret.

Prefer a managed identity where the collector runs inside Azure. Those types store no account key at all — the collector resolves a short-lived token from Entra ID.

Note: Whichever identity you use must hold Storage Blob Data Reader on the container (or account) scope, and Storage Blob Data Contributor if Delete Blobs After Reading is on. A missing role assignment surfaces as a 403 on the first listing, not a configuration error.

The Event Hub connection string must contain EntityPath=<event hub name>. Copy it from the Event Hub instance's shared access policy, not the namespace's. A namespace-level RootManageSharedAccessKey string has no EntityPath, so there is no way to tell which hub to read. Praxis rejects such a string when the pipeline is saved.

Basic Configuration

Parameter	Type	Default	Required	Description
container	string	none	Yes	The blob container to read. It must already exist — this source does not create it.
prefix	string	none	No	Read only blobs whose name starts with this, e.g. `year=2026/month=09/`. Strongly recommended on a busy container.
encoding	string	`otlp_json`	No	How each blob is decoded (Enum: `otlp_json`, `otlp_proto`, `json`, `json_array`, `text`).

Set a prefix on any container that is not tiny. Listing is the first thing the source does and it pages through the container 5,000 blobs at a time. On a container holding a year of diagnostic output an unfiltered listing can take longer than the poll interval, and the node ingests nothing while it runs. A prefix that pins the year, or the year and month, turns that into a handful of pages. It is not mandatory, because a dedicated drop box does not need one.

Blob Format

Value	Reads	Use for
OTLP JSON (`otlp_json`)	OTLP `ExportLogsServiceRequest` as JSON	What an Azure Blob Storage destination writes. The default.
OTLP protobuf (`otlp_proto`)	The same thing, binary	A destination configured to write protobuf.
JSON records (`json`)	A stream of JSON objects, one record each, newline-delimited or concatenated	Azure diagnostic archives. This is the shape a diagnostic setting produces.
JSON array (`json_array`)	One JSON array per blob, one record per element	An exporter that wraps a whole blob in `[ ... ]`.
Plain text (`text`)	The blob as text, optionally split on a regex	Anything unstructured. Parse it with a downstream processor.

json, json_array and text are served by an encoding extension the platform emits alongside the receiver; otlp_json and otlp_proto are built in.

With Plain text the Record Separator field appears. It is a regular expression each blob is split on — \r?\n for one record per line. Leave it empty to read the whole blob as a single record.

A format mismatch is silent. A blob that decodes cleanly but yields zero records is not an error: the poll succeeds, the blob is marked read, and the destination stays empty. The collector logs a hint naming the likely fix, and prx_src_records_dropped_total{reason="format_mismatch"} counts it. If a pipeline looks healthy and produces nothing, check this field first.

Polling vs notification mode

Polling is the default: the container is listed every Poll Interval and anything new is read. Nothing needs to be configured in Azure beyond the credential.

Notification mode activates as soon as a connection_string credential is attached. Azure Event Grid publishes a BlobCreated event per new blob to an Event Hub, and the source reads that hub instead of listing. Blobs are fetched the moment they appear, and a large container costs nothing to watch. It needs an Event Grid subscription on the storage account with the Event Hub as its endpoint.

In notification mode Poll Interval, Max Blobs Per Scan, Settle For, Start From, Dedup Cache Size and Persist Offsets do not apply — the notification itself is the position.

Notification mode does not survive a collector restart. The consumer starts at the latest event and checkpoints nothing, so every notification published while the collector was down is missed — and because this mode never lists the container, those blobs are not picked up by a later pass. Use polling where a gap is unacceptable.

Advanced Configuration

Parameter	Type	Default	Description
delete_after_read	boolean	`false`	Delete each blob after reading it. Cannot be undone. Requires Storage Blob Data Contributor.
poll_interval	duration	`60s`	How often the container is listed. Ignored in notification mode.
timeout	duration	`60s`	Maximum time one Azure call may take before it is retried on the next poll.
decompress	string	`auto`	Whether to decompress blobs (Enum: `auto`, `none`, `gzip`, `zstd`). `auto` keys off the blob's magic bytes, so it is a no-op on uncompressed data.
max_blobs_per_scan	integer	`500`	Maximum number of blobs processed in a single poll cycle. Caps the first pass over an existing archive.
settle_for	duration	`0s`	Wait until a blob has been untouched this long before reading it. Use for append blobs, which a writer keeps extending after creating them.
start_time	date-time	none	Skip blobs last modified at or before this instant. RFC3339, e.g. `2026-09-28T00:00:00Z`.
persist_offsets	boolean	`true`	Checkpoint the read offset to disk so a restart resumes instead of re-reading. Not used in notification mode, or when Delete Blobs After Reading is on.
seen_cache_size	integer	`4096`	How many recently-read blob names are remembered for deduplication.
unmarshaling_separator	string	none	Plain text only. A regular expression each blob is split on, e.g. `\r?\n`.

Leave persist_offsets on. With it off, the read position lives in memory only and a restart re-reads everything newer than start_time — the whole container if start_time is unset. When on, offsets are written under <collector home>/metadata/azure_blob_offsets/<node name>, one directory per node, so two blob sources never overwrite each other's positions. No offset file is written when Delete Blobs After Reading is on — deleting the blob is the position — or in notification mode.

decompress matters for round-tripping a destination. The Azure Blob Storage destination gzips by default and appends .gz. Leaving this on auto is what makes reading its output work without any further configuration.

Turn on settle_for for append blobs. An append blob is readable the instant it is created and keeps growing afterwards. Reading it immediately captures a prefix and marks it done. A settle window of a minute or two makes the source wait until the writer has stopped touching it.

Example Configuration

Polling an Azure diagnostic archive — one azure_blob credential, no Event Hub:

{
  "container": "insights-logs-auditlogs",
  "prefix": "resourceId=/SUBSCRIPTIONS/.../y=2026/m=09/", // narrow the listing
  "encoding": "json",                                     // Azure writes JSON records

  "advanced": {
    "poll_interval": "60s",   // default
    "timeout": "60s",         // default
    "decompress": "auto",     // default
    "max_blobs_per_scan": 500, // default
    "persist_offsets": true,   // default
    "start_time": "2026-09-28T00:00:00Z",
  },
}

Replaying what an Azure Blob Storage destination wrote — the defaults already fit:

{
  "container": "praxis-archive",
  "prefix": "year=2026/month=09/",
  "encoding": "otlp_json", // default; decompress: auto handles the .gz
}

Notification mode — attach a connection_string credential as well; the polling fields no longer apply:

{
  "container": "logs",
  "encoding": "otlp_json",

  "advanced": {
    "timeout": "60s",
    "decompress": "auto",
  },
}

Metrics

Metric Name	Description
`prx_src_records_received_total{signal="logs"}`	Log records decoded from blobs and forwarded to the pipeline.
`prx_src_bytes_received_total`	Bytes read from the container, after decompression.
`prx_src_pipeline_latency_seconds`	Each record's observed timestamp → downstream handoff. On an archive this is the age of the data, not the time the source took to read it. Only recorded when the decoded records carry an observed timestamp.
`prx_src_errors_total{reason}`	Source errors by reason — `list_failed`, `read_failed`, `decompress_failed`, `consume_failed`, `delete_failed`. A poll cancelled by a collector restart is deliberately not counted.
`prx_src_parse_errors_total{parser}`	Blob whose contents could not be decoded with the configured Blob Format.
`prx_src_records_dropped_total{reason}`	`format_mismatch` — the blob decoded but produced zero records. `downstream_error` — the pipeline refused them.
`prx_health_status{component="receiver"}`	Whether the receiver started and holds a usable client.

Overview​

Authentication Configuration​

Basic Configuration​

Blob Format​

Polling vs notification mode​

Advanced Configuration​

Example Configuration​

Metrics​