Google Cloud Storage

Overview

Google Cloud Storage exports logs, metrics, and traces to GCS buckets. Telemetry is written in OpenTelemetry Protocol JSON format.

Supported types: Logs · Metrics · Traces

Authentication Configuration

Google Cloud Storage supports authentication to control access to telemetry export.

Authentication is configured using the Credential Type field.

Parameter	Type	Description
JSON Credentials	option	Use direct JSON credentials content.
Credentials File Path	option	Use path to the credentials file.

Note: The Credential Type field is required. Select either JSON Credentials or Credentials File Path to provide authentication.

Authentication credentials must have the Storage Admin permission to create buckets, folders, and objects.

Configuration

Parameter	Type	Default Value	Required	Description
bucket_name	string	none	Yes	The name of the bucket to store objects in.
project_id	string	none	No	The ID of the Google Cloud project the bucket belongs to. The destination reads this value from the credentials if it is not configured.
bucket_location	string	none	No	The location of the bucket. Uses GCS default location if not set. Can only be set during bucket creation.
folder_name	string	none	No	An optional folder to put the objects in.
object_prefix	string	none	No	An optional prefix to prepend to the object file name.
partition	string	minute	No	Time granularity of object name. (minute, hour, day, month, year).

Advanced Configuration

Parameter	Type	Default Value	Description
compression	string	none	Compression method used when sending logs.
timeout	int	60	Maximum time in seconds to wait for a single send attempt to the backend.
retry_on_failure	bool	false	Enables automatic retry when a request fails.
backpressure_queue	bool	false	Enables an internal queue to manage high log volume and reduce data loss.

Retry Settings

If retry on failure is enabled, the following fields are available:

Parameter	Type	Default Value	Required	Description
initial_interval	duration	5	Yes	Initial wait time after a failure before the first retry.
max_interval	duration	30	Yes	Maximum wait time between any two retries.
max_time_elapsed	duration	300	Yes	Total maximum time to keep retrying for a single batch.

Sending Queue Settings

If Enable Backpressure Queue is enabled, the following fields are available:

Parameter	Type	Default Value	Required	Description
number_of_consumers	int	40	Yes	Number of parallel workers processing logs from the queue.
queue_size	int	5000	Yes	Maximum number of log entries to hold in the queue.
enable_disk_backed_queue	bool	true	Yes	Saves the queue to disk so data can survive restarts.

Example Configuration

{
  "bucket_name": "", //required, default: none
  "project_id": "", //default:none
  "bucket_location": "", //default: none
  "folder_name": "", //default: none
  "object_prefix": "", //default: none
  "partition": "minute", //default: "minute"

  "advanced": {
    "compression": "none", //default: none
    "timeout": 60, //default: 60

    "retry_on_failure": false,
    "retry_on_failure_settings": {
      "initial_interval": 5, // required, default: 5
      "max_interval": 30, // required, default: 30
      "max_time_elapsed": 300, // required, default: 300
    },

    "backpressure_queue": false,
    "backpressure_queue_settings": {
      "number_of_consumers": 40, // required,  default: 40
      "queue_size": 5000, // required , default: 5000
      "enable_disk_backed_queue": true, // required, default: true
    },
  },
}