GCP Inventory

Overview

GCP Inventory is a pull-based source. It takes a periodic snapshot of Google Cloud assets and identities from [Cloud Asset Inventory] and sends one record per asset into a Praxis pipeline.

These records are entity context: they describe what exists (VMs, disks, networks, buckets, service accounts, roles), not what happened. A SIEM uses them to enrich events.

Resource	Cloud Asset Inventory asset types	gcp.inventory.resource → Google SecOps log type
Compute: VM instances, disks, VPC networks, subnetworks, firewall rules, forwarding rules	compute.googleapis.com/Instance, Disk, Network, Subnetwork, Firewall, ForwardingRule	compute → GCP_COMPUTE_CONTEXT
Storage: Cloud Storage buckets	storage.googleapis.com/Bucket	storage → GCP_STORAGE_CONTEXT
IAM: service accounts, service account keys, custom roles	iam.googleapis.com/ServiceAccount, ServiceAccountKey, Role	iam → GCP_IAM_CONTEXT

Supported types: Logs

Supported platforms: Linux, Windows, macOS

Minimum collector version: 0.5.71

Authentication

The credential is optional.

Credential type	When to use
Google Auth (googleauth)	Recommended. Supports service account JSON (inline or a file path), Workload Identity Federation, Application Default Credentials, and service account impersonation.
GCS (gcs)	Reuse an existing Google Cloud Storage service account credential.
None	The collector uses Application Default Credentials on its host: GOOGLE_APPLICATION_CREDENTIALS, gcloud login, or the GCE / GKE identity.

On a Compute Engine VM with no credential, the VM must have the cloud-platform access scope ("Allow full access to all Cloud APIs"). With the default VM access scopes, Cloud Asset Inventory calls fail even when the IAM roles are granted.

Required IAM roles

All calls are read-only. Grant the identity these roles on every scope (project, folder or organization):

Role	Why it's needed
roles/cloudasset.viewer (Cloud Asset Viewer)	Read asset metadata: cloudasset.assets.listResource for ListAssets.
roles/serviceusage.serviceUsageConsumer (Service Usage Consumer)	Use the Cloud Asset Inventory API in the project. Google requires it with every Cloud Asset Inventory call.

Also enable the Cloud Asset Inventory API (cloudasset.googleapis.com) in the project the credential belongs to. See Cloud Asset Inventory roles and permissions.

Basic configuration

Parameter	Type	Required	Default	Description
scopes (Scope)	list	Yes		Which Google Cloud project to collect from: projects/ followed by the project ID, for example projects/test. To collect all projects in a folder or organization, use folders/<folder-id> or organizations/<organization-id> instead.
resources	list	No	all three	What to collect. compute: VMs, disks, networks and firewall rules. storage: buckets. iam: service accounts, their keys and custom roles.
mode	scheduled | manual	No	scheduled	scheduled: take a snapshot every interval. manual: take a snapshot only with Fetch Inventory Now.
interval	integer (seconds)	No	21600 (6h)	Time between full snapshots in scheduled mode. Minimum 900 (15 minutes). Hidden in manual mode.
sync_request	integer	No	0	Fetch Inventory Now: shown and used in manual mode only.

Errors

Error	What happens
Quota exceeded, service unavailable, timeout, pipeline busy	Resume from the checkpoint after backoff.
Permission denied or not found on one scope	That scope/resource is skipped, the rest are sent, and it is tried again at the next snapshot.
Every scope and resource fails	The snapshot is retried with backoff.

Scheduled and manual fetching

Fetch Mode	When snapshots run
scheduled (default)	At start-up, then every Snapshot Interval. A restart waits out the rest of the interval, unless a snapshot was unfinished, in which case it resumes immediately.
manual	Only when you raise Fetch Inventory Now. Nothing runs on a timer.

Fetching on demand

1. Set Fetch Mode to manual.
2. In Fetch Inventory Now, enter a number higher than the current one (for example 1 → 2).
3. Deploy. The collector takes one full snapshot.

The completed number is saved on the collector, so a restart or a later config change with the same number does not fetch again.

Record format

Each asset becomes one log record. Its body is the asset in the JSON shape Cloud Asset Inventory exports, which is what the Google SecOps GCP_*_CONTEXT parsers read:

Sending to Google SecOps

One GCP Inventory source produces records for three Google SecOps log types. Between the source and the Google SecOps destination, add one Google SecOps Standardization processor per log type, each with a condition:

Log Type	Condition
GCP_COMPUTE_CONTEXT	log.attributes["gcp.inventory.resource"] == "compute"
GCP_STORAGE_CONTEXT	log.attributes["gcp.inventory.resource"] == "storage"
GCP_IAM_CONTEXT	log.attributes["gcp.inventory.resource"] == "iam"

Each processor sets the log type only on the records its condition matches, so the three can run one after another.

Example configuration

{
  "scopes": ["projects/my-project"],     // required
  "resources": ["compute", "storage", "iam"],
  "mode": "scheduled",                   // default: scheduled (or "manual")
  "interval": 21600,                     // default: 21600 (6h), minimum 900; scheduled mode only
  "sync_request": 0                      // manual mode: raise and deploy to fetch now
}

Operational notes

• Snapshots, not streaming. Every snapshot re-sends the whole inventory. Deleted resources stop appearing in the next snapshot.
• Many projects. Use a folder or organization scope rather than listing many projects. One scope covers everything under it.
• Quota. The ListAssets quota belongs to the credential's project and is shared with anything else using it.

Metrics

Metric Name	Description
prx_src_records_received_total{signal="logs"}	Log records emitted, one per asset. A page re-sent after a failure is counted again.
prx_src_bytes_received_total	Bytes of the emitted record bodies.
prx_src_pipeline_latency_seconds	Arrival → downstream handoff.
prx_src_errors_total{reason}	Cloud Asset Inventory or snapshot failures by reason: auth, permission_denied, timeout, io.
prx_src_records_dropped_total{reason="backpressure"}	Records the downstream pipeline refused. They are re-sent from the checkpoint.