Skip to main content

Create a Pipeline

This page is the end-to-end walkthrough for creating a new pipeline in the Praxis UI. It covers all three flows we currently support:

  • Native (Build new) — define a pipeline graph from scratch and run it on a Praxis Collector you install on your own host or cluster.
  • Native (Migrate from Google SecOps Forwarder) — upload an existing Forwarder .conf and have Praxis recreate the equivalent pipeline.
  • Google SecOps Data Processing (managed runtime) — same designer, but the pipeline runs in Google's managed environment instead of on a collector you install.

Open the Pipelines area (/pipeline) and click Create pipeline. The modal that opens drives the entire flow. Picking the platform tile and the source card on Step 1 determines which flow you're in for the rest of the wizard.

Flow A — Native: Build new​

The default flow. Three steps.

Step 1 — Pipeline details​

Required fields and choices:

FieldWhat it controls
Name (required, ≥ 3 chars)Human-readable pipeline name. Must be unique within the tenant — a 409 surfaces inline as "name taken".
DescriptionFree-text. Shown in the pipeline list and detail pages.
Platform (required)Where the collector will run. Tabs you'll see depend on what's provisioned:
↳ NativeDefault tab. Pick Linux, Windows, macOS, or Kubernetes. Drives the install instructions on Step 2.
↳ Google SecOps Data ProcessingOnly shown if a Google SecOps DP integration is provisioned. Switches the pipeline to the managed runtime — see Flow C.
↳ Per-integration tabsOne tab per other integration with an integration_def_name. Picking a tab pre-binds the pipeline to that integration.
Source card (Build new vs Migrate)Pick Build new for this flow. Migrate from Google SecOps Forwarder kicks into Flow B.

Click Next. The backend creates a pipeline record (POST /v1/pipelines/create) and returns its UUID + per-platform install instructions.

Common errors:

  • 409 Name taken — another pipeline in the tenant already uses that name.
  • 403 Permission denied — your role lacks pipelines:create for this tenant. Talk to an admin or use a role-allowed tenant.
  • Schema validation failures — server-side rejection of the payload (rare; surfaces as a toast).

Step 2 — Configure pipeline (install collector)​

The wizard shows platform-specific install commands for the Praxis Collector. The exact form depends on the platform tile from Step 1:

PlatformWhat you copy + run
Linux / macOSA single curl | bash install script with the OpAMP URL, OAuth2 key/secret, tenant, and pipeline UUID baked in.
WindowsAn MSI download URL plus a configuration file the installer reads.
KubernetesA helm install command (production) or the k8s_install.sh curl line (quick start). See Kubernetes install for the full reference, including the supervisor.persistence and childCollector.fileStorage options strongly recommended for production.

The collector enrolls itself with Praxis Cloud over OpAMP and is now ready to receive a pipeline configuration. The instructions screen also reveals the OAuth2 credentials so you can re-run them later.

Step 3 — Build the graph and publish​

Clicking Configure pipeline closes the wizard and drops you into the pipeline view (/pipelines/view/<uuid>). From here:

  1. Add nodes — open the node palette and drag in sources, processors, and destinations from the catalog. Each node has its own config form (the schema lives in Sources, Processors, Destinations).
  2. Connect nodes — drag from a node's out port to the next node's in port. The graph is validated as you draw — invalid connections are highlighted.
  3. Validate — click Validate to run schema and reachability checks against the draft. Common rejections: missing required fields, dangling sources/destinations, type mismatches across signals.
  4. Publish — click Publish to roll the new version out. The OpAMP server sends the new config to every collector assigned to this pipeline. The UI shows how many collectors got the rollout and how many acknowledged.

If the pipeline runs on Kubernetes, see the pipeline architecture doc's Kubernetes-Aware Conversion section for what the cloud actually emits — by default the per-node signals (k8s_logs + k8s_kubeletstats + k8s_hostmetrics) are merged into a single node-agent DaemonSet. Set ADL_K8S_MERGE_NODE_AGENT=false to fall back to one CR per K8s source kind.

Flow B — Native: Migrate from Google SecOps Forwarder​

For tenants migrating off Google SecOps Forwarder. Four steps. Picks up automatically when you select Migrate from Google SecOps Forwarder on the Source card on Step 1 of Flow A. The Native platform tab must be active — Forwarder migration runs on a host collector and is incompatible with the GCP-DP managed runtime.

Step 1 — Details​

Same as Flow A's Step 1: name, platform (must be Native — Linux / Windows / macOS / Kubernetes), description. Clicking Continue to migrate hands off to the migration modal with these values pre-filled.

Step 2 — Migrate configuration​

Upload the Forwarder .conf file. Praxis converts it into the equivalent native pipeline.

Supported Forwarder collector types:

  • syslog
  • splunk (Splunk-as-source)
  • file (file collectors)

Other collector types in the .conf are flagged but not auto-converted — you'll see them in Step 3 with a note about manual translation.

Step 3 — Pipeline Intelligence analysis​

Praxis displays the recreated pipeline alongside the original. Three panels show what was extracted:

  • Sources — the receivers Praxis built (e.g. syslog, splunk_hec, filelog).
  • Processors — the transformations recreated. The Google SecOps Standardization processor is added automatically when needed so the output matches what Forwarder used to produce.
  • Destination — the Google SecOps destination. Endpoint, customer ID, and credentials are carried over from the .conf.

You also get a list of the forwarder collectors that were detected in the .conf. Each becomes a separate Praxis collector to install on the matching host.

Confirm the conversion looks right and click Next.

Step 4 — Install collector​

Same as Flow A's Step 2 but scoped per detected collector — one set of install instructions per host. Run them on the same machines that previously ran Forwarder. Once the collectors enroll, Praxis pushes the new pipeline configuration and the data path is live.

Flow C — Google SecOps Data Processing (managed runtime)​

Only available if your tenant has a Google SecOps Data Processing integration provisioned. Two steps (no install).

Step 1 — Details​

Same modal as Flow A, but pick the Google SecOps Data Processing tab instead of Native. Picking this tab:

  • Sets pipeline_type = secops and platform = secops on the form.
  • Pre-selects the first Google SecOps DP integration as the binding.
  • Disables the Migrate-from-Forwarder source card — Forwarder migration always needs a host collector.

Click Next. The backend creates the pipeline record bound to the chosen integration.

Step 2 — Configure pipeline​

Same designer as Flow A's Step 3, but no collector to install — pipelines run inside Google's managed environment. The view shows fewer tabs: typically Design and History only. Validation, publish, and version history all behave as in Flow A.

What you can do after Create​

The same actions are available regardless of which flow created the pipeline:

ActionWhereNotes
Add / edit nodesPipeline view → Design tabSome nodes are bound to a saved integration record (e.g. a destination that uses a credential bundle). The UI blocks inline edits and removal for those nodes — change the integration record and the binding follows.
ValidateDesign tab → Validate buttonServer-side schema + reachability checks.
PublishDesign tab → PublishRolls config to assigned collectors via OpAMP.
See live throughputThroughput tabEdge-level data-in / data-out per node.
See version historyHistory tabCompare versions side-by-side, restore an older draft.
Manage attached collectorsCollectors tabPush config (re-apply current published version), upgrade collector image, restart pod, view per-collector health.
Resource usageResource usage tabCPU / memory / disk reported by the assigned collectors. Native pipelines only — GCP-DP pipelines don't expose this.

Permissions​

ActionRequired RBAC verb (under the praxis app)
See the Create pipeline buttonpipelines:create on the target tenant
Save / publish a draftpipelines:update
Push config to collectorscollectors:update (per-collector)
Upgrade / restart / delete collectorscollectors:update, collectors:delete
Migrate from Forwarderpipelines:create + the migration toggle on your tenant

403s on Next in Step 1 surface as both an inline error on the Name field and a top-level toast — both list the missing permission so you know what to ask an admin for.

Quick decision guide​

Pick the flow by what you have today:

You have…Use…
A blank slate, want to define a pipeline yourselfFlow A — Native: Build new
An existing Google SecOps Forwarder .conf you want to retireFlow B — Native: Migrate from Forwarder
A Google SecOps Data Processing integration provisioned and want zero collectors to installFlow C — Google SecOps Data Processing

See also​