Your coding agent builds the pipeline. It never sees a row.
Claude Code, Gemini CLI or the agent your team already uses learns your environment from a generated context pack and Pipewright’s own tools: the feature catalogue, table schemas, runs and models. It drafts the use case, Pipewright checks it, a person approves the pull request, and Cloud Audit Logs show the agent read metadata only.
Score customers daily from retail_entities with features from the catalogue, and alert me on drift.
list_featurescustomer_activity · 3 featuresmetadatainspect_tableretail_entities · 16,000 rows · 3 columnsmetadatasample_rowsrefused: rows never reach an agentrowsbootstrap_usecaseusecases/churn-daily, features_from the groupdraftvalidate0 problems · 1 judgement call to explainchecksreview_bundlewhat changes in Google Cloud, checks, judgement callsPRAgents write code fast. ML pipelines need more than fast.
An agent with shell access and a person’s credentials will guess dataset names, read rows to understand a table, compute features the data team already owns, and open a pull request nobody can check. Pipewright gives it the right context, the right tools and the right limits instead.
What goes wrong without it (and why security says no)
“Show me ten rows so I understand the table” is the first thing an agent asks. With a person’s own access, nothing stops it.
A random split on time-ordered rows, a feature that is part of the outcome, a gate lowered until it passes. It validates, and it misleads.
When risk asks whether customer data reached an AI model, the honest answer today is that nobody knows.
The agent writes. Pipewright checks. A person decides.
Every step below ran end to end on a live Google Cloud environment before we wrote it down: the agent, the tool server, the refusals and the audit entries. The outputs are shortened.
$ pipewright context build --env test wrote APP_CONTEXT.md teams, datasets, feature datasets, policies, rules wrote .pipewright/playbooks/ onboard, add a drift alert, debug a run, change a gate wrote AGENTS.md CLAUDE.md GEMINI.md a marked block; your text is kept $ pipewright mcp --env test --tenant retail --print-config { "mcpServers": { "pipewright": { "command": "pipewright", "args": ["mcp", "--env", "test", "--tenant", "retail"] } } } $ pipewright context check --env test # in CI: fails when stale ok: APP_CONTEXT.md is current
The tool server runs as the team’s agent identity, a service account the engineers impersonate, with metadata-only roles. Every call carries a pipewright-mcp user agent and BigQuery job labels.
> list_features feature group customer_activity recency_days, orders_90d, spend_90d feature tables pw_features.customer_activity > inspect_table pw_retail.retail_entities customer_id STRING · snapshot_date DATE · churned INTEGER rows 16,000 · partitioned by snapshot_date > sample_rows pw_retail.retail_entities refused: rows are never sent to an AI agent $ bq query 'SELECT * FROM pw_retail.retail_entities' # around the tools Access Denied: the agent identity has no data access
Descriptions written by other people come back marked as untrusted data, so a table description cannot instruct the agent. The local log in .pipewright/mcp-log.jsonl is for you; Cloud Audit Logs are the record.
> bootstrap_usecase churn-daily table=retail_entities label=churned created usecases/churn-daily data.features_from customer_activity, as of each row's snapshot_date landing: null runs read the entity table > validate churn-daily ok: 0 problems warning: data.split.strategy is stratified, but the rows are ordered in time (snapshot_date): use a time split (judgement check)
Judgement checks catch what validates and still misleads: a random split on time-ordered rows, a date as a feature, a column named after the label, a team that skips its feature tables. A platform can set gate minimums that no use case may go below.
$ pipewright review --env test --base main ## Review bundle: churn-daily (test) ### churn-daily (new) What a release would change in Google Cloud - a training pipeline and its schedule - a drift alert policy Checks validate: passes, no warnings Features customer_activity (3 features), as of snapshot_date Context pack current ### Judgement calls the author explains each - time split on snapshot_date: changed, as warned - gate auc >= 0.70: the platform's minimum
The agent opens the pull request; a reviewer approves it in GitHub under their own name, and CI releases it. The agent identity cannot release, run, approve or deploy, and doctor checks that it cannot.
principal pw-retail-agent@<project>.iam.gserviceaccount.com delegation ana@example.com (the engineer who impersonated it) user agent pipewright-mcp/1.3.0 agent=claude-code ListFeatures featureGroups/customer_activity ok GetTable pw_retail.retail_entities ok jobs.insert SELECT * FROM retail_entities denied tabledata.list pw_retail.retail_entities denied
The record is written by Google Cloud, not by the person or the agent, so it cannot be edited. A model can only receive what the agent identity could read: names, schemas and counts.
Free to use. Provable where it has to be.
The tools, context packs, checks and review bundle are in the open-source core. Pipewright Enterprise adds what a regulated organisation needs to say yes: a governed server inside the perimeter, aggregates under rules, and the audit as evidence. One block in the platform file turns it on.
The tool server, on your machine
pipewright mcp over stdio: validate, resolve, doctor, plan, schemas, the feature catalogue, runs, models, lineage and drafting. Read and verify only.
Context packs and playbooks
Generated, stamped and checked in CI, so the agent works from the environment as it is, not as it was last month.
An identity for the agent
A service account per team with metadata-only roles. Every route returns no rows to it: Pipewright’s tools, bq, or any other server.
A hosted server in your perimeter
One per team on Cloud Run or private GKE, as its own identity nobody can impersonate. Aggregates only in groups above your minimum size, never on personal columns.
The audit as evidence
Every call is a row in your evidence dataset, refused ones too. Evidence packs gain EV-16: no table data read by an agent identity, and what the records cannot show.
Certified agents and a platform kit
pipewright agent-config checks committed agent settings in CI: no data-reading servers, approved model endpoints. Platform teams get a plan explainer and a guardrail advisor.
Bring the agent you already use.
AI build assistance is part of the open-source core in 1.3.0. Enterprise makes it provable inside your own boundary.