The use case files

A use case is a folder. Its YAML holds the decisions; its code holds how the model learns. This page covers the settings you will change most. For every key, run pipewright explain --list, or read the churn example, which shows every default as a # default line.

usecases/churn/
  usecase.yaml              what it is, who owns it, how it is promoted and served
  config/triggers.yaml      what starts a run: landing files, schedules, API
  config/runtime.yaml       pipeline parameters, compute per step, the test command
  config/tables.yaml        BigQuery tables the use case owns
  config/data.yaml          data contract, label, split, features, personal data
  config/training.yaml      hyperparameters, experiment, tuning
  config/evaluation.yaml    metrics, gates, champion, slices, explainability, model card
  config/operations.yaml    inference, monitoring, retraining
  config/synthetic.yaml     synthetic data, off until you need it
  environments/<env>.yaml   the few settings that differ per environment
  training_pipeline.py, inference_pipeline.py, components/, schemas/, tests/

usecase.yaml

apiVersion: pipewright/v1
kind: UseCase
metadata: { name: churn, owner: team-retention }
spec:
  tenant: retention                       # given to you by the platform team
  include: [config/*.yaml]
  pipelines:
    training:  { file: training_pipeline.py,  function: churn_training_pipeline }
    inference: { file: inference_pipeline.py, function: churn_inference_pipeline }
  model:
    name: churn-classifier
    promotion: model                      # promote the approved files, or retrain in each environment
    serving: { enabled: false, machine: n1-standard-2, canary_percent: 10 }
  approvals:
    promote_model: manual                 # a passing model is a candidate until approved
    deploy_model: manual                  # also auto, ci, on_register

config/data.yaml

data:
  contract:                               # checked before anything learns from the data
    columns:
      - { name: customer_id,   type: string,  required: true, max_null_fraction: 0 }
      - { name: tenure_months, type: integer, min: 0, max: 1000 }
      - { name: churned,       type: integer, allowed: [0, 1] }
    min_rows: 500
    unique: [customer_id]
  label: { column: churned, values: [0, 1], positive: 1 }
  split: { strategy: time, time_column: snapshot_date, test_fraction: 0.2 }

Columns declared as personal data are never used as features, and pipewright data erase removes a customer by key across the team’s tables. data.features_from joins features from feature tables or Feature Store groups as of each row’s cut-off; data.label_source builds labels from an events table and a window; data.snapshot_table keeps exactly what each run read.

config/evaluation.yaml

evaluation:
  primary_metric: auc
  gates:                                  # every gate must pass
    - { metric: auc, min: 0.75 }
    - { metric: calibration_error, max: 0.1 }
  champion: { enabled: true, metric: auc, min_improvement: 0.0 }
  slices: [contract_length_months]
  slice_gates:
    - { metric: auc, max_gap: 0.25, min_rows: 100 }

Fairness gates compare groups you name; explainability adds feature importance to the report; model_card records intended use and limits.

config/operations.yaml

monitoring:
  alerts: { notify: [ml-team@example.com], events: [monitoring.alert, training.rejected, run.failed] }
  drift: { enabled: true, method: psi, threshold: 0.2 }
  performance:
    enabled: true
    labels: { table: churn_outcomes, key: customer_id, label: churned }
    metric: auc
    min: 0.7
retraining:
  mode: propose                           # off | propose | candidate (trains, never promotes)
  when: [drift, performance]
  decisions: { notify: [ml-team@example.com] }
  approvals: { required: [ml-leads, model-risk] }
  snooze: { max_days: 30, reopen_when_worse_by: 0.1 }

pipewright alerts sync turns monitoring.alerts into Cloud Monitoring alert policies that email the team.

config/triggers.yaml

A landing trigger starts a pipeline when a file lands in the team’s prefix (training by default). schedules become Cloud Scheduler jobs, or Kubernetes CronJobs on private GKE, when you run pipewright schedules sync.

The bqml section

A use case with a bqml section has no pipeline code: Pipewright’s own pipelines submit BigQuery jobs. See Models in SQL.

spec:
  bqml:
    model_type: logistic_reg              # boosted_tree_classifier, kmeans, pca, arima_plus, ...
    options: { l2_reg: 0.1 }
    prepare: sql/prepare.sql              # optional SQL over {rows}, the run's rows
    max_bytes_billed: 107374182400        # every job has a byte limit

config/synthetic.yaml

Off until a team needs it. Switched on per environment, Pipewright runs its own synthetic pipeline: an engine learns inside your environment (MOSTLY AI’s SDK in local mode by default), keys and personal data are never learned, and the candidates are scored on fidelity, utility and privacy before the best that passes every gate is published. Without real data, it can also write rows from the data contract alone. Run it with pipewright run --pipeline synthetic; pipewright synthetic gates lists the gates a release must pass, and pipewright synthetic generate writes rows from the contract and rules alone, with no real data.

environments/<env>.yaml

Overrides for one environment, on a fixed list of keys: compute, schedules, notification addresses, serving machines and the like. The data contract, gates and features may not differ, so production judges models the way test did. pipewright explain <key> says whether a key may vary.