Serving and canaries

Batch scoring answers overnight. Some answers are needed in a request: at a checkout, in an app. This tutorial puts the champion behind an endpoint, rolls a new version out to a share of the traffic first, and then lets callers send only a customer ID.

1. Say how it is served

model:
  name: churn-classifier
  serving: { enabled: true, machine: n1-standard-2, canary_percent: 10 }
approvals:
  deploy_model: manual                    # also auto, ci (a GitHub-approved gate), on_register

2. Deploy the champion

pipewright deploy-model   --env dev --usecase churn --approve
pipewright model endpoint --env dev --usecase churn       # the URL, the permission, a sample request
pipewright model predict  --env dev --usecase churn --instances request.json

deploy-model serves the registry’s default version. It checks the version against the gates first, refuses a version not yet approved, and ends by telling you how to call it. Repeating it is safe: it says when the version is already served.

3. A canary

Train and approve a better version, then send it a share of the traffic while the current one keeps the rest:

pipewright deploy-model --env dev --usecase churn --canary --approve     # 10% to the new version
pipewright deploy-model --env dev --usecase churn --promote              # all traffic to it
pipewright deploy-model --env dev --usecase churn --undeploy-canary      # or abandon it

4. Features from the online store

A caller at a checkout knows the customer, not their 90-day spend. With an online store (the platform team creates it; it uses Bigtable and bills by the hour), the endpoint looks the features up itself, from the same definitions training used:

model:
  serving: { enabled: true, features: online }
data:
  features_from:
    sources:
      - feature_group: customer_profile   # served by the feature view of the same name
        features: [tenure_months, monthly_charges]

A request is then {"instances": ["c-42"]}. An empty feature gets its training median, as in batch scoring, unless you set model.serving.fill_missing: false.