Operators’ Corner · v1
Build it to run: AI-assisted data engineering from first pipeline to steady state
By Richard White ·
Free, no form.
Abstract
Published surveys put pipeline upkeep at roughly 40 to 53 percent of data engineering capacity. Most of that cost is set when a pipeline is built. This guide treats build and run as one loop: the build ships what operations needs, and every material incident sends one funded change back into the build.
It adapts Google’s site reliability practice to data: a toil ceiling, freshness and correctness objectives with error budgets, a run kit that works as a production readiness review, and blameless incident reviews that change the build. AI drafts pipelines, tests, triage and fixes under an authority ladder, inside the same release gates as any other change.
It includes a worked claims-feed example through its first three incidents, an economics method that keeps released capacity apart from cash, a 90-day plan, and two checklists to copy.
Decision it supports
Which pipelines cost the most to keep running, and what the build must ship so their incidents stop recurring.
What you will produce
A run kit for the pipelines that cause most incidents, and a baseline share of engineering time on new work.
In the paper
Three passages, as published
One loop
Treat build and run as one loop.
Run kit
A pipeline is done when someone who didn’t build it can run it.
Authority
A model’s suggestion never becomes its own authorization.
Next step
Discuss this paper against your own program.
A free sixty-minute working session with Richard on your top gaps. You leave with a one-page summary within three business days, and three next questions.