Siddarth Nyati · the mechanism behind the five gates
One I built and shipped inside an enterprise, on a platform with real controls. One I built alone, where nobody else was going to catch my mistakes — so I had to build the quality system myself. Neither is complete on its own. The method is what carries across.
Act one · the enterprise system
A release page goes in. Customer-facing documentation and a notification to the client-facing teams come out. Four narrow model steps, one human gate, and a single scoped tool with permission to send anything.
It is deliberately not autonomous. My code decides the sequence; the model reasons inside each step and never chooses what runs next. This runs close to a production release, so predictability was worth more than flexibility. Autonomy earns its place where the path can't be enumerated — support triage, a coding agent. Publishing a release note isn't that shape.
The seam
A gate is not an eval, and conflating the two is how teams feel safe while flying blind.
A gate catches errors one at a time on the way out. An eval tells you whether version N+1 is better or worse than version N. I had the first and not the second, which meant that when I changed a prompt, I had no way of knowing whether I'd quietly broken something that used to work.
What I'd build first, and it costs almost nothing: capture the diff between what the model produced and what the human published. That gives two things at once — a regression set that builds itself from real production inputs, and edit rate as an online metric that falls as the system improves. Edit rate is also the answer to "when do you remove the human": by class of output, once it has been low and stable long enough to trust, and never all at once.
Act two · the system where I owned the whole quality problem
On my own editorial pipeline there was no reviewer but me, so the checking had to be built rather than staffed. The organising rule I wrote for it:
Each layer is evaluated by the cheapest method that can catch its failure mode.
| When the contract is… | Test with | In practice | Cost |
|---|---|---|---|
| Objective | Deterministic rules | Nine encoded rules that run as a script and exit non-zero on failure | Free, seconds, no network |
| Labelled | Golden-set scoring | A classifier scored against a hand-labelled catalogue; the gate is a hard threshold | ~$0.01 smoke, ~$0.10 full |
| Editorial | Judge against a written rubric | Six boolean checks and a three-way verdict the orchestrator branches on | ~$0.02 a run |
| Personal taste | Human gate + user signal | A person picks; downstream feedback demotes bad output | Minutes, and it doesn't scale |
revise pass on the second attempt"How accurate is it" is the wrong question — accurate at what, and what does an error there cost? I made the same call twice in unrelated domains. On a document-processing platform in a regulated healthcare setting, the confidence threshold routed low-confidence extractions to a human, biased toward precision because auto-filing a wrong authorisation is worse than making a person look at more documents. On the editorial pipeline, one classification dimension is a hard gate because getting it wrong breaks the output, and a second is reported only because getting it wrong changes nothing.
One instance is an anecdote. Two is a way of working.
Some output has no single right answer, and the honest response isn't to give up on measuring it — it's to layer cheap checks ahead of expensive judgement so the costly step only sees what survived.
"Isn't a model judge just marking its own homework?" It would be if it were scoring prose on general quality. It's checking one output against an external document — the written standard is the ground truth, not the model. I keep it boolean rather than scored, I grade named dimensions separately rather than asking for one holistic score, I deliberately refused an "uncertain" verdict because models use it to avoid committing, and there's a human gate downstream that continuously samples whether the judge is being too lenient.
Judge bias is real and documented — position, verbosity, self-preference. Most of that literature concerns comparing two candidate outputs to each other. Comparing one output to a fixed rubric is a different and smaller failure surface.
Because a platform's constraint is capacity, not ideas
Normal software costs the same whether one person uses it or a million. These don't. Someone has to own the cost per unit of value, and that someone is the product owner.
Containment
You can ask a model nicely not to do something, or you can arrange things so it can't. Only the second one is a control.
| Layer | What it does | In the pipeline |
|---|---|---|
| Output | Validate before anything is shown or sent | Sanitisation before publish; schema validation on every structured step |
| Tool scope | Limit what the system can do at all | A single scoped tool is the only send path — the model cannot notify arbitrarily |
| Deployment | Inherit the firm's controls rather than reinvent them | Hosted on the internal self-service platform, behind enterprise MFA |
| Input | Treat ingested content as data, never instruction | Not built. See below. |
Indirect prompt injection. These steps read pages and ticket descriptions written by other people, then generate customer-facing documentation and send a notification. Anyone who can write a ticket description can put text in front of a model that also has a send capability. There was no input guardrail against that.
Two things limited the blast radius by architecture rather than by intent: the human gate on the path to anything customer-facing, and the fact that the only send path is one narrowly scoped tool. Those are two of the four real mitigations. The others — structurally separating fetched content from instructions, and filtering output for exfiltration — are what I'd add, alongside logging attempts so it becomes a detection signal rather than a silently blocked request.
No prompt-level defence is complete. That's precisely why the containment has to be architectural.
Act three
No single role hands you the whole surface. The enterprise system gave me deployment, scoped tools, real controls and a human gate. It didn't give me evals or cost engineering — so I built something where I'd have to own those myself.
| Release pipeline · enterprise | Editorial pipeline · built alone | |
|---|---|---|
| Multi-step decomposition, output contracts | yes | yes |
| Human review gate before anything ships | yes | two of them |
| Deployed on an enterprise platform, behind MFA | yes | no |
| Tool scoping as a security control | yes | not needed |
| Evals with a verdict the orchestrator branches on | no | yes |
| Golden set gating every change | no | yes |
| Cost caps and per-stage cost reporting | no | yes |
| Model routing by consequence | no | yes |
| Retry, backoff, fallback, salvage | no | yes |
| Per-step traceability and resumable runs | no | yes |
| Grounding verified at a gate | no | yes |
| Input guardrail against injection | no | no |
The column on the left is what a team gets when it builds inside an organisation that already has controls. The column on the right is what a team has to build for itself when nobody else will catch its mistakes. A central AI platform's job is to make the right-hand column something spoke teams inherit rather than rebuild — an eval harness, a cost standard, a trace format, a routing rule — so that shipping responsibly is the cheap path rather than the disciplined one.
That's the same argument as the five gates, one level down: gates decide what should be built, this decides what every build gets for free.