$ ctrlaltroot
Analysis

Evals are the release gate enterprise AI is still missing

Every enterprise AI programme I see has a demo that works and no way to prove it still works tomorrow. That's not a model problem — it's a missing release gate.

There is a sentence that should cost a lot of money to say, and currently costs nothing:

“It worked when we showed it to the business.”

I hear it in almost every conversation about agents right now, and it always means the same thing: there is no repeatable way to answer “is this system still behaving the way it did when we approved it?” Everything downstream — pilot to production, security sign-off, a CFO who will trust the thing with a real workflow — is blocked on that one unanswered question.

What we already solved, and forgot

A decade of cloud-native work gave us the discipline for this. We learned that you don’t ship a service because it responded correctly to a request in staging. You ship it because you have tests, a CI pipeline that runs them on every change, a deployment you can roll back, and dashboards that tell you something changed before your customers do.

Every one of those has an analogue in agentic systems, and almost nobody has built it:

Cloud-native Agents today
Unit and integration tests Regression suites over fixed inputs, judged deterministically where possible
CI on every merge The same suite running on every prompt, model or tool change
Rollback Pinned model versions and a documented path back
Dashboards Traces of every agent run: inputs, tool calls, tokens, cost, verdict
Staging environment A replay harness that re-runs yesterday’s traffic against today’s build

The gap is not conceptual. Everyone nods along. The gap is that the third party in the middle — the judge that decides whether a given output is acceptable — is much harder to define for a non-deterministic system, so teams quietly skip it and call their demo a validation.

Evals as engineering, not a vibe check

What works, in my experience, is treating evaluation as a product with a score, not a meeting with an opinion:

  • Deterministic first. A large share of agent quality is checkable without another model: did it call the right tool, did the output parse, did it cite a real record, did it stay in budget, did it refuse when it should. Cheap, fast, and it catches most regressions.
  • Model-as-judge second, with a rubric written by the domain owner and versioned like code — and calibrated against human-labelled samples so you know how often the judge itself is wrong.
  • A scored number, per change. Not “quality improved”, but 87/100 on the 40-case suite, down 3 from the previous build, failures concentrated in the refund flow.
  • Failures kept forever. The most valuable asset in an eval suite is the pile of cases that once went wrong. Every production incident becomes a permanent test case.

The point of the number is political as much as technical. You cannot get an enterprise to sign off on “the agent seems good”. You can get them to sign off on a gate that blocks deployment when the score drops, because that is a control they already understand from every other system they run.

Why this is the wedge in regulated markets

Swiss enterprises in particular — finance, insurance, pharma — are not short of AI enthusiasm. They’re short of a defensible answer to their own risk function. The reason pilots stall isn’t the model; it’s that nobody can produce evidence the system is being controlled.

An eval harness with a release gate is exactly that evidence. It’s also, conveniently, the same shape as the compliance artefacts these organisations already accept.

That’s the work I’m doing now, and what most of these posts will be about.

← all notes · subscribe via RSS