1
Sit in observe for a fortnight
Against real traffic, not a staging replay. You are looking for the gap
between what you believe the agent does and what it attempts.
2
Move to shadow and feed it your incumbent
Shadow exercises the full decision path, still refusing nothing. Pass what
your existing system or reviewer decided, and the divergence is the
signal:Where the policy and your incumbent disagree is where one of them is wrong.
Usually it is the policy, and finding that out here costs nothing.
3
Canary
The first rung that refuses. Pick the grant where a mistake is most
expensive rather than the one that is easiest, because the cheap one teaches
you nothing about the hard case.Watch refusal volume for a fortnight. A rate near zero usually means the
bounds are drawn too wide to be doing work, not that everything is fine.
4
Enforce
Refusals apply to everything under the grant. Nothing changes in your code;
the exceptions you already handle start being raised for real.
What to check before each step up
- Are refusals being handled by type? A bare
except Blockedthat swallowsExhaustedandGatedis survivable in observe and is an incident in enforce. - Does the on-call team know what a refusal looks like? The console names the gate and the reason. Somebody should have seen one before it wakes them.
- Is the failure mode agreed? Enforcement fails closed. If your team cannot accept that, decide it now and record it, rather than discovering it during an outage.

