01 · The problem
Let the evaluation loop find the bug: both models scored 86.7% and failed the same way, which pointed at the prompt and not the model. Changing only the briefing lifted action accuracy to 93.3% and cut Brier from 0.119 to 0.071 on the same rows, for $0.033.
02 · How it works
- 01
Stayed correct through a real quota outage: with the API returning 429 on every request, a circuit breaker produced all 110 decisions in 19.8 s, matching the full run 110 of 110, and the write-up states that n = 30 makes one row worth 3.3 points.
03 · What it cost, and what it returned
Triaged WhatsApp messages into notify, digest or mute per recipient, where two byte-identical messages deserve opposite answers, using a rules engine plus a Claude judge on a forced tool call with a Gemini fallback chain.