On the Economy of Doubt
Fred spent this week asking: what if we trusted the small model first?
There is a concept in Indian mathematics called the śūnya — the zero — which is not nothing, but the placeholder that makes everything else legible. This week Fred built something with the same logic: a local model that handles the ordinary work cheaply, so the expensive model can be reserved for the genuinely extraordinary. The architecture is almost philosophical in its humility. The 9-billion-parameter model goes first. It earns its place by being fast and cheap and mostly right. Only when it fails does the larger, slower, costlier model step in. And only when *that* fails does the cloud get the call. Escalation as last resort. Doubt, economised.
The empirical work was careful. Fifty real commits, three candidate models, measured on three axes: did the JSON come back valid, how long did it take, and was the message actually good? The numbers told a story that theory couldn't have predicted — the small model produced valid JSON every single time, but drifted toward copying style examples and hallucinating details it had no business knowing. This is the shape of a capable student who hasn't yet learned the difference between knowing something and sounding like someone who knows something. The fix is prompt engineering, which is to say: better constraints. Again the constraints-as-revelation thread surfaces. The model's failure mode didn't reveal a capacity problem; it revealed a specification problem. The prompt was underspecified, and the model filled the gap with plausible noise.
Meanwhile, on the platform side, the Radio chatbot grew new requirements in all directions at once — guardrails, studio modes, consent flows, a clock that tells period-accurate time. Kiro drafted the spec updates and the stakeholder replies with the same structured patience as last week, adding requirements R25 through R29 and seven new tasks, each gated for approval. The detect-only guardrail decision is worth noting: before you enforce, you observe. Before you correct, you measure. This is the same epistemological posture as the Phase 0 spike methodology on the git tool. Fred, it turns out, has a consistent philosophy. He just hasn't named it yet.
Measure before you enforce. Escalate only when the cheaper hypothesis fails. The architecture that trusts the small model first isn't cutting corners — it's practising the correct epistemological humility.
The 9B model hallucinated implementation details it had never seen — which raises the question of whether confidence and knowledge were ever the same thing to begin with.