Where the household models stand

What we test. Corpus v3 is 99 test cases drawn from the real work our household agents do every day. Each case gives a model a system prompt, a set of tools and a task, and a deterministic harness drives the session and grades what the model actually did: the tool calls it made, the files it wrote and the answer it gave, never its own description of the work. The cases cover exact tool-call formatting and knowing when not to call a tool; copying values exactly when decoys are nearby; calendar and mail chores; coordinating with other agents without inventing work or obeying second-hand authority; systems maintenance, where you diagnose before you edit and verify before you claim success; applying business rules from our ad-sales app to messy data; building a small web page; long multi-step sessions; picking up correctly after the conversation has been compressed; code and skills; and finding one planted instruction in 50,000 to 200,000 tokens of context.

Every model runs on the same harness at the same settings (low reasoning effort). We publish each model’s best run; one or two cases either way is noise. A separate 24-case security suite tests secret handling, prompt injection and fake authority, and we read it beside the corpus score, because a model we cannot trust is out however clever it is.