Agnes 3.0 Flash Overclock reviewed: a checker, a QA panel and an honest benchmark

What the repo is

The repo is described as an experimental prompt-engineering and validation kit for Agnes 3.0 Flash, built around Russian short-video scripts, source-preserving rewrites and constrained JSON. Its README defines "overclock" as tuning the workflow, not changing model weights or proving maximum quality. It states that it is not installed in a production pipeline.

The code needs Python 3.10 or newer and the standard library only. The checker needs no API key and makes no network call. The LICENSE file is the MIT license, and the repo vendors two skills from other authors with their own license files.

What the benchmark showed

The README reports six tasks with two samples per condition, 35 of 36 planned calls completed on 10 October 2026. These are the published numbers.

The README calls these directional constraint-check scores. Empty or incomplete outputs were excluded, denominators differ, there was no blind quality judge, and the reference was a self-written answer set, not a verified run of another model. It says no parity claim is supported. The benchmark report adds that the aggregates cannot be regenerated from the repo, because the raw outputs, full references, the original scorer and the call ledger are absent, and that 4 of the 36 planned samples are missing from the denominators with their identities unrecoverable. My judgment: that disclosure is rare and is the reason the numbers are usable at all, as a hint rather than a result.

The five parts

• Output checker. check.py checks word bands, strict JSON, scene durations, paragraph constraints, numeric literals and wrapper text for six preset tasks. I ran the example draft and it passed. The README says a pass does not prove truth, and that numeric checks compare literal sets, not associations.
• 25-agent QA panel. A method with 25 lenses in 5 waves of 5, plus an offline CLI that renders briefs, consolidates reports and gates on fix status. The repo cannot launch agents; the helpers check bookkeeping. The docs say the run found 1 blocker and 11 majors, and that those counts come from the lead agent's own list, not an independent audit.
• Unslop UI filters. A prompt and a regex scanner for generic-looking frontend code. On the repo's synthetic sloppy.css I got 12 findings, 9 of them high or critical; on clean.css, none. The docs say the rules are paraphrased taste, the upstream licenses were not checked, and a clean scan is not a pass.
• Proof-of-delivery gate. An opt-in JSON envelope with file hashes, command ledger, test records and visual-inspection assertions. The docs are explicit that it checks presence and shape, not truth, and that a plausible fabricated ledger can pass. The example I ran passed.
• Smart router and MCP server. route.py returns only the guidance for one task and phase by keyword matching, with no model. The MCP server exposes six tools: agnes_check, agnes_list_tasks, agnes_get_prompt, agnes_build_messages, agnes_route and agnes_review_messages.

What I ran

At commit 077e4ab, the full suite reported 113 tests, OK, 1 skipped (the opt-in live-network test), in about 8 seconds. The README gives the same count. The checker, scanner, router and proof gate examples ran as documented. I did not run any host integration (Cursor, Codex, Claude Code and others) and I had no Agnes access, so the prompts, the editor pass and the v2 repair loop are unverified by me. The repo says the same.

Where it is strong

• Claims match evidence. Each extension is labelled untested, and the benchmark caveats sit next to the numbers.
• Small and auditable. Standard library only, about 2,800 lines of Python including tests, no keys.
• Careful remote fetching. Its remote routes use a fixed origin, no redirects, a time limit, a size cap and an untrusted-content marker, and the tests exercise them with a stubbed network.

Where it is thin

• Evidence base. The only measured result is one small benchmark on Russian short-video tasks that cannot be reproduced from the repo.
• Structure, not truth. The checker, proof gate and scanner verify form. Human review of facts and voice is still required, as the repo says.
• Host support is unproven. The per-host MCP configs were checked against vendor docs, and no host was run end to end.
• Narrow checker. The presets target Russian video scripts; other uses mean writing your own constraints.

A disclosure

Three routes in this repo, jg-prompts, jg-mcp and jg-extensions, fetch small pages from this site, the one you are reading. I have no say in how the repo is written, and I have tried to review it by the same standard as any other. You should still weigh that.

How I would evaluate it

Run the offline checker on outputs you already have, and see whether its violations match what an editor would flag. Measure the editor pass on your own tasks, with failed and empty runs kept in the count. The numbers will be yours; I have none to add and I am not publishing simulated ones.