Anthropic counted how much AI builds AI. The hard part is the denominator.

DAILY RESEARCH NOTE · PRIMARY METHODOLOGY

Twenty-six percent sounds like a verdict. It is a measurement of a specific basket. Anthropic’s August snapshot says Claude “leads” 26% of measured AI R&D tasks. More than 90% reached at least “collaborates.” None reached fully autonomous. That is a defined set of work, not the fraction of a company that AI can replace.

Where the 26% comes from

Researchers sampled 20% of relevant staff for each July week. Claude extracted about 15,000 tasks and organized 542 nodes with 378 leaves. Another model rated AL0–AL5. Person-time set weights. A frozen basket supports trends but can miss new work created after automation.

Raters disagree, and that is fine if you say so

Hand a model and a person the same task and they pick the same level 59% of the time. Two people agree 35% of the time. Within one level, model and human agree 97% of the time. So the AL scale is usable, and it is subjective, most of all on the line between AL3 “collaborates” and AL4 “leads.”

Watching everything is not catching everything

About 30,000 agents were active at once on the main internal platform in August. Anthropic says 100% of actions passed through online monitoring and entered offline monitoring. Over one billion decisions, 0.002% were blocked. Offline systems flag about 100,000 transcripts weekly; roughly 50 reach people. Routing every action does not prove every danger is recognized.

A dashboard that refuses orphan percentages

The original code below attaches scope and period to each rate. It rejects a percentage without numerator and denominator. This is an explanatory pattern, not Anthropic software.

def rate(n, d, scope, period):
    if d <= 0 or n > d: raise ValueError("bad denominator")
    return {"rate": n/d, "n": n, "d": d,
            "scope": scope, "period": period}

Original explanatory pattern, not Anthropic software.

A drill you can run this week

Plant 100 known violations across agents, permissions and time horizons. Then measure whether each one arrived, was detected, was blocked, was escalated and was resolved correctly, and how long that took. Change the model or the monitor and run it again. Publish the false negatives too.

Six percent of what, exactly?

For July 13–20, about 6% of AI R&D compute was classified as safety, and 12% of AI-driven AI R&D compute. About 14% of almost 10,000 runs were sampled, weighted toward large runs. Mixed-purpose work counted as general R&D. This is one company-defined week, not an industry benchmark.

What is known, what I think, what is open

Fact: Anthropic published these methods and snapshots. It notes model judging, incomplete labels, one-platform oversight, one-week compute data, and disputed boundaries. Analysis: open denominators are valuable, but cross-lab comparison needs common definitions and outside verification. Unknown: later trends, independent replication, and false-negative rates.

What I would want published next

Version the task tree. Give the numerator, denominator, time window and scope for every rate. Report missing data and the seeded detection tests. Publish where outside reviewers disagree. A number becomes governable once outsiders can reproduce its boundary.

Primary methodology, results, and limitations