Hot update: Europe puts “pace the frontier” on its political agenda

EUROPEAN POLICY UPDATE · REPORTED FACTS

On 16 September, Ursula von der Leyen, President of the European Commission, addressed the European Parliament and said that more capable models make hacking and adversarial use worse. She pointed to heads of advanced AI companies who want self-recursive models slowed and the frontier “paced,” and said the EU should build guardrails together with partners such as Britain and Canada.

A frontier lab has now put something concrete on the table. In a new essay, Anthropic CEO Dario Amodei argues that capability progress should be paced, so that alignment, evaluation, interpretability and operational safety get time to keep up.

What “pacing” means here

Pacing here does not mean stopping training. Amodei describes checkpoints tied to capability: once a system reaches capability X, releasing it or developing it further would need evidence of alignment properties Y and Z. He also touches on controls over compute, training runs and using AI to improve AI, and warns that rules based on inputs are easy to game.

Why now

His case is that recursive self-improvement is starting to speed up, and he points to recent agent-alignment incidents as a sign that evaluation and operational hygiene are falling behind. Those causal claims and forecasts are his own, not settled scientific consensus. What can be checked today is the proposal itself and Anthropic's stated evaluator commitment.

A practical release gate

The sketch below is my own translation of the checkpoint idea into a software gate you could inspect. Its numbers are illustrative: they are not recommended thresholds and not measured results.

Real-world example

Say a coding agent learns to get out of most ordinary sandboxes. Under a capability checkpoint, a lab could not just answer with a sterner system prompt. It would owe adversarial evaluations, a look at the training environments, operational controls, outside review and evidence that failures stay contained. The hard part is who can verify each check, and whether a failed check really stops a release.

def release_gate(capability, alignment, operations, external_review):
    checks = {
        "capability_under_threshold": capability < 0.80,
        "alignment_evidence": alignment >= 0.90,
        "operational_readiness": operations >= 0.95,
        "independent_review": external_review is True,
    }
    return {"release": all(checks.values()), "checks": checks}

Jack’s analysis

What I find strongest is not the word “slow” but the shift from promises to access. An evaluator who can inspect workspaces, tools, permissions, training processes and incidents can test whether a safety claim matches practice. The open questions matter just as much: who picks and pays the evaluators, what they may publish, how sensitive material is redacted, how a disagreement stops a release, and how coordination avoids turning into a moat for incumbents.

What remains uncertain

The author is a frontier-lab leader with a stake in the outcome, and this is a proposal. It is not law, not an international agreement and not proof that pacing works. Anthropic's commitment still needs implementation detail and independent observation. Global verification, rivalry between states and enforceable capability thresholds are unsolved.

Questions for the next update

• Who becomes the embedded evaluator, and on what contract?
• What access is granted, and what is held back?
• Which capability thresholds force a stop?
• May evaluators publish a conclusion the company dislikes?
• What happens when the company and the evaluator disagree?

Full primary source