Project Swap: the agents bargained well and knew their owners poorly
Anthropic’s Project Swap found a preference gap in a 201-person book market: 61% pairwise agreement and a larger loss at intake than in bargaining.
PROJECT SWAP · AI AGENTS · 25 SEP 2026
A book-trading agent can win the negotiation and still go home with the wrong book. In Anthropic’s September 24 Project Swap report, that is the central failure: a short interview gave agents imperfect preferences before any trade began. Anthropic, Sep 24, 2026.
IN BRIEF
- 201 employees in six offices joined a controlled book-swap market. Among 188 ranked responses, a five-minute intake produced 61% pairwise agreement with human rankings. The reported outcome averaged 0.55 on a 0–1 preference scale, against a feasible optimum of 0.89. Project Swap.
Reported facts
Each participant brought a book, spoke with Claude about reading tastes and privately ranked ten books. Agents received Claude-generated full-pool rankings, not the withheld human rankings, and traded for books. The team reran markets with different models and prosocial or ruthless prompts; this is a company-run experiment, not an independent field trial. Source.
What they ran
201 employees in six offices joined a controlled book-swap market. Among 188 ranked responses, a five-minute intake produced 61% pairwise agreement with human rankings. The reported outcome averaged 0.55 on a 0–1 preference scale, against a feasible optimum of 0.89.
Where the value leaks
The authors decompose a 0.34-point shortfall from the 0.89 optimum. Imperfect preference representation accounts for about 0.29 points in their reported comparison, far more than the bargaining layer. A numerical ranking can be internally consistent and still represent the wrong person. Source.
A held-out check you can run
Before a shopping agent is allowed to buy, ask the person to rank a small held-out set of alternatives. Compare the agent’s ordering with the person’s pairs, look into any reversals, and ask again if confidence is low. The code below is an illustrative local check, not Project Swap’s scoring method and not a production safety gate.
Illustrative code; not the study’s method
from itertools import combinations
def pair_agreement(human, agent):
# Both lists contain the same unique item IDs, best first.
assert len(human) == len(agent) == len(set(human)) == len(set(agent))
assert set(human) == set(agent)
h, a = {x:i for i,x in enumerate(human)}, {x:i for i,x in enumerate(agent)}
pairs = list(combinations(human, 2))
return sum((h[x] < h[y]) == (a[x] < a[y]) for x,y in pairs) / len(pairs)
assert pair_agreement(['A','B','C'], ['A','C','B']) == 2/3Analysis (Jack Gross)
My reading: agent benchmarks often grade negotiation or task completion against the agent’s own objective. Project Swap shows why that can flatter a system. The missing input is not more persuasive bargaining; it is a check that the objective fits the human. A low-stakes book trade makes that defect visible without assuming it will transfer unchanged to housing, finance or healthcare.
Where this stops holding
The participants were Anthropic employees, not a representative sample. Rankings of ten books may be noisy; a book exchange is not a cash marketplace. The authors also studied cooperative Claude agents on a rule-bound floor, not adversarial sellers. The later survey’s roughly 30% average book-budget delegation is a stated willingness among respondents, not observed spending or a public adoption rate. Source.
What to take from it
The design lesson is small. Hold back some decisions the agent never sees, test preference fit against them, and keep costly commitments behind human review. A better model improved results on the agents’ own rankings, but a strong negotiation score cannot repair a wrong goal.
FAQ
Did the agents trade poorly?
The report attributes most of the outcome gap to preference representation rather than bargaining.
Does 61% mean 61% of books were correct?
No. It is pairwise agreement between the agent and person over ranked book pairs.
Were real purchases tested?
No. The experiment used book barter among Anthropic employees.
Anthropic, Project Swap, Sep 24, 2026. Article dated Sep 25, 2026.