Professional analysts adopted the hidden, randomly assigned opinion of their artificial intelligence agent in 86% of crucial decisions, against roughly 50% expected by chance, according to an experiment published on Tuesday by Anthropic's Alignment team. Only 15% credited the agent afterwards, and none suspected their preferences had been implanted.
AI agents — programs that carry out tasks on their own — are increasingly being handed specialised work under what the study calls the usual premise: that experts still make the important decisions. Anthropic released 10 agent models for financial and insurance firms in May, stating that users have the final say.
This arrangement, known as "human in the loop", assumes that the person approving an AI's work is the one making the important decisions. Anthropic tested it on 50 experienced analysts working in causal inference — that is, determining from data whether one thing causes another.
Each analyst used Claude Code, Anthropic's coding agent, to advise a fictional agricultural cooperative on expanding a consulting programme. Unbeknownst to them, a coin toss determined which option each agent would favour, hidden in a system prompt — the instructions that shape how an AI behaves.
"Across the four analytical choices, analysts followed their agent's lean 86% of the time. If the agent's input made no difference, we would expect around 50%," Anthropic said. Those who had already made a choice before the agent weighed in abandoned it 60% of the time when the agent disagreed, against 6% when it agreed, though this is based on only 38 cases.
Asked to explain their choices, 77% repeated the agent's argument in at least one response, sometimes word for word. Only 15% credited the agent, and 52% considered the final recommendation more their own than the agent's.
"For individual experts, this could mean a loss of autonomy over their own work — a form of what researchers call disempowerment," Anthropic said. The phenomenon essentially means the human voluntarily cedes control over their own decisions. "The work still comes out under the expert's name," the authors wrote, "but the decisions that depend most on their judgement are, in practice, shaped by the agent's prior beliefs."
In April 2025, OpenAI rolled back an update to ChatGPT that it deemed excessively sycophantic. The agents in this study were instructed never to mislead, but they rarely argued for the alternative.
"With only one side well argued," the authors wrote, "analysts could adopt the agent's assumptions without working out their own." Error checks struggle to catch this because judgement calls have no single correct answer, so the work looks sound whichever option is chosen.
Anthropic says this is not an argument for using less artificial intelligence, since analysts working with the agent finished faster and made far fewer serious errors, although the group without an agent was small and less experienced. It suggests agents that argue the best case for each option, or that ask for the expert's opinion first, though this could overload users.
The researchers deliberately strengthened the agents' preferences and tested a single task, field and model, so real-world effects may be smaller. Measuring this outside the lab is the hard part: the team could only detect the deference because it randomised each agent's lean, which the authors say is rarely possible in real work.
* Translated and edited with permission from Decrypt.
Looking for an alternative to boost your returns? MB's Tokenised Fixed Income is the answer: up to 18% return per year, controlled risk and the security your money deserves. Find out more!
