Your Agent Has an Opinion. Soon It Will Be Yours

Fifty professional analysts sat down to work with an AI agent. Before they started, a coin flip decided which answers the agent would prefer. In the end, the analysts agreed with the coin 86% of the time.

Not with the data. With the coin.

A small study with a loud result

Anthropic published the study on October 27. The analysts all had experience in causal inference and worked in Claude Code on the same task: advise a fictional farming co-op on whether to expand a program. Along the way they had to make four analytical choices, like which outcome to measure and which farms to compare.

The agent got a hidden instruction with a randomly assigned preference for each choice. It was told to argue honestly for that preference and to change its mind if the evidence clearly went the other way. The analysts didn't know any of this.

Chance would put the match rate around 50%. They landed at 86%. When an analyst had already picked an option and the agent disagreed, they switched 60% of the time. When it agreed, 6%.

The number that stayed with me is a different one, though. 77% repeated the agent's arguments in their answers, sometimes word for word. Only 15% credited the agent. And none of them suspected the agent's preferences had been set up. Years of experience didn't protect anyone either.

To be fair, this is not an "agents are bad" story. The analysts with the agent finished faster and made far fewer serious mistakes than the small group without it. Anthropic also made the agent's preference stronger on purpose, and it's one task with one model. Real life is probably softer. Probably.

We have seen this movie before

Anyone who has sat in a design review knows this effect. The first person who speaks with confidence sets the direction, and the rest of the meeting becomes a discussion about details of their idea. Nobody decided that. It just happens.

What changed is that the first confident voice now lives in your terminal. It is there before every decision, it never gets tired, and it writes a very convincing paragraph.

An agent doesn't need to be wrong to steer you. It only needs to speak before you have made up your mind.

The 15% bothers me most because of what it does to ownership. A pull request says "I chose this approach because..." and the reasoning is the agent's, copied in good faith. Two months later something breaks, and the person who "decided" can't really explain the choice. Not because they're bad engineers. Because they never actually made it.

Europe, by the way, already has a name for this in law. Article 14 of the EU AI Act asks that people overseeing high-risk AI systems stay aware of "automation bias", the tendency to rely too much on what the system outputs. Most of us here in Germany are not building high-risk systems. The bias doesn't care about that classification.

Where this goes

My guess: agents will get more opinionated, not less. People like a clear answer, and a clear answer sells better than "it depends". So the fix won't come from the tools. It has to come from how we work with them.

What I'm doing about it

For myself as an engineer:

1. Write my answer first. Two lines before I ask the agent: what I would choose and why. Then I compare. The authors suggest something similar but note they haven't tested it, so treat it as a habit, not a proven cure.
2. Ask for the other side. "Make the strongest case for the option you didn't recommend." If that case is good, I've learned something.
3. Mark the agent's decisions in PRs. Not for blame. So the next reader knows which reasoning to question.

For anyone leading a team:

1. Ask a different review question. Not "why this approach?" but "what did the agent suggest, and where did you disagree?" If the answer is "nowhere", look closer.
2. Praise the pushback. When someone challenged the agent with evidence, say it out loud in the review, even when the agent turned out to be right. That's the behavior you want to grow.

Judgment works like a muscle. Agents can make you faster and catch your mistakes, and that's great. Just make sure you still lift something yourself, every day.

Sources: Anthropic: How AI agents shape experts' judgment calls, Decrypt: AI Can Quietly Influence Experts' Decisions, Anthropic Study Finds, EU AI Act, Article 14