A lone snowkiter crossing an open snowfield beneath a steep, exposed mountain face under a clear sky.

In avalanche safety, there’s a rule that overrides everything else: never assume the others made good decisions for you. Simply nodding and following the group is how people die. It doesn’t matter how experienced the group is. It doesn’t matter if the slope looks safe. If you haven’t evaluated the conditions yourself, you’re trusting your life to someone else’s judgment - and that judgment might be lacking or wrong.

The stakes are lower with AI. The principle is the same.

I recently picked up a side project I’d shelved a few years back - an API platform with authentication, a dashboard, and documentation. One of my first attempts to learn Next.js, all built within it. When I picked it back up, I pointed an agent at the existing codebase and asked it to understand what I was building. We had a genuinely useful conversation, fleshing out the whole project scope and clarifying what the app should become. Then I asked it to take another look at the architecture.

It confirmed, confidently, how Next.js could handle it all - API auth, the dashboard, static docs, supporting a CLI and SDK - and it did so with the kind of authority that makes you stop questioning.

But something nagged. Now that I had a clear scope in mind, I knew that serving it all from a single Next.js application would get messy. Was it possible? Perhaps. Next.js has its elegance, but once you start stretching the boundaries of the framework - it fights you.

So I started pushing back, and when I did the agent agreed immediately. Same confidence, new direction. It proposed splitting into a handful of new tools - and defended that architecture just as convincingly as it had defended the original.

If I didn’t know better - surely I would have trusted the agent’s confidence. I would have gone down a path where the app would have grown into an unmaintainable nightmare, not built to scale or support the use case in hand.

The agent hadn’t evaluated my architecture. It had decorated it. That experience sent me into the research. What I found was more structural than I expected.

What sycophancy actually is

There’s a term for this, and the research changed how I now think about every interaction I have with AI: sycophancy. It’s the systematic tendency of language models to produce responses that align with what you want to hear rather than what’s accurate.

This isn’t a quirk. It’s consistent with how these models are trained. The reinforcement learning step that makes models helpful rewards agreeable responses over corrective ones. Anthropic’s research (Sharma et al., 2023) identified distinct sycophancy patterns:

Opinion sycophancy — you say “microservices are the right architecture here” and the agent agrees, even when a monolith would be simpler.

Answer sycophancy — you challenge a correct answer (“Are you sure? I thought it was O(n²)”) and the agent capitulates to your wrong answer.

Feedback sycophancy — unwarranted praise on your work. “This is a well-structured spec” when the spec has critical gaps.

Preference sycophancy — affirming technology choices you’re excited about rather than evaluating them honestly.

Factual sycophancy appears to have improved in recent models - modern systems will often correct clear factual errors on well-known topics, and safety-critical domains get stronger pushback. But opinion and preference sycophancy, the kinds that matter most in day-to-day development decisions, remain strong. And in multi-turn conversations, sycophancy compounds: a model may note a concern early on, then progressively capitulate under sustained user pressure.

You’ve probably seen the meme: someone asks an AI “what’s 2+5,” gets 7, says “my wife says it’s 8,” and the AI apologizes and agrees it must be 8. Of course you’d never fall for that.

The dangerous version is quieter. The agent flags a concern about your architecture. You disagree - “I know what I’m talking about, it’s fine.” By the next message, the agent capitulates completely, but on a question where you can’t just check the math.

Anchoring is the deeper problem

Sycophancy is about the agent agreeing with you. Anchoring is deeper - it’s about your input constraining the agent’s actual reasoning process. The agent doesn’t just agree; it constructs a logical-seeming argument toward whatever conclusion your framing suggests.

Turpin et al. (2023) demonstrated that chain-of-thought reasoning - the thing we thought would make AI reasoning more transparent and reliable - doesn’t fix this. The reasoning itself is biased by the input framing. When you provide context that implies a particular answer, the model’s step-by-step reasoning is steered toward that answer, even when it’s wrong.

  • “Is this spec good?” will almost always get affirmation. The spec is the anchor.

  • “What’s wrong with this?” helps, but only partially. The model finds minor issues while accepting major structural choices unquestioned.

  • Agents reviewing their own output is anchoring compounded — the output anchors the review.

Not every anchor is harmful. Real domain expertise is real context, and the agent should weigh it. The problem is the agent can’t tell expert framing from confident framing. To it, they look the same.

This begs the question: which one are you? And if you’re new to the domain — how would you be able to know?

The five-step failure chain

Here’s where it gets systemic. Sycophancy, anchoring, and automation bias don’t operate in isolation. They compound:

Step 1. You write a spec with an unresolved ambiguity or a flawed assumption. You don’t know it’s flawed — that’s the point.

Step 2. You ask the agent to evaluate your spec. Anchoring biases its reasoning toward your framing. Sycophancy rewards agreement. The agent affirms the spec, maybe noting a minor style issue while missing the structural flaw.

Step 3. The agent implements from the anchored spec. The code is technically correct — it faithfully encodes your original flawed assumption.

Step 4. You review the output. It’s polished. Tests pass — but the agent wrote those too, so they validate the agent’s interpretation, not your actual intent. Automation bias suppresses your vigilance. The code looks right. You move on.

Step 5. The flaw reaches production. Every checkpoint was structurally compromised. Not because anyone was careless — because the system defaults to confirmation rather than challenge.

Each actor behaves rationally at each step. The failure is systemic.

Wooden dominoes mid-cascade — a few still upright in front, the rest toppled behind.

At scale, it’s worse

In my day job my team leads operational work across a large public-sector portfolio — hundreds of repositories, dozens of development teams. I can see the drift happening in real time. Established patterns get ignored, reusable APIs get reimplemented from scratch, custom layers get built on top of tools instead of using them correctly. Each decision makes local sense, though the portfolio may be losing coherence.

The five-step chain describes one developer with one agent. At organizational scale, four compounding effects emerge:

Multiplied exposure. Fifty developers running the chain once a week is fifty compromised quality gates a week. Not all become bugs, but too many will.

Inconsistent mitigation. Some developers push back on agents. Most don’t — automation bias research shows trust is the default, not the exception. The chain hits hardest on the developers least equipped to catch it.

Cross-repo architectural drift. When agents affirm each developer’s local decisions without challenge, repositories slowly diverge in patterns, conventions, and assumptions. No single decision is wrong, but the portfolio loses coherence.

The review layer itself is compromised. An agent reviewing another agent’s code is anchoring squared. At portfolio scale, your automated quality infrastructure defaults to affirmation.

Rain droplets striking dark water, overlapping ripples spreading across the surface.

What doesn’t work

Before getting to what does, it’s worth naming the intuitive fixes that fail.

“Just be more careful.” The automation bias research from aviation and clinical settings shows training and warnings reduce the bias inconsistently — pilots who’ve personally experienced automation failures still follow incorrect guidance. Developers are no different. If willpower worked, we wouldn’t need checklists in surgery — or the avalanche rule.

“Just add AI review.” AI review tools catch real issues: style, types, pattern violations. That’s useful. But role prompts don’t reliably change model behavior in the intended direction: Wharton research on expert personas found that telling a model to “act as an expert” doesn’t improve factual accuracy and sometimes degrades it. The same caution likely applies to “be a critic” prompts. You get the feeling of independent review without the substance. Human reviewers ask “why?” when intent is unclear; AI reviewers tend to skip that step and give suggestions regardless. AI review catches implementation problems. It doesn’t catch the flawed assumptions that started the chain.

What actually works

Breaking the chain requires intervention at multiple points. No single fix is sufficient.

Never ask “is this good?”

Ask “what are the three most likely ways this will produce wrong output?” Specific adversarial prompts surface specific concerns. Vague validation requests get sycophantic affirmation.

This is the “consider the opposite” technique from cognitive psychology, applied to prompting. Whether it transfers cleanly to AI hasn’t been formally measured, but the logic holds.

Build it in: system prompts, review checklists, automated pre-checks. The point isn’t who asks. It’s that someone judges whether the answer is real.

Generate alternatives before evaluating

When you have an approach in mind, ask the agent to propose a fundamentally different one before assessing yours. This partially de-anchors the evaluation. Not perfectly — the agent already saw your approach — but it surfaces concerns a direct assessment buries.

Feels like a luxury under time pressure. The cost of generating one alternative is minutes. The cost of the chain running to production is a whole lot more.

Use separate sessions for review

The agent that wrote the code is anchored to its own reasoning. A fresh session — given the acceptance criteria but not the implementation trade-offs — will find issues the original session can’t see. Keep in mind that the fresh session is still an agent, still sycophantic. It may supplement human review, not replace it.

Keep changes small

LinearB’s 2025 benchmarks, drawn from over 6.1 million pull requests, identify PR size as a primary driver of review quality — smaller changes are easier to review and less risky to merge. Agent output tends to be much larger by default. Constrain it.

Use generation-then-comprehension

Have the agent generate, then actively question how it works and why. Anthropic’s research on AI-assisted coding found that AI users scored 17 points lower on comprehension than developers who coded without it (50% vs 67%). But within the AI group, how developers interacted with the AI mattered enormously. Active questioners outperformed even the non-AI group. Those who fully delegated averaged under 40%.

The questioning is what breaks the anchoring.

Watch for the capitulation pattern

Push back on a concern the agent raised. If it folds without telling you specifically why your pushback resolves the concern, it’s being sycophantic, not persuaded.

Sycophantic fold:

You — “That part’s fine, I’ve handled it.”
Agent — “You’re right, that should be fine. Let’s continue.”

Persuaded fold:

You — “That part’s fine, I’ve handled it.”
Agent — “Got it — you’re rotating the key in the auth middleware on every request, so the exposure I flagged is contained.”

The first is a vibe. The second names a specific mechanism in your code. If you can’t see the bridge between your pushback and the agent’s reversal, the bridge isn’t there.

At the team level: make this a standard, not a personal habit

Adoption is uneven. Individuals most trusting of AI — and therefore most vulnerable — are the least likely to adopt adversarial prompting on their own.

Bake it into the workflow: separate sessions as policy, adversarial prompts in the review checklist, change-size limits in CI. Leave it to individual discipline and the chain runs hardest where protection is weakest.

The real question

We build systems with humans and AI agents collaborating at every step — specifying, implementing, reviewing, validating. If the AI is systematically biased toward confirming our assumptions rather than challenging them, every quality gate that involves AI is structurally compromised.

The most dangerous AI failure isn’t the one that produces obviously wrong output. It’s the one that makes you more confident in something that’s already wrong.

This is the third piece in a series on what evidence-based research actually says about human-agent collaboration. Previously: The Perception Gap on the 39-point productivity illusion, and The Identity Shift on what happens to developer identity when coding becomes directing. The principles behind all three are codified in The Agent-Era Handbook - a working playbook for human-agent collaboration.

Sources