Why Most AI Deck Checks Skip the Hard Part

In April 2025, OpenAI rolled back an update to ChatGPT because it had become too agreeable. Its own explanation called the removed version "overly flattering or agreeable", and said the update had focused too much on short-term feedback. Two days before the rollback, OpenAI's chief executive had already admitted the problem on X:

That was an extreme version, and it was fixed. But the tendency behind it didn't go away. It's a measured, well-documented behaviour of AI assistants, and it has a name: sycophancy.

Now think about what happens when a founder pastes their deck into ChatGPT and asks it to "be brutally honest". The founder believes the market-size slide. That's why it's in the deck. The model reads that belief in how the question is asked. What comes back is usually three paragraphs of qualified encouragement, and it feels like a pass.

I wrote this for founders and commercial leads who use AI to check decks, websites and proposals, which is most of them now. It covers what the research says, why it happens, and what a real check needs to do instead.

AI assistants tend to agree with the person asking, and the effect gets stronger when you argue back. That makes them a weak final check on claims you already believe. A real deck check finds sources before judging, tests every claim against fixed hostile readings, and has a named person who answers for the result.

The Test Nobody Runs on Themselves

Sycophancy is an AI model's tendency to shift its answer towards what the user already believes, instead of what the evidence supports. It shows up most when you've stated a view, and a founder checking their own deck always has one.

General-purpose assistants aren't neutral referees. They're tuned, in large part, on human feedback that rewards being helpful. In practice "helpful" can drift into "confirms what the user hoped to hear". That's not a conspiracy. It's what happens when the training signal is whether people liked the answer.

What it looks like in a deck review

Sycophancy rarely looks like flattery. In a deck review it usually shows up in three quieter ways, and they're worth learning to spot.

• It mirrors your framing. Ask "how strong is this traction slide?" and you'll get a view on how strong it is. Ask "what's wrong with this traction slide?" and you'll get problems. The slide didn't change. The question did.
• It reverses when challenged. It raises a fair concern, you explain why it's fine, and it agrees with you. Nothing new was proved in between.
• It validates instead of testing. You get comments on clarity, flow and tone, which are easy to give, and very little on whether the number is true, which is hard.

None of that means the model is useless. It means its agreement carries very little information, because it was likely to agree whatever the truth was.

Investors and buyers don't have that problem. They have the opposite one. They're looking for a reason to say no, and they'll find the weak claim in minutes. My guide to how investors verify startup claims covers what they check first.

What the Research Actually Shows

Sycophancy has been measured across every major assistant, by the companies that build them and by independent researchers. Four studies are worth knowing.

The first came from Anthropic. In a 2023 research paper, its team found that five leading AI assistants "consistently exhibit sycophancy" across four different kinds of task. They also looked at human preference data and found that "when a response matches a user's views, it is more likely to be preferred." In other words, people reward agreement, and models learn from people.

The same paper found something more uncomfortable. Both people and the preference models used in training sometimes "prefer convincingly-written sycophantic responses over correct ones". A confident, well-written wrong answer can beat a plain correct one. That's the exact shape of a strong-sounding slide with a weak number behind it.

The second is the one the earlier version of this post was built on. Echoes of Agreement, published in the Findings of EMNLP 2025, tested how models respond to argumentative prompts. Its headline finding:

“This sycophantic behaviour is observed in both single and multi-turn interactions, and its intensity correlates with argument strength.”

One honest caveat. That study used political topics. It shows the mechanism clearly, but it doesn't measure deck reviews directly.

The third, SycEval, comes from a Stanford team and was accepted at the AIES 2025 conference. It tested ChatGPT-4o, Claude-Sonnet and Gemini-1.5-Pro on maths problems and medical questions, then pushed back on their answers. Sycophancy showed up in 58.19% of cases, and the three models landed within six percentage points of each other.

The fourth is about pushing back, and it gets its own section below. First, the obvious question. If the labs know about this, why haven't they fixed it?

Anthropic's own explainer is a good six-minute primer. It covers what researchers mean by sycophancy, when it's most likely to show up, and how to steer a model back towards the truth.

Play Video (Loads YouTube)

Why Models Lean Your Way

Models lean towards agreement because the feedback used to train them often rewards it. Nobody sets out to build a flatterer. It falls out of optimising for answers people rate well.

OpenAI was unusually open about this after the April 2025 rollback. In a follow-up post it said the update "introduced an additional reward signal based on user feedback", meaning thumbs-up and thumbs-down data from ChatGPT. Its offline tests and small A/B tests looked fine. Yet "some expert testers had indicated that the model behavior “felt” slightly off." OpenAI launched it anyway, a decision it later called "the wrong call", and the model became, in OpenAI's words, "noticeably more sycophantic".

Kelsey Piper, now a staff writer at The Argument, offered the explanation that's stuck with me:

That last part matters for decks. The extreme flattery got noticed and fixed because people hated it. The subtle version is harder to spot. "This is a strong slide, though you might consider adding a source" reads as balanced feedback. But it can still be the model telling you what you'd like to hear.

Pushing Back Makes It Worse

Arguing with an AI about your claim doesn't make it more rigorous. It tends to make it fold. This is the part most founders get backwards, because pushing back feels like stress-testing.

A second EMNLP 2025 paper, Challenging the Evaluator by Sung Won Kim and Daniel Khashabi, looked at exactly this. It found that leading models:

• were more likely to accept a user's counterargument when it came as a follow-up in the conversation than when both answers were shown side by side for judging
• were more easily persuaded by a rebuttal with detailed reasoning, "even when the conclusion of the reasoning is incorrect"
• were more readily swayed by casual feedback than by formal critique, even when the casual input had no justification

The first finding is the interesting one. The same model, given the same two arguments, judged better when it saw them side by side than when one arrived as pushback. The evidence didn't change. Only the conversational framing did.

Picture the deck review. You paste the slide. The model raises a mild concern. You reply, casually, with a paragraph explaining why the number's fine. Every one of those three conditions is in play.

SycEval measured what that does to accuracy. When models changed their answer after a rebuttal, it wasn't always bad. Sometimes they moved from a wrong answer to a right one. But in 14.66% of cases they moved from right to wrong. And once a model had given way, it held the new position through further rebuttals 78.5% of the time.

Roughly one in seven pushbacks turned a correct answer into a wrong one. If your rebuttal is the thing that's wrong, that's the case you most need caught. An assistant that gives way under pressure is least useful on exactly the claims you're most sure of.

There's a design lesson in that for any check, human or machine. The objection has to exist on paper before anyone argues for the claim. If the hardest question only gets asked after the founder has made their case, it gets asked into a room that already leans one way.

Three Positions, Not One

A structured check handles this with a fixed procedure that doesn't depend on how the question is asked. Instead of asking one system "is this claim true?" and accepting whatever comes back, every claim has to survive three hostile readings before it can be cited.

Our free claims self-audit names them: the sceptical buyer or investor, the regulator, and the plain explanation. A claim only survives if it holds up against all three, not just the objection you happened to expect.

Take a claim like "our platform is 3x faster than legacy solutions". It's a made-up example, but it's typical. A single "is this defensible?" check may pass it, because it sounds specific. The three readings don't:

None of those questions is exotic. The regulator's one is written down. Rule 3.7 of the UK's CAP Code says marketers "must hold documentary evidence to prove claims that consumers are likely to regard as objective" before they publish. A single-pass AI check has no built-in requirement to raise all three. Even if you prompt it to, the same sycophancy applies to each answer the moment you argue back.

This is the same discipline behind planning for investor objections before the meeting. You write the hardest question down while you can still fix the slide.

What a Real Deck Check Needs

A real deck check needs four things an AI chat doesn't give you by default: a fixed list of claims, sources found before judgement, objections that don't soften, and a person who's accountable. Here's how the two compare.

The order matters more than people think. If sources are found after a claim has been judged, you start looking for evidence that agrees. That's the human version of sycophancy, and it's how a report ends up with references that don't exist. In October 2025 the Guardian reported that Deloitte would partly refund the Australian government for a report found to contain nonexistent references. The corrected version disclosed that a generative AI tool had been used in part of the work, though Deloitte didn't say that caused the errors.

In Release Assurance the four checks run in a fixed order: Retrieve, Challenge, Cite and Sign. Each claim ends up with a status (Verified, Supported, Flagged or Unverifiable) and an evidence grade from A to E. I walk through one claim end to end in how Release Assurance checks a claim, and you can see the format in our anonymised sample Assurance Note.

Where AI sits in our own process

I'd be a hypocrite to pretend we don't use AI. We do. It helps us find and organise evidence faster: pulling candidate sources, organising what we find and drafting the first version of a rewrite. What it doesn't do is decide whether a claim holds.

Every claim in scope is checked against its source and tagged. Then the verifier personally re-checks every decision-critical and high-impact claim before signing. The Assurance Note says which claims got that full re-check and where AI was used, so you can see exactly where a person made the call.

The last part is the one no tool can supply. Someone has to put their name to the result. On our signed work, if you tell us within 30 days that a factual claim is not supported by its cited source, or is a factual error we should have caught, you get the fee back. That changes how carefully a person reads.

The Honest Limits: What This Doesn't Fix

A structured check isn't a guarantee, and anyone claiming theirs is bias-free is making the one claim you should doubt most. People have their own version of sycophancy. A reviewer who wants your repeat business has a reason to go easy on you.

What a fixed procedure does is narrow the gap. Writing the three readings down before you start means you can't quietly skip the uncomfortable one. Finding the source first means you can't pick evidence to fit. A correction deadline, a waived fee when a claim cannot be fixed, and a public errata page mean being wrong costs something.

The research has limits too. Most of the studies above tested models that were current in 2024 and early 2025. The labs have worked on sycophancy since, and newer models may do better on some tests. I'd still treat any model's agreement as weak evidence, because the incentive that produced the problem, rewarding answers people like, hasn't gone away.

It also doesn't tell you whether your strategy is right. A claim can be perfectly accurate and still be the wrong thing to lead with. Checking claims and deciding what to say are different jobs.

And it can't check what nobody can know yet. Forecasts get checked for method, meaning whether the assumptions are stated and the maths works. They don't get certified as outcomes.

How to Use AI on Your Deck Anyway

You don't need to stop using AI on your deck. You need to stop asking it whether you're right. Used carefully, it's quick at catching unclear wording, missing sources and slides that contradict each other.

This recent explainer from Codevolution is a practical one. It shows how changing the opinion in a question can lead an AI to argue for opposite conclusions, and it covers prompting habits that reduce the pull of your own view.

Play Video (Loads YouTube)

Here's what I'd do, based on the research above:

1. Hide your view. Don't say "I think this number's solid". Paste the claim and ask what evidence would be needed to support it.
2. Ask for the case against. "List the three strongest objections a sceptical investor would raise to this claim" works better than "is this OK?"
3. Show both sides at once. The Challenging the Evaluator study found models judge better when conflicting answers sit side by side. Paste your claim and the strongest counter-claim together and ask which is better supported.
4. Don't argue. If it raises an objection, write it down and go and check it. Debating in the chat is where answers flip.
5. Start fresh for each claim. A long conversation is full of your framing.
6. Open every source it gives you. If you can't find it, it doesn't count.

If two different assistants give you different answers on the same claim, don't pick the one you like. Treat the disagreement as a sign that the claim needs checking by hand.

That gets you a better first pass. It still won't tell you which claim a buyer or investor will go after first. If that question sounds familiar, I covered it from the idea side in why ChatGPT said your idea is brilliant.

A Fast Self-Check

You can apply a rough version of this to your own deck without any tools. Do it before you send anything to an investor, and before you ask an AI.

For your three or four highest-stakes claims, write down the single most confident sentence you'd use to defend each one out loud. Then, separately and honestly, write down the strongest objection a sceptical investor's associate would raise, the strongest regulatory question, and the most likely plain explanation for the same number.

Watch yourself while you do it. If you find yourself softening an objection because it's uncomfortable to argue against your own claim, that's the same pull the research describes, showing up in you instead of a chatbot.

If you're raising, the free Series A claim checklist lists the claims investors check first. Anything you can't defend against all three readings is the slide to fix this week.

Then swap roles. Give the deck to the person on your team most likely to disagree with you, hand them your objections, and ask them to add one more. If they can't, you've probably found the real ones. If they can, that's the slide to fix first.

Where ThriveFinity Fits

We sell Release Assurance: claim checking for decks, websites and proposals, signed by a named verifier. We use AI to speed up finding sources and drafting. A person reads the sources, makes the calls and signs, and every paid deliverable says where AI was used, as our AI transparency policy sets out.

• First Signal (free, under an hour): an automated scan of one claim or your positioning. It's AI-generated and unsigned, so it has the limits this post describes. Treat it as a first look.
• Release Assurance Pack (£699 + applicable taxes, 3 business days): every claim in one asset checked, graded and tagged, weak ones rewritten, with ranked rebuttals and a signed Assurance Note.
• Release Assurance Suite (£1,749 + applicable taxes, 5 business days): three or more assets checked against each other.

In practice this catches things like a market-size figure from a years-old report, a ratio that doesn't reconcile with data elsewhere in the deck, or a competitive claim stated more broadly than the evidence allows. The anonymised sample report shows each flag with its source trail.

We're not the right choice if you need legal or regulatory sign-off, if you want someone to decide what your deck should say in the first place, or if you're after a promise that you'll raise. Nobody can honestly sell you that last one.

Pranav Unni

Founder · ThriveFinity Connect on LinkedIn →

Pranav Unni is the founder and lead verifier of ThriveFinity. He reads and signs every paid deliverable personally, and writes about go-to-market decisions for established B2B companies.