Last week OpenAI updated GPT-6 in ChatGPT to give more concise answers and to "reduce overly flattering language". In other words, the world's most popular chatbot had been telling its users they were geniuses a little too often... We talked about exactly this a few months ago, in a Critical thinking workshop I ran for the HR leaders of a large organization, along with a few other related topics. So here are some tips on using AI without being fooled by it.
Confidently Wrong
AI mistakes stopped being funny a long time ago: they now come with a price tag. In July 2025 a US federal judge fined two lawyers for MyPillow CEO Mike Lindell $3,000 each, after they submitted a court filing full of errors, including cases that AI tools had simply invented. In October 2025 Poland's National Appeal Chamber disqualified the lowest bid in a road maintenance tender, worth about €3.66 million, because the bid's 280-page explanation cited tax rulings that don't exist. The chamber's view: the company simply failed to verify what the AI wrote.
Neither of these was a technical failure. In both cases experienced professionals received a fluent, confident and well-formatted answer, and simply didn't check it. That is the real risk: AI is fast and sounds sure of itself, and we get used to not checking, especially under time pressure.
Why AI Makes Things Up
A large language model doesn't look facts up in a database. It predicts the next likely word, based on patterns in the texts it was trained on. It's like a student who didn't study the material but hands in a paper that looks excellent, except that some of the facts are wrong...
Three things make it worse. Training rewards answers that sound confident and helpful, not answers that say "I don't know". Rare facts, such as a specific clause in your local labor law, appear so seldom in the training data that the model has to guess. And anything that happened after the training cutoff forces it to fill in the gaps, with exactly the same confidence.
Five Ways to Get Fewer Made-Up Answers
1. Use a reasoning model
A standard model writes the first plausible answer and moves on. A reasoning model (the "Thinking" option in ChatGPT, Copilot and the other chatbots) plans, checks and only then answers. It takes longer, but it's much less impulsive. Let the fast model write drafts and brainstorm, and give the thinking model every task where accuracy matters more than speed.
2. Ask it to check itself
Chatbots are often better at reviewing text than at writing it. After an important answer, ask: "Are you sure? Review your reasoning and point out any gaps or uncertainty." In the second round the model reads its first answer as if someone else wrote it, and often finds mistakes in it. For any important task, don't settle for the first answer.
3. Give it your sources
Paste the policy, the contract or the data into the prompt and add: "Use only the information I have provided." The model switches from creative prediction to summarizing, and the number of errors drops sharply.
4. Stay within your expertise
Only use AI to produce content you are able to check. In a field you don't know, the made-up parts will sound exactly like the truth. It's like reviewing a legal document in a language you're not fluent in.
5. Activate search and verify sources
This is the tip with the highest return. Start your prompt with "Search the web for…" (or turn on web search, if your chatbot has this option), then click every link the AI cites: check that the page exists, that it's current, and that the site is credible. Then read the source yourself, because AI summaries can drop or twist details. Yes, this takes much longer than copying the AI's answer as is, but it takes much less time than finding the sources on Google yourself and summarizing them. A quick check of the sources makes the answer far more reliable, and in the end, you are the one responsible for what you write!
The Chatbot That Always Agrees
Made-up facts are only part of the problem. Chatbots are trained on human feedback, and we rate agreeable answers higher, so the models learn to agree with us even when we're wrong. Tell a chatbot that an employee complained about unfair treatment but you think they're overreacting, and it may answer "if it doesn't seem serious to you, it's probably fine to let it go". Share your hiring strategy and ask what it thinks, and it will call it "innovative and forward-thinking". Research from MIT in February 2026 found that personalization and long conversations make this worse: over time the AI starts mirroring your worldview.
The fix is to ask for criticism explicitly. These are the prompts I give participants in my workshops, and you can also add them to your chatbot's custom instructions:
- "Play devil's advocate and test this idea."
- "Which assumptions did I make, and which of them might be flawed?"
- "Present 3 well-reasoned counter-arguments to the strategy I just described."
- "What are the 5 most likely ways this plan could fail? For each, give an early warning sign and a mitigation."
- "Don't compliment me. Give me your honest opinion, no sugar-coating."
And one more tip: don't reveal your own opinion first. Ask neutrally, and share your view only after you've seen the AI's answer.
The Critical Thinking Checklist
Before you act on an AI answer, especially on topics that involve money, the law or your organization's reputation, work through this checklist:
- Think first: did I use a thinking model?
- Ground it: did I provide documents, data or web search results?
- Audit it: did I open and read the cited sources myself?
- Challenge it: did I ask for a devil's advocate or a critique of my assumptions?
- Own it: is this within my expertise, so I can actually verify it?
We practice these habits hands-on, with your own tasks, in our Advanced AI workshop, and we also run a version for university faculty and lecturers.
Summary
AI is a powerful engine, but you are the one at the wheel. The people who get the most value from it are not the ones who use it the most, but the ones who know when to question it, how to check it, and when to decide otherwise. Make checking a habit of the whole team, not just yours, and the next confident mistake will stop with you instead of reaching a judge or a tender committee.
