Blog ·
Ask AI the same thing twice, get a different answer
Ask a friend “Where’s a good bakery?” three times. You might get three different answers. AI does this too. So if an AI answer changes once, it does not mean much. We wanted to know how much AI answers change all by themselves.
- 11%
- About 1 in 10 answers named the company once, then didn’t
- 20%
- 1 in 5 answers showed the company’s website once, then didn’t
- 44%
- Asked twice, AI used the same websites only this often
- 2%
- Ask twice and wait until both agree: mix-ups drop to this
What did we do?
On 30 October 2026 we picked three companies. They all sell software to other businesses. None of them changed their website while we tested.
For each company we wrote eight questions, the kind a shopper might ask. Five questions did not say the company’s name, like “What tool is good for a small team?”. Three did.
We asked every question to four AIs: ChatGPT’s maker OpenAI, Google Gemini, Claude and Perplexity. We asked each one twice. Then we did it all again. And again. Three times in half an hour.
Then we compared the answers. Did the AI say the company’s name both times? Or only once? If only once, we call that a flip.
What happened?
About 1 time in 10 (11%), the AI said the company’s name once and then left it out.
The company’s website flipped even more: 1 time in 5 (20%).
The AI also looked at different websites each time. Only 44% were the same the second time. So more than half changed.
When the question said the company’s name, the AI always named it. Some AIs changed more than others. Claude and OpenAI changed the most. Gemini changed the least.
Which AI changed its mind most?
| AI | Said the name | Showed the website |
|---|---|---|
| Claude | ||
| OpenAI | ||
| Perplexity | ||
| Gemini | ||
| All |
— We didn’t save which websites Perplexity used this time. So the “All” website number counts the other three AIs (270 answers).
What does this mean?
One AI answer is like one roll of a dice. Say your company is missing. You fix your website, ask again, and now it shows up. Was it your fix, or just luck? You can’t tell.
So don’t trust just one answer. Ask twice, and only count a change when both answers agree. When we did that, mix-ups fell to 2%. That is how FwdLink checks. It is also why FwdLink checks once a month, and 2 and 4 weeks after you fix something, not every day.
These three companies are already well known. AI names them about 9 times out of 10. A smaller company may flip even more.
Next, we will ask the same questions every week until the end of November and tell you what changes.
Asking twice means fewer mix-ups
| Look once: website | |
|---|---|
| Look once: name | |
| Both tries agree (how FwdLink checks) |
How often the answer changed, even though nothing was changed.
Next step