Last updated: 30 October 2026
AI answers change far more often than most business professionals assume: Google's AI Overviews have an average persistence of just 2.15 days, and repeated identical prompts to ChatGPT produced consistent answers only 73% of the time in a controlled study. For UK firms relying on AI for research, compliance or customer-facing answers, this volatility is measurable, not anecdotal.
Key Takeaways
- Google AI Overviews change on average every 2.15 days, according to Ahrefs (2026), based on a study of 43,000 keywords.
- Only 54.5% of cited URLs overlap between consecutive AI Overview responses to the same query, meaning roughly one citation changes every time a query is re-run, per Ahrefs (2026).
- ChatGPT gave consistent answers only 73% of the time across ten identical prompts testing scientific hypotheses, found Washington State University researchers (2026).
- Perplexity's top recommended brand flipped 84% of the time over 90 days, versus just 41% for Claude, the most stable platform tested, according to MaxAEO (2026).
- Newer model versions are measurably more consistent: ChatGPT 4 held 77.8% accuracy across repeated medical exam questions versus 44.9% for ChatGPT 3.5, per the NCBI/PMC study (2024).
What does "AI answer drift" mean?
AI answer drift is the tendency of a generative AI system — a chatbot or search feature that produces natural-language responses rather than a fixed list of links — to return a different answer to the same, or near-identical, question over time or across sessions. It covers everything from a single word changing in phrasing to a completely different brand, fact or source being cited.
Drift happens for several distinct reasons, and separating them matters because each has a different fix. Some drift is structural — built into how large language models (LLMs, the neural network architectures behind tools like GPT and Gemini) generate text. Some is operational — caused by retraining, re-indexing or live web retrieval. And some is contextual — caused by memory, personalisation or account tier.
For a UK business using AI answers to brief a client, check a regulation, or monitor how it's described online, the practical question isn't whether drift exists — it clearly does — but how much, how fast, and where it matters most.
Why do AI chatbots give different answers to the same question?
AI chatbots give different answers to identical questions primarily because of probabilistic text generation: at each step, the model samples from a distribution of likely next words rather than selecting one fixed output. Mesut Cicek, Associate Professor at Washington State University's Carson College of Business, documented this directly: "We used 10 prompts with the same exact question. Everything was identical. It would answer true. Next, it says it's false. It's true, it's false, false, true."
This isn't a bug so much as a design feature of how these systems work. Cicek's broader conclusion was stark: "We're not just talking about accuracy, we're talking about inconsistency, because if you ask the same question again and again, you come up with different answers."
In that same study, ChatGPT's accuracy on hypothesis-testing tasks also shifted year to year — 76.5% correct when run in 2024, rising to 80% when repeated in 2025 — showing drift isn't only within a single session but compounds across model versions over time, according to Washington State University (2026).
Three mechanical causes sit underneath this:
- Sampling randomness — the model doesn't always pick the statistically "best" next word, by design.
- Prompt sensitivity — tiny wording changes (a comma, a synonym) can shift the probability distribution enough to change the output.
- Model updates mid-study — providers patch models silently between a user's first and second attempt at the same question.
How often do GPT, Gemini and Claude get retrained or updated?
Large AI model providers update their systems on rolling, mostly undisclosed schedules — ranging from near-daily minor patches to major version releases every several months — rather than on a fixed public calendar. OpenAI, Google DeepMind and Anthropic all ship incremental updates to underlying models, safety filters and retrieval systems far more frequently than they announce named version releases such as GPT-4 or GPT-5.
This matters because an update doesn't need to be a headline-grabbing new model to change your answers. A quiet adjustment to a ranking algorithm, a safety guardrail, or the data a model retrieves from the live web can shift output overnight, with no changelog visible to the end user.
The WSU hypothesis-testing study captured exactly this effect: accuracy moved from 76.5% to 80% simply by re-running the same experiment a year apart, per Washington State University (Mesut Cicek et al.) (2026) — evidence that version-to-version drift is real and measurable, even without the user doing anything differently.
For search-integrated tools, the update cycle is even tighter. Google's AI Overviews refresh their underlying content with an average persistence of 2.15 days, based on tracking 43,000 keywords by Ahrefs (2026) — meaning the answer a customer saw on Monday may already look different by Wednesday.
How does temperature affect AI answer consistency?
Temperature is a configurable setting in large language models that controls how much randomness is injected into word selection, with a low temperature (near 0) producing near-deterministic, repetitive output and a high temperature producing more varied, creative — and less predictable — text. Most consumer chatbot interfaces (the free, public-facing version of ChatGPT or Gemini, for instance) run at a moderate-to-high default temperature to sound natural and conversational, which is precisely what makes repeat answers vary.
Businesses building on the API (application programming interface, the developer-facing route into a model that bypasses the consumer chat interface) can usually set temperature to zero or close to it for more repeatable output. This doesn't eliminate variability entirely — model updates and retrieval changes still apply — but it removes one major source of randomness.
The practical takeaway for a UK operations or compliance team: a public-facing chatbot answer and an API-driven internal tool running the same underlying model can behave very differently in terms of stability, purely because of this one setting.
Does a new chat session produce a different answer than the original?
Yes — starting a fresh chat session can produce a materially different answer to the identical question asked moments earlier in another session, even on the same AI platform and the same day. This happens because most consumer AI tools don't carry context, memory or prior reasoning between separate conversations unless memory features are explicitly enabled, so each new session effectively re-samples the model from scratch.
Layer on live retrieval — where a chatbot pulls fresh information from the internet rather than only its training data — and the effect compounds. A query about current interest rates, a regulatory deadline, or a competitor's pricing can return different cited sources within hours, simply because the web content it's drawing from has updated.
This connects directly to the citation instability seen in search-integrated AI. Ahrefs (2026) found that only 54.5% of cited URLs overlap between two consecutive runs of the same query in Google AI Overviews — in practice, close to one source swapped out every single time the query is re-run. Separately, Ahrefs' analysis found that only 38% of pages cited in AI Overviews also rank in the top 10 organic results, down sharply from 76% in a July 2025 study — showing that AI citation selection is drifting away from traditional search rankings over time, not converging with them.
Comparing answer stability: ChatGPT, Gemini, Copilot and Claude
Independent tracking shows meaningful differences in how often AI platforms change their top recommendation for the same prompt, with Perplexity the least stable and Claude the most stable of the platforms measured.
| Platform | Flip rate for #1 recommended brand (90 days) | Notes |
|---|---|---|
| Perplexity | 84% | Most volatile platform tracked |
| Claude | 41% | Most stable platform tracked |
| ChatGPT (3.5) | 44.9% accuracy on repeated medical exam questions | Earlier model version |
| ChatGPT (4) | 77.8% accuracy on repeated medical exam questions | Marked improvement over 3.5 |
Source: MaxAEO (2026) for flip rates; NCBI/PMC (2024) for ChatGPT version accuracy.
Across the full 90-day tracking window of 1,247 prompts spanning 8 AI platforms, MaxAEO (2026) found that on any given day, 17% of prompts change at least one recommended brand compared with the day before, and the wording of answers is phrased materially differently 54% of the time, day over day. Zoomed out across the full 90 days, the #1 recommended brand flipped at least once for 65% of prompts overall.
For UK businesses tracking how they're mentioned across AI engines, this platform-level variation is a practical reason not to rely on a single tool's output as representative of "what AI says" about your brand. Monitoring across ChatGPT, Gemini, Copilot, Claude and Perplexity in parallel — which is the approach Aether AI's citation tracking takes — gives a far more accurate picture than checking one assistant occasionally.
What are the risks for UK businesses relying on AI answers for compliance?
AI answer volatility creates direct exposure for UK firms using chatbots to check regulatory, legal or financial information, because an answer that was accurate last week may not be accurate — or citeable — today. This is particularly acute for sectors governed by bodies such as the Financial Conduct Authority (FCA), the Information Commissioner's Office (ICO) for data protection matters under UK GDPR, or the Health and Safety Executive (HSE) for workplace compliance, where guidance changes and an AI tool trained on older data may not reflect it.
A compliance officer asking a chatbot the same question twice and getting different answers — as documented in the WSU study at a 73% consistency rate — cannot treat either answer as authoritative without independent verification. The risk isn't theoretical: it's the same mechanism that makes Washington State University's hypothesis-testing results flip between true and false on identical prompts.
Sensible practice for UK businesses includes:
- Never citing an AI chatbot's answer as the sole source in a client-facing compliance document.
- Cross-checking any regulatory claim against the primary source — legislation.gov.uk, the relevant regulator's own guidance, or a qualified adviser.
- Logging the date and exact prompt used when an AI answer informs a business decision, so it can be re-checked if challenged later.
- Treating AI-generated answers about your own organisation (how it's described, recommended or cited by AI engines) as something to monitor continuously rather than check once.
Your AI-answer monitoring checklist
- Re-run business-critical prompts weekly, not once, to catch drift early.
- Record the date, exact wording and platform used for any AI answer informing a decision.
- Cross-check regulatory or legal answers against the primary source (gov.uk, the relevant regulator, or legislation.gov.uk) before acting on them.
- Test the same question across at least two platforms (for example ChatGPT and Claude) given their differing stability rates.
- Set temperature to zero on any API-based internal tool used for repeatable tasks.
- Track how your own brand is cited across AI engines, not just how general queries perform.
- Flag any AI answer that changes materially between checks for manual review rather than silent acceptance.
- Re-verify high-stakes outputs (pricing, compliance, legal wording) immediately before publishing or client use, regardless of when they were last checked.
FAQ
How often do AI answers change for the same question?
AI answers can change within hours for search-integrated tools and vary between sessions even on the same day for chatbots. Google's AI Overviews shift on average every 2.15 days, according to Ahrefs (2026), while ChatGPT gave inconsistent answers to roughly 27% of identical repeated prompts in a controlled study by Washington State University (2026).
Why does ChatGPT give different answers to the same exact question?
ChatGPT generates each response by sampling probable next words rather than retrieving a fixed stored answer, so identical prompts can produce different outputs. Researcher Mesut Cicek of Washington State University described testing this directly with ten identical prompts, noting the model flipped between "true" and "false" answers on the same question.
Does asking in a new chat session produce a different answer?
Yes, a new session typically starts without memory of a prior conversation, so the model effectively re-samples from scratch. Combined with live web retrieval, this means the same question asked in two separate sessions, even minutes apart, can return different facts, sources or recommendations.
Which AI platform gives the most consistent answers?
Among platforms tracked over a 90-day period, Claude was the most stable with a 41% flip rate for its top recommended brand, while Perplexity was the most volatile at 84%, according to MaxAEO (2026). Newer model versions also tend to be more consistent than older ones, as shown by ChatGPT 4's 77.8% accuracy versus ChatGPT 3.5's 44.9% on repeated medical questions, per NCBI/PMC (2024).
How often should businesses fact-check AI-generated answers?
Business-critical AI answers, particularly those touching compliance, legal or financial matters, should be re-verified every time they're used in a new document or decision, not relied on from a single check. Given that only 54.5% of cited URLs overlap between consecutive identical AI Overview queries, per Ahrefs (2026), treating any single AI answer as permanently valid is a measurable risk.
Can personalisation or memory make AI answers more consistent for one user?
Memory features, where enabled, allow an AI assistant to retain context from previous conversations with the same account, which can make answers feel more tailored and occasionally more consistent for that specific user. However, this doesn't eliminate underlying model randomness or the effect of provider-side updates, and answers can still diverge between the free and memory-enabled versions of the same tool.
Are paid or enterprise AI tools more consistent than free versions?
Enterprise and API-based tools generally offer more consistency because they allow fixed settings such as temperature control and don't carry the same consumer-facing update cadence as free chatbot interfaces. The clearest evidence of version-driven consistency gains comes from the jump between ChatGPT 3.5 and ChatGPT 4, which improved repeated-question accuracy from 44.9% to 77.8% in the NCBI/PMC study (2024).
Tracking AI answer volatility with Aether AI
This kind of drift is precisely why UK businesses can no longer treat a single AI chatbot check as reliable evidence of how they're described, recommended or cited online. Aether AI's citation tracking monitors brand mentions across six AI engines — ChatGPT, Perplexity, Google AI Overviews, Claude, Gemini and Copilot — because a business appearing consistently in one tool may be invisible, or inconsistently cited, in another.
Aether AI's own measured baseline illustrates the scale of the gap: across 700 live AI engine scans of four UK brands' target keywords between July and August 2026, those brands were named in just 23 answers — 3.3% — with even the longest-established brand appearing in only 6.3% of its 270 scans. Separately, on two of the same brands' own websites, 82.1% and 88.3% of Google Search Console clicks between 19 July and 15 August 2026 came from queries Google itself does not name — a reminder that both visibility and attribution are harder to pin down than they first appear.
If your organisation wants to understand how often your own brand's AI answers are shifting, Aether AI offers a free AI-visibility audit at /audit, with full pricing published at /pricing for ongoing monitoring and tracking.