Refresh the page, ask the exact same thing again, and the AI Overview sitting at the top of your search can look noticeably different — different sources, different wording, sometimes a different take on the facts entirely. That’s not a bug you stumbled into. It’s close to how the feature is built to behave, and once you understand why, the constant shifting stops feeling random.
It’s Not Your Imagination — Someone Actually Measured This
An Ahrefs team member noticed it happen live while writing about the feature: ask the same question, refresh, and get a visibly different AI Overview back. So Ahrefs ran the comparison properly, tracking pairs of consecutive AI Overviews for the same query and counting how many named entities — brands, people, products, whatever the overview mentioned — stayed the same between the two versions.
The result: about 54 percent of entities carried over from one version to the next. The other 46 percent changed. That’s essentially a coin flip on any given name or fact staying put between two back-to-back generations of the same answer. Interestingly, the researchers found the churn in cited sources moved at almost exactly the same rate as the churn in entities — off by just half a percentage point — which points to Google reshuffling both the “who’s being talked about” layer and the “who’s being linked to” layer of the answer together, as one unit, rather than one staying fixed while the other drifts.
The Answer Isn’t Sitting There Waiting for You — It’s Assembled on the Spot
The reason a refresh can change things so much comes down to what an AI Overview actually is under the hood. It isn’t a saved snapshot Google wrote once and now serves to everyone who searches the same thing. We’ve gone through the mechanics of this before — Google breaks your search into a handful of related sub-questions, pulls candidate passages for each one from its index, and has Gemini stitch them into a written answer with citations attached. Every part of that pipeline can shift slightly between runs: which sub-questions get generated, which passages get pulled back for each one, and how the model phrases the result from whatever it retrieved that time.
Nothing about that process guarantees the same output twice. It’s closer to Google re-doing a small research task from scratch each time you ask, rather than opening a file it already has saved.
Some Questions Move More Than Others
Not every search wobbles equally. According to Search Engine Land’s guide to how AI Overviews get triggered, the feature pulls from different kinds of pages depending on what you’re asking — straightforward informational queries tend to draw from blog-style explainer content, while comparison or judgment-based questions lean more on list-style and third-party pages. That difference in source material is itself a difference in how much room there is for the answer to shift: a query with one settled, factual answer has less to disagree with itself about than a query asking which of two options is “better,” where reasonable sources genuinely differ.
Broad, simple definitional searches tend to hold together fairly consistently between refreshes. Anything involving comparison, opinion, or a topic where the web itself doesn’t agree is where you’re more likely to catch the overview meaningfully changing its story.
Sometimes It’s Not Just Different — It’s Wrong, Then Right, Then Wrong Again
This is the part that matters most if you’re actually relying on what the overview tells you. A widely covered analysis, commissioned by The New York Times and carried out by an AI research firm called Oumi, found that AI Overviews were accurate roughly 90 percent of the time. Ninety percent sounds solid until you do the math on Google’s actual search volume — as Popular Science laid out, Google handles more than five trillion searches a year, which means even a 10 percent error rate works out to tens of millions of wrong answers every hour. And because the feature regenerates itself on demand rather than settling into one fixed answer, a search that returns something false the first time can come back correct on the very next attempt — with no way to tell, in the moment, which version you’re looking at.
There’s a genuinely funny example of this same unreliability turning on itself. Someone asked Google’s AI Overview about how often AI systems hallucinate, and it confidently cited a figure claiming AI Overviews specifically get things wrong 60 percent of the time — except that number actually came from an unrelated experiment testing different chatbots on an entirely different task, which the overview appears to have simply misread and misattributed, according to Yahoo Tech’s writeup of the mix-up. Google, for its part, maintains that AI Overviews are built on its most accurate Gemini models and held to the same quality standards as the rest of Search.
The Model Underneath Keeps Getting Swapped Too
On top of the moment-to-moment regeneration, there’s a slower kind of change happening in the background. Google continuously updates the models and systems powering AI Overviews — new Gemini versions roll out, and improvements first tested in the more conversational AI Mode gradually work their way into ordinary AI Overviews over time, according to Google’s own comments reported by Yahoo Tech. A question that got one answer last month can get a structurally different one now, not because your search changed, but because the system answering it did.
What Actually Matters If You’re On the Receiving End of This
If you’re a searcher, the practical takeaway is simple: don’t treat a single AI Overview as a settled, permanent fact, especially on anything with real stakes. If it matters — health, money, a decision you can’t easily undo — refresh the question or check the actual sources it cited before you act on it. A second look costs you nothing and catches exactly the kind of flip-flop described above.
If you run a website and you’re chasing citations, this changes the calculus a bit too. It’s not a one-time win where you optimize a page and then coast — since the pool of cited sources and entities reshuffles constantly even for identical searches, staying visible looks more like an ongoing presence than a single achievement. That’s also worth keeping in mind alongside the fact that a citation doesn’t even require a page-one ranking to begin with; the whole system is more fluid, in both directions, than it first appears.
Frequently Asked Questions
Why does Google’s AI Overview give a different answer when I refresh the page?
Because it isn’t pulling from a saved, fixed response — it regenerates the answer each time, re-running its retrieval and writing process. Research tracking this found roughly 46 percent of named entities changed between two back-to-back versions of the same AI Overview.
Is Google’s AI Overview ever just wrong?
Yes, regularly. An analysis commissioned by The New York Times found AI Overviews accurate about 90 percent of the time, which at Google’s search volume still works out to tens of millions of incorrect answers every hour.
Can the same question get a wrong answer once and a correct one the next time?
Yes, and this is one of the more frustrating parts of how the feature works — because each AI Overview is generated fresh, an inaccurate response can flip to an accurate one (or vice versa) on a later attempt, with no visible way to know which version you’re seeing.
Do all types of search questions change equally often?
No. Straightforward, factual, definitional questions tend to stay more consistent between attempts. Questions involving comparison, opinion, or topics where sources genuinely disagree tend to show more variation.
Does Google update the AI model behind AI Overviews over time?
Yes. Google continuously updates the Gemini models powering the feature, and capabilities first introduced in its more conversational AI Mode are gradually being folded into standard AI Overviews as well.
Should I trust an AI Overview answer without double-checking it?
For anything low-stakes, it’s usually fine. For anything that actually matters, it’s worth a quick refresh or a check of the linked sources — given how often the answer changes and how often it’s simply wrong, a second look is a small cost for real peace of mind.
