Why ChatGPT gives a different answer every time you ask.
If you asked an assistant to recommend a business like yours, asked again, and got a different list, nothing broke. This page explains why it happens, what one large study found, and how to measure a business's AI visibility when no single answer can be trusted on its own.
Each answer is a new sample.
Why does ChatGPT give different answers to the same question?
An assistant writes each answer fresh. It chooses its words with some randomness built in, and when web search is on it reads whatever pages its search returns at that moment. Change the wording, the assistant or the location and the list changes again. Two askings of the same question are two separate samples, not two looks at one fixed ranking.
Four reasons the list changes.
1. The answer is written, not looked up
There is no stored list of the best dentists in your town that the assistant reads out. Each answer is generated word by word, and the model is built to vary its choices slightly each time. On a question with many reasonable answers, such as which local business to choose, that small variation is enough to change who gets named.
2. Live search returns different pages
When an assistant searches the web before it answers, it reads some of the pages that come back, and what comes back changes over time. A business named on a page the assistant happened to read is in the answer. The same business, missing from the pages read the next time, is not.
3. The wording and the place change the question
Best orthodontist near me, best reviewed orthodontist in a named town, and how much braces cost nearby are three different questions to an assistant, and each one sends it to different sources. Where the person is, or which town the question names, changes the candidates before anything else happens.
4. Each assistant is a different system
ChatGPT, Gemini, Claude and Perplexity are built by different companies, search the web through different systems and can rely on different sources. A business named by one is not automatically named by the others, which is why each one has to be checked on its own.
What a 2026 study of dentist recommendations found.
Dental.me, a dental directory, asked ChatGPT, Gemini and Claude one question, who are the best dentists, in each of the 50 largest US cities. DrBicuspid reported the results on 28 October 2026.
| Finding | Result |
|---|---|
| Different practices named as best, across all three assistants | 560 |
| Practices named by all three | 30 |
| Cities where the three had no consensus on the best practice | 28 |
| Recommended practices that changed when the same question was asked again, in the 10 largest cities | 38% |
| A Baltimore practice Gemini included in its five highly rated picks | 3.1 stars on 24 Google reviews |
| Baltimore practices in the directory's own data at 4.8 stars or higher with more than 100 reviews | 49 |
Two cautions. The study comes from a dental directory, and it asked the broadest question possible. The article reports a repeat question only for the 10 largest cities. The Baltimore rows are the ones for a business owner to note: one of Gemini's five picks had 3.1 stars on 24 reviews, in a city where 49 practices have 4.8 stars or higher on more than 100 reviews. A strong rating on its own did not decide who was named.
Variable is not the same as random.
Are AI recommendations for local businesses random?
Not entirely. An assistant can only name a business it can find and read, and the study below shows the best rated are not always the ones named. A site the assistant can fetch, pages that say plainly what the business does and where, and outside listings that agree are the parts of being named that a business controls.
The work behind those parts is described in GEO for local businesses.
Is one AI answer good evidence either way?
No. One answer is one sample. A screenshot showing your business named proves it can be named, not that it usually is, and a screenshot showing a competitor proves just as little. Anyone selling or judging AI visibility from a single answer is reporting a sample of one.
Why did my AI referrals go down this month?
Check four things before concluding anything. Answers vary, so one month can be noise; compare several. AI apps often send visitors with no referrer, so they read as direct visits. The analytics tag on your site may report to a different account than the one you are reading, which can happen when more than one agency has worked on the site. And many categories are seasonal. Only a trend across months, measured the same way each time, says the recommendations themselves changed.
Tracking leads from ChatGPT covers the measurement side.
How to get a number you can trust.
- Write the questions the way customers ask them. Include the town, the service and the price questions, not only the business name. A business can be recognised by name and still be missing from every recommendation.
- Ask each question more than once, on a schedule. The same set, the same way, every time. A change in the set is a change in the instrument, and it breaks the comparison with last month.
- Report a share with its denominator. Named in 7 of 20 answers says what was measured. A percentage with no count behind it does not.
- Keep each assistant separate. A combined score hides the assistant where you are missing.
- Read the trend, not the day. A single run is a snapshot. Movement that holds across several runs is a finding.
The other guides in this set.
- Local SEO vs GEO vs AEO for local businesses Three acronyms, one plain comparison table, and which one to fund first.
- GEO for local businesses: does yours need it? Three questions that decide it, what the work actually is, and how to prove it worked.
- What sources ChatGPT uses to recommend local businesses What is observable about its retrieval, and what is only claimed.
- AI visibility audit for local businesses: what we check and why Every check in the free audit, with the reason it exists.
- How to choose an AI SEO or GEO agency for a local business Nine questions to ask, and the answers that should end the call.
- Which leads found you from ChatGPT, and how to track them The three records that show an AI lead, what each can prove, and what to check when referrals dip.
Find out whether assistants can read your site.
The free audit checks whether AI retrieval can fetch and read your site, the part of being named that you control. A person emails it to you within 2 business days.