The short answerNo. With identical prompt text, ChatGPT named a different set of agencies on 474 of 488 consecutive-day pairs where both days named an agency. Across 1,636 answers from ChatGPT, Perplexity and Google AI Overviews to 39 B2B agency questions, 5 to 18 October 2026 (Peec AI), ChatGPT named the same first agency as the day before 38.5% of the time.
| Same question, next day (5 to 18 Oct 2026) | ChatGPT | Perplexity | Google AI Overviews |
|---|---|---|---|
| Identical list of agencies | 2.9% (14 of 488) | 20.3% (88 of 433) | 5.8% (27 of 462) |
| Same agency named first | 38.5% (188 of 488) | 62.1% (269 of 433) | 30.5% (141 of 462) |
| Agencies from day one named again | 55.9% | 66.2% | 48.0% |
| Different agencies one question surfaced in 14 days (average per question) | 15.4 | 8.4 | 16.3 |
| Agencies named per answer (empty answers counted as zero) | 4.9 | 2.9 | 4.1 |
A single screenshot of an AI answer tells you what one engine said once. It does not tell you what your buyers see.
How we ran the study
We track buyer questions for LoudFace in Peec AI, a tool that sends the same prompt text to ChatGPT, Perplexity and Google AI Overviews once a day and records each answer, its sources and the brands it names.
For this study we took the 39 questions we treat as our main ones: generic and category-level agency questions such as "Agency that helps SaaS companies rank in AI search" and "Which agencies do LLM optimization for B2B tech companies?". We pulled every answer from 5 to 18 October 2026, 14 days. Peec listed 1,638 answers and we read 1,636 of them (two failed to load): 545 from ChatGPT, 545 from Perplexity and 546 from Google AI Overviews.
"Named" means one of the 53 brands Peec tracks in our project, almost all of them agencies. An agency outside that list does not count, so the study measures how stable the known names are. It does not score every word of the answer.
We used four measures, each comparing an engine with itself on the same question on two consecutive days where both answers named at least one agency:
- Identical list. Both days named exactly the same agencies.
- Same first pick. The first agency named was the same on both days.
- Kept. The share of agencies named on day one that were named again on day two.
- Overlap. Agencies named on both days divided by agencies named on either day. 1.0 means identical lists, 0 means nothing in common.
If you ask ChatGPT the same question twice, you get a different shortlist
Across 488 consecutive-day pairs, ChatGPT repeated the same set of agencies 14 times. That is 2.9%. It kept 55.9% of the agencies it had named the day before, so of all agencies named, 44.1% did not return the next day. Average overlap was 0.391.
Here is one question, "Agency that helps SaaS companies rank in AI search", on ChatGPT over five days:
| Date | Agencies ChatGPT named, in order |
|---|---|
| 12 Oct | Omniscient, PipeRocket Digital, First Page Sage |
| 13 Oct | LoudFace, Derivatex, Omniscient, Obility |
| 14 Oct | Omniscient, Minuttia, Siege Media, iPullRank |
| 15 Oct | Omniscient, Directive Consulting, iPullRank, Skale, LoudFace, Omnius |
| 16 Oct | LoudFace, Omniscient, Skale, iPullRank |
Omniscient made all five lists. No two lists matched. A buyer who asked on Tuesday and a colleague who asked on Wednesday would have walked into the same meeting with different shortlists.
The first agency named held on better than the rest. On ChatGPT, the agency named first one day was named again somewhere in the next day's answer 68.6% of the time (335 of 488), against 52.7% for the other agencies on the list. Keeping the first spot was rarer: ChatGPT named the same agency first on 188 of 488 day pairs, 38.5%.
Which AI engine gives the most consistent answers?
Perplexity, on all four day-to-day measures. Google AI Overviews was the least consistent on three of the four measures: same first pick (30.5%), agencies kept (48.0%) and overlap (0.350). ChatGPT had the lowest identical-list rate, 2.9% against 5.8%.
| Engine | Day pairs | Identical list | Same first pick | Kept next day | Overlap |
|---|---|---|---|---|---|
| ChatGPT | 488 | 2.9% | 38.5% | 55.9% (1,353 of 2,421) | 0.391 |
| Perplexity | 433 | 20.3% | 62.1% | 66.2% (932 of 1,407) | 0.541 |
| Google AI Overviews | 462 | 5.8% | 30.5% | 48.0% (948 of 1,977) | 0.350 |
One thing helps Perplexity's lead. Among answers that named at least one agency, it names fewer per answer (3.3, against 5.0 on ChatGPT), and a short list is easier to repeat. Separately, Perplexity left more answers without any tracked agency: 57 of 545, against 10 of 545 on ChatGPT and 29 of 546 on Google AI Overviews. Those answers sit outside the day-pair measures above, which only compare days where both answers named an agency.
The engines also differ in what they read. We compared the source domains Peec returned for each answer, the pages the engine read, with the same engine's answer the day before. Perplexity's source domains overlapped by 60.1% day to day (503 pairs). ChatGPT's overlapped by 22.0% (315 pairs) and Google AI Overviews' by 27.4% (498 pairs). These source figures count only day pairs where both answers returned at least one source domain. That is why ChatGPT has 315 pairs here: 111 of its 545 answers returned no source domains. Perplexity, the engine with the most similar sources from one day to the next, also gave the most similar answers. The ordering did not hold between the other two: Google AI Overviews' sources overlapped more than ChatGPT's (27.4% against 22.0%), yet its answers overlapped less (0.350 against 0.391). Steadier sources went with steadier answers on Perplexity, and they do not explain the whole gap.
That fits what we found in our own answers in September: engines named LoudFace almost only when they had read one of our pages. LoudFace was named 631 times in a week, and 624 of those answers had read a loudface.co page. If naming follows reading, an engine that reads a different set of pages tomorrow can name a different set of agencies tomorrow.
One question, two weeks, 8 to 16 different agencies on average
The daily churn adds up. On an average question, ChatGPT named 4.9 agencies per answer but 15.4 different agencies across the 14 days. Google AI Overviews surfaced 16.3 over the same window with 4.1 per answer. Perplexity surfaced 8.4.
| Engine (average per question) | Agencies per answer (empty answers as zero) | Different agencies in 14 days | Named on 75%+ of days | Named on one day only |
|---|---|---|---|---|
| ChatGPT | 4.9 | 15.4 | 1.8 | 4.5 |
| Perplexity | 2.9 | 8.4 | 1.3 | 2.3 |
| Google AI Overviews | 4.1 | 16.3 | 0.7 | 5.8 |
So each question has a small core and a long tail. On average, fewer than two agencies per question held a place on three days in four on ChatGPT. On Google AI Overviews, fewer than one did. The rest of each day's list came from agencies named on fewer than three days in four.
Only 12 of the 39 questions had an agency ChatGPT named on every day we read. On Perplexity it was 5 of 39, and on Google AI Overviews 3 of 39.
Even the most-named agency in the category was not a fixture. Omniscient appeared in 71.9% of ChatGPT's answers (392 of 545), 58.7% of Perplexity's and 56.8% of Google AI Overviews'. It was the leader everywhere and still missing from more than a quarter of ChatGPT's answers. LoudFace was named in 28.1% of ChatGPT's answers (153 of 545), fifth of the 53 brands tracked.
What one check gets wrong, and what a week of checks fixes
Take every agency an engine named on at least half of a question's days. Check that question once, on a random day, and you miss the agency about a quarter to a third of the time.
| Engine | Single check misses an agency named on at least half of its days |
|---|---|
| ChatGPT | 26.8% (557 of 2,082 checks) |
| Perplexity | 26.3% (371 of 1,413) |
| Google AI Overviews | 33.9% (461 of 1,358) |
We then asked how far a short reading lands from the days it did not cover. For every agency a question named at least once, we compared its named rate in one window with its rate on that question's other days:
| Reading compared | ChatGPT | Perplexity | Google AI Overviews |
|---|---|---|---|
| One day against the question's other days | 30.6 points off | 30.7 points off | 30.6 points off |
| Week one (5 to 11 Oct) against week two (12 to 18 Oct) | 18.5 points off | 20.3 points off | 19.0 points off |
These are average differences. One day's reading sat 30.6 to 30.7 points from the rest of the fortnight on every engine, so a single day is a noisy number. Week one and week two sat 18.5 to 20.3 points apart, so even a week of readings for one agency on one question moves from week to week.
The practical rule for anyone tracking AI visibility:
- Never judge a question from one answer. A screenshot from a sales call or a founder's phone is one draw.
- Compare week with week. Read a week against a week, and judge a trend over several weeks, not one day.
- Read each engine on its own line. Their consistency differs, and a blended number hides which engine moved.
- Read many questions. One question's week-to-week swing is large, so judge visibility across a set of questions, not one.
If you hire someone to do this, the AEO ROI guide covers what to ask for.
Why AI answers change
Several things move an answer, and they stack.
Sampling and batching
Even with randomness turned down, language model servers are not fully repeatable. Thinking Machines Lab traced most of it to load: "the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies!" In its test, 1,000 runs of one prompt at temperature 0 on an open model produced 80 different completions. The test ran an open model on its own inference setup rather than ChatGPT, but it shows that identical input does not guarantee identical output.
Live search
When an engine searches the web, the pages it retrieves change from day to day. OpenAI's help centre says ChatGPT may search the web automatically when a question benefits from current information. Google says AI Overviews and AI Mode may use a query fan-out technique, issuing several related searches, and notes that its two AI features differ from each other: "AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary" (Google Search Central). Our earlier fan-out queries explainer covers how those searches work.
Memory, location and account
Our study holds these still. Peec's tracker sends the same prompt text each day through the web interface, and Peec describes it as a logged-out user. The 39 prompts are set to a US location. Real users bring their own memory and location. OpenAI's Memory in ChatGPT page says that when Memory is on, ChatGPT can remember preferences and details from your past chats and other sources to personalise responses, and may use them to shape its web searches. Its search help page says ChatGPT may use an approximate location from your IP address for local results. Two buyers asking the same question can get answers shaped by their own histories and locations, on top of the day-to-day churn we measured.
Does ChatGPT give the same answer to everyone?
No. In our data, ChatGPT named a different set of agencies on 97.1% of consecutive days (474 of 488 pairs) when we sent it identical prompt text. We did not compare two users, and OpenAI documents Memory and approximate location as further personalisation, so two people can differ by more than we measured. What does carry across answers is a small core: agencies named on three days in four averaged 1.8 per question on ChatGPT, 1.3 on Perplexity and 0.7 on Google AI Overviews.
How this compares with other studies
| Who measured it | What they ran | What they found |
|---|---|---|
| LoudFace, this study | 1,636 answers, 39 B2B agency questions, 3 engines, 14 days | Identical agency list on consecutive days: ChatGPT 2.9%, Perplexity 20.3%, Google AI Overviews 5.8% |
| SparkToro, January 2026 | 600 volunteers, 12 prompts, 2,961 runs, Nov to Dec 2025 | "there's a <1 in 100 chance that ChatGPT or Google's AI, if asked 100X, will give you the same list of brands in any two responses." |
| Thinking Machines Lab, September 2025 | 1,000 runs of one prompt at temperature 0, open model | 80 unique completions |
| Atil et al. (arXiv preprint) | 5 models, 8 tasks, 10 runs, deterministic settings | "accuracy variations up to 15% across naturally occurring runs" |
| LoudFace, 15 October 2026 | 987 same-day answer sets, 1 to 7 Oct | Next-day overlap: ChatGPT 0.421, Perplexity 0.423, Google AI Overviews 0.313 |
SparkToro's figure is lower than ours. Its volunteers ran the prompts in the AI tools themselves, copied and pasted the responses into survey forms, and kept their usual settings, personalised or default. Its twelve prompts ranged from chef's knives to cloud computing providers for SaaS startups. Its lists count every brand, product or entity an answer recommended. Ours count only 53 tracked agencies, which makes a repeated list more likely, and our prompts run without personal history. Both studies land on the same advice. SparkToro's Rand Fishkin now believes "visibility % across dozens to hundreds of prompts run multiple times is a reasonable metric."
Our own one-week read on 15 October found the same ordering for ChatGPT and Google AI Overviews. Perplexity read steadier on these 39 main questions (0.541) than on that study's 141 non-branded questions (0.423).
Limits of this study
- It counts only the 53 brands Peec tracks in our project, so it measures how stable the known names are. It does not score whole answers.
- It covers one category, B2B SaaS questions about SEO, AEO and growth agencies, asked with a US location. Consumer and product questions may behave differently.
- Each engine ran once per question per day, so it measures day-to-day change. It says nothing about two runs a minute apart.
- Peec describes its tracker as a logged-out user, and it is fixed to a US location, so it never measured two different users. See the memory and location section above.
- 14 days is long enough to see the churn but too short to see seasonal or model-release effects.
If you want this read for your own category, our GEO service tracks AI citations across ChatGPT, Perplexity and Google AI Overviews in a live Peec panel, reported per engine every week.
An AI answer is one draw from a moving list, so judge your visibility over several weeks of answers, engine by engine.
