Every measurement product has a boundary, and most of them keep it in a footnote. Ours is worth reading before you trust a single number on the dashboard, so here it is in full.
An AI answer is not a document sitting on a server that we fetch. It is generated, once, for whoever asked. To measure it at all you have to ask the question yourself — and every decision about how you ask changes what you can honestly claim afterwards.
We capture the interface, not the API
Each question is put to the product a reader uses — chatgpt.com, Gemini, Google's AI Overviews and AI Mode — in the reader's country, and the answer is stored as it was shown: the text, and the sources the interface attached to it. Perplexity is the exception: it is asked through its own API, which returns the pages its answer cites. The model's API is otherwise used only when the interface cannot be reached, and every stored answer records which path it came from.
The two paths are not interchangeable, even with the API's web search on.
We asked ChatGPT 100 buying questions through its own web interface, twice, and the same questions through OpenAI's API three ways. The brands came out mostly the same. The sources did not: the API cited fewer than half as many sites, leaned on the brands' own pages, and shared only 8% of its cited pages with what a reader of chatgpt.com was shown.
What we found
- The interface is its own noise floor. Asked the same question twice in the same minute, ChatGPT named the same brands 78% of the time and cited the same pages only 30% of the time.
- The API names roughly the right brands, in a different order. Brand overlap with the interface was 65% for the API with web search, against 78% for the interface with itself. It picked the same top brand 57% of the time, against 73%.
- The sources are a different list. The API's cited pages overlapped the interface's 8% of the time — under a third of the interface's agreement with itself.
- The API cites the brands; the interface cites reviewers. 64% of the domains the API cited belonged to a brand in the category, against 41% in the interface. TechRadar and Tom's Guide were cited in 18% vs 3% and 16% vs 3% of answers respectively.
- Per-brand figures can be far apart. In running shoes, the interface named Hoka in 80% of answers and the API in 10%.
The interface does not agree with itself
Before comparing the API with the interface, we compared the interface with itself. Every question went to chatgpt.com twice, within seconds of each other, from the same location and with web search on. That pair is the yardstick for everything else: a difference between the API and the interface only means something by however much it exceeds the difference between two interface answers.
The two interface answers named an average of 3.8 tracked brands each and shared 78% of them. They chose the same brand first 73% of the time. Their sources moved much more: 47% of cited domains were shared, and only 30% of cited pages. One answer is a sample, not a measurement — which is why any single screenshot of "what ChatGPT says about us" should be read as one draw from a distribution.
Brands: close, but not the same order
Brand overlap is the share of brands named by either answer that both named. Each API arm is compared with both interface answers and the two figures averaged. The interval is a 95% bootstrap interval over the 100 questions.
- Brand overlap
- Same top pick
Show the data
| Compared with the interface | Brand overlap | Same top pick |
|---|---|---|
| The interface, asked again | 78% (73%–82%) | 73% (64%–81%) |
| API, gpt-6-sol with web search | 65% (60%–70%) | 57% (49%–66%) |
| API, gpt-5.6-luna with web search | 72% (67%–76%) | 68% (61%–76%) |
| API, gpt-6-sol, no search | 67% (62%–72%) | 65% (56%–73%) |
Every API arm agreed with the interface less than the interface agreed with itself, but not by a wide margin, and one result ran against expectation: the smaller model with search tracked the interface slightly more closely than the larger one. Its answers were longer and named more brands — 3.9 on average against 2.9 — and a longer list simply has more chances to overlap. The larger API model wrote 196 words on average; the interface wrote 589.
Sources: where the two part ways
The interface cited 7.6 domains per answer; the API with search cited 3.2. Measured the same way as brands:
- Cited domains
- Cited pages
Show the data
| Compared with the interface | Cited domains | Cited pages |
|---|---|---|
| The interface, asked again | 47% (43%–51%) | 30% (27%–33%) |
| API, gpt-6-sol with web search | 27% (24%–30%) | 8% (6%–10%) |
| API, gpt-5.6-luna with web search | 30% (27%–33%) | 10% (8%–11%) |
The difference is not only how many sources, but which kind. Counting a cited domain as brand-owned when it carries the name of a brand in the category, 64% of the API's citations went to the brands themselves, against 41% of the interface's. The interface leaned on review publishers and comparison articles the API rarely touched:
- Interface
- API with search
Show the data
| Domain | Interface | API with search |
|---|---|---|
| rtings.com | 15% | 13% |
| hubspot.com | 13% | 11% |
| techradar.com | 18% | 3% |
| zoho.com | 12% | 8% |
| tomsguide.com | 16% | 3% |
| bitwarden.com | 10% | 8% |
| tryprofound.com | 8% | 7% |
| peec.ai | 8% | 7% |
| otterly.ai | 7% | 8% |
| protonvpn.com | 8% | 6% |
| semrush.com | 8% | 6% |
| quickbooks.intuit.com | 8% | 5% |
| xero.com | 7% | 6% |
| waveapps.com | 7% | 5% |
| clickup.com | 6% | 5% |
Among the domains the interface cited and the API never did: dupple.com (8%), technologyadvice.com (7%), brevo.com (6%), surfshark.com (6%), dickssportinggoods.com (5%), itechguides.com (5%), dreametech.com (5%), reddit.com (4%).
Category by category
Share of voice here is the share of answers in a category naming a brand. The gap is the average, over every brand named, of the absolute difference between two arms' shares. With ten questions per category these are directional, and the interface-against-itself bar says how much of any gap is noise.
- Interface vs itself
- Interface vs API with search
Show the data
| Category | Interface vs itself | Interface vs API with search |
|---|---|---|
| CRM | 6% | 8% |
| Project management | 5% | 10% |
| Email marketing | 5% | 8% |
| Password managers | 12% | 12% |
| VPNs | 10% | 17% |
| Running shoes | 8% | 14% |
| Robot vacuums | 8% | 17% |
| Small-business accounting | 8% | 9% |
| Web hosting | 9% | 10% |
| AI visibility tools | 8% | 9% |
| Category | Interface top 3 | API top 3 | Widest gap |
|---|---|---|---|
| CRM | HubSpot, Pipedrive, Zoho CRM | HubSpot, Zoho CRM, Pipedrive | Pipedrive: 65% vs 40% |
| Project management | ClickUp, Trello, Jira | Trello, ClickUp, Asana | Monday: 45% vs 10% |
| Email marketing | Brevo, Mailchimp, MailerLite | ConvertKit, Klaviyo, Omnisend | Brevo: 55% vs 30% |
| Password managers | Bitwarden, 1Password, Proton Pass | Bitwarden, 1Password, Proton Pass | Proton Pass: 70% vs 30% |
| VPNs | Proton VPN, NordVPN, Surfshark | Proton VPN, Mullvad, NordVPN | Surfshark: 60% vs 20% |
| Running shoes | Asics, Hoka, Brooks | Asics, Brooks, Nike | Hoka: 80% vs 10% |
| Robot vacuums | Roborock, Dreame, Eufy | Roborock, Dreame, Mova | Eufy: 50% vs 10% |
| Small-business accounting | QuickBooks, Xero, Wave | QuickBooks, Wave, Xero | Xero: 80% vs 60% |
| Web hosting | SiteGround, Hostinger, Kinsta | SiteGround, Hostinger, Kinsta | Bluehost: 40% vs 10% |
| AI visibility tools | Semrush, Profound, Peec AI | Otterly.AI, Peec AI, Semrush | Rankscale: 25% vs 0% |
In password managers, CRM, email marketing, accounting, web hosting and AI visibility tools, the API's gap was within a few points of the interface's own. In project management, VPNs, running shoes and robot vacuums it was close to double, and the brands that moved most — Monday, Surfshark, Hoka, Eufy — were named far more often by the interface than by the API.
What it means if you track AI visibility
- A brand-level number from the API is a rough guide. It will usually find the same leaders, and it can be badly wrong about one brand in one category — which is the brand you are paying to watch.
- A source list from the API is not the one readers see. If the plan is to get onto the pages ChatGPT cites, the API points at brand sites, and the interface points at the reviewers and comparison articles that actually carry the answer.
- One answer is not a measurement. Even the interface disagrees with itself on most of its sources. A figure is only as good as the number of answers behind it, and should come with an interval. We wrote about how we count that in what AI search visibility is.
Method and limits
- Questions. 100 buying questions, ten in each of ten categories — CRM, project management, email marketing, password managers, VPNs, running shoes, robot vacuums, small-business accounting, web hosting and AI visibility tools — phrased as people ask them: "best", "alternatives to", "X vs Y", "cheapest".
- Five answers per question, 500 in all. Two from the chatgpt.com interface, captured by DataForSEO's LLM Scraper with web search forced, located in the United States in English; the interface reported its model as gpt-6. Three from the API through OpenRouter: gpt-6-sol with OpenAI's native web search, gpt-5.6-luna with the same search, and gpt-6-sol with no search. The API answers were sent the question alone, with no system prompt and no location. All were collected on 12 October 2026.
- Brands. Each category has a fixed list of 16 to 20 brands, matched by the same deterministic detector our product uses, with aliases. A brand missing from a list is invisible to every arm equally, so it understates the counts without favouring an arm.
- Sources. The interface's sources are the citations a reader is shown; the API's are its URL citations. Tracking parameters were stripped before comparing pages. "Brand-owned" is a name match on the domain and is approximate.
- Limits. One engine, one country, one language, one day. The interface does not say which gpt-6 tier it runs, so the main API arm is our best match, not a certain one. Ten questions per category is enough to see a pattern, not to rank brands.
- Cost. The whole run cost $12.94, most of it the larger API model with search.
The full dataset — every question, arm, model, brand named and cited URL, 500 rows — is free to download as CSV. You are welcome to use it with a link back to this page.
A capture is still not your screen. It carries no account history and no memory of earlier conversations, so an answer shaped by one reader's past can differ from ours. What it closes is the larger gap — the retrieval, and the sources that come with it. The docs on fidelity say how each answer was obtained, and how sampling works walks through one cycle.
Sources are the ones the interface showed
A captured answer arrives with its own list of sources — the pages the interface linked beside or beneath the text — and that list is what counts as cited. Each address is reduced to a canonical form first, because one answer will happily cite the same page as both https://x.com and https://www.x.com/, and counting that as two citations of two sources is wrong twice over.
The first version of this sampler parsed links out of the answer text instead, and missed every source an engine shows in its own panel rather than writing into the prose. Reading the interface's list is what removed that blind spot.
Named and cited are two different lists
An answer that links your domain without ever writing your name has cited you; it has not mentioned you. We keep those apart, down to the detail of ignoring brand names that appear only inside a URL — the characters around "acme" in https://www.acme.com/ are dots, so a naive match reads the domain as prose and credits you with a mention nobody made.
A brand the answer only linked to is on the citation list. A brand the answer recommended by name is on the other one. The difference is the whole point of the measurement.
The engines we sample, and the ones we don't
ChatGPT, Gemini, AI Overviews, Google AI Mode and Perplexity are sampled on every pass, and the product records which engine and which market produced each answer it stores. Claude is not sampled at all: it has no consumer search interface to capture, and an answer from its API is a measurement of something no buyer reads.
Nothing before the day you start
There is no archive of what ChatGPT said about your category last March. Nobody has one — not us, not the tools that imply otherwise. Tracking begins when a prompt is added, which is the least convenient honest answer and the only one available.
Find out what AI says about your brand
Add a domain and the first pass runs while you read the next one.