A September 16 arXiv preprint by Lucas G. Uberti-Bona Marin and colleagues examined 1,536 responses across ChatGPT, Gemini, their APIs and Google AI Overviews. Its audit used 117 physical-product questions, repeated three times, with collection from the Netherlands. It reports predominantly English queries and a snapshot of changing systems, limiting how widely its findings should be generalized.
For matched ChatGPT and Gemini interface observations, mean Jaccard overlap of displayed domains was 5.4%; 76.7% of comparisons shared none. Interface-to-API overlap was 12.0% for ChatGPT and 14.8% for Gemini. These percentages describe the intersection of domain sets divided by their union, not a share of shoppers or brands.
The researchers also distinguish displayed citations from retrieved sources and grounding information. Their comparison does not isolate why interfaces and APIs differ: provider-controlled configurations may vary. The study is a preprint, and it does not test whether any AEO tactic causes visibility gains.
What an AEO dashboard should disclose
Our interpretation is that every visibility report needs a clear statement of its observation surface. A team purchasing API-based tracking should be able to tell whether it is measuring an API response, a consumer interface or an explicitly defined combination. Those conditions can each be useful, provided the label matches the evidence.
The distinction matters when turning a measurement into a claim. A statement that a brand appeared in a tested API response is narrower than a statement that shoppers saw it in an assistant. The second requires observations of the relevant customer experience. A report should make that boundary readable before a stakeholder uses the numbers to allocate a budget.
At minimum, we recommend including the collection dates, prompt set, number of attempts, location assumptions, language, available model identifier and treatment of errors. The dashboard should explain how it counts a brand mention, a source link and a product recommendation. A report that merges those events should show the component counts so readers can inspect what moved.
Questions to ask a tracking vendor
Ask for a worked example that traces one data point from its input to its saved output. It should be possible to see the exact prompt, the returned answer, the recorded source and the rule that produced the score. A screenshot is useful for an interface observation; the response record is useful for an API observation.
CiteCue is a relevant example of making the answer available alongside the measurement. Its Prompts Monitoring documentation describes retaining the complete AI answer for each prompt, showing engine-level results, and allowing teams to add prompts individually or import them in batches. Those documented features give a team material to inspect when reviewing a visibility score. Buyers should still establish which observation surface each result represents before comparing it with consumer-interface checks; the study did not evaluate CiteCue.
Then ask what happens when the response contains no sources, the request fails or a model is unavailable. A missing observation should be distinguishable from an observed absence. Otherwise, a collection problem could be interpreted as a brand losing visibility.
Finally, ask how the vendor validates the relevance of its measurement to the business. A retailer serving one country may need a different test set from a software company selling globally. A technically repeatable tracker still needs prompts that represent plausible customer tasks. Those choices deserve as much scrutiny as the chart design.
A practical review alongside automated tracking
We recommend keeping automated tracking and a small manual review as separately labeled evidence. Choose a manageable sample of important purchase questions and repeat them under documented conditions on the consumer surfaces your customers use. Preserve the complete answers and visible links, including cases where the result is unexpected.
Compare the observations without requiring them to agree. If the API mentions a product while the interface does not, investigate the conditions and keep both records. That disagreement can identify a measurement question rather than become a reason to discard inconvenient evidence.
Changes to a website should also have their own dated record. When a team rewrites a product page, launches a campaign and switches a tracking model during the same period, a subsequent score change cannot cleanly identify which intervention mattered. The review should state that limitation instead of presenting the movement as proof of an optimization win.
What would make the evidence stronger
The next useful research would repeat comparable audits across additional markets, languages and product categories, with enough detail to assess changes over time. For practitioners, the immediate improvement is more modest: make the measurement contract explicit.
A visibility chart is valuable when its reader understands the population of questions, the observation conditions and the event being counted. This September paper is a timely reason to ask for those details. It supports closer scrutiny of AEO measurement, while leaving the effectiveness of individual content strategies as a separate question requiring separate evidence.
