How to evaluate an AI visibility study
An AI visibility report should let you trace its main conclusion to identified observations and understand what the evidence does not establish. It should separate the measured finding, the explanation proposed for it and the business response recommended.
Verify the research basis
When evaluating an AI visibility study, a good study should state both the population it intended to observe and the observations it could actually use. Missing answers, unresolved identity matches and incompatible collection conditions need explicit treatment. A score calculated from a restricted eligible set must keep that denominator visible. Ask whether named-company prompts and open discovery questions are analyzed separately before interpreting the result as market visibility.
Start with the question. Which organization, market, buyer needs, language, location, systems and period does the study cover? Are the observations from Google Search, an AI Overview, a consumer assistant or an API? Was web access used, and what is unknown about the configuration?
Check how questions were chosen. A proprietary measurement panel can be useful without being a sample of human search demand. A report must not silently turn repeated model tests into statements about customer popularity.
Measurement and uncertainty explains how the population, exclusions and missing evidence determine what the reported number can establish.
Inspect one important finding
When evaluating an AI visibility study, trace more than the headline number. If the conclusion is that a rival is recommended more often, inspect whether the classification distinguishes a passing mention, a source reference, a shortlist entry and an endorsement. Then check the comparator's actual commercial role. The website of a publisher may be cited frequently without making that publisher your business rival.
Choose a conclusion that matters to your decision. Ask for the retained answer or a permitted excerpt, the entity attribution, the calculation and its denominator. If the answer cites a source, follow that reference and assess what it actually supports.
Ask for a counterexample or the strongest limitation. An unresolved name match, missing answer or different observation mode can change the interpretation. Evidence access may be restricted, but the restriction and its consequence for independent checking should be explicit.
When verification requires restricted material, the access guidance identifies the object, recipients and permitted use that need agreement before inspection.
Check entity identity if the finding depends on who was counted, and source provenance if it depends on what a citation substantiates. These are distinct ways a plausible aggregate can fail inspection.
Evaluate action and outcome claims
When evaluating an AI visibility study, the practical test is whether the proposed response follows from the finding. A source mismatch may justify checking a cited article; a verified service error may justify correcting owned information; a well-supported strength may merit protection. Neither an impressive score nor an alarming screenshot establishes the budget, implementation priority or expected commercial return.
A proposed action needs more than an interesting difference. Is the issue material to the organization? Can the team change it? What supports the proposed response rather than another one? What evidence would change the recommendation?
Claims of causal improvement, revenue or return on investment need their own design and evidence. A rank change after a content change is not enough to establish either.
The evidence and claims standard separates a measured finding from an explanatory hypothesis and from advice, so the action claim is assessed on its own basis.
Use a buyer review worksheet
When evaluating an AI visibility study, AAI Kernel can help assess a disputed conclusion or scope a new investigation around the decision it affects. The useful review does not end with a verdict on presentation quality: it identifies which findings can support a brief, which need clarification and which cannot yet be relied on. Your leadership or agency decides how that assessment changes the plan.
For each major conclusion, record the claim, observation scope, source, denominator, attribution check, counter-signal, interpretation and proposed owner. Mark what is verified, uncertain or unavailable. This is a review framework, not a sample client result.
Bring the decision and any evidence you are permitted to share. We can assess whether a new commission, a narrower validation or a better use of existing research would help.
A useful review meeting can follow one finding all the way through: capture, attribution, source support, interpretation and proposed action. Ask the reviewer to explain what would change the conclusion. If the answer relies on information that cannot be shared, agree a permitted verification route and record the remaining limit. The executive decision is whether that degree of support is sufficient for the planned use, not whether every private artifact must become public.
| Review item | Ask to inspect | Decision consequence |
|---|---|---|
| Scope | Question set, surface, dates, eligible population | Know where the claim applies |
| Attribution | Verified company, role and exact source | Avoid counting the wrong object |
| Calculation | Inputs, denominator, exclusions and missingness | Understand what the score measures |
| Claim support | Capture, source passage and strongest counterexample | Retain supported wording and qualifications |
| Action | Business relevance, owner and alternatives | Approve only a justified response |
An adviser preparing the response can use the agency research pathway to agree a reviewable handoff. Bring a specific unresolved claim through Contact, identifying the decision it prevents.
Common questions
- What should an auditable report include?
- An auditable report should identify the question, scope, captures, attribution rules, derivations, denominator, limitations and access conditions behind its main findings.
- How do we verify an agency’s GEO case-study claims?
- Check the exact outcome claim, baseline, population, intervention, comparison conditions and alternative explanations. A before-and-after change alone does not prove causal uplift.
Discuss a study you need to evaluate
Describe the conclusion you need to rely on and the evidence you are permitted to share. We can assess a focused validation or a new research scope.
07 Where this leads
The findings that survive review still need a decision. The prioritization guide considers which justify protection, clarification, investigation or a test. If a remaining uncertainty prevents that choice, the commissioning guide helps define the evidence needed to resolve it, using existing research where it can already answer the question.