AI Visibility Significance Test

Presence, citation and visibility are three different measurements, and most GEO reporting collapses them into one number and loses the diagnosis. Presence is the share of prompts where a brand is mentioned at all. Citation is the share where the answer actually links to its site. Visibility weights each mention by where it appears, so being named first counts for more than being named fifth.

The gap between them is the useful part. High presence with low citation usually means the engine knows of a brand but is sourcing the claim from somewhere else - a competitor's comparison page, a directory, a review site. That is a different problem from not being mentioned at all, and it has a different fix: the missing thing is a page that deserves to be the source, not more brand mentions.

The prompt set has to be sized before it is run. A small set swings on its own: going from four mentions in twenty prompts to six is a jump from 20 to 30 per cent, and it is almost certainly nothing. The test computes the number of prompts needed from the current presence rate and the change worth detecting, using the same two-proportion power calculation used to size an A/B test. Sometimes the answer is that detecting a five-point move needs more prompts than anyone is willing to run - which is a real answer, and better learned before a quarter of reporting is built on it.

A useful prompt set mixes four shapes. Category prompts, of the best-tools-for-X kind, are where shortlists get built and are the hardest to enter. Problem prompts are where a brand can earn a mention on usefulness rather than brand strength. Comparison prompts name competitors explicitly and reveal who the engine pairs a brand with. Branded prompts act as a control: failing there points at broken entity resolution rather than weak content. The set stays fixed between runs, because changing the prompts changes what is being measured.

Scoring a run needs no API key. Ask each engine the prompts, record where the brand appeared, how often the answer linked to it, how it was framed, and who else was named. The read-out reports presence, citation and rank-weighted visibility per engine, a share-of-voice table against every competitor the engines named, and, when a previous run is supplied, a two-proportion test that states plainly whether the change is detectable or whether a set that size would produce that swing on its own.

If a tracker already collects the answers for you, import its export instead of typing the counts. The test reads a CSV with one row per prompt per engine, from AAJ's citation-run template or a tool export with similar columns (engine, prompt, mentioned, rank, cited, framing, competitors), and fills the form in the browser. Where a mention has no rank, it is placed after the competitors named in that answer, and the read-out says how many were placed that way. The file never leaves the browser. The point is the same either way: whatever collects the answers, the question that matters is whether this month's change is bigger than the noise.

All free tools

Run this in your agent

The free AAJ agent skill GEO Citation Tracker does this job inside a coding agent, on your own data. Install it with npx skills add sarojkjha/aaj-marketing-skills --skill geo-citation-tracker.

Using Claude or ChatGPT instead? Add the AAJ engines as a custom connector, with no install: https://aaj-engines-mcp.sarojkjha10.workers.dev/mcp