AI search visibility has become the new vanity metric, and most teams are measuring the wrong number. The AI visibility tools multiplying across the market count how often a model mentions you, or cites you, when someone types a prompt. That number feels like progress because it looks like the rank tracking we have done for 20 years. It is not the same thing, and the gap between what these tools count and what actually moves your business keeps widening. This is a guide to the metrics that matter in AI search, the ones that only look like they matter, and how to tell them apart, built from six No Hacks podcast conversations with people who do this for a living and the data that landed this month.
Why AI Visibility Is The Wrong Number
The dominant way to measure AI search right now is prompt tracking, and as most teams use it, it is the wrong instrument. The pitch is familiar: a tool types a set of prompts into ChatGPT, Perplexity, and Google’s AI Overviews, and reports how often your brand shows up. It looks like rank tracking, so it sells. I get more pitches every week from companies that built one and want to come on the podcast. There are far more of them than the market needs, and the attention is rushing toward the most visible, most obvious thing to count, whether or not it is the thing that matters.
Jono Alderson, a technical SEO consultant, put the objection this way when he came on the show. “We need to instead try and influence how the machine perceives us. And that’s not prompt tracking, which is what everyone is doing at the moment,” he said. “There is a place for that, but it’s far smaller than I think.” His diagnosis of why these dashboards feel off is sharper still: “It’s copy-paste the current modality of rank tracking into a new thing. It doesn’t really fit, but it’s better than nothing.”
Prompt tracking also rests on a guess about user behavior that mostly is not grounded. You invent a list of prompts you hope your customers type, then measure yourself against them. For most brands, that list has little to do with what real people are actually asking, and an AI prompt is not a keyword. Ground those prompts in real search data instead, and you hit a second problem: AI is corrupting that data faster than you can read it.
I have a first-hand reason. Last year, I helped investigate a strange leak: real users’ ChatGPT prompts were turning up inside Google Search Console, the report website owners use to track search traffic. I worked on it with the analytics consultant Jason Packer, who published the findings on Quantable, and Ars Technica, and a long list of outlets covered it. We traced it to a bugged prompt box that made ChatGPT search almost every time, with a ChatGPT URL leading the queries that Google then tokenized, so websites ranking for those terms saw strangers’ private prompts in their dashboards. My own concern at the time, which Ars Technica quoted, was that the leak was contributing to “crocodile mouth” in Search Console, the pattern where impressions spike while clicks dip.
The leak was the visible version of a problem that is now everywhere and impossible to see. AI systems hit Google constantly to ground their answers, fanning a single prompt out into several parallel queries and reading results no human ever lays eyes on. Every one of those searches lands as an impression on the pages that rank for it. So when your impressions climb while your clicks do not, that is not rising human demand.
