Trakkr traced 1,285 statistics cited by ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews back through citation chains up to five pages deep, finding only 40% reached an original study matching the claimed figure, while 8% reached a study where the number or context had mutated. The remaining 51% dead-ended: 40% stopped at unverifiable sources (paywalled, missing, or silent on the figure) and 11% led nowhere. AI models predominantly cited intermediary pages—company blogs (over half), news stories, and statistics roundups—rather than primary research (only 21% of citations). Starting from a research firm or government page yielded a matching study 61% of the time versus 37% from company blogs and 35% from roundups; even research firms failed 38% of the time, often due to gated reports. Among statistics reaching a study, 17% had altered en route: the digit changed in 26% of mutations, while context shifts dominated (measured variable 20%, year 17%, source 16%, population 14%). In a test of 71 questions where all five assistants provided percentages, 28 produced four or five different figures with a median spread of 16 percentage points; consensus did not indicate reliability—only 2 of 5 unanimous answers traced to a study, and 11 of 29 multi-assistant agreements did. ChatGPT cited fewest statistics per answer (1.3 vs. Claude’s 6.8) and performed best on sourcing rates. The authors recommend treating AI statistics as leads: follow citation chains to primary studies, verify sample, date, and population, and publish methodology alongside figures on open pages to prevent zombie stats—numbers endlessly recycled without empirical origin.
Read original
hackernews