Security researcher HD Moore of Dragos conducted a large-scale experiment seeding nearly two million unique "canary" credentials—including unused AWS access keys, SSH passwords, private git repository URLs, and DNS names—into individual prompts sent through the AI inference supply chain to track data leakage. The study revealed systemic failures across AI routers (e.g., OpenRouter, Vercel), hosting providers, and model resellers. At least three hosting providers were compromised by attackers who intercepted prompts at the ingress point, exfiltrating "a couple hundred" AWS keys within hours; stolen keys were used tens of thousands of times to probe S3 buckets, Secrets Manager, and IAM, with evidence suggesting a single adversary compromised two providers. Multiple resellers enabled request logging to harvest prompts, identifiable because credentials leaked only when routed through specific sellers. A Chinese AI lab scraped secret URLs from over half of prompts sent to its models between July and October, including roughly two-thirds marked Zero Data Retention (ZDR) or "no data collection," with scrapers appearing days later; US resellers of this lab's models inadvertently forwarded prompts back to the lab, violating data locality expectations. One US provider crawled a secret URL over two months post-inference. Moore concludes that on-premises hosting is the only reliable mitigation, while using first-tier providers directly minimizes—but does not eliminate—exposure in the opaque, multi-hop inference marketplace.

Read original