Keenable AI Open-Sources NEEDLE: A Live Search Benchmark That Rebuilds Its Query Set Every Hour

Article automatically generated from technical news.

Most search benchmarks are a fixed question set with a public answer key. That's not a benchmark for agents — because an agent with a fetch tool can just download the key mid-evaluation. Keenable open-sourced NEEDLE — a live search benchmark that rebuilds its query set instead of freezing one. News queries regenerate every hour from RSS feeds and Google Trends. Finance, scholar, and legal gold regenerates daily from public registries, SEC XBRL, arXiv, Europe PMC, CourtListener