Researchers have introduced Benchmark Radar, a comprehensive living database and search engine designed to streamline the discovery and evaluation of AI benchmarks. Addressing the challenges of locating relevant datasets, code, and specific evaluation settings for large language models (LLMs), the system provides a centralized catalog covering diverse domains including reasoning, coding, safety, and agentic tool-use. The architecture utilizes a daily discovery mechanism that aggregates data from 37 distinct sources, comprising 13 direct connectors and 24 first-party research and engineering feeds. Currently, the database contains 1,283 source records aggregated from four existing catalogs, alongside 12,916 numeric observations across 790 records. Beyond simple retrieval, the platform enables users to track score histories, analyze benchmark saturation, and observe adoption trends. To facilitate reproducible research, the authors released a web dashboard featuring a leaderboard, a Pareto frontier view comparing scores against measured usage, and a command-line interface (CLI) for offline querying. By maintaining original source identities and citations, Benchmark Radar allows developers to audit evaluation evidence and inspect the specific settings behind reported model performance, mitigating the difficulties inherent in comparing disparate benchmark results.

Read original