An analysis of 1,629 published benchmark results found that AI labs report scores averaging 6.4% higher than independent testers for the same models and tests. The study compared 611 results across 180 models and 103 benchmarks, with the dataset released under CC‑BY‑SA 4.0 and the code under MIT.
Read original
reddit/r/LocalLLaMA