EPSBench evaluates LLM agents on quarterly earnings forecasting across 149 firm-quarters from 86 US companies, requiring agents to estimate company-specific income-statement drivers from a fixed corpus of SEC filings, earnings-call transcripts, and news articles. A shared accounting calculator converts these driver estimates into GAAP diluted EPS, enabling direct comparison of forecast assumptions against date-matched analyst consensus. Six models were tested under a common harness: three showed no statistically detectable paired difference in point forecast error relative to consensus, while three exhibited higher error. Prediction intervals proved poorly calibrated; nominal 80% intervals captured reported EPS in only 37–70% of runs that produced an interval, with most misses occurring above the upper bound due to systematically low forecasts. Driver-level attribution traced the majority of the error gap between models to operating-income estimates. The benchmark isolates forecasting skill by standardizing evidence access and the translation of drivers to EPS, revealing that current agents struggle with both point accuracy and uncertainty quantification, particularly in modeling core operating profitability.

Read original