Research by Liu et al. (2024) reveals that current LLM-based agent benchmarks are fundamentally unreliable due to the non-deterministic nature of black-box evaluation endpoints. By replaying byte-identical agent logs from frameworks like LangChain, CrewAI, and Agentic-Security-Lab through OpenAI and Anthropic APIs, the study demonstrates that even at zero temperature, LLM judges exhibit significant self-inconsistency. Specifically, the agreement rate for deterministic outputs drops as low as 89%, meaning up to one in ten verdicts is inconsistent upon rerunning. Quantitative analysis shows mean self-agreement rates ranging from 88% to 91% across different frameworks, with instability occurring even when using "static" endpoints. This volatility is exacerbated by undocumented vendor-side model updates and the inherent randomness of LLM responses, which prevents fuzzy matching or strict formatting from ensuring stability. Consequently, when agent performance differences are marginal, the resulting leaderboard rankings become statistically insignificant and non-reproducible. The findings suggest that without strict endpoint versioning and reproducible rater outputs, current leaderboards for tool-use, collaboration, and security tasks lack scientific substance.