The article argues that requesting continuous confidence scores (e.g., 0–100) from Large Language Models (LLMs) is technically invalid and provides a false sense of security. The author asserts that LLMs lack a reliable mechanism to quantify their own internal states, noting that even when models demonstrate "emergent introspective awareness," such capabilities are highly context-dependent and unreliable for assessing factual correctness. A significant issue is that models do not treat the 0–100 scale as a true continuous spectrum; empirical data shows they tend to cluster on round-number anchors, with one model reporting exactly 100 in 68% of instances. Furthermore, collapsing complex token-level probabilities into a single scalar value loses critical information regarding whether a model is uncertain about a specific factual claim or merely its linguistic phrasing. While research suggests that calibrated verbal confidence or semantic entropy (measuring meaning clusters via repeated sampling) can provide better insights, these require intensive post-hoc calibration, fine-tuning, or external measurement. The author concludes that instead of seeking "psychological safety tricks" via self-reported scores, developers should focus on improving retrieval quality and addressing the underlying grounding and synthesis problems that cause inaccuracies.

Read original