A new preprint introduces Spanda, a fast lexical entropy method for uncertainty quantification that matches DeBERTa-based Semantic Entropy performance without requiring GPU-intensive clustering. Benchmarks across 1.5B to 120B parameter models show Spanda effectively replicates Semantic Entropy's hallucination detection capabilities, though the approach breaks down at frontier model scales, suggesting limitations in simpler entropy-based methods as model complexity grows. The work provides an open-source benchmark and codebase for comparing self-consistency and uncertainty quantification techniques.
Read original
reddit/r/machinelearningnews