Researchers have introduced CALVER (Causal Axiom-Level VERification), a training-free symbolic verification method designed to improve causal reasoning in Large Language Models (LLMs). Traditional self-consistency methods, which rely on majority voting among sampled reasoning traces, often fail in causal contexts because samples may repeat identical confounding errors or split votes among multiple valid answers. To address this, CALVER scores structured reasoning traces against Pearl's causal criteria, specifically assessing d-separation, backdoor adjustment, and interventions. This approach allows the system to select the highest-scoring candidate without requiring a reference answer. On the CLEAR benchmark's "find-one-valid" queries, where multiple graph-valid answers exist, CALVER achieved a 42.1% success rate, significantly outperforming plurality voting, reward models, LLM judges, and model confidence, which all hovered near 30%. Notably, scaling an LLM judge to 72B parameters failed to close this performance gap. The method's robustness was demonstrated across ten Bayesian networks and settings requiring text-to-graph construction. Furthermore, CALVER improved thresholded average-treatment-effect decisions and generalized to logic tasks via truth-table checkers. The system is highly efficient, capable of scoring candidates in milliseconds on a CPU, provided the causal structure is available either directly or extracted from the text.

Read original