This research investigates how instruction tuning influences the relationship between model confidence and the lexical diversity of generated rationales. By evaluating three matched pairs of base and instruction-tuned models across various question-answering benchmarks, the authors examine whether verbalized overconfidence correlates with reduced rationale consistency. The findings reveal that instruction tuning consistently elevates verbalized answer confidence even when predictive accuracy remains relatively stable, which simultaneously leads to a degradation in likelihood-based calibration. Regarding lexical characteristics, the study identifies a non-uniform impact on rationale diversity: cross-rationale diversity shows a consistent decrease following instruction tuning, whereas surface-level lexical diversity exhibits variable changes in both magnitude and direction depending on the specific model and benchmark used. Crucially, the researchers demonstrate that these shifts in confidence and rationale diversity persist even when controlling for rationale length and answer selection. This confirms that instruction tuning exerts distinct, independent effects on a model's self-reported confidence and the linguistic variety of its supporting explanations, rather than these phenomena being mere artifacts of response length or accuracy levels.
Read original
huggingface/daily-papers