Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Researchers have identified the "Knowing--Using Gap," a phenomenon where fine-tuned LLMs memorize new facts but fail to apply them to downstream reasoning tasks. The study explores this failure by analyzing the temporal …
→ View original source