Researchers from T-Tech presented LIMe at EMNLP 2026, a method reducing LLM training compute by approximately 15% without quality loss. The approach addresses representation collapse in standard Transformers, where reliance on the previous layer's hidden state forces all contextual information through a bottleneck. LIMe mitigates this pressure by altering information flow across layers.

Read original