Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference
The paper examines layer dropout (stochastic depth) and its benefits for faster training, higher accuracy, and robustness to zero‑shot pruning in transformers. It notes that despite these advantages, layer dropout has la…
→ View original source