Researchers have addressed the English-centric bias in reasoning language models by focusing on L2 reasoning, which enables models to reason consistently within the language of a user's prompt. To investigate how data composition and scheduling during Supervised Fine-Tuning (SFT) impact reasoning generalization, the authors developed Tiny Aya L2-Thinker, a model scaled at 3.35B parameters. The study demonstrates that reasoning is a language-agnostic capability that can be transferred across typologically diverse languages through strategic data mixing, rather than requiring direct reasoning supervision for every target language. By utilizing a robust English reasoning backbone combined with broad multilingual non-reasoning data, the model achieved an L2 reasoning rate exceeding 93% across 60 different languages. This performance was validated across six distinct benchmarks covering math, commonsense reasoning, instruction following, open-ended generation, and cultural reasoning. The findings suggest that optimizing the mix of available multilingual data and English-centric reasoning data is the primary driver for generalizing reasoning capabilities to held-out languages. The researchers have released the model weights and the multilingual reasoning dataset to facilitate further advancements in accessible, in-language reasoning capabilities.

Read original