Researchers introduce Cultivar, a contrastive and locale-oriented translation benchmark derived from a localized subset of FLORES, designed to address contamination and localization challenges in multilingual translation evaluation. Unlike conventional benchmarks that treat language pairs as the unit of evaluation and source content from English, Cultivar enables source-contrastive evaluation by comparing localized and unlocalized translations, revealing performance discrepancies tied to cultural and regional nuances. The benchmark aims to improve robustness and cultural sensitivity in machine translation systems.

Read original