Researchers introduce NarrativeBench, a benchmark evaluating large language models (LLMs) on six capabilities required for diverse multilingual information seeking: narrative inclusion, exclusion, and missing narrative identification as foundational tasks, plus presenting diverse narratives, preserving coverage under confirmatory queries, and maintaining information parity across languages as higher-level tasks. The benchmark comprises 1,333 news articles in 23 publication languages spanning 23 contested topics and 460 Reddit-grounded queries. Evaluating 14 LLMs across 24 query languages, the study finds current models lack the capabilities to reliably navigate diverse narratives, limiting user control over received perspectives and leaving substantial gaps in exposure to competing accounts even when diversity is explicitly requested. Results show information access varies across query framings and languages, creating risks of echo chambers, linguistic filter bubbles, and information disparities that undermine common ground for democratic discourse. NarrativeBench provides a foundation for developing systems that grant users across languages and regions agency to navigate diverse narratives.

Read original