Researchers introduce DailySafe, a benchmark designed to evaluate dynamic safety awareness in multimodal large language models (MLLMs) during realistic, everyday multi-turn interactions. Unlike existing benchmarks that focus on isolated adversarial prompts, DailySafe comprises 195 naturalistic, multi-image dialogues spanning 10 everyday domains and 42 task categories, grounded in authentic user topics and documented safety knowledge. The benchmark captures scenarios where safety-critical evidence emerges progressively across conversational turns and visual observations, requiring models to track evolving task conditions, recognize implicit contextual risks, and provide safe assistance without abandoning legitimate user goals. Evaluation uses case-specific safety annotations to assess whether MLLMs can appropriately address these evolving risks. DailySafe is publicly available.
Read original
huggingface/daily-papers