Researchers introduce NPCBench, a clinical apprenticeship benchmark for nasopharyngeal carcinoma (NPC) designed to evaluate multimodal care-pathway reasoning beyond static question-answering. The benchmark structures assessment across progressive stages mirroring specialist training: guideline knowledge, case planning, longitudinal review, CarePath integration, and open-ended consultation. It comprises 4,777 component and consultation items alongside a 100-patient CarePath cohort containing 417 scored diagnostic and care tasks. A key innovation is episode-level evaluation, which tests whether diagnostic and care answers are jointly correct within a patient's care trajectory, requiring models to ground decisions in visual evidence, interpret longitudinal state changes, and satisfy clinical consultation requirements. Evaluation of 30 models reveals a significant composition gap: strong performance on isolated tasks does not translate to jointly correct diagnostic and care reasoning for the same patient. Among 21 multimodal models, strict CarePath success rates reach at most 23%. Paired controls and physician calibration further assess evidence grounding, longitudinal revision capabilities, and consultation judgment quality. NPCBench establishes a testbed for evidence-based, temporally coherent reasoning in oncology workflows.
Read original
huggingface/daily-papers