AI 中文总结
本文提出OncoTriad-QA多模态癌症问答基准及OncoVLM模型，实验显示OncoVLM经微调后在泛癌问答任务上优于现有模型，该基准可用于相关模型的训练与评估。
AI 中文摘要
癌症的诊断与特征表征需要整合放射学、病理学、基因组学及临床元数据的互补证据。然而，多数医学大语言模型（LLM）与视觉-语言模型（VLM）基准聚焦于孤立模态或狭窄的图像-文本任务，导致跨多证据流的患者级肿瘤学评估大多未被测试。本文提出OncoTriad-QA，这是一个用于泛癌问答的患者级放射学-病理学-基因组学基准。OncoTriad-QA包含来自32个癌症队列的9281例TCGA患者病例，共8.61万个语义问题，对齐了CT/MRI放射学、全切片组织病理学、体细胞突变、拷贝数变异、DNA甲基化、 bulk RNA-seq及临床元数据。病例特定注释通过基于源的LLM辅助流程构建，该流程以 curated 标签、诊断报告、分子谱及模态衍生证据为主要真实来源，辅以自动一致性检查与临床医生审核。本文还提出OncoVLM，这是一个参考多模态模型，通过学习到的投影器将模态原生的放射学、病理学、DNA甲基化及RNA-seq证据映射至LLM接口。实验显示，现有通用及医学LLM在综合泛癌问答上仍存在局限，尤其在问题需整合影像发现、肿瘤形态学与分子证据时表现更差。在OncoTriad-QA上微调后，OncoVLM在多项选择准确率与BERTScore-F1指标上平均超过MedGemma-4B 10.7分，且在仅放射学、仅病理学及全可用设置下的多项选择与开放式问题中均保持一致提升。这些结果证明了该基准对训练和评估整合型癌症问答模型的价值。
英文摘要
Cancer diagnosis and characterization require integrating complementary evidence from radiology, pathology, genomics, and clinical metadata. However, most medical large language model (LLM) and vision-language model (VLM) benchmarks focus on isolated modalities or narrow image-text tasks, leaving patient-level oncology assessment across multiple evidence streams largely untested. We introduce OncoTriad-QA, a patient-level radiology-pathology-genomics benchmark for pan-cancer question answering. OncoTriad-QA contains 86.1k semantic questions across 9,281 TCGA patient cases from 32 cancer cohorts, aligning CT/MRI radiology, whole-slide histopathology, somatic mutations, copy-number alterations, DNA methylation, bulk RNA-seq, and clinical metadata. Case-specific annotations are constructed through a source-grounded LLM-assisted pipeline using curated labels, diagnostic reports, molecular profiles, and modality-derived evidence as primary sources of truth, with automated consistency checks and clinician review. We also introduce OncoVLM, a reference multimodal model that maps modality-native radiology, pathology, DNA methylation, and RNA-seq evidence into an LLM interface through learned projectors. Experiments show that existing general-purpose and medical LLMs remain limited on comprehensive pan-cancer QA, especially when questions require integrating imaging findings, tumor morphology, and molecular evidence. After fine-tuning on OncoTriad-QA, OncoVLM exceeds MedGemma-4B by an average of 10.7 points when using MCQ accuracy and BERTScore-F1, with consistent gains across multiple-choice and open-ended questions under radiology-only, pathology-only, and all-available settings. These results demonstrate the benchmark's value for training and evaluating models for integrated cancer question answering.
