arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00376cs.LG
收藏

MATRIX: 一种用于材料科学的多模态基准和后训练框架

MATRIX: A Multimodal Benchmark and Post-Training Framework for Materials Science

Delia McGrath, Curtis Chong, Rohil Kulkarni, Gerbrand Ceder, Adeesh Kolluru

AI总结:

MATRIX通过多模态后训练提升材料科学推理,展示视觉引导对实验解释和文本任务的显著改进。

摘要:

材料科学中的科学推理需要将多模态实验证据与基础物理理论相结合。现有基准难以评估在后训练过程中整合视觉实验数据是否能超越纯文本监督的机制基础解释推理。我们引入MATRIX,一种用于材料科学推理的多模态基准,该基准评估基础理论、研究级推理以及多种表征模式下的实验制品解释。使用MATRIX作为受控诊断,我们通过比较仅使用结构化材料科学文本进行后训练与整合配对实验图像进行后训练,来隔离视觉引导的影响。尽管使用相对少量的多模态数据,视觉监督将实验解释能力提高了10-25%,并在纯文本科学推理任务上带来了5-16%的提升。我们的结果表明,这些改进依赖于后训练过程中正确的图像-文本对齐,突显了跨模态表示迁移。我们还观察到在ScienceQA和PubMedQA上的一致性改进,证明了结构化多模态后训练的益处不仅限于材料科学。MATRIX数据集可在https://huggingface.co/datasets/radical-ai/MATRIX获取,模型可在https://huggingface.co/radical-ai/MATRIX-PT获取。

英文摘要:

Scientific reasoning in materials science requires integrating multimodal experimental evidence with underlying physical theory. Existing benchmarks make it difficult to assess whether incorporating visual experimental data during post-training improves mechanism-grounded explanation reasoning beyond text-only supervision. We introduce MATRIX, a multimodal benchmark for materials science reasoning that evaluates foundational theory, research-level reasoning, and the interpretation of real experimental artifacts across multiple characterization modalities. Using MATRIX as a controlled diagnostic, we isolate the effect of visual grounding by comparing post-training on structured materials science text alone with post-training that incorporates paired experimental images. Despite using relatively small amounts of multimodal data, visual supervision improves experimental interpretation by 10-25% and yields 5-16% gains on text-only scientific reasoning tasks. Our results demonstrate that these improvements rely on correct image-text alignment during post-training, highlighting cross-modal representational transfer. We also observe consistent improvements on ScienceQA and PubMedQA, demonstrating that the benefits of structured multimodal post-training extend beyond materials science. The MATRIX dataset is available at https://huggingface.co/datasets/radical-ai/MATRIX and the model at https://huggingface.co/radical-ai/MATRIX-PT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑