QuPAINT:一种面向量子材料发现的物理感知指令微调方法
QuPAINT: Physics-Aware Instruction Tuning Approach to Quantum Material Discovery
AI总结:
QuPAINT通过物理感知指令微调方法,提升量子材料发现的多模态表征能力,建立标准化评估基准。
摘要:
从光学显微镜图像表征二维量子材料具有挑战性,因为存在层依赖的对比度细微差别、标注数据有限以及实验室和成像设置之间的显著变化。现有的视觉模型在这一领域难以应对,因为它们缺乏物理先验知识,无法推广到新材料或硬件条件。本文提出了一种新的物理感知多模态框架,从数据和模型两个方面解决这些限制。我们首先介绍了Synthia,一种基于物理的合成数据生成器,模拟量子材料碎片在薄膜干涉下的真实光学响应。Synthia生成多样化且高质量的样本,有助于减少对专家手动标注的依赖。我们介绍了QMat-Instruct,首个大规模的量子材料指令数据集,包含多模态、物理指导的问题-答案对,旨在教导多模态大语言模型(MLLMs)理解碎片的外观和厚度。然后,我们提出了物理感知指令微调(QuPAINT),一种多模态架构,其中包含一个物理指导的注意力模块,将视觉嵌入与光学先验融合,使碎片表示更加稳健和判别性。最后,我们建立了QF-Bench,一个涵盖多种材料、基底和成像设置的综合基准,提供标准化协议以实现公平和可重复的评估。
英文摘要:
Characterizing two-dimensional quantum materials from optical microscopy images is challenging due to the subtle layer-dependent contrast, limited labeled data, and significant variation across laboratories and imaging setups. Existing vision models struggle in this domain since they lack physical priors and cannot generalize to new materials or hardware conditions. This work presents a new physics-aware multimodal framework that addresses these limitations from both the data and model perspectives. We first present Synthia, a physics-based synthetic data generator that simulates realistic optical responses of quantum material flakes under thin-film interference. Synthia produces diverse and high-quality samples, helping reduce the dependence on expert manual annotation. We introduce QMat-Instruct, the first large-scale instruction dataset for quantum materials, comprising multimodal, physics-informed question-answer pairs designed to teach Multimodal Large Language Models (MLLMs) to understand the appearance and thickness of flakes. Then, we propose Physics-Aware Instruction Tuning (QuPAINT), a multimodal architecture that incorporates a Physics-Informed Attention module to fuse visual embeddings with optical priors, enabling more robust and discriminative flake representations. Finally, we establish QF-Bench, a comprehensive benchmark spanning multiple materials, substrates, and imaging settings, offering standardized protocols for fair and reproducible evaluation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。