AMF-MedIT：一种用于医学图像-表格数据的高效对齐-调制-融合框架
AMF-MedIT: An Efficient Align-Modulation-Fusion Framework for Medical Image-Tabular Data
AI总结：
针对医学图像与表格数据融合中的跨模态差异和噪声问题，提出AMF-MedIT框架，通过自适应调制与融合模块及FT-Mamba编码器，在数据稀缺条件下实现了高准确性、鲁棒性和数据效率。
摘要：
结合图像与表格数据的多模态医学分析日益受到关注。然而，由于特征维度与模态贡献的跨模态差异，以及高维表格输入带来的噪声，有效融合仍具挑战。为解决这些问题，我们提出了AMF-MedIT，一种用于医学图像与表格数据整合的高效对齐-调制-融合框架，尤其适用于数据稀缺条件。基于自监督学习策略，我们引入了自适应调制与融合（AMF）模块，这是一种新颖且简化的融合范式，可协调维度差异并动态平衡模态贡献。它整合先验知识以指导融合中模态贡献的分配，并采用特征掩码以及幅度与泄漏损失来调整单模态特征的维度和幅度。此外，我们开发了FT-Mamba，这是一种强大的表格编码器，利用选择性机制高效处理含噪医学表格数据。包括临床噪声模拟在内的大量实验表明，AMF-MedIT在多模态分类任务中实现了卓越的准确性、鲁棒性和数据效率。可解释性分析进一步揭示了FT-Mamba如何塑造多模态预训练并增强图像编码器的注意力，突显了该框架对于可靠且高效的临床人工智能应用的实用价值。
英文摘要：
Multimodal medical analysis combining image and tabular data has gained increasing attention. However, effective fusion remains challenging due to cross-modal discrepancies in feature dimensions and modality contributions, as well as the noise from high-dimensional tabular inputs. To address these problems, we present AMF-MedIT, an efficient Align-Modulation-Fusion framework for medical image and tabular data integration, particularly under data-scarce conditions. Built upon a self-supervised learning strategy, we introduce the Adaptive Modulation and Fusion (AMF) module, a novel, streamlined fusion paradigm that harmonizes dimension discrepancies and dynamically balances modality contributions. It integrates prior knowledge to guide the allocation of modality contributions in the fusion and employs feature masks together with magnitude and leakage losses to adjust the dimensionality and magnitude of unimodal features. Additionally, we develop FT-Mamba, a powerful tabular encoder leveraging a selective mechanism to handle noisy medical tabular data efficiently. Extensive experiments, including simulations of clinical noise, demonstrate that AMF-MedIT achieves superior accuracy, robustness, and data efficiency across multimodal classification tasks. Interpretability analyses further reveal how FT-Mamba shapes multimodal pretraining and enhances the image encoder's attention, highlighting the practical value of our framework for reliable and efficient clinical artificial intelligence applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
