TNF: 用于多模态医学数据分类的三分支神经融合方法
TNF: Tri-branch Neural Fusion for Multimodal Medical Data Classification
AI总结：
针对多模态医学数据分类中标签不一致、特征互斥导致准确率低的问题，提出三分支神经融合TNF方法，通过图像、表格、混合三分支加集成决策，在多数据集多架构上优于传统方法。
摘要：
本文提出了一种三分支神经融合（Tri-branch Neural Fusion，TNF）方法，旨在对多模态医学图像与表格数据进行分类，同时还提出了两种解决方案，以应对多模态分类中的标签不一致难题。传统多模态医学数据分类方法通常采用单标签策略，一般会融合两种不同输入模态的特征；当特征互斥或不同模态的标签存在差异时，这种方法就会出现问题，导致分类准确率下降。为解决该问题，我们提出的TNF方法采用三分支框架，分别处理三类独立输出：一类对应图像模态，一类对应表格模态，第三类为融合图像与表格数据的混合输出；最终决策通过集成方法整合三个分支的似然值得出。我们通过大量实验验证了TNF的有效性，结果表明，在多个数据集上，该方法在各类卷积神经网络与基于transformer的架构中，性能均优于传统融合与集成方法。
英文摘要：
This paper presents a Tri-branch Neural Fusion (TNF) approach designed for classifying multimodal medical images and tabular data. It also introduces two solutions to address the challenge of label inconsistency in multimodal classification. Traditional methods in multi-modality medical data classification often rely on single-label approaches, typically merging features from two distinct input modalities. This becomes problematic when features are mutually exclusive or labels differ across modalities, leading to reduced accuracy. To overcome this, our TNF approach implements a tri-branch framework that manages three separate outputs: one for image modality, another for tabular modality, and a third hybrid output that fuses both image and tabular data. The final decision is made through an ensemble method that integrates likelihoods from all three branches. We validate the effectiveness of TNF through extensive experiments, which illustrate its superiority over traditional fusion and ensemble methods in various convolutional neural networks and transformer-based architectures across multiple datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
