iStructTab：面向图像与表格数据多模态学习的结构化特征排序
iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data
查看机构详情
- West Virginia University（西弗吉尼亚大学）
- The University of Utah（犹他大学）
- Scientific Computing and Imaging Institute（科学计算与成像研究所）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
iStructTab基于图增强描述子排序算法（GEDS），将其融入感知顺序的高效Transformer框架，在多模态基准上有效减少特征分散，提升了图像与表格数据多模态学习的预测性能及鲁棒性。
中文摘要 AI 辅助
图像与表格数据的多模态学习常因表征效果不佳而出现冗余、分散和泛化问题。为应对这一挑战，本文提出基于列排列问题（CPP）原理的图增强描述子排序算法（GEDS），该算法通过基于相似度图的计算优化特征的统计描述子，系统确定有效的特征排序。我们将GEDS集成到感知顺序的高效Transformer框架中，利用感知顺序的记忆令牌，通过专用损失函数明确遵循推导的特征排序。在多模态基准上的实验结果表明，iStructTab可有效最小化特征分散，提升预测性能与鲁棒性，凸显结构化特征排序在多模态学习中的重要性。
英文摘要
Multimodal learning of images and tabular data is often impaired by ineffective representations, resulting in redundancy, dispersion, and generalization problems. To tackle this challenge, we introduce Graph-Enhanced Descriptor Sequencing (GEDS), a structured feature sequencing algorithm grounded in principles from the Column Permutation Problem (CPP). GEDS refines statistical descriptors of the features through similarity graph-based computations, systematically determining an effective feature sequencing. We incorporate GEDS within an order-aware efficient transformer framework, utilizing order-aware memory tokens that explicitly adhere to the derived feature sequencing via a dedicated loss function. Experimental results across multimodal benchmarks demonstrate that iStructTab effectively minimizes feature dispersion, improving predictive performance and robustness, and highlighting the significance of structured feature sequencing in multimodal learning.
