通过游戏学习：表格知识的不对称迁移以增强图像模型
Learning Through Game: Skewed Transfer of Tabular Knowledge to Strengthen Image Model
浏览论文内容
中文总结 AI 辅助
针对测试时表格数据缺失及模态不平衡问题，提出偏斜知识迁移（SKT），通过共享头与两步纳什谈判策略不对称迁移表格知识，提升图像模型性能。
中文摘要 AI 辅助
多模态表格-图像学习正受到越来越多的关注，但由于测试时表格数据不可用，它面临着挑战。一种实用的解决方案是在训练期间将表格知识迁移到图像，以增强图像模型在推理时的性能。然而，被忽视但重要的挑战在于图像和表格之间的模态不平衡，以及它们在跨模态迁移中的不对称模态关系，这限制了表格数据的辅助作用。为了解决这些问题，我们提出了偏斜知识迁移（SKT），它通过自适应整合共享参数空间中的模态梯度，不对称地将表格知识迁移以改进图像模型。具体来说，我们首先引入一个多模态共享头，使模型能够在不增加额外参数的情况下受益于跨模态结构。然后，我们设计了一个两步纳什谈判策略，以有效利用表格梯度。在第一步中，SKT寻求模态平衡点，并在第二步中使用偏好感知将组合梯度引导至对图像有益的方向。此外，我们从理论上分析了SKT的帕累托改进和收敛性。为此，表格知识被明确迁移以增强图像模型。在广泛使用的表格-图像数据集上的实证实验表明，SKT通过使用表格数据作为辅助信息，持续提高了图像单模态性能。
英文摘要
Multimodal tabular-image learning is gaining growing attention, yet it faces challenges due to tabular data unavailable at test time. A practical solution involves transferring tabular knowledge to images during training to enhance the performance of image models at inference. However, the overlooked yet important challenges lie in the modality imbalance between images and tables, as well as their asymmetric modality relationship in cross-modal transfer, which limits the auxiliary role of tabular data. To address these issues, we propose Skewed Knowledge Transfer (SKT), which asymmetrically transfers tabular knowledge to improve the image model by adaptive integration of modality gradients in a shared parameter space. Specifically, we first introduce a multimodal shared head, which allows the model to benefit from cross-modal structure without adding additional parameters. We then design a two-step Nash Bargaining strategy to effectively leverage tabular gradients. In the first step, SKT seeks a point of modality balance and uses preference awareness in the second step to steer combined gradients toward image-beneficial directions. Furthermore, we theoretically analyze the Pareto improvement and convergence of SKT. To this end, tabular knowledge is explicitly transferred to enhance image models. Empirical experiments on widely used tabular-image datasets reveal that SKT consistently improves image unimodal performance by using tabular data as auxiliary information.
