基于$β$-泛化前门准则的干预式不平衡多模态表示学习
Interventional Imbalanced Multi-Modal Representation Learning via $β$-Generalization Front-Door Criterion
AI总结：
针对不同模态对任务预测的贡献不平衡导致多模态方法判别性能下降的问题，从因果视角提出基于$β$-泛化前门准则的干预式不平衡多模态表示学习方法，解决基准方法理论可解释性不足等缺陷，提升判别性能。
摘要：
多模态方法相比单模态方法具有全面优势，但不同模态对任务相关预测的贡献不平衡会持续降低典型多模态方法的判别性能。根据对任务相关预测的贡献，模态可分为主导模态和辅助模态。基准方法提出了一种易处理的解决方案：在训练期间增强贡献较小的辅助模态。然而，我们的实证探索对这种做法背后的基本思想提出了挑战，并进一步发现基准方法存在理论可解释性不足、判别知识探索能力有限等缺陷。为此，我们从因果视角重新审视多模态表示学习，构建结构因果模型；在考虑辅助模态的同时，捕捉主导模态判别知识与预测标签间的真实因果关系，进而引入$β$-泛化前门准则，并提出一种新网络以充分探索多模态判别知识。最后通过严谨的理论分析和多样的实证评估验证了所提方法内在机制的有效性。
英文摘要：
Multi-modal methods establish comprehensive superiority over uni-modal methods. However, the imbalanced contributions of different modalities to task-dependent predictions constantly degrade the discriminative performance of canonical multi-modal methods. Based on the contribution to task-dependent predictions, modalities can be identified as predominant and auxiliary modalities. Benchmark methods raise a tractable solution: augmenting the auxiliary modality with a minor contribution during training. However, our empirical explorations challenge the fundamental idea behind such behavior, and we further conclude that benchmark approaches suffer from certain defects: insufficient theoretical interpretability and limited exploration capability of discriminative knowledge. To this end, we revisit multi-modal representation learning from a causal perspective and build the Structural Causal Model. Following the empirical explorations, we determine to capture the true causality between the discriminative knowledge of predominant modality and predictive label while considering the auxiliary modality. Thus, we introduce the $β$-generalization front-door criterion. Furthermore, we propose a novel network for sufficiently exploring multi-modal discriminative knowledge. Rigorous theoretical analyses and various empirical evaluations are provided to support the effectiveness of the innate mechanism behind our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
