arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.16639cs.LG
收藏

SPICE: 基于协同与部分信息的课程演化

SPICE: Synergy and Partial Information Based Curriculum Evolution

New York Institute of Technology(纽约理工学院);New York University(纽约大学)

Ankush Pratap Singh, Houwei Cao, Yong Liu

AI总结:

提出SPICE框架,利用部分信息分解理论动态量化样本复杂度,设计渐进式课程使模型从学习共享跨模态线索过渡到模态特定模式再到复杂协同交互,在多个多模态基准上取得一致改进。

摘要:

多模态学习利用异构模态间的互补信息。每种模态的信息量在不同样本和训练阶段可能差异很大。现有的多模态课程学习策略通常假设样本的相对复杂度在训练过程中保持不变,因此无法适应模型的演化。我们提出了SPICE(基于协同与部分信息的课程演化),一种新颖的渐进式课程框架,用于多模态交互学习。在部分信息分解(PID)理论的指导下,我们的方法将多模态交互分解为冗余、独特和协同信息成分,从而实现对样本复杂度的可解释且动态的表征。基于这种分解,我们设计了一个在训练过程中不断演化的渐进式课程,使模型能够从学习共享的跨模态线索过渡到模态特定模式,最后到复杂的协同交互。为了适应模型演化,样本排序通过从单模态和多模态预测中得出的PID信息估计进行实时优化。在多个多模态基准上的实验表明,与传统训练和最先进基线相比,该方法取得了持续改进,凸显了PID信息分解和自适应样本排序在多模态课程学习中的有效性。

英文摘要:

Multimodal learning exploits complementary information across heterogeneous modalities. The informativeness of each modality can vary widely across samples and training stages. Existing multimodal curriculum learning strategies often assume that the relative complexity of samples remains unchanged throughout training and therefore cannot adapt to model evolution. We propose SPICE (Synergy and Partial Information based Curriculum Evolution), a novel progressive curriculum framework for multimodal interaction learning. Guided by Partial Information Decomposition (PID) theory, our approach decomposes multimodal interactions into redundant, unique, and synergistic information components, enabling an interpretable and dynamic characterization of sample complexity. Building on this decomposition, we design a progressive curriculum that evolves throughout training, allowing the model to transition from learning shared cross-modal cues to modality-specific patterns and, finally, to complex synergistic interactions. Adapting to model evolution, sample ordering is refined in real-time using PID information estimates derived from unimodal and multimodal predictions. Experiments across multiple multimodal benchmarks demonstrate consistent improvements over conventional training and state-of-the-art baselines, highlighting the effectiveness of PID information decomposition and adaptive sample ordering for multimodal curriculum learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑