样本级多模态交互的高效量化
Efficient Quantification of Multimodal Interaction at Sample Level
AI总结：
针对多模态交互样本级量化难题，提出基于逐点信息论的轻量级LSMI估计器，实现高效精确的冗余性估计，并支持样本划分、知识蒸馏和模型集成等应用。
摘要：
模态间的交互——冗余性、独特性和协同性——共同决定了多模态信息的构成。理解这些交互对于分析多模态系统中的信息动态至关重要，然而其准确的样本级量化在理论和计算上均面临重大挑战。为解决此问题，我们引入了轻量级样本级多模态交互（LSMI）估计器，其严格基于逐点信息论。我们首先开发了一个冗余性估计框架，采用适当的逐点信息度量来量化这种最可分解且可测量的交互。在此基础上，我们提出了一种通用的交互估计方法，采用高效的熵估计，专门针对连续分布中的样本级估计进行定制。在合成和真实数据集上的大量实验验证了LSMI的精度和效率。关键在于，我们的样本级方法揭示了多模态数据中细粒度的样本级和类别级动态，从而支持诸如基于冗余性的样本划分、目标知识蒸馏和交互感知的模型集成等实际应用。代码可在 https://github.com/GeWu-Lab/LSMI_Estimator 获取。
英文摘要：
Interactions between modalities -- redundancy, uniqueness, and synergy -- collectively determine the composition of multimodal information. Understanding these interactions is crucial for analyzing information dynamics in multimodal systems, yet their accurate sample-level quantification presents significant theoretical and computational challenges. To address this, we introduce the Lightweight Sample-wise Multimodal Interaction (LSMI) estimator, rigorously grounded in pointwise information theory. We first develop a redundancy estimation framework, employing an appropriate pointwise information measure to quantify this most decomposable and measurable interaction. Building upon this, we propose a general interaction estimation method that employs efficient entropy estimation, specifically tailored for sample-wise estimation in continuous distributions. Extensive experiments on synthetic and real-world datasets validate LSMI's precision and efficiency. Crucially, our sample-wise approach reveals fine-grained sample- and category-level dynamics within multimodal data, enabling practical applications such as redundancy-informed sample partitioning, targeted knowledge distillation, and interaction-aware model ensembling. The code is available at https://github.com/GeWu-Lab/LSMI_Estimator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
