arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.08263cs.CVcs.AI
收藏

ImagebindDC:基于ImageBind的压缩框架压缩多模态数据

ImagebindDC: Compressing Multi-modal Data with Imagebind-based Condensation

EPIC Lab, SJTU(上海交通大学EPIC实验室);Bosch Corporate Research Asia Pacific(博世亚太公司研究部);HKUST(香港科技大学)

Yue Min, Shaobo Wang, Jiaze Li, Tianle Niu, Junxin Fan, Yongliang Miao, Lijin Yang, Linfeng Zhang

AI总结:

针对多模态数据压缩中保留模态间依赖的挑战,提出ImageBindDC框架,在ImageBind特征空间用特征函数损失实现三级分布对齐,在NYU-v2上仅用5个类内压缩数据点即获无损性能,压缩时间减少超4倍。

摘要:

数据压缩技术旨在从大型数据集中合成紧凑数据集以实现高效模型训练,虽在单模态场景中成功,但在需保留复杂模态间依赖关系的多模态场景中常失败。为此,我们提出ImageBindDC,一种在ImageBind统一特征空间内运行的新型数据压缩框架。我们的方法超越传统分布匹配,采用强大的特征函数(CF)损失,在傅里叶域通过精确无限矩匹配实现更精确的统计对齐。我们设计目标以确保三个关键分布一致性级别:(i)单模态对齐,匹配合成与真实数据在各模态内的统计特性;(ii)跨模态对齐,通过匹配混合真实-合成数据对的分布保留成对语义;(iii)联合模态对齐,通过对齐真实数据对与其合成对应物的联合分布捕获完整多变量数据结构。广泛实验凸显ImageBindDC的有效性:在NYU-v2数据集上,每个类仅用5个压缩数据点训练的模型实现与全数据集训练模型相当的无损性能,取得新的最先进结果,比之前最佳方法绝对提升8.2%,且压缩时间减少超4倍。

英文摘要:

Data condensation techniques aim to synthesize a compact dataset from a larger one to enable efficient model training, yet while successful in unimodal settings, they often fail in multimodal scenarios where preserving intricate inter-modal dependencies is crucial. To address this, we introduce ImageBindDC, a novel data condensation framework operating within the unified feature space of ImageBind. Our approach moves beyond conventional distribution-matching by employing a powerful Characteristic Function (CF) loss, which operates in the Fourier domain to facilitate a more precise statistical alignment via exact infinite moment matching. We design our objective to enforce three critical levels of distributional consistency: (i) uni-modal alignment, which matches the statistical properties of synthetic and real data within each modality; (ii) cross-modal alignment, which preserves pairwise semantics by matching the distributions of hybrid real-synthetic data pairs; and (iii) joint-modal alignment, which captures the complete multivariate data structure by aligning the joint distribution of real data pairs with their synthetic counterparts. Extensive experiments highlight the effectiveness of ImageBindDC: on the NYU-v2 dataset, a model trained on just 5 condensed datapoints per class achieves lossless performance comparable to one trained on the full dataset, achieving a new state-of-the-art with an 8.2\% absolute improvement over the previous best method and more than 4$\times$ less condensation time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑