互补子模信息度量用于平衡和鲁棒的数据选择
Complement Submodular Information Measures for Balanced and Robust Data Selection
AI总结:
提出互补子模信息(CSI)目标函数,通过建模子集与其补集之间的共享结构信息,实现平衡且鲁棒的数据选择,并在理论上证明其近似单调性和贪心近似保证,实验表明在鲁棒隐藏切片感知子集选择中优于经典子模目标。
摘要:
子模优化已成为数据选择、检索、摘要和表示学习的基本范式,因为它能够建模覆盖度、多样性和代表性。然而,经典子模目标仅优化所选子集,并未明确保留所选子集与剩余数据之间的结构信息。在许多现代机器学习应用中,包括训练/验证/测试分割、基准构建和鲁棒子集选择,选择的质量关键取决于在所选子集及其补集之间保持平衡结构。在这项工作中,我们引入了互补子模信息(CSI),这是一类新的互补感知子模目标,用于量化子集与其补集之间的共享结构信息。我们的框架产生了几个经典子模函数的互补感知变体,包括设施选址、图割、LogDet、饱和覆盖、集合覆盖、概率集合覆盖和基于特征函数。我们分析了CSI目标的理论性质,并表明它们在有限曲率条件下表现出近似单调性,从而得到接近$(1-1/e)$的贪心近似保证。实验上,CSI目标在鲁棒隐藏切片感知子集选择中始终优于标准子模目标。特别是,CSI目标显著改善了相干稀有/尾部语义结构的保留,同时抑制了噪声和孤立异常值,从而显著提高了下游预测性能。合成实验进一步说明了不同的CSI实例如何捕获代表性、多样性、连通性和平衡邻域保留的互补概念。
英文摘要:
Submodular optimization has become a fundamental paradigm for data selection, retrieval, summarization, and representation learning due to its ability to model coverage, diversity, and representativeness. However, classical submodular objectives optimize only the selected subset and do not explicitly preserve structural information between the selected subset and the remaining data. In many modern machine learning applications, including train/validation/test splitting, benchmark construction, and robust subset selection, the quality of a selection depends critically on preserving balanced structure across both the selected subset and its complement. In this work, we introduce Complement Submodular Information (CSI), a new class of complement-aware submodular objectives that quantify shared structural information between a subset and its complement. Our framework induces complement-aware variants of several classical submodular functions including Facility Location, Graph Cut, LogDet, Saturated Coverage, Set Cover, Probabilistic Set Cover, and Feature Based Functions. We analyze the theoretical properties of CSI objectives and show that they exhibit approximate monotonicity under bounded curvature conditions, leading to near-$(1-1/e)$ greedy approximation guarantees. Empirically, CSI objectives consistently outperform standard submodular objectives on robust hidden-slice-aware subset selection. In particular, CSI objectives significantly improve preservation of coherent rare/tail semantic structure while simultaneously suppressing noisy and isolated outliers, leading to substantially improved downstream predictive performance. Synthetic experiments further illustrate how different CSI instantiations capture complementary notions of representativeness, diversity, connectivity, and balanced neighborhood preservation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。