发表机构
HKU; CUHK; Stanford（香港大学; 香港中文大学; 斯坦福大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
从结果角度重审数据集蒸馏，提出影响匹配方法，通过可微估计器量化参数变化，学习合成集使影响与真实数据集匹配，在分类和视觉语言蒸馏任务中表现出色，超越基线。
AI 中文摘要
我们从以结果为中心的角度重新审视数据集蒸馏。影响匹配（Inf-Match）不是对齐过程代理（逐步骤梯度或训练轨迹），而是对齐训练的最终结果：它学习一个紧凑的合成集，其对收敛参数的影响与完整数据集的影响相匹配。具体而言，我们引入了一个完全可微的样本级影响估计器，通过展开优化动态并应用一阶泰勒近似，在线性时间内量化添加或删除数据导致的参数变化。然后通过最小化合成集与真实数据集影响之间的不匹配来学习合成集，实现结果对齐而非启发式过程模仿。Inf-Match在标准分类基准上实现了最佳准确率，在图像/文本检索任务上也优于强过程匹配基线。代码将通过此https URL发布。
英文摘要
We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5\%, a +4.7\% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5\%. The code will be released via https://github.com/hrtan/infmatch.
Journal refCVPR 2026
