arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.07488cs.AIcs.LG
收藏

通过增量优化效用实现多模态模型的数据选择效率

Efficient Data Selection for Multimodal Models via Incremental Optimization Utility

Jinhao Jing, Qiannian Zhao, Chao Huang, Zhan Su

AI总结:

本文提出OST框架,将数据选择转化为增量优化效用排名问题,通过轻量代理模拟单步更新估算样本边际效用,实验证明在减少训练成本的同时提升性能,优于现有基线方法。

摘要:

大规模多模态模型(LMMs)的扩展受到合成数据中质量和数量的权衡限制。先前方法如LLM-as-a-Judge在解决此问题上有效,但计算成本高且缺乏可解释性。为此,我们提出One-Step-Train(OST),将数据选择重新表述为增量优化效用排名问题。不同于依赖语义启发式,OST通过轻量代理模拟单步更新估算每个样本的边际效用。在Qwen系列多模态数学推理基准上的实验表明,OST实现了帕累托最优效率。通过选择前50个样本,OST将训练成本降低了43%(总时间消耗降低17),并在1.8个点上超越了强LLM-as-a-Judge基线。此外,在固定计算预算下,仅使用前20个样本的我们的方法在LLM-as-a-Judge上实现了5.6个点的提升,优于启发式评分基线如DEITA,并在8.8个点上超越Full-SFT基线。值得注意的是,尽管Full-SFT因噪声导致性能下降,我们的基于优化的方法有效识别了有毒样本,成功逆转了复杂推理任务中频繁观察到的负迁移。

英文摘要:

The scaling of Large Multimodal Models (LMMs) is constrained by the quality-quantity trade-off inherent in synthetic data. Previous approaches, such as LLM-as-a-Judge, have proven their effectiveness in addressing this but suffer from prohibitive computational costs and lack of interpretability. To bridge this gap, we propose One-Step-Train (OST), a framework that reformulates data selection as an incremental optimization utility ranking problem. Instead of relying on semantic heuristics, OST estimates the marginal utility of each sample via a simulated single-step update on a lightweight proxy. Experiments on the Qwen series across multimodal mathematical reasoning benchmarks demonstrate that OST achieves Pareto-optimal efficiency. By selecting the top-50 subset, OST reduces training costs by 43% (and total time consumption by 17) while surpassing the strong LLM-as-a-Judge baseline by 1.8 points. Furthermore, under a fixed compute budget, our method using only the top-20 subset achieves a 5.6 point gain over LLM-as-a-Judge, improves upon heuristic scoring baselines like DEITA, and outperforms the Full-SFT baseline by 8.8 points. Notably, while Full-SFT suffers from performance degradation due to noise, our optimization-grounded approach effectively identifies toxic samples, successfully reversing the negative transfer frequently observed in complex reasoning tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑