揭示内在能力：一种用于视觉-语言模型数据整理的范式
Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Models
AI总结：
CADC通过揭示视觉-语言模型的内在能力，提供了一种基于能力分析的数据整理范式，以提升指令微调的效果。
摘要：
大型视觉-语言模型（VLMs）在基准测试中表现强劲，但通过指令微调控制其行为仍然困难。减少指令微调数据集的预算往往会导致退化，因为启发式策略将模型视为黑箱，并忽略了支配学习的潜在能力。我们引入了能力属性数据整理（CADC），一种将数据整理从任务特定的启发式方法转向内在能力分析的框架。CADC通过基于梯度的学习轨迹无监督地发现内在能力，并通过影响估计将训练数据归因于这些能力，并通过平衡选择和分阶段排序来整理能力感知的课程。这将黑箱指令微调转变为可控的能力驱动过程。使用原始数据的5%，CADC在多模态基准上超过了全数据训练。这些结果验证了内在能力作为模型学习的基本构建块，并确立了CADC作为指令数据整理的原则范式。
英文摘要：
Large vision-language models (VLMs) achieve strong benchmark performance, but controlling their behavior through instruction tuning remains difficult. Reducing the budget of instruction tuning dataset often causes regressions, as heuristic strategies treat models as black boxes and overlook the latent capabilities that govern learning. We introduce Capability-Attributed Data Curation (CADC), a framework that shifts curation from task-specific heuristics to intrinsic capability analysis. CADC discovers intrinsic capabilities in an unsupervised manner from gradient-based learning trajectories, attributes training data to these capabilities via influence estimation, and curates capability-aware curricula through balanced selection and staged sequencing. This transforms black-box instruction tuning into a controllable, capability-driven process. With as little as 5% of the original data, CADC surpasses full-data training on multimodal benchmarks. These results validate intrinsic capabilities as the fundamental building blocks of model learning and establish CADC as a principle paradigm for instruction data curation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
