arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.11643cs.CL
收藏

通过多格式训练提升语言模型的跨格式鲁棒性

Improving Cross-Format Robustness in Language Models with Multi-Format Training

Ant Group(蚂蚁集团);International Digital Economy Academy (IDEA)(国际数字经济学院(IDEA))

June M. Liu, Shaomian Zheng, He Cao, Dingnan Jin, Qing Cui, Jun Zhou

AI总结:

提出FormatMix方法,通过将部分训练数据扩展为多种等价格式,显著提升大语言模型在不同答案格式下的一致性,仅需30%数据即可接近全格式训练效果。

摘要:

大型语言模型通常对答案格式仍然敏感:一种格式下正确解答的问题可能在另一种语义等价的格式下失败。为了研究这一差距,我们将跨格式鲁棒性定义为模型在不同格式下一致回答相同潜在问题的程度。然后,我们比较了全格式训练与FormatMix,后者使用随机或目标选择将仅一部分训练项扩展为多种等价格式。在GLM4和Llama-3.1上,多格式监督一致地提升了任务性能和跨格式鲁棒性,而仅使用多项选择题(MCQ)监督几乎无益,甚至可能降低鲁棒性。我们进一步发现,仅将约30%的训练集扩展为多种格式通常能恢复全格式训练的大部分收益,并且这一效果在我们研究的模型族和规模中均存在。这些结果表明,格式多样性(而非额外的监督本身)是鲁棒性的关键驱动因素。轻量级的多格式增强是一种实用的方法,可以在不改变基础模型的情况下使LLM对答案格式不那么敏感。

英文摘要:

Large language models often remain sensitive to answer format: a question solved correctly in one form may fail in another semantically equivalent form. To study this gap, we define cross-format robustness as the extent to which a model answers the same underlying question consistently across formats. We then compare full-format training with FormatMix, which expands only a subset of training items into multiple equivalent formats using either random or targeted selection. Across GLM4 and Llama-3.1, multi-format supervision consistently improves both task performance and cross-format robustness, whereas Multiple-choice question (MCQ)-only supervision alone brings little benefit and can even reduce robustness. We further find that expanding only about 30% of the training set into multiple formats often recovers most of the gain from full-format training, and this effect appears across the model families and sizes we study. These results suggest that format diversity, rather than additional supervision alone, is the key driver of robustness. That lightweight multi-format augmentation is a practical way to make LLMs less sensitive to answer format without changing the base model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑