arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.14499cs.CL
收藏

“我的答案是C”:指令微调语言模型中首token概率与文本答案并不一致

"My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models

LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Bocconi University(博科尼大学)

Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, Barbara Plank

AI总结:

本文系统评估了指令微调大语言模型中首token概率评估与最终文本答案的一致性,发现两者在多维度上严重不匹配,不匹配率超过60%,提示不能仅依赖首token评估。

摘要:

语言生成的开放性使得自回归大语言模型(LLMs)的评估具有挑战性。一种常见的评估方法使用多项选择题(MCQ)来限制响应空间,然后通过首个token预测的对数概率对候选答案进行排序来评估模型。然而,由于模型具有多样的响应风格,例如以“Sure”开头或拒绝回答,首token可能无法一致地反映最终响应输出。因此,MCQ评估并不能代表模型与用户交互时的行为。但差异有多大?我们从多个维度评估了首token评估与文本输出之间的一致性,即最终选项选择、拒绝率、选择分布以及提示扰动下的鲁棒性。我们的结果表明,两种方法在所有维度上都严重不一致,不匹配率超过60%。在对话或安全数据上重度微调的模型尤其受影响。至关重要的是,即使我们不断增加对提示的约束,即强制模型以选项字母或示例模板开头,模型仍然不一致。我们的发现i)强调了同时检查文本输出的重要性,ii)提醒不要仅依赖首token评估。

英文摘要:

The open-ended nature of language generation makes the evaluation of autoregressive large language models (LLMs) challenging. One common evaluation approach uses multiple-choice questions (MCQ) to limit the response space. The model is then evaluated by ranking the candidate answers by the log probability of the first token prediction. However, first-tokens may not consistently reflect the final response output, due to model's diverse response styles such as starting with "Sure" or refusing to answer. Consequently, MCQ evaluation is not indicative of model behaviour when interacting with users. But by how much? We evaluate how aligned first-token evaluation is with the text output along several dimensions, namely final option choice, refusal rate, choice distribution and robustness under prompt perturbation. Our results show that the two approaches are severely misaligned on all dimensions, reaching mismatch rates over 60%. Models heavily fine-tuned on conversational or safety data are especially impacted. Crucially, models remain misaligned even when we increasingly constrain prompts, i.e., force them to start with an option letter or example template. Our findings i) underscore the importance of inspecting the text output as well and ii) caution against relying solely on first-token evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑