强化符号绑定使大语言模型成为可靠的多项选择题选择器
Strengthened Symbol Binding Makes Large Language Models Reliable Multiple-Choice Selectors
AI总结:
本文针对大语言模型在监督微调阶段仍存在的多项选择符号绑定能力不足问题,提出逐点智能反馈算法PIF,通过构造负样本和逐点损失增强符号与选项内容的关联,从而显著降低选择偏差并提升多项选择题准确率。
摘要:
多项选择题(MCQs)是大语言模型(LLMs)研究中的一个关键领域。以往的工作已经研究了少样本场景下多项选择题中的选择偏差问题,即大语言模型的性能可能受到答案选项呈现方式的影响,但监督微调(SFT)阶段的选择偏差尚未被探索。在本文中,我们揭示了选择偏差在SFT阶段依然存在,其主要原因是大语言模型的多项选择符号绑定(MCSB)能力不足。这一局限性意味着模型难以有效地将答案选项与其对应符号(例如A/B/C/D)关联起来。为了增强模型的MCSB能力,我们首先将选项内容纳入损失函数,随后调整选项符号和内容的权重,引导模型理解当前符号所对应的选项内容。在此基础上,我们提出了一种用于多项选择题的高效SFT算法,称为逐点智能反馈(PIF)。PIF通过将错误选项内容与所有候选符号随机组合来构造负样本,并提出一种逐点损失,将这些负样本的反馈提供给大语言模型。我们的实验结果表明,PIF通过提升模型的MCSB能力,显著降低了模型的选择偏差。值得注意的是,PIF在多项选择题的准确率上表现出显著提升。
英文摘要:
Multiple-Choice Questions (MCQs) constitute a critical area of research in the study of Large Language Models (LLMs). Previous works have investigated the selection bias problem in MCQs within few-shot scenarios, in which the LLM's performance may be influenced by the presentation of answer choices, leaving the selection bias during Supervised Fine-Tuning (SFT) unexplored. In this paper, we reveal that selection bias persists in the SFT phase , primarily due to the LLM's inadequate Multiple Choice Symbol Binding (MCSB) ability. This limitation implies that the model struggles to associate the answer options with their corresponding symbols (e.g., A/B/C/D) effectively. To enhance the model's MCSB capability, we first incorporate option contents into the loss function and subsequently adjust the weights of the option symbols and contents, guiding the model to understand the option content of the current symbol. Based on this, we introduce an efficient SFT algorithm for MCQs, termed Point-wise Intelligent Feedback (PIF). PIF constructs negative instances by randomly combining the incorrect option contents with all candidate symbols, and proposes a point-wise loss to provide feedback on these negative samples into LLMs. Our experimental results demonstrate that PIF significantly reduces the model's selection bias by improving its MCSB capability. Remarkably, PIF exhibits a substantial enhancement in the accuracy for MCQs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。