arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.07301cs.CLcs.AI
收藏

小语言模型可实现自我修正

Small Language Model Can Self-correct

Shanghai Institute of AI for Education and School of Computer Science and Technology, East China Normal University(华东师范大学上海智能教育研究院与计算机科学与技术学院);School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University(复旦大学计算机科学学院上海市数据科学重点实验室)

Haixia Han, Jiaqing Liang, Jie Shi, Qianyu He, Yanghua Xiao

AI总结:

针对语言模型易生成错误信息的问题,本文提出内在自我修正(ISC)机制,通过自修正数据构建流程与部分答案掩码(PAM)微调,让60-130亿参数的小语言模型也能自触发修正输出,提升生成质量。

摘要:

ChatGPT等生成式语言模型(LM)已在各类下游任务中展现出卓越性能。然而,其最突出的缺陷之一是会以自信的语气生成不准确或错误的信息。过往研究设计了复杂的流程与提示词,以引导大语言模型具备自我修正能力。但大语言模型是被明确提示分别验证和修改其答案,而非像人类那样自发完成所有步骤。此外,这类复杂提示对小语言模型来说极难遵循。本文提出了生成式语言模型中的内在自我修正(Intrinsic Self-Correction,ISC)机制,旨在以自触发的方式修正语言模型的初始输出,即便对于参数量为60亿的小语言模型也适用。具体而言,我们设计了一套构建自修正数据的流程,并提出部分答案掩码(Partial Answer Masking,PAM)方法,旨在通过微调赋予模型内在自我修正能力。我们使用参数量从60亿到130亿不等的语言模型,在常识推理和事实知识推理两项任务上开展了实验。实验结果表明,采用ISC生成的输出优于未进行自修正的输出。我们认为,即便小语言模型,通过赋予其内在自我修正能力,输出质量也可得到进一步提升。

英文摘要:

Generative Language Models (LMs) such as ChatGPT have exhibited remarkable performance across various downstream tasks. Nevertheless, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. Previous studies have devised sophisticated pipelines and prompts to induce large LMs to exhibit the capability for self-correction. However, large LMs are explicitly prompted to verify and modify its answers separately rather than completing all steps spontaneously like humans. Moreover, these complex prompts are extremely challenging for small LMs to follow. In this paper, we introduce the \underline{I}ntrinsic \underline{S}elf-\underline{C}orrection (ISC) in generative language models, aiming to correct the initial output of LMs in a self-triggered manner, even for those small LMs with 6 billion parameters. Specifically, we devise a pipeline for constructing self-correction data and propose Partial Answer Masking (PAM), aiming to endow the model with the capability for intrinsic self-correction through fine-tuning. We conduct experiments using LMs with parameters sizes ranging from 6 billion to 13 billion in two tasks, including commonsense reasoning and factual knowledge reasoning. Our experiments demonstrate that the outputs generated using ISC outperform those generated without self-correction. We believe that the output quality of even small LMs can be further improved by empowering them with the ability to intrinsic self-correct.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑