arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.00902cs.CL
收藏

Instruct-of-Reflection:通过动态元指令增强大型语言模型的迭代反思能力

Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction

Beijing University of Posts and Telecommunications(北京邮电大学);AI Lab of Lenovo Research(联想研究院AI实验室);Tianjin University(天津大学);The Hong Kong University of Science and Technology(香港科技大学)

Liping Liu, Chunhong Zhang, Likang Wu, Chuang Zhao, Zheng Hu, Ming He, Jianping Fan

AI总结:

针对LLMs静态反思存在的冗余、漂移和固执问题,提出Instruct-of-Reflection框架,利用元思维和自洽性分类器驱动的动态元指令指导迭代反思,在数学与常识推理任务上较基线平均提升10.1%。

摘要:

大型语言模型(LLMs)的自我反思引起了广泛关注。现有方法基于LLMs的内部反思能力或外部反馈,让模型迭代并改进先前的回答。然而,近期研究对无外部反馈的内在自我纠正是否会降低性能提出了质疑。基于经验证据,我们发现当前的静态反思方法可能导致冗余、漂移和固执问题。为缓解此问题,我们引入了Instruct-of-Reflection(IoRT),这是一种新颖且通用的反思框架,利用动态元指令增强LLMs的迭代反思能力。具体而言,我们提出由元思维和自洽性分类器驱动的指令生成器,生成包括刷新、停止和选择在内的各种指令,以指导下一次反思迭代。实验表明,IoRT在数学和常识推理任务中较已有基线平均提升10.1%,突显了其有效性与适用性。

英文摘要:

Self-reflection for Large Language Models (LLMs) has gained significant attention. Existing approaches involve models iterating and improving their previous responses based on LLMs' internal reflection ability or external feedback. However, recent research has raised doubts about whether intrinsic self-correction without external feedback may even degrade performance. Based on our empirical evidence, we find that current static reflection methods may lead to redundant, drift, and stubborn issues. To mitigate this, we introduce Instruct-of-Reflection (IoRT), a novel and general reflection framework that leverages dynamic-meta instruction to enhance the iterative reflection capability of LLMs. Specifically, we propose the instructor driven by the meta-thoughts and self-consistency classifier, generates various instructions, including refresh, stop, and select, to guide the next reflection iteration. Our experiments demonstrate that IoRT achieves an average improvement of 10.1% over established baselines in mathematical and commonsense reasoning tasks, highlighting its efficacy and applicability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑