arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.18679cs.CLcs.AIcs.LG
收藏

N-Critics:基于评论家集成的大语言模型自我优化方法

N-Critics: Self-Refinement of Large Language Models with Ensemble of Critics

Hewlett Packard Enterprise(慧与科技)

Sajad Mousavi, Ricardo Luna Gutiérrez, Desik Rengarajan, Vineet Gundecha, Ashwin Ramesh Babu, Avisek Naug, Antonio Guillen, Soumyendu Sarkar

AI总结:

本研究提出N-Critics自我修正机制,通过评论家集成与模型自身反馈优化大语言模型输出,缓解毒性与事实幻觉问题,该方法模型无关且能多领域提升可信度,实验证实其在减毒纠错上有效。

摘要:

我们提出了一种针对大语言模型(LLMs)的自我修正机制,用于缓解毒性内容、事实幻觉等问题。该方法通过一组评论家(critics)的集成与模型自身的反馈来优化模型输出。受人类行为启发,我们探究LLMs能否模拟人类的自我修正过程——人类在理解复杂主题时,常会进行自我反思并征求他人意见以完善认知。该方法具有模型无关性,可应用于多个领域,通过解决公平性、偏见和鲁棒性问题提升可信度。我们持续观察到,LLMs在降低毒性、纠正事实错误方面的性能均有提升。

英文摘要:

We propose a self-correction mechanism for Large Language Models (LLMs) to mitigate issues such as toxicity and fact hallucination. This method involves refining model outputs through an ensemble of critics and the model's own feedback. Drawing inspiration from human behavior, we explore whether LLMs can emulate the self-correction process observed in humans who often engage in self-reflection and seek input from others to refine their understanding of complex topics. Our approach is model-agnostic and can be applied across various domains to enhance trustworthiness by addressing fairness, bias, and robustness concerns. We consistently observe performance improvements in LLMs for reducing toxicity and correcting factual errors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑