arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.01064cs.AI
收藏

真实还是欺骗?一种贝叶斯解码博弈增强一致性与可靠性

Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability

Imperial College London(伦敦帝国学院);University of Tokyo(东京大学);Friedrich-Alexander University Erlangen-Nürnberg(埃尔朗根-纽伦堡弗里德里希-亚历山大大学)

Weitong Zhang, Chengqi Zang, Bernhard Kainz

AI总结:

本文提出多阶段贝叶斯解码博弈,通过正确性对齐和模糊性校准,在无人工反馈或额外训练下区分有效与似是而非输出,使小模型可靠性超过更大模型。

摘要:

大型语言模型(LLM)通常会生成看似合理、但可能缺乏一致性和可靠性的输出,尤其是在模糊或复杂场景中。挑战在于确保输出同时符合事实正确性和人类意图。现有方法往往以降低准确性为代价来提升一致性,这存在问题。为缓解这些挑战,我们提出一种新颖的博弈论方法,用于在 LLM 输出生成的解码阶段增强一致性和可靠性。我们的方法将解码过程建模为一个多阶段贝叶斯解码博弈。该方法通过正确性对齐确保一致性,并通过模糊性校准增强可靠性。模型会动态地就最可靠的输出达成共识,并在没有人类反馈或额外训练的情况下区分 {Valid, Specious} 输出。我们的博弈设计使较小模型能够通过博弈机制超过大得多的模型(例如,78.1 的 LLaMA13B 对比 76.6 的 PaLM540B),同时还能整合各种 LL 策略和模型,展示了博弈论工具在提升 LLM 真实性与可靠性方面的潜力。

英文摘要:

Large Language Models (LLMs) often produce outputs that -- though plausible -- can lack consistency and reliability, particularly in ambiguous or complex scenarios. Challenges arise from ensuring that outputs align with both factual correctness and human intent. This is problematic in existing approaches that trade improved consistency for lower accuracy. To mitigate these challenges, we propose a novel game-theoretic approach to enhance consistency and reliability during the decoding stage of LLM output generation. Our method models the decoding process as a multistage Bayesian decoding game. This ensures consistency through Correctness Alignment and enhances reliability via Ambiguity Calibration. The model dynamically converges to a consensus on the most reliable outputs and distinguishes {Valid, Specious} outputs without human feedback or additional training. Our game design allows smaller models to outperform much larger models through game mechanisms (e.g., 78.1 LLaMA13B vs 76.6 PaLM540B), as well as integrating various LL strategies and models, demonstrating the potential of game-theoretic tools to improve the truthfulness and reliability of LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑