arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.23263cs.CLcs.AI
收藏

小语言模型帮助解决LLM提示的语义歧义

Small Language Model Helps Resolve Semantic Ambiguity of LLM Prompt

School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);Department of Electronic Engineering, Kyung Hee University(庆熙大学电子工程系)

Zhenzhen Huang, Chaoning Zhang, Fachrina Dewi Puspitasari, Jiaquan Zhang, Yitian Zhou, Shuxu Chen, Yang Yang

AI总结:

本文提出通过显式提示消歧机制解决LLM提示的语义歧义问题,利用小语言模型高效计算提升推理性能,实验表明方法在多个基准上提升2.5点,成本仅0.02美元。

摘要:

大型语言模型(LLMs)因其出色的指令遵循能力被广泛应用于各种复杂推理任务中。然而,模型性能高度依赖于用户输入提示的开放性特征。自然提示往往不遵循严格的语法规则,导致产生具有多种解释的模糊查询。此类模糊提示使模型在选择正确推理路径以回答问题时产生困惑。先前的工作通过在LLM推理过程中应用查询编辑来解决这一挑战,但并未明确解决歧义的根本原因。为解决这一限制,我们提出了一种预推理提示优化机制,通过显式的提示消歧。特别地,我们识别提示中的语义风险,检查其多视角一致性,并解决由此产生的任何语义冲突。最后,我们将解决的歧义以逻辑结构的方式组织成干净的输入,供LLM使用。通过显式解决语义歧义,我们的方法可以产生更集中的注意力分布,专注于语义关键的标记。我们还利用小语言模型(SLMs)作为提示消歧的主要执行者,以受益于其高效的计算。通过在多个基准上的全面实验,我们证明了我们的方法在成本仅0.02美元的情况下,将推理性能提高了2.5个点。我们的研究将显式提示消歧作为有效的提示优化方法推广,而不影响LLM推理的内部机制。

英文摘要:

Large language models (LLMs) are increasingly utilized in various complex reasoning tasks due to their excellent instruction following capability. However, the model's performance is highly dependent on the open-ended characteristics of the users' input prompt. Natural prompts often do not follow proper syntactic rules, which creates ambiguous queries that yield multiple interpretations. Such ambiguous prompts confuse the model in choosing the correct reasoning paths to answer questions. Prior works address this challenge by applying query editing during the LLM inference process without explicitly solving the root cause of the ambiguity. To address this limitation, we propose a pre-inference prompt optimization mechanism via explicit prompt disambiguation. Particularly, we identify semantic risks in the prompt, check their multi-perspective consistency, and resolve any semantic conflicts that arise. Finally, we organize the resolved ambiguities in a logically structured manner as a clean input to the LLM. By explicitly resolving semantic ambiguity, our method can produce a more focused attention distribution to the semantically essential tokens. We also leverage small language models (SLMs) as the main executor of prompt disambiguation to benefit from their efficient computation. Through comprehensive experiments on multiple benchmarks, we demonstrate that our method improves reasoning performance by 2.5 points at a cost of only \$0.02. Our study promotes explicit prompt disambiguation as an effective prompt optimization method without disturbing the internal mechanism of LLM inference.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑