面向电信问答的首词概率引导检索增强生成（RAG）
First Token Probability Guided RAG for Telecom Question Answering
AI总结：
针对电信领域多项选择问答中RAG检索质量不足、幻觉问题突出的挑战，提出首词概率引导的RAG框架，通过选项概率归一化的置信度迭代优化超参数、动态调整上下文，提升了领域问答准确率。
摘要：
大语言模型（LLM）凭借其出色的通用能力已获得广泛关注。对于需要复杂领域知识的应用场景，检索增强生成（RAG）在将领域特定信息融入大语言模型方面展现出显著优势。然而，现有RAG研究尚未充分解决电信领域多项选择问答（MCQA）的挑战，尤其是在检索质量和缓解幻觉问题方面。为应对这些挑战，我们提出了一种新颖的首词概率引导RAG框架。该框架利用置信度分数优化关键超参数（如块数量、块窗口大小），同时动态调整上下文。我们的方法首先检索最相关的文本块，生成单个token作为潜在答案，随后将所有选项的概率进行归一化处理，作为置信度分数以指导上下文的动态调整。通过基于这些置信度分数迭代优化超参数，我们能够持续提升RAG的性能。我们开展了实验以验证该框架的有效性，结果表明其有望提升领域特定多项选择问答任务的准确率。
英文摘要：
Large Language Models (LLMs) have garnered significant attention for their impressive general-purpose capabilities. For applications requiring intricate domain knowledge, Retrieval-Augmented Generation (RAG) has shown a distinct advantage in incorporating domain-specific information into LLMs. However, existing RAG research has not fully addressed the challenges of Multiple Choice Question Answering (MCQA) in telecommunications, particularly in terms of retrieval quality and mitigating hallucinations. To tackle these challenges, we propose a novel first token probability guided RAG framework. This framework leverages confidence scores to optimize key hyperparameters, such as chunk number and chunk window size, while dynamically adjusting the context. Our method starts by retrieving the most relevant chunks and generates a single token as the potential answer. The probabilities of all options are then normalized to serve as confidence scores, which guide the dynamic adjustment of the context. By iteratively optimizing the hyperparameters based on these confidence scores, we can continuously improve RAG performance. We conducted experiments to validate the effectiveness of our framework, demonstrating its potential to enhance accuracy in domain-specific MCQA tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
