更健康的语言模型：用于公共卫生问答的检索增强生成
Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering
AI总结：
研究针对大型语言模型在公共卫生问答受幻觉等限制的问题，采用检索增强生成方法，通过扩展基准并系统评估检索与生成选择，比较多种检索方式，引入评判方法，突出检索对可靠公共卫生问答的作用并提供实用指导。
摘要：
大型语言模型（LLMs）在医学问答基准测试中取得了不错的成果，但在公共卫生领域的应用受到幻觉和官方指南快速演变的限制。检索增强生成（RAG）通过在明确维护的语料库中进行响应来减轻这些风险，但端到端性能关键取决于检索配置和超出多项选择格式的评估。我们将PubHealthBench扩展为检索增强设置，该基准包含7929个源自英国政府公共卫生指南的问答问题，并系统评估检索和生成选择。我们比较了多种嵌入模型和语料库变体中的密集、稀疏和混合检索，表明混合检索能持续提高召回率和排名质量，块长度和主题与排名性能相互作用。提供检索到的上下文显著提高了多种语言模型的多项选择准确率，使较小的开放权重模型能够匹配或超越未使用检索的较大模型，收益主要由检索质量和精心的上下文选择驱动。为评估实际的自由形式回答，我们引入了基于评分标准的语言模型评判方法，涵盖忠实性、完整性、清晰度和事实一致性，并通过双人注释进行验证。评判者与人类在忠实性和完整性方面的一致性最强，而事实一致性和清晰度的再现性较差，这在大规模解释这些维度时需谨慎。总体而言，我们的结果突出了检索作为可靠公共卫生问答的主要杠杆作用，并为构建和评估基于官方指南的RAG系统提供了实用指导。
英文摘要：
Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hallucinations and the rapid evolution of official guidance. Retrieval-Augmented Generation (RAG) mitigates these risks by grounding responses in an explicitly maintained corpus, but end-to-end performance depends critically on retrieval configuration and on evaluation beyond multiple-choice formats. We extend PubHealthBench, a question answering (QA) benchmark of 7,929 questions derived from UK Government public health guidance, into a retrieval-augmented setting and systematically evaluate retrieval and generation choices. We compare dense, sparse, and hybrid retrieval across multiple embedding models and corpus variants, and show that hybrid retrieval consistently improves recall and ranking quality, with chunk length and topic interacting with ranking performance. Providing retrieved context substantially increases multiple-choice accuracy across a diverse set of LLMs, enabling smaller open-weight models to match or outperform larger models used without retrieval, with gains primarily driven by retrieval quality and careful context selection. To assess realistic free-form answering, we introduce a rubric-based LLM-as-a-judge covering faithfulness, completeness, clarity, and factual consistency, and validate it against dual human annotations. Judge-human agreement is strongest for faithfulness and completeness, while factual consistency and clarity are less reliably reproduced, motivating caution when interpreting those dimensions at scale. Overall, our results highlight retrieval as a primary lever for reliable public health QA and provide practical guidance for building and evaluating RAG systems grounded in official guidance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
