语言模型能否自我批评？探究BioASQ 2025检索增强生成中的自我反馈机制
Can Language Models Critique Themselves? Investigating Self-Feedback for Retrieval Augmented Generation at BioASQ 2025
AI总结：
本研究在BioASQ 2025挑战中探究了LLM通过自我反馈机制迭代优化检索增强生成输出的效果，发现该策略在不同模型和任务上表现不一，为智能体搜索系统的自我修正研究提供了见解。
摘要：
智能体检索增强生成（RAG）和“深度研究”系统旨在实现自主搜索过程，使大型语言模型（LLMs）能够迭代地优化输出。然而，将这些系统应用于特定领域的专业搜索（如生物医学研究）面临挑战，因为自动化系统可能减少用户参与，并与专家的信息需求不一致。专业搜索任务通常要求高水平的用户专业知识和透明度。BioASQ CLEF 2025挑战赛使用专家制定的问题，可作为研究这些问题的平台。我们探索了当前推理型和非推理型LLM（如Gemini-Flash 2.0、o3-mini、o4-mini和DeepSeek-R1）的性能。我们方法的一个关键方面是自我反馈机制，即LLM生成、评估并优化其输出，用于查询扩展以及多种答案类型（是/否、事实型、列表型、理想型）。我们研究了这种迭代式自我修正是否能提高性能，以及推理模型是否更能生成有用的反馈。初步结果表明，自我反馈策略在不同模型和任务上的表现各异。这项工作为LLM自我修正提供了见解，并为未来在这些搜索系统中比较LLM生成反馈与直接人类专家输入的有效性研究提供了参考。
英文摘要：
Agentic Retrieval Augmented Generation (RAG) and 'deep research' systems aim to enable autonomous search processes where Large Language Models (LLMs) iteratively refine outputs. However, applying these systems to domain-specific professional search, such as biomedical research, presents challenges, as automated systems may reduce user involvement and misalign with expert information needs. Professional search tasks often demand high levels of user expertise and transparency. The BioASQ CLEF 2025 challenge, using expert-formulated questions, can serve as a platform to study these issues. We explored the performance of current reasoning and nonreasoning LLMs like Gemini-Flash 2.0, o3-mini, o4-mini and DeepSeek-R1. A key aspect of our methodology was a self-feedback mechanism where LLMs generated, evaluated, and then refined their outputs for query expansion and for multiple answer types (yes/no, factoid, list, ideal). We investigated whether this iterative self-correction improves performance and if reasoning models are more capable of generating useful feedback. Preliminary results indicate varied performance for the self-feedback strategy across models and tasks. This work offers insights into LLM self-correction and informs future work on comparing the effectiveness of LLM-generated feedback with direct human expert input in these search systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
