自学习智能体长上下文理解
Self-Taught Agentic Long Context Understanding
AI总结：
提出AgenticLU框架，通过澄清链（CoC）结合树搜索和两阶段微调，增强LLM对复杂长上下文问题的理解与检索能力，在多任务上显著超越现有方法。
摘要：
回答复杂的长上下文问题仍然是大型语言模型（LLMs）面临的一项重大挑战，因为这需要有效的问题澄清和上下文检索。我们提出了智能体长上下文理解（AgenticLU），这是一个旨在通过在智能体工作流中整合有针对性的自我澄清与上下文基础来增强LLM对此类查询理解的框架。AgenticLU的核心是澄清链（CoC），模型通过自我生成的澄清问题及其相应的上下文基础来完善其理解。通过将推理扩展为树搜索，其中每个节点代表一个CoC步骤，我们在NarrativeQA上实现了97.8%的答案召回率，搜索深度最高为三，分支因子为八。为了将此搜索过程的高成本分摊到训练中，我们利用CoC工作流获得的每个步骤的偏好对，并执行两阶段模型微调：（1）监督微调以学习有效的分解策略，以及（2）直接偏好优化以增强推理质量。这使得AgenticLU模型能够在单次推理中有效且高效地生成澄清并检索相关上下文。在七个长上下文任务上的广泛实验表明，AgenticLU显著优于最先进的提示方法和专门的长上下文LLM，实现了稳健的多跳推理，同时随着上下文长度的增长保持一致的性能。
英文摘要：
Answering complex, long-context questions remains a major challenge for large language models (LLMs) as it requires effective question clarifications and context retrieval. We propose Agentic Long-Context Understanding (AgenticLU), a framework designed to enhance an LLM's understanding of such queries by integrating targeted self-clarification with contextual grounding within an agentic workflow. At the core of AgenticLU is Chain-of-Clarifications (CoC), where models refine their understanding through self-generated clarification questions and corresponding contextual groundings. By scaling inference as a tree search where each node represents a CoC step, we achieve 97.8% answer recall on NarrativeQA with a search depth of up to three and a branching factor of eight. To amortize the high cost of this search process to training, we leverage the preference pairs for each step obtained by the CoC workflow and perform two-stage model finetuning: (1) supervised finetuning to learn effective decomposition strategies, and (2) direct preference optimization to enhance reasoning quality. This enables AgenticLU models to generate clarifications and retrieve relevant context effectively and efficiently in a single inference pass. Extensive experiments across seven long-context tasks demonstrate that AgenticLU significantly outperforms state-of-the-art prompting methods and specialized long-context LLMs, achieving robust multi-hop reasoning while sustaining consistent performance as context length grows.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
