AMOR：通过过程反馈构建自适应模块化知识智能体的配方
AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback
AI总结：
本文提出基于开源 LLM 与有限状态机的模块化知识智能体框架 AMOR，通过模块级过程反馈和预热、适应两阶段微调，实现跨知识环境泛化与领域适配，并在多领域实验中优于强基线。
摘要：
大语言模型（LLM）的显著成功掀起了构建语言智能体以完成各类复杂任务的热潮。我们提出 AMOR，这是一个基于开源 LLM 的智能体框架，能够利用外部知识库进行推理，并通过人类对推理过程的监督适应特定领域。AMOR 在有限状态机（FSM）之上构建推理逻辑，通过在解耦模块上的自主执行与状态转移来解决问题。这使得人类可以向单个模块提供直接反馈，从而自然形成过程监督。基于这一推理与反馈框架，我们通过两个阶段的微调开发 AMOR：预热与适应。前者利用从各类公共数据集自动构建的示例对 LLM 进行微调，使 AMOR 能够泛化到不同的知识环境；后者则利用过程反馈使 AMOR 适配特定领域。跨多个领域的大量实验证明，得益于基于 FSM 的推理和过程反馈机制，AMOR 相比强基线具有优势。代码和数据公开发布于 https://github.com/JianGuanTHU/AMOR。
英文摘要：
The notable success of large language models (LLMs) has sparked an upsurge in building language agents to complete various complex tasks. We present AMOR, an agent framework based on open-source LLMs, which reasons with external knowledge bases and adapts to specific domains through human supervision to the reasoning process. AMOR builds reasoning logic over a finite state machine (FSM) that solves problems through autonomous executions and transitions over disentangled modules. This allows humans to provide direct feedback to the individual modules, and thus naturally forms process supervision. Based on this reasoning and feedback framework, we develop AMOR through two-stage fine-tuning: warm-up and adaptation. The former fine-tunes the LLM with examples automatically constructed from various public datasets, enabling AMOR to generalize across different knowledge environments, while the latter tailors AMOR to specific domains using process feedback. Extensive experiments across multiple domains demonstrate the advantage of AMOR to strong baselines, thanks to its FSM-based reasoning and process feedback mechanism. The code and data are publicly available at \url{https://github.com/JianGuanTHU/AMOR}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
