MicroRemed：在微服务修复中对 LLMs 进行基准测试
MicroRemed: Benchmarking LLMs in Microservices Remediation
AI总结：
本文提出了首个端到端微服务修复基准 MicroRemed，要求 LLMs 直接根据诊断报告生成可执行的 Ansible playbooks，并设计了模拟 SREs 推理的多智能体框架 ThinkRemed，通过迭代推理与系统反思显著提升了修复性能。
摘要：
结合基于智能体推理框架的大型语言模型（LLMs）近期在自主决策和系统级操作方面展现出巨大潜力。一个极具前景但尚未充分探索的方向是微服务修复，其目标是自动恢复发生故障的微服务系统。然而，现有方法仍依赖于站点可靠性工程师（SREs）人工编写的提示，LLMs 仅将文本指令转换为可执行代码。为推进该领域研究，我们推出了 MicroRemed，这是首个用于评估 LLMs 在端到端微服务修复中的基准，模型必须直接根据诊断报告生成可执行的 Ansible playbooks 以恢复系统功能。我们进一步提出了 ThinkRemed，这是一个模拟 SREs 反思性与感知性推理的多智能体框架。实验结果表明，MicroRemed 对当前的 LLMs 提出了巨大挑战，而 ThinkRemed 通过迭代推理与系统反思提升了端到端的修复性能。该基准可在 https://github.com/LLM4AIOps/MicroRemed 获取。
英文摘要：
Large Language Models (LLMs) integrated with agent-based reasoning frameworks have recently shown strong potential for autonomous decision-making and system-level operations. One promising yet underexplored direction is microservice remediation, where the goal is to automatically recover faulty microservice systems. Existing approaches, however, still rely on human-crafted prompts from Site Reliability Engineers (SREs), with LLMs merely converting textual instructions into executable code. To advance research in this area, we introduce MicroRemed, the first benchmark for evaluating LLMs in end-to-end microservice remediation, where models must directly generate executable Ansible playbooks from diagnosis reports to restore system functionality. We further propose ThinkRemed, a multi-agent framework that emulates the reflective and perceptive reasoning of SREs. Experimental results show that MicroRemed presents substantial challenges to current LLMs, while ThinkRemed improves end-to-end remediation performance through iterative reasoning and system reflection. The benchmark is available at https://github.com/LLM4AIOps/MicroRemed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
