PLANET：评估LLM规划能力的基准集合
PLANET: A Collection of Benchmarks for Evaluating LLMs' Planning Capabilities
- Computer Science Department, Emory University（埃默里大学计算机科学系）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文系统考察并分类了规划基准，识别测试平台与空白，为不同算法推荐合适基准，以指导未来开发。
AI中文摘要：
规划是智能体及智能体AI的核心。规划能力，例如在预算内创建旅行行程，在科学和商业领域都具有巨大潜力。此外，与临时方法相比，最优规划往往需要更少的资源。迄今为止，对现有规划基准的全面理解似乎仍然缺乏。缺乏这种理解，跨领域比较规划算法的性能或为新场景选择合适的算法仍然具有挑战性。在本文中，我们考察了一系列规划基准，以识别常用于算法开发的测试平台并突出潜在空白。这些基准被分类为具身环境、网页导航、调度、游戏与谜题以及日常任务自动化。我们的研究为各种算法推荐了最合适的基准，并为指导未来基准开发提供了见解。
英文摘要：
Planning is central to agents and agentic AI. The ability to plan, e.g., creating travel itineraries within a budget, holds immense potential in both scientific and commercial contexts. Moreover, optimal plans tend to require fewer resources compared to ad-hoc methods. To date, a comprehensive understanding of existing planning benchmarks appears to be lacking. Without it, comparing planning algorithms' performance across domains or selecting suitable algorithms for new scenarios remains challenging. In this paper, we examine a range of planning benchmarks to identify commonly used testbeds for algorithm development and highlight potential gaps. These benchmarks are categorized into embodied environments, web navigation, scheduling, games and puzzles, and everyday task automation. Our study recommends the most appropriate benchmarks for various algorithms and offers insights to guide future benchmark development.
