AgentSearchBench：面向真实世界的AI代理搜索基准
AgentSearchBench: A Benchmark for AI Agent Search in the Wild
AI总结：
本文提出AgentSearchBench，一个基于真实世界代理的大规模基准，通过执行任务查询和高层任务描述，评估代理相关性，揭示语义相似性与实际性能间的差距，并展示轻量行为信号对排名质量的提升作用。
摘要：
人工智能代理生态系统快速成长，正在改变复杂任务的委托与执行方式，带来了识别适合任务的代理的新挑战。不同于传统工具，代理能力通常是组合性和执行依赖性的，仅凭文本描述难以评估。然而，现有研究和基准通常假设功能明确、候选池受控或仅能执行任务查询，导致真实代理搜索场景研究不足。我们引入AgentSearchBench，一个大规模的真实世界代理搜索基准，包含近10000个来自多个提供者的代理。该基准将代理搜索形式化为检索和重排序问题，同时在可执行任务查询和高层任务描述下进行评估，使用基于执行的性能信号来衡量相关性。实验揭示了语义相似性与实际代理性能之间的持续差距，暴露了基于描述的检索和重排序方法的局限性。我们进一步表明，轻量级行为信号，包括执行感知探测，可以显著提高排名质量，强调了将执行信号纳入代理发现的重要性。我们的代码可在https://github.com/Bingo-W/AgentSearchBench获取。
英文摘要：
The rapid growth of AI agent ecosystems is transforming how complex tasks are delegated and executed, creating a new challenge of identifying suitable agents for a given task. Unlike traditional tools, agent capabilities are often compositional and execution-dependent, making them difficult to assess from textual descriptions alone. However, existing research and benchmarks typically assume well-specified functionalities, controlled candidate pools, or only executable task queries, leaving realistic agent search scenarios insufficiently studied. We introduce AgentSearchBench, a large-scale benchmark for agent search in the wild, built from nearly 10,000 real-world agents across multiple providers. The benchmark formalizes agent search as retrieval and reranking problems under both executable task queries and high-level task descriptions, and evaluates relevance using execution-grounded performance signals. Experiments reveal a consistent gap between semantic similarity and actual agent performance, exposing the limitations of description-based retrieval and reranking methods. We further show that lightweight behavioral signals, including execution-aware probing, can substantially improve ranking quality, highlighting the importance of incorporating execution signals into agent discovery. Our code is available at https://github.com/Bingo-W/AgentSearchBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
