JAF：裁判代理森林
JAF: Judge Agent Forest
AI总结：
JAF通过裁判代理森林框架，利用联合推理和集合学习原理，提升主代理的自我改进能力，通过高效的哈希算法优化多样示例选择，改进推理路径探索。
摘要：
裁判代理是代理AI框架中的基本组成部分：它们提供自动评估，并使推理过程能够进行迭代的自我完善。我们介绍了JAF：裁判代理森林，这是一种框架，其中裁判代理在一组由主代理生成的查询-响应对上进行联合推理，而不是单独评估每个对。这种范式将裁判从局部评估者提升为整体学习者：通过同时评估相关响应，裁判能够发现跨实例的模式和不一致，其汇总反馈使主代理能够通过裁判的集体视角来改进其自身输出。从概念上讲，JAF连接了信念传播和集合学习原则：重叠的上下文邻近区域诱导出一个知识图结构，这有助于传播批评，并且重复的、随机的评估会产生一个稳健的、基于情境的判断集合。JAF可以完全通过ICL实现，其中裁判在每个查询中被提示使用其关联的主代理响应以及一个小型、可能有噪声的同行示例集。虽然kNN在嵌入空间中是一个自然的起点，但这种方法忽略了分类结构、领域元数据或现代LLM可访问的细微差别。为了克服这些限制，我们开发了一种灵活的局部敏感哈希（LSH）算法，该算法通过整合语义嵌入、LLM驱动的哈希谓词、来自分类标签的监督以及相关附加信息来学习信息丰富的二进制代码。这些哈希代码支持高效、可解释且关系感知的多样化示例选择，并进一步优化CoT推理路径的探索。我们通过在大规模云环境中云配置错误处理的严苛任务上进行实证研究来验证JAF。
英文摘要：
Judge agents are fundamental to agentic AI frameworks: they provide automated evaluation, and enable iterative self-refinement of reasoning processes. We introduce JAF: Judge Agent Forest, a framework in which the judge agent conducts joint inference across a cohort of query--response pairs generated by a primary agent, rather than evaluating each in isolation. This paradigm elevates the judge from a local evaluator to a holistic learner: by simultaneously assessing related responses, the judge discerns cross-instance patterns and inconsistencies, whose aggregate feedback enables the primary agent to improve by viewing its own outputs through the judge's collective perspective. Conceptually, JAF bridges belief propagation and ensemble-learning principles: overlapping in-context neighborhoods induce a knowledge-graph structure that facilitates propagation of critique, and repeated, randomized evaluations yield a robust ensemble of context-sensitive judgments. JAF can be instantiated entirely via ICL, with the judge prompted for each query using its associated primary-agent response plus a small, possibly noisy set of peer exemplars. While kNN in embedding space is a natural starting point for exemplars, this approach overlooks categorical structure, domain metadata, or nuanced distinctions accessible to modern LLMs. To overcome these limitations, we develop a flexible locality-sensitive hashing (LSH) algorithm that learns informative binary codes by integrating semantic embeddings, LLM-driven hash predicates, supervision from categorical labels, and relevant side information. These hash codes support efficient, interpretable, and relation-aware selection of diverse exemplars, and further optimize exploration of CoT reasoning paths. We validate JAF with an empirical study on the demanding task of cloud misconfigs triage in large-scale cloud environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
