EPC：一种用于测量LLM智能体系统中评估者偏好动态的标准协议
EPC: A Standardized Protocol for Measuring Evaluator Preference Dynamics in LLM Agent Systems
AI总结：
提出EPC协议，通过四阶段隔离范式标准化测量LLM智能体系统中评估者偏好耦合现象，并提供参考快照和版本约定以支持复现、比较和衰减检测。
摘要：
当LLM智能体在闭环中使用评估者反馈来调整其行为时，评估者偏差会通过智能体的策略分布传播——这种现象称为评估者偏好耦合。先前的工作已经记录了跨多个评估者家族和模型版本的耦合，但该领域缺乏一种标准化的协议，使第三方研究人员能够（i）复现耦合测量，（ii）跨评估者和时间点比较结果，以及（iii）在专有评估者静默更新时检测测量衰减。本文提供了该协议。我们详细说明了EPC（评估者偏好耦合）——一种详细的、RFC风格的协议规范，用于四阶段隔离范式，涵盖执行者和评估者配置、策略和任务设计、TTRL更新规则、指标计算（gamma、JSD、ECE、Brier）以及输出模式。我们随协议附带了一个版本化的参考快照v1.0：来自五项独立研究的八种评估者条件（N=122个独特的实验重复，涵盖GPT-4o、Qwen、DeepSeek等）的耦合测量，并附有评估者版本标识符、API端点和测量日期。该快照明确有时间限制：所有值都依赖于特定模型版本，并预计随着专有评估者更新而衰减。我们定义了一个版本约定（vX.Y-Z，编码协议版本、快照版本和评估者代次），并提供了涵盖采用、解释和已知陷阱的使用指南。该协议、参考快照和实现代码作为开放基础设施发布。
英文摘要：
When LLM agents use evaluator feedback to adapt their behavior in closed loops, evaluator biases propagate through the agent's strategy distribution -- a phenomenon known as evaluator preference coupling. Prior work has documented coupling across multiple evaluator families and model versions, but the field lacks a standardized protocol that enables third-party researchers to (i) reproduce coupling measurements, (ii) compare results across evaluators and time points, and (iii) detect measurement decay as proprietary evaluators silently update. This paper provides the protocol. We specify EPC (Evaluator Preference Coupling) -- a detailed, RFC-style protocol specification for the four-phase isolation paradigm, covering executor and evaluator configuration, strategy and task design, the TTRL update rule, metric computation (gamma, JSD, ECE, Brier), and output schema. We accompany the protocol with a versioned Reference Snapshot v1.0: coupling measurements for eight evaluator conditions (N=122 unique experimental repetitions across GPT-4o, Qwen, DeepSeek, and others) derived from five independent studies, annotated with evaluator version identifiers, API endpoints, and measurement dates. The snapshot is explicitly time-bound: all values are conditional on specific model versions and are expected to decay as proprietary evaluators update. We define a versioning convention (vX.Y-Z, encoding protocol version, snapshot version, and evaluator generation) and provide a usage guide covering adoption, interpretation, and known pitfalls. The protocol, reference snapshot, and implementation code are released as open infrastructure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
