审计示范策展指标：仅动作评分器在降低模仿策略的结构缺陷上失败
Auditing Demonstration Curation Metrics: Action-Only Scorers Fail on the Structural Defects That Degrade Imitation Policies
AI总结：
本研究构建受控测试平台，注入两类示范缺陷（细微扰动和结构错误），审计七种策展指标，发现仅动作指标无法检测结构错误，且部分指标评分倒置，而状态轨迹指标能部分检测但下游性能恢复有限。
摘要：
模仿学习策略继承了其训练示范的质量，越来越多的策展指标声称能自动评分和过滤低质量示范。这些指标各自在不同协议的不同数据上验证，因此不清楚哪些指标真正识别出损害策略的示范。我们构建了一个受控测试平台，其中示范缺陷以已知类型注入，并沿两个轴审计七种策展指标：每个指标区分缺陷示范与清洁示范的效果，以及基于每个指标策展的子集训练行为克隆策略是否提高任务成功率。我们研究两种缺陷机制。细微扰动（相关动作噪声、震颤、截断）可通过多变量离群值评分检测，一旦移除，可恢复全部下游差距。结构错误，即示范在关键时刻执行错误动作，对我们测试的每个仅动作指标都是不可见的，其中两个指标是倒置的：它们将缺陷示范评分为更高质量，并用于策展时，往往使策略处于或低于未策展基线，而非高于基线。只有检查状态轨迹的指标能检测结构错误，即使最好的指标也只能恢复三分之一的下游差距。高检测准确性并不保证下游改进。我们发布了测试平台和所有策展实现。
英文摘要：
Imitation-learning policies inherit the quality of the demonstrations they are trained on, and a growing set of curation metrics promise to score and filter low-quality demonstrations automatically. These metrics are each validated on different data with different protocols, so it is unclear which of them actually identify the demonstrations that harm a policy. We build a controlled testbed in which demonstration defects are injected with known type, and audit seven curation metrics along two axes: how well each separates defective from clean demonstrations, and whether training a behavior-cloning policy on each metric's curated subset improves task success. We study two defect regimes. Subtle perturbations (correlated action noise, tremor, truncation) are detectable by multivariate outlier scoring and, once removed, recover the full downstream gap. Structural errors, where the demonstration executes a wrong action at a key moment, are invisible to every action-only metric we test, and two of them are inverted: they score defective demonstrations as higher quality and, used for curation, tend to leave the policy at or below the uncurated baseline rather than above it. Only metrics that examine the state trajectory detect structural errors, and even the best of them recovers just a third of the downstream gap. High detection accuracy does not guarantee downstream improvement. We release the testbed and all curation implementations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
