arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.13616cs.ROstat.AP
收藏

超越二元成功:样本高效且统计严谨的机器人策略比较

Beyond Binary Success: Sample-Efficient and Statistically Rigorous Robot Policy Comparison

University of Pennsylvania(宾夕法尼亚大学);Toyota Research Institute (TRI)(丰田研究院);Princeton University(普林斯顿大学)

David Snyder, Apurva Badithela, Nikolai Matni, George Pappas, Anirudha Majumdar, Masha Itkina, Haruki Nishimura

AI总结:

本文提出一种样本高效且统计严谨的机器人策略比较框架,适用于多种评价指标,通过安全 anytime-valid 推断实现早期停止,减少评估负担并提升统计严谨性。

摘要:

通用机器人操作策略正变得越来越有能力,但其评估受限于少量硬件运行。这种现实测试中的强资源限制需要更 informative 的性能指标和可靠高效的评估程序。本文提出一种新颖的机器人策略比较框架,具有样本高效、统计严谨且适用于广泛评价指标。基于安全 anytime-valid 推断(SAVI),我们的测试程序是顺序的,允许评估者在足够统计证据积累时提前停止以达到预设置信水平。与以往针对二元成功的二元工作不同,我们的统一方法处理了广泛的 informative 指标:从离散部分信用任务进度到连续的 episodic 奖励或轨迹平滑度测量,涵盖参数和非参数比较问题。通过广泛的模拟和现实数据验证,我们展示了与标准批量方法相比评估负担减少高达 70%,与针对二元结果设计的最先进顺序程序相比减少高达 50%,且无统计严谨性损失。值得注意的是,我们的实证结果表明,使用细粒度任务进度而非二元成功指标时,竞争策略可以更快被区分。

英文摘要:

Generalist robot manipulation policies are becoming increasingly capable, but are limited in evaluation to a small number of hardware rollouts. This strong resource constraint in real-world testing necessitates both more informative performance measures and reliable and efficient evaluation procedures to properly assess model capabilities and benchmark progress in the field. This work presents a novel framework for robot policy comparison that is sample-efficient, statistically rigorous, and applicable to a broad set of evaluation metrics used in practice. Based on safe, anytime-valid inference (SAVI), our test procedure is sequential, allowing the evaluator to stop early when sufficient statistical evidence has accumulated to reach a decision at a pre-specified level of confidence. Unlike previous work developed for binary success, our unified approach addresses a wide range of informative metrics: from discrete partial credit task progress to continuous measures of episodic reward or trajectory smoothness, spanning both parametric and nonparametric comparison problems. Through extensive validation on simulated and real-world evaluation data, we demonstrate up to 70% reduction in evaluation burden compared to standard batch methods and up to 50% reduction compared to state-of-the-art sequential procedures designed for binary outcomes, with no loss of statistical rigor. Notably, our empirical results show that competing policies can be separated more quickly when using fine-grained task progress than binary success metrics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑