arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.07941cs.ROcs.AIcs.LG
收藏

基于学习奖励函数的挖掘机操作员自动评价

Automatic Evaluation of Excavator Operators using Learned Reward Functions

École de Technologie Supérieure(高等技术学院);Mila(米拉研究所);CM Labs(CM实验室);CIFAR(加拿大高等研究院)

Pranav Agarwal, Marek Teichmann, Sheldon Andrews, Samira Ebrahimi Kahou

摘要:

培训新手用户操作挖掘机以学习不同的技能需要专家教师的存在。考虑到问题的复杂性,寻找熟练的专家相对昂贵,因为这个过程很耗时,需要精确的重点。此外,由于人类往往有偏见,评估过程是嘈杂的,并会导致具有类似技能的不同操作员的最终得分差异很大。在这项工作中,我们解决这些问题,并提出了一种新的策略,挖掘机操作员的自动评价。我们考虑到挖掘机的内部动力学和安全标准,在每个时间步骤来评估性能。为了进一步验证我们的方法,我们使用这个分数预测模型作为强化学习代理的奖励来源,以学习在模拟环境中操纵挖掘机的任务,该模拟环境密切复制了现实世界的动态。对于使用这些外部奖励预测模型学习的策略,我们的结果表明,与仅使用基于任务的奖励函数训练的策略相比,遵循所需的动态约束的解决方案更安全,使其更接近现实生活中的采用。对于未来的研究,我们将在https://github.com/pranavAL/InvRL_Auto-Evaluate上发布我们的代码库,并在https://drive.google.com/file/d/1jR1otOAu8zrY8mkhUOUZW9jkBOAKK71Z/view?上发布视频结果。usp=share_link .

英文摘要:

Training novice users to operate an excavator for learning different skills requires the presence of expert teachers. Considering the complexity of the problem, it is comparatively expensive to find skilled experts as the process is time-consuming and requires precise focus. Moreover, since humans tend to be biased, the evaluation process is noisy and will lead to high variance in the final score of different operators with similar skills. In this work, we address these issues and propose a novel strategy for the automatic evaluation of excavator operators. We take into account the internal dynamics of the excavator and the safety criterion at every time step to evaluate the performance. To further validate our approach, we use this score prediction model as a source of reward for a reinforcement learning agent to learn the task of maneuvering an excavator in a simulated environment that closely replicates the real-world dynamics. For a policy learned using these external reward prediction models, our results demonstrate safer solutions following the required dynamic constraints when compared to policy trained with task-based reward functions only, making it one step closer to real-life adoption. For future research, we release our codebase at https://github.com/pranavAL/InvRL_Auto-Evaluate and video results https://drive.google.com/file/d/1jR1otOAu8zrY8mkhUOUZW9jkBOAKK71Z/view?usp=share_link .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑