arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1610.00850cs.ROcs.LG
收藏

从演示中比较以人为中心和以机器人为中心的机器人深度学习采样

Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations

University of California, Berkeley(加州大学伯克利分校)

Michael Laskey, Caleb Chuck, Jonathan Lee, Jeffrey Mahler, Sanjay Krishnan, Kevin Jamieson, Anca Dragan, Ken Goldberg

摘要:

受机器人控制深度学习最新进展的启发,本文考虑了两种学习算法如何获取演示。“以人为中心”(HC)采样是标准的监督学习算法,其中人类监督者通过远程操作机器人来提供由状态控制对组成的轨迹来演示任务。“以机器人为中心”(RC)采样是Dagger等算法中越来越流行的替代方案,其中人类主管观察机器人执行学习的策略,并为访问的每个状态提供纠正控制标签。RC采样对于人类监督员来说可能具有挑战性,并且容易出现错误标记。RC采样也可能导致策略性能的错误,因为它重复访问状态空间中难以学习的区域。虽然对于标准学习模型(例如线性SVM),使用RC采样学习的策略可能优于HC采样,但使用HC采样学习的策略可能与具有很小模型误差的高表达性学习模型(例如深度学习和超参数决策树)相当。我们比较HC和RC使用网格世界和物理机器人分离任务,其中在后者的输入是一个二进制图像的一组连接的对象在一个平面的工作表面和政策产生的运动的抓手,以分离一个对象从其余的。我们在模拟中观察到,对于线性SVM,使用RC学习的策略优于使用HC学习的策略,但对于深度模型,这种优势消失了。我们还发现,与RC,由人类提供的纠正控制标签可以是高度不一致的。我们证明了存在一类例子,在极限,HC是保证收敛到一个最优的政策,而RC可能无法收敛。

英文摘要:

Motivated by recent advances in Deep Learning for robot control, this paper considers two learning algorithms in terms of how they acquire demonstrations. "Human-Centric" (HC) sampling is the standard supervised learning algorithm, where a human supervisor demonstrates the task by teleoperating the robot to provide trajectories consisting of state-control pairs. "Robot-Centric" (RC) sampling is an increasingly popular alternative used in algorithms such as DAgger, where a human supervisor observes the robot executing a learned policy and provides corrective control labels for each state visited. RC sampling can be challenging for human supervisors and prone to mislabeling. RC sampling can also induce error in policy performance because it repeatedly visits areas of the state space that are harder to learn. Although policies learned with RC sampling can be superior to HC sampling for standard learning models such as linear SVMs, policies learned with HC sampling may be comparable with highly-expressive learning models such as deep learning and hyper-parametric decision trees, which have little model error. We compare HC and RC using a grid world and a physical robot singulation task, where in the latter the input is a binary image of a connected set of objects on a planar worksurface and the policy generates a motion of the gripper to separate one object from the rest. We observe in simulation that for linear SVMs, policies learned with RC outperformed those learned with HC but that with deep models this advantage disappears. We also find that with RC, the corrective control labels provided by humans can be highly inconsistent. We prove there exists a class of examples where in the limit, HC is guaranteed to converge to an optimal policy while RC may fail to converge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑