arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.06227cs.ROcs.LG
收藏

面向未知任务中触觉共享控制的深度强化学习

Deep Reinforcement Learning for Haptic Shared Control in Unknown Tasks

Pontificia Universidade Católica do Rio de Janeiro(里约热内卢天主教大学)

Franklin Cardeñoso Fernandez, Wouter Caarls

AI总结:

本文提出 TAHSC,将 DDPG 与 CNN 结合用于未知任务的触觉共享控制,依据用户反馈阻力学习个性化引导力策略,从而稳定辅助用户更快完成拾取与放置任务。

摘要:

近年来,在远程操作系统中使用触觉共享控制(HSC)受到越来越多的关注。在 HSC 中,虚拟引导力的施加减少了用户的控制负担,并改善了多种任务中的执行时间,与直接远程操作相比是一种良好的替代方案。尽管 HSC 已表现出良好性能,但它也带来了一个新的缺口:如何设计引导力。因此,挑战在于开发控制器,以为正在执行的任务提供最优引导力。本工作通过设计一个控制器来应对这一挑战:该控制器基于深度确定性策略梯度(DDPG)算法提供辅助,并使用卷积神经网络(CNN)执行任务检测,称为 TAHSC(Task Agnostic Haptic Shared Controller,任务无关触觉共享控制器)。智能体学习最小化人类执行期望任务所需的时间,同时最小化其对所提供反馈的抵抗。因此,这种抵抗为学习算法提供了有关人类正试图沿哪个方向运动的信息;在本研究中,该任务为拾取与放置任务。多项结果表明,所提方法通过为每位受邀测试该系统的用户学习定制化策略而得到成功应用。它表现出稳定收敛,并帮助用户以尽可能少的时间完成任务。

英文摘要:

Recent years have shown a growing interest in using haptic shared control (HSC) in teleoperated systems. In HSC, the application of virtual guiding forces decreases the user's control effort and improves execution time in various tasks, presenting a good alternative in comparison with direct teleoperation. HSC, despite demonstrating good performance, opens a new gap: how to design the guiding forces. For this reason, the challenge lies in developing controllers to provide the optimal guiding forces for the tasks that are being performed. This work addresses this challenge by designing a controller based on the deep deterministic policy gradient (DDPG) algorithm to provide the assistance, and a convolutional neural network (CNN) to perform the task detection, called TAHSC (Task Agnostic Haptic Shared Controller). The agent learns to minimize the time it takes the human to execute the desired task, while simultaneously minimizing their resistance to the provided feedback. This resistance thus provides the learning algorithm with information about which direction the human is trying to follow, in this case, the pick-and-place task. Diverse results demonstrate the successful application of the proposed approach by learning custom policies for each user who was asked to test the system. It exhibits stable convergence and aids the user in completing the task with the least amount of time possible.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑