Reinforcement learning methods have been developed to achieve great success in training control policies in various automation tasks. However, a main challenge of the wider application of reinforcement learning in practical automation is that the training process is hard and the pretrained policy networks are hardly reusable in other similar cases. To address this problem, we propose the cascade attribute network (CAN), which utilizes its hierarchical structure to decompose a complicated control policy in terms of the requirement constraints, which we call attributes, encoded in the control tasks. We validated the effectiveness of our proposed method on two robot control scenarios with various add-on attributes. For some control tasks with more than one add-on attribute attribute, by directly assembling the attribute modules in cascade, the CAN can provide ideal control policies in a zero-shot manner.
Georgia Institute of Technology（佐治亚理工学院）；Google（谷歌）
Real-world autonomous missions often require rich interaction with nearby objects, such as doors or switches, along with effective navigation. However, such complex behaviors are difficult to learn because they involve both high-level planning and low-level motor control. We present a novel framework, Cascaded Compositional Residual Learning (CCRL), which learns composite skills by recursively leveraging a library of previously learned control policies. Our framework learns multiplicative policy composition, task-specific residual actions, and synthetic goal information simultaneously while fr…
Policy learning for partially observed control tasks requires policies that can remember salient information from past observations. In this paper, we present a method for learning policies with internal memory for high-dimensional, continuous systems, such as robotic manipulators. Our approach consists of augmenting the state and action space of the system with continuous-valued memory states that the policy can read from and write to. Learning general-purpose policies with this type of memory representation directly is difficult, because the policy must automatically figure out the most sali…
Deep reinforcement learning has been applied to solve a variety of control problems in isolation. However, the learned latent representations cannot be optimally reused for other analogous tasks and/or control systems without additional training or tuning. In this regard, we propose a novel framework that can be used to learn a single control policy for a whole class of analogous control systems. The framework is abbreviated as CASNET and it leverages the similarities in the designs of analogous control-systems to learn general-purpose abstract system-representations. The framework uses a casc…
Aalto University（阿尔托大学）；Örebro University（厄勒布鲁大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
Policy search reinforcement learning allows robots to acquire skills by themselves. However, the learning procedure is inherently unsafe as the robot has no a-priori way to predict the consequences of the exploratory actions it takes. Therefore, exploration can lead to collisions with the potential to harm the robot and/or the environment. In this work we address the safety aspect by constraining the exploration to happen in safe-to-explore state spaces. These are formed by decomposing target skills (e.g., grasping) into higher ranked sub-tasks (e.g., collision avoidance, joint limit avoidance…
Policy Decomposition (PoDec) is a framework that lessens the curse of dimensionality when deriving policies to optimal control problems. For a given system representation, i.e. the state variables and control inputs describing a system, PoDec generates strategies to decompose the joint optimization of policies for all control inputs. Thereby, policies for different inputs are derived in a decoupled or cascaded fashion and as functions of some subsets of the state variables, leading to reduction in computation. However, the choice of system representation is crucial as it dictates the suboptima…
EmbodiedSkills: World Action Model-augmented Cascaded Skills for Vision-Language-Action Agent
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Cornell University（康奈尔大学）；Universal Ubiquitous AI Co., Ltd.（Universal Ubiquitous AI有限公司）；Hangzhou DEEP Robotics Technology Co., Ltd.（杭州深度机器人科技有限公司）；National University of Singapore（新加坡国立大学）
BiKC+: Bimanual Hierarchical Imitation with Keypose-Conditioned Coordination-Aware Consistency Policies
Centre for Transformative Garment Production, The University of Hong Kong（变革性服装生产中心，香港大学）；School of Computing and Data Science, The University of Hong Kong（计算与数据科学学院，香港大学）；Advanced Manufacturing Lab, Huawei Technologies（先进制造实验室，华为技术）
Fundation Model Research Center, Institute of Automation, Chinese Academy of Sciences（基础模型研究中心，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究院（AIR），清华大学）；The Chinese University of Hong Kong（香港中文大学）；Shanghai Jiao Tong University（上海交通大学）；Peking University（北京大学）；Xiaomi EV（小米电动车）
Many real-world applications can be formulated as multi-agent cooperation problems, such as network packet routing and coordination of autonomous vehicles. The emergence of deep reinforcement learning (DRL) provides a promising approach for multi-agent cooperation through the interaction of the agents and environments. However, traditional DRL solutions suffer from the high dimensions of multiple agents with continuous action space during policy search. Besides, the dynamicity of agents' policies makes the training non-stationary. To tackle the issues, we propose a hierarchical reinforcement l…
