Service robots are appearing more and more in our daily life. The development of service robots combines multiple fields of research, from object perception to object manipulation. The state-of-the-art continues to improve to make a proper coupling between object perception and manipulation. This coupling is necessary for service robots not only to perform various tasks in a reasonable amount of time but also to continually adapt to new environments and safely interact with non-expert human users. Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object in predefined settings. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, the knowledge of such robots is fixed after the training phase, and any changes in the environment require complicated, time-consuming, and expensive robot re-programming by human experts. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In such environments, no matter how extensive the training data used for batch learning, a robot will always face new objects. Therefore, apart from batch learning, the robot should be able to continually learn about new object categories and grasp affordances from very few training examples on-site. Moreover, apart from robot self-learning, non-expert users could interactively guide the process of experience acquisition by teaching new concepts, or by correcting insufficient or erroneous concepts. In this way, the robot will constantly learn how to help humans in everyday tasks by gaining more and more experiences without the need for re-programming.
CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigation
Purdue University（普渡大学）；Indian Institute of Technology, Kharagpur（印度理工学院卡哈拉格普尔分校）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）；National University of Singapore（新加坡国立大学）
Learning from Mistakes: Rollout-Retrieval Lifelong Policy Learning for Autonomous Driving
School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械工程学院）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航空航天工程学院）
Perception is one of the crucial module of the autonomous driving system, which has made great progress recently. However, limited ability of individual vehicles results in the bottleneck of improvement of the perception performance. To break through the limits of individual perception, collaborative perception has been proposed which enables vehicles to share information to perceive the environments beyond line-of-sight and field-of-view. In this paper, we provide a review of the related work about the promising collaborative perception technology, including introducing the fundamental concep…
The formation of eyes led to the big bang of evolution. The dynamics changed from a primitive organism waiting for the food to come into contact for eating food being sought after by visual sensors. The human eye is one of the most sophisticated developments of evolution, but it still has defects. Humans have evolved a biological perception algorithm capable of driving cars, operating machinery, piloting aircraft, and navigating ships over millions of years. Automating these capabilities for computers is critical for various applications, including self-driving cars, augmented reality, and arc…
Intel Labs（英特尔实验室）；City University of Hong Kong（香港城市大学）；Tsinghua University（清华大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Wuhan University（武汉大学）；University of Bologna（博洛尼亚大学）；Dublin City University（都柏林城市大学）；Imperial College London（帝国理工学院）
The recent breakthroughs in computer vision have benefited from the availability of large representative datasets (e.g. ImageNet and COCO) for training. Yet, robotic vision poses unique challenges for applying visual algorithms developed from these standard computer vision datasets due to their implicit assumption over non-varying distributions for a fixed set of tasks. Fully retraining models each time a new task becomes available is infeasible due to computational, storage and sometimes privacy issues, while naïve incremental strategies have been shown to suffer from catastrophic forgetting.…
The role of deep learning (DL) in robotics has significantly deepened over the last decade. Intelligent robotic systems today are highly connected systems that rely on DL for a variety of perception, control, and other tasks. At the same time, autonomous robots are being increasingly deployed as part of fleets, with collaboration among robots becoming a more relevant factor. From the perspective of collaborative learning, federated learning (FL) enables continuous training of models in a distributed, privacy-preserving way. This paper focuses on vision-based obstacle avoidance for mobile robot…
Rutgers, the State University of New Jersey（罗格斯，新泽西州立大学）
This work proposes a robotic pipeline for picking and constrained placement of objects without geometric shape priors. Compared to recent efforts developed for similar tasks, where every object was assumed to be novel, the proposed system recognizes previously manipulated objects and performs online model reconstruction and reuse. Over a lifelong manipulation process, the system keeps learning features of objects it has interacted with and updates their reconstructed models. Whenever an instance of a previously manipulated object reappears, the system aims to first recognize it and then regist…
WHALES: A Multi-Agent Scheduling Dataset for Enhanced Cooperation in Autonomous Driving
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（清华大学智能绿色车辆与移动性国家重点实验室）
Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Exploration
Robotics and Control Laboratory, School of Advanced Manufacturing and Robotics, and the State Key Laboratory of Turbulence and Complex Systems, Peking University（机器人与控制实验室、先进制造与机器人学院，以及湍流与复杂系统国家重点实验室，北京大学）；Defense Innovation Institute, Academy of Military Sciences, Beijing（国防科技创新研究院，军事科学院，北京）；Tianjin Artificial Intelligence Innovation Center, Tianjin（天津人工智能创新中心，天津）；Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen)（计算与智能学院，哈尔滨工业大学（深圳））
SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）
Latest research in industrial robotics is aimed at making human robot collaboration possible seamlessly. For this purpose, industrial robots are expected to work on the fly in unstructured and cluttered environments and hence the subject of perception driven motion planning plays a vital role. Sampling based motion planners are proven to be the most effective for such high dimensional planning problems with real time constraints. Unluckily random stochastic samplers suffer from the phenomenon of 'narrow passages' or bottleneck regions which need targeted sampling to improve their convergence r…
Sparse 3D Perception for Rose Harvesting Robots: A Two-Stage Approach Bridging Simulation and Real-World Applications
School of Computer Engineering, Iran University of Science and Technology（伊朗科学技术大学计算机工程学院）；George Mason University Department of Computer Science（乔治·梅奥大学计算机科学系）
Capstan-driven Continuum Surgical Robot: Design, Modeling, and Perception
The Chinese University of Hong Kong（香港中文大学）；School of Mechanical and Electronic Engineering, Wuhan University of Technology（武汉理工大学机电工程学院）；Huazhong University of Science and Technology（华中科技大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究院）；Multi-Scale Medical Robotics Center（多尺度医疗机器人中心）
G-MAPP: GPU-accelerated Multi-Agent Planning and Perception for Reactive Motion Generation
Department of Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University（多伦多都会大学电气、计算机与生物医学工程系）；Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM)（慕尼黑工业大学慕尼黑机器人与机器智能研究所）；Institute for Experiential Robotics, Northeastern University（东北大学体验式机器人研究所）；Idiap Research Institute（Idiap 研究所）；EPFL（瑞士联邦理工学院洛桑）；CHART Group at the School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院 CHART 小组）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）
NEC Corporation（日本电气股份有限公司）；Stanford University（斯坦福大学）；UC San Diego（加州大学圣迭戈分校）
Today's robotic fleets are increasingly measuring high-volume video and LIDAR sensory streams, which can be mined for valuable training data, such as rare scenes of road construction sites, to steadily improve robotic perception models. However, re-training perception models on growing volumes of rich sensory data in central compute servers (or the "cloud") places an enormous time and cost burden on network transfer, cloud storage, human annotation, and cloud computing resources. Hence, we introduce HarvestNet, an intelligent sampling algorithm that resides on-board a robot and reduces system …
