Unsupervised robotic sorting: Towards autonomous decision making robots
Arts et Métiers ParisTech（巴黎高等工程技术学院）
Joris Guérin, Stéphane Thiery, Eric Nyiri, Olivier Gibaru
备注：
Paper published in International Journal of Artificial Intelligence and Applications (IJAIA), March 2018, Volume 9, Number 2 17 pages, 5 figures, 7 tables. arXiv admin note: text overlap with arXiv:1707.01700
发表信息：
International Journal of Artificial Intelligence and Applications (IJAIA), March 2018, Volume 9, Number 2
Autonomous sorting is a crucial task in industrial robotics which can be very challenging depending on the expected amount of automation. Usually, to decide where to sort an object, the system needs to solve either an instance retrieval (known object) or a supervised classification (predefined set of classes) problem. In this paper, we introduce a new decision making module, where the robotic system chooses how to sort the objects in an unsupervised way. We call this problem Unsupervised Robotic Sorting (URS) and propose an implementation on an industrial robotic system, using deep CNN feature extraction and standard clustering algorithms. We carry out extensive experiments on various standard datasets to demonstrate the efficiency of the proposed image clustering pipeline. To evaluate the robustness of our URS implementation, we also introduce a complex real world dataset containing images of objects under various background and lighting conditions. This dataset is used to fine tune the design choices (CNN and clustering algorithm) for URS. Finally, we propose a method combining our pipeline with ensemble clustering to use multiple images of each object. This redundancy of information about the objects is shown to increase the clustering results.
Anomaly detection in Minimally-Invasive Surgery (MIS) traditionally requires a human expert monitoring the procedure from a console. Data scarcity, on the other hand, hinders what would be a desirable migration towards autonomous robotic-assisted surgical systems. Automated anomaly detection systems in this area typically rely on classical supervised learning. Anomalous events in a surgical setting, however, are rare, making it difficult to capture data to train a detection model in a supervised fashion. In this work we thus propose an unsupervised approach to anomaly detection for robotic-ass…
University of Nevada, Reno（内华达大学雷诺分校）；Swansea University（斯旺西大学）
Robot manipulation and grasping mechanisms have received considerable attention in the recent past, leading to the development of wide range of industrial applications. This paper proposes the development of an autonomous robotic grasping system for object sorting application. RGB-D data is used by the robot for performing object detection, pose estimation, trajectory generation, and object sorting tasks. The proposed approach can also handle grasping certain objects chosen by users. Trained convolutional neural networks are used to perform object detection and determine the corresponding poin…
Mixed integer convex and nonlinear programs, MICP and MINLP, are expressive but require long solving times. Recent work that combines learning methods on solver heuristics has shown potential to overcome this issue allowing for applications on larger scale practical problems. Gathering sufficient training data to employ these methods still present a challenge since getting data from traditional solvers are slow and newer learning approaches still require large amounts of data. In order to scale up and make these hybrid learning approaches more manageable we propose ReDUCE, a method that exploi…
University of Cambridge（剑桥大学）；Universität Bremen（不来梅大学）
Deep learning approaches have become the standard solution to many problems in computer vision and robotics, but obtaining sufficient training data in high enough quality is challenging, as human labor is error prone, time consuming, and expensive. Solutions based on simulation have become more popular in recent years, but the gap between simulation and reality is still a major issue. In this paper, we introduce a novel method for augmenting synthetic image data through unsupervised image-to-image translation by applying the style of real world images to simulated images with open source frame…
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）
Owing to human labor shortages, the automation of labor-intensive manual waste-sorting is needed. The goal of automating waste-sorting is to replace the human role of robust detection and agile manipulation of waste items with robots. To achieve this, we propose three methods. First, we provide a combined manipulation method using graspless push-and-drop and pick-and-release manipulation. Second, we provide a robotic system that can automatically collect object images to quickly train a deep neural-network model. Third, we provide a method to mitigate the differences in the appearance of targe…
Monocular depth estimation plays a critical role in various computer vision and robotics applications such as localization, mapping, and 3D object detection. Recently, learning-based algorithms achieve huge success in depth estimation by training models with a large amount of data in a supervised manner. However, it is challenging to acquire dense ground truth depth labels for supervised training, and the unsupervised depth estimation using monocular sequences emerges as a promising alternative. Unfortunately, most studies on unsupervised depth estimation explore loss functions or occlusion ma…
Naval Research Laboratory（海军研究实验室）；California Polytechnic State University, San Luis Obispo（加州州立理工大学圣路易斯奥比斯波分校）
We introduce a convolutional neural network model for unsupervised learning of depth and ego-motion from cylindrical panoramic video. Panoramic depth estimation is an important technology for applications such as virtual reality, 3D modeling, and autonomous robotic navigation. In contrast to previous approaches for applying convolutional neural networks to panoramic imagery, we use the cylindrical panoramic projection which allows for the use of the traditional CNN layers such as convolutional filters and max pooling without modification. Our evaluation of synthetic and real data shows that un…
Learning an accurate model of the environment is essential for model-based control tasks. Existing methods in robotic visuomotor control usually learn from data with heavily labelled actions, object entities or locations, which can be demanding in many cases. To cope with this limitation, we propose a method, dubbed DMotion, that trains a forward model from video data only, via disentangling the motion of controllable agent to model the transition dynamics. An object extractor and an interaction learner are trained in an end-to-end manner without supervision. The agent's motions are explicitly…
SurgT challenge: Benchmark of Soft-Tissue Trackers for Robotic Surgery
Imperial College London（伦敦帝国理工学院）；Jmees（Jmees公司）；Electronics and Telecommunications Research Institute (ETRI)（电子通信研究院）；Ajou University（亚洲大学）；University of Toronto（多伦多大学）；University College London（伦敦大学学院）
VitaTouch: Property-Aware Vision-Tactile-Language Model for Robotic Quality Inspection in Manufacturing
School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京邮电大学智能工程与自动化学院）；Zhongguancun Academy（中关村学院）；School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）；School of AI, Beijing Institute of Technology（北京理工大学人工智能学院）；Beijing Key Laboratory of Lightweight Intelligent System, Beijing Institute of Technology（北京理工大学北京市轻量化智能系统重点实验室）
First Lessons Learned of an Artificial Intelligence Robotic System for Autonomous Coarse Waste Recycling Using Multispectral Imaging-Based Methods
Hamburg University of Applied Sciences（汉堡应用技术大学）；University of the West of Scotland（西苏格兰大学）；Institute for Energy, Recycling and Environmental Protection at Bremen University of Applied Sciences（不来梅应用技术大学能源、回收与环境保护研究所）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
This document is an introduction to designing a multiple degree of freedom robotic manipulator. The goal of the robot is to sort an object based upon the objects color. The robot will be a synthesis of several linkages, servo motors, an Arduino system, and an end-effector. The design must be able to autonomously determine what object to move and where to place it. The robot will have a maximum reach of 24 horizontally. The operating conditions include being able to sort several different colored spheres within the area of a 180° up to 16 from the base of the arm. Sorting consists of picking up…
In this paper, we explore the problem of interesting scene prediction for mobile robots. This area is currently underexplored but is crucial for many practical applications such as autonomous exploration and decision making. Inspired by industrial demands, we first propose a novel translation-invariant visual memory for recalling and identifying interesting scenes, then design a three-stage architecture of long-term, short-term, and online learning. This enables our system to learn human-like experience, environmental knowledge, and online adaption, respectively. Our approach achieves much hig…
RAISE: A Robot-Assisted Selective Disassembly and Sorting System for End-of-Life Phones
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University（J. Mike Walker ’66 机械工程系，德克萨斯A&M大学）；College of Engineering, University of Georgia（工程学院，佐治亚大学）；Idaho National Laboratory（爱达荷国家实验室）；Sunnking Sustainable Solutions（Sunnking可持续解决方案公司）；Zachry Department of Civil and Environmental Engineering, Texas A&M University（扎克里土木与环境工程系，德克萨斯A&M大学）
Digital Twin Driven Textile Classification and Foreign Object Recognition in Automated Sorting Systems
Institute of Smart Systems Technologies（智能系统技术研究所）；University of Klagenfurt（克雷格弗特大学）；AAU SAL USE Laboratory（AAU SAL USE实验室）；Silicon Austria Labs（硅 Austria 实验室）
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）
Autonomous robots frequently need to detect "interesting" scenes to decide on further exploration, or to decide which data to share for cooperation. These scenarios often require fast deployment with little or no training data. Prior work considers "interestingness" based on data from the same distribution. Instead, we propose to develop a method that automatically adapts online to the environment to report interesting scenes quickly. To address this problem, we develop a novel translation-invariant visual memory and design a three-stage architecture for long-term, short-term, and online learn…
Express companies are deploying more robotic sorting systems, where mobile robots are used to sort incoming parcels by destination. In this study, we propose an integrated assignment and path-finding method for robots in such sorting systems. The method has two parts: offline and online. In the offline part, we represent the system as a traffic flow network, develop an approximate delay function using stochastic models, and solve the min-cost network flow problem. In the online part, robots are guided through the system according to the calculated optimal flow split probability. The online cal…
Google（谷歌）；Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）；Stanford University（斯坦福大学）；UC Berkeley（加州大学伯克利分校）
We consider the problem of learning useful robotic skills from previously collected offline data without access to manually specified rewards or additional online exploration, a setting that is becoming increasingly important for scaling robot learning by reusing past robotic data. In particular, we propose the objective of learning a functional understanding of the environment by learning to reach any goal state in a given dataset. We employ goal-conditioned Q-learning with hindsight relabeling and develop several techniques that enable training in a particularly challenging offline setting. …
ROBOCYCLE: Autonomous Dual-Arm Robotic Manipulation and Coordination for Recycling Applications
Tecnologico de Monterrey, School of Engineering and Sciences, Department of Computer Science（墨西哥蒙特雷理工大学工程学院与科学学院计算机科学系）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）
Massachusetts Institute of Technology（麻省理工学院）；Vicarious AI（维卡里亚人工智能公司）
Visual servoing enables robotic systems to perform accurate closed-loop control, which is required in many applications. However, existing methods either require precise calibration of the robot kinematic model and cameras or use neural architectures that require large amounts of data to train. In this work, we present a method for unsupervised learning of visual servoing that does not require any prior calibration and is extremely data-efficient. Our key insight is that visual servoing does not depend on identifying the veridical kinematic and camera parameters, but instead only on an accurat…
Current reinforcement learning (RL) in robotics often experiences difficulty in generalizing to new downstream tasks due to the innate task-specific training paradigm. To alleviate it, unsupervised RL, a framework that pre-trains the agent in a task-agnostic manner without access to the task-specific reward, leverages active exploration for distilling diverse experience into essential skills or reusable knowledge. For exploiting such benefits also in robotic manipulation, we propose an unsupervised method for transferable manipulation skill discovery that ties structured exploration toward int…
Interestingness recognition is crucial for decision making in autonomous exploration for mobile robots. Previous methods proposed an unsupervised online learning approach that can adapt to environments and detect interesting scenes quickly, but lack the ability to adapt to human-informed interesting objects. To solve this problem, we introduce a human-interactive framework, AirInteraction, that can detect human-informed objects via few-shot online learning. To reduce the communication bandwidth, we first apply an online unsupervised learning algorithm on the unmanned vehicle for interestingnes…
School of Aerospace, Mechanical and Mechatronic Engineering, The University of Sydney（悉尼大学航空航天、机械与机电工程学院）；Sydney Institute for Robotics and Intelligent Systems（悉尼机器人与智能系统研究院）
While an exciting diversity of new imaging devices is emerging that could dramatically improve robotic perception, the challenges of calibrating and interpreting these cameras have limited their uptake in the robotics community. In this work we generalise techniques from unsupervised learning to allow a robot to autonomously interpret new kinds of cameras. We consider emerging sparse light field (LF) cameras, which capture a subset of the 4D LF function describing the set of light rays passing through a plane. We introduce a generalised encoding of sparse LFs that allows unsupervised learning …
Georgia Tech（佐治亚理工学院）；Penn State University（宾夕法尼亚州立大学）
This paper describes current progress on developing an ethical architecture for robots that are designed to follow human ethical decision-making processes. We surveyed both regular adults (folks) and ethics experts (experts) on what they consider to be ethical behavior in two specific scenarios: pill-sorting with an older adult and game playing with a child. A key goal of the surveys is to better understand human ethical decision-making. In the first survey, folk responses were based on the subject's ethical choices ("folk morality"); in the second survey, expert responses were based on the ex…
In three laboratory experiments, we examine the impact of personally relevant failures (PeRFs) on perceptions of a collaborative robot. PeR is determined by how much a specific issue applies to a particular person, i.e., it affects one's own goals and values. We hypothesized that PeRFs would reduce trust in the robot and the robot's Likeability and Willingness to Use (LWtU) more than failures that are not personal to participants. To achieve PeR in human-robot interaction, we utilized three different manipulation mechanisms: A) damage to property, B) financial loss, and C) first-person versus …
Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation
NAVER LABS Europe（NAVER欧洲实验室）；TU Darmstadt（达姆施塔特工业大学）；EPITA Research Laboratory (LRE)（EPITA研究实验室）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Center for Cognitive Science, TU Darmstadt（达姆施塔特工业大学认知科学中心）；Hessian Center for Artificial Intelligence (Hessian.ai)（黑森州人工智能中心）
Streaming Scene Maps for Co-Robotic Exploration in Bandwidth Limited Environments
Applied Ocean Physics and Engineering Department at the Woods Hole Oceanographic Institution (WHOI)（伍兹霍尔海洋研究所（WHOI）应用海洋物理与工程系）；MIT Media Lab（麻省理工学院媒体实验室）；MIT-WHOI Joint Program in Applied Ocean Science and Engineering（麻省理工学院-伍兹霍尔海洋研究所应用海洋科学与工程联合项目）；Mechanical Engineering Department at Northeastern University（东北大学机械工程系）
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）
Automated surgical gesture recognition is of great importance in robot-assisted minimally invasive surgery. However, existing methods assume that training and testing data are from the same domain, which suffers from severe performance degradation when a domain gap exists, such as the simulator and real robot. In this paper, we propose a novel unsupervised domain adaptation framework which can simultaneously transfer multi-modality knowledge, i.e., both kinematic and visual data, from simulator to real robot. It remedies the domain gap with enhanced transferable features by using temporal cues…
