Model-free vision-based shaping of deformable plastic materials
ARC Centre of Excellence for Robotic Vision, Queensland University of Technology（昆士兰科技大学澳大利亚研究委员会机器人视觉卓越中心）；ARC Centre of Excellence for Robotic Vision, Monash University（莫纳什大学澳大利亚研究委员会机器人视觉卓越中心）
Andrea Cherubini, Valerio Ortenzi, Akansel Cosgun, Robert Lee, Peter Corke
备注：
Accepted to The International Journal of Robotics Research (IJRR)
We address the problem of shaping deformable plastic materials using non-prehensile actions. Shaping plastic objects is challenging, since they are difficult to model and to track visually. We study this problem, by using kinetic sand, a plastic toy material which mimics the physical properties of wet sand. Inspired by a pilot study where humans shape kinetic sand, we define two types of actions: \textit{pushing} the material from the sides and \textit{tapping} from above. The chosen actions are executed with a robotic arm using image-based visual servoing. From the current and desired view of the material, we define states based on visual features such as the outer contour shape and the pixel luminosity values. These are mapped to actions, which are repeated iteratively to reduce the image error until convergence is reached. For pushing, we propose three methods for mapping the visual state to an action. These include heuristic methods and a neural network, trained from human actions. We show that it is possible to obtain simple shapes with the kinetic sand, without explicitly modeling the material. Our approach is limited in the types of shapes it can achieve. A richer set of action types and multi-step reasoning is needed to achieve more sophisticated shapes.
Deformable solid objects such as clay or dough are prevalent in industrial and home environments. However, robotic manipulation of such objects has largely remained unexplored in literature due to the high complexity involved in representing and modeling their deformation. This work addresses the problem of shaping elasto-plastic dough by proposing to use a novel elastic end-effector to roll dough in a reinforcement learning framework. The transition model for the end-effector-to-dough interactions is learned from one hour of robot exploration, and doughs of different hydration levels are roll…
This paper addresses the problem of contact-based manipulation of deformable linear objects (DLOs) towards desired shapes with a dual-arm robotic system. To alleviate the burden of high-dimensional continuous state-action spaces, we model the DLO as a kinematic multibody system via our proposed keypoint detection network. This new perception network is trained on a synthetic labeled image dataset and transferred to real manipulation scenarios without conducting any manual annotations. Our goal-conditioned policy can efficiently learn to rearrange the configuration of the DLO based on the detec…
Soft Contact Simulation and Manipulation Learning of Deformable Objects with Vision-based Tactile Sensor
School of Mechanical Engineering, Anhui University of Technology（安徽工业大学机械工程学院）；School of Engineering and Technology, China University of Geosciences (Beijing)（中国地质大学（北京）工程技术学院）
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）
In this paper, we presented a new method for deformation control of deformable objects, which utilizes both visual and tactile feedback. At present, manipulation of deformable objects is basically formulated by assuming positional constraints. But in fact, in many situations manipulation has to be performed under actively applied force constraints. This scenario is considered in this research. In the proposed scheme a tactile feedback is integrated to ensure a stable contact between the robot end-effector and the soft object to be manipulated. The controlled contact force is also utilized to r…
In this paper, a B-spline chained multiple random matrices representation is proposed to model geometric characteristics of an elongated deformable object. The hyper degrees of freedom structure of the elongated deformable object make its shape estimation challenging. Based on the likelihood function of the proposed model, an expectation-maximization (EM) method is derived to estimate the shape of the elongated deformable object. A split and merge method based on the Euclidean minimum spanning tree (EMST) is proposed to provide initialization for the EM algorithm. The proposed algorithm is eva…
Robotic manipulation of deformable objects gains great attention due to its wide applications including medical surgery, home assistance, and automatic food preparation. The ability to deform soft objects remains a great challenge for robots due to difficulties in defining the problem mathematically. In this paper, we address the problem of shaping a piece of dough-like deformable material into a 2D target shape presented upfront. We use a 6 degree-of-freedom WidowX-250 Robot Arm equipped with a rolling pin and information collected from an RGB-D camera and a tactile sensor. We present and com…
Autonomy for robots interacting with sand will enable a wide range of beneficial behaviors, from earth moving for construction and farming vehicles to navigating rough terrain for Mars rovers. The goal of this work is to shape sand into desired forms. Unlike other common autonomous tasks of achieving desired state of a robot, achieving a desired shape of a continuously deformable environment like sand is a much more challenging task. The state of robot can be described with a couple of states-x, y, z, roll, pitch, yaw-but the desired shape of sand can not be described with just a few values. S…
Stanford University（斯坦福大学）；UC San Diego（加州大学圣迭戈分校）；MIT（麻省理工学院）
Modeling and manipulating elasto-plastic objects are essential capabilities for robots to perform complex industrial and household interaction tasks (e.g., stuffing dumplings, rolling sushi, and making pottery). However, due to the high degree of freedom of elasto-plastic objects, significant challenges exist in virtually every aspect of the robotic manipulation pipeline, e.g., representing the states, modeling the dynamics, and synthesizing the control signals. We propose to tackle these challenges by employing a particle-based representation for elasto-plastic objects in a model-based planni…
Lattice-based shape tracking and servoing of elastic objects
CNRS（法国国家科学研究中心）；Clermont Auvergne INP（克莱蒙奥弗涅国立应用科学学院）；Institut Pascal（帕斯卡研究所）；Université Clermont Auvergne（克莱蒙奥弗涅大学）；Instituto de Investigación en Ingeniería de Aragón (I3A)（阿拉贡工程研究所（I3A））；Universidad de Zaragoza（萨拉戈萨大学）
