Performing a large volume of experiments in Chemistry labs creates repetitive actions costing researchers time, automating these routines is highly desirable. Previous experiments in robotic chemistry have performed high numbers of experiments autonomously, however, these processes rely on automated machines in all stages from solid or liquid addition to analysis of the final product. In these systems every transition between machine requires the robotic chemist to pick and place glass vials, however, this is currently performed using open loop methods which require all equipment being used by the robot to be in well defined known locations. We seek to begin closing the loop in this vial handling process in a way which also fosters human-robot collaboration in the chemistry lab environment. To do this the robot must be able to detect valid placement positions for the vials it is collecting, and reliably insert them into the detected locations. We create a single modality visual method for estimating placement locations to provide a baseline before introducing two additional methods of feedback (force and tactile feedback). Our visual method uses a combination of classic computer vision methods and a CNN discriminator to detect possible insertion points, then a vial is grasped and positioned above an insertion point and the multi-modal methods guide the final insertion movements using an efficient search pattern. Through our experiments we show the baseline insertion rate of 48.78% improves to 89.55% with the addition of our "force and vision" multi-modal feedback method.
VTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);Samsung R&D Institute China–Beijing(三星中国北京研发中心);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
Needle Biopsy And Fiber-Optic Compatible Robotic Insertion Platform
Department of Mechatronics and Robotics(机械电子与机器人系);Department of Mechanical Science and Engineering(机械科学与工程系);Beckman Institute for Advanced Science and Technology(先进科学与技术研究所);Division of Research and Economic Development(研究与经济开发部门)
Multi-Modal Manipulation via Multi-Modal Policy Consensus
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);Massachusetts Institute of Technology(麻省理工学院);Harvard University(哈佛大学)
Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Shenzhen Taobotics Co., Ltd.(深圳桃机器人有限公司);Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院(TeleAI))