Mobile Manipulator for Autonomous Localization, Grasping and Precise Placement of Construction Material in a Semi-structured Environment
Czech Technical University in Prague（布拉格捷克理工大学）；New York University（纽约大学）；University of Pennsylvania（宾夕法尼亚大学）
Petr Stibinger, George Broughton, Filip Majer, Zdenek Rozsypalek, Anthony Wang, Kshitij Jindal, Alex Zhou, Dinesh Thakur, Giuseppe Loianno, Tomas Krajnik, Martin Saska
备注：
8 pages, 11 figures, submitted to IEEE ICRA and RA-L 2021, under review
Mobile manipulators have the potential to revolutionize modern agriculture, logistics and manufacturing. In this work, we present the design of a ground-based mobile manipulator for automated structure assembly. The proposed system is capable of autonomous localization, grasping, transportation and deployment of construction material in a semi-structured environment. Special effort was put into making the system invariant to lighting changes, and not reliant on external positioning systems. Therefore, the presented system is self-contained and capable of operating in outdoor and indoor conditions alike. Finally, we present means to extend the perceptive radius of the vehicle by using it in cooperation with an autonomous drone, which provides aerial reconnaissance. Performance of the proposed system has been evaluated in a series of experiments conducted in real-world conditions.
In this paper, we present the design, simulation and experimental validation of a control architecture for a flying hand, i.e., a system made of an unmanned aerial vehicle, a robotic manipulator and a gripper, which is grasping an object fixed on a vertical wall. The goal of this work is to show that the overall control allows the flying hand to approach the wall, to dock on the object by means of the gripper, take the object and fly away. The control strategy has been implemented and validated in the simulated model and in experiments on the complete flying hand system.
Aerial robotic systems have raised emerging interests in recent years. In this article, we propose a novel aerial manipulator system that is significantly different from conventional aerial discrete manipulators: An Aerial Continuum Manipulator (AeCoM). The AeCoM compactly integrates a quadrotor with a tendon-driven continuum robotic manipulator. Due to the compact design and the payload bearing ability of tendon-driven continuum robotic arms, the proposed system solved the conflict between payload capacity and dexterity lying in conventional aerial manipulators. Two contributions are made in …
Rigid grippers used in existing aerial manipulators require precise positioning to achieve successful grasps and transmit large contact forces that may destabilize the drone. This limits the speed during grasping and prevents "dynamic grasping", where the drone attempts to grasp an object while moving. On the other hand, biological systems (e.g., birds) rely on compliant and soft parts to dampen contact forces and compensate for grasping inaccuracy, enabling impressive feats. This paper presents the first prototype of a soft drone -- a quadrotor where traditional (i.e., rigid) landing gears ar…
As roles for unmanned aerial vehicles (UAV) continue to diversify, the ability to sense and interact closely with the environment becomes increasingly important. Within this paper we report on the initial flight tests of a novel adaptive compliant actuator which will allow a UAV to carry out such tasks as the "pick and placement" of remote sensors, structural testing and contact-based inspection. Three key results are discussed and presented; the ability to physically compensate impact forces or apply interaction forces by the UAV through the use of the active compliant manipulator; to be able…
Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克理工大学电气工程学院）
This paper presents a system for autonomous cooperative wall building with a team of Unmanned Aerial Vehicles (UAVs). The system was developed for Challenge 2 of the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The wall-building scenario of Challenge 2 featured an initial stack of bricks and wall structure where the individual bricks had to be placed by a team of three UAVs. The objective of the task was to maximize collected points for placing the bricks within the restricted construction time while following the prescribed wall pattern. The proposed approach uses initial…
Arm-Constrained Curriculum Learning for Loco-Manipulation of the Wheel-Legged Robot
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）
KTH Royal Institute of Technology（KTH皇家理工学院）；Yale University（耶鲁大学）；Örebro University（厄勒布鲁大学）
We address the problem of motion planning for a robotic manipulator with the task to place a grasped object in a cluttered environment. In this task, we need to locate a collision-free pose for the object that a) facilitates the stable placement of the object, b) is reachable by the robot manipulator and c) optimizes a user-given placement objective. Because of the placement objective, this problem is more challenging than classical motion planning where the target pose is defined from the start. To solve this task, we propose an anytime algorithm that integrates sampling-based motion planning…
Institut de Recherche en Communications et Cybernétique de Nantes（南特通信与控制研究所）
This paper deals with the optimal path placement for a manipulator based on energy consumption. It proposes a methodology to determine the optimal location of a given test path within the workspace of a manipulator with minimal electric energy used by the actuators while taking into account the geometric, kinematic and dynamic constraints. The proposed methodology is applied to the Orthoglide~3-axis, a three-degree-of-freedom translational parallel kinematic machine (PKM), as an illustrative example.
Quasi-static models of robotic motion with frictional contact provide a computationally efficient framework for analysis and have been widely used for planning and control of non-prehensile manipulation. In this work, we present a novel quasi-static model of planar manipulation that directly maps commanded manipulator velocities to object motion. While quasi-static models have traditionally been unable to capture grasping and jamming behaviors, our approach solves this issue by explicitly modeling the limiting behavior of a velocity-controlled manipulator. We retain the precise modeling of sur…
AERMANI-PLACE: Language Guided Object Placement with Aerial Manipulators
Robotics Research Center, IIIT Hyderabad（海得拉巴国际信息技术学院机器人研究中心）；Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Newcastle University（纽卡斯尔大学）
Robot arm placements are oftentimes a limitation in surgical preoperative procedures, relying on trained staff to evaluate and decide on the optimal positions for the arms. Given new and different patient anatomies, it can be challenging to make an informed choice, leading to more frequently colliding arms or limited manipulator workspaces. In this paper, we develop a method to generate the optimal manipulator base positions for the multi-port da Vinci surgical system that minimizes self-collision and environment-collision, and maximizes the surgeon's reachability inside the patient. Scoring f…
AeroPlace-Flow: Language-Grounded Object Placement for Aerial Manipulators via Visual Foresight and Object Flow
Robotics Research Center, IIIT Hyderabad, India（印度IIIT海得拉巴机器人研究中心）；Department of Computer Science, University of Manchester, UK（英国曼彻斯特大学计算机科学系）；Newcastle University, UK（英国新castle大学）
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
Aiming at the traditional grasping method for manipulators based on 2D camera, when faced with the scene of gathering or covering, it can hardly perform well in unstructured scenes that appear as gathering and covering, for the reason that can't recognize objects accurately in cluster scenes from a single perspective and the manipulators can't make the environment better for grasping. In this case, a novel method of pushing-grasping collaborative based on the deep Q-network in dual perspectives is proposed in this paper. This method adopts an improved deep Q network algorithm, with an RGB-D ca…
Lan-grasp: Using Large Language Models for Semantic Object Grasping and Placement
Department of Computer Science and Artificial Intelligence, University of Technology Nuremberg（计算机科学与人工智能系，图恩技术大学）；Department of Engineering ``Enzo Ferrari'' (DIEF), University of Modena and Reggio Emilia（恩佐·费尔拉蒂工程系（DIEF），摩德纳和雷吉奥艾米利亚大学）；Technical University of Munich（慕尼黑技术大学）
Developing personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic grasping systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn complex grasping strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. The system is informed of the desired target …
ROG-Grasp: Root-Oriented Geometry for Robotic Grasping and Placement
Department of Electrical and Computer Engineering, Drexel University（德雷塞尔大学电气与计算机工程系）；Virginia Seafood Agricultural Research and Extension Center, and Department of Biological Systems Engineering, Virginia Tech（弗吉尼亚理工学院生物系统工程系和弗吉尼亚海鲜农业研究与推广中心）；Amazon Store Foundation AI (SFAI)（亚马逊商店基金会人工智能（SFAI））
The Grasp Reset Mechanism: An Automated Apparatus for Conducting Grasping Trials
Collaborative Robotics and Intelligent Systems (CoRIS) Institute（协作机器人与智能系统研究所）；Oregon State University（俄勒冈州立大学）；Southwest Research Institute (SwRI)（西南研究院）；UMass Lowell Nerve Center（马萨诸塞大学洛厄尔分校神经中心）
B*: Efficient and Optimal Base Placement for Fixed-Base Manipulators
Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；School of Psychological and Cognitive Sciences, Peking University（北京大学心理与认知科学学院）；Beijing Key Laboratory of Behavior and Mental Health, Peking University（北京大学行为与心理健康北京重点实验室）；LeapZenith AI Research（LeapZenith人工智能研究）；University of Chinese Academy of Sciences（中国科学院大学）
Hook-Based Aerial Payload Grasping from a Moving Platform
Systems and Control Lab, HUN-REN Institute for Computer Science and Control（系统与控制实验室，HUN-REN计算机科学与控制研究所）；National Laboratory for Autonomous Systems（自主系统国家实验室）；Control Systems Group of the Eindhoven University of Technology（埃因霍温理工大学控制系统组）
Task-Space Clustering for Mobile Manipulator Task Sequencing
Singapore Centre for 3D Printing (SC3DP)（新加坡3D打印中心（SC3DP））；Nanyang Technological University (NTU)（南洋理工大学（NTU））；HP-NTU Digital Manufacturing Corporate Lab（惠普-南洋理工大学数字制造联合实验室）；Eureka Robotics（优傲机器人（Eureka Robotics））
University of Birmingham（伯明翰大学）；CNRS at Irisa and Inria Rennes Bretagne Atlantique（法国国家科学研究中心（CNRS）IRISA 与 Inria 雷恩布列塔尼大西洋研究中心）
This paper addresses the problem of mixed initiative, shared control for master-slave grasping and manipulation. We propose a novel system, in which an autonomous agent assists a human in teleoperating a remote slave arm/gripper, using a haptic master device. Our system is designed to exploit the human operator's expertise in selecting stable grasps (still an open research topic in autonomous robotics). Meanwhile, a-priori knowledge of: i) the slave robot kinematics, and ii) the desired post-grasp manipulative trajectory, are fed to an autonomous agent which transmits force cues to the human, …
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））
In this paper, we present a planner that plans a sequence of base positions for a mobile manipulator to efficiently and robustly collect objects stored in distinct trays. We achieve high efficiency by exploring the common areas where a mobile manipulator can grasp objects stored in multiple trays simultaneously and move the mobile manipulator to the common areas to reduce the time needed for moving the mobile base. We ensure robustness by optimizing the base position with the best clearance to positioning uncertainty so that a mobile manipulator can complete the task even if there is a certain…
Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery
Institute of Science and Technology for Brain-inspired Intelligence (ISTBI), Fudan University（复旦大学类脑智能科学与技术研究院）；School of Data Science, Fudan University（复旦大学大数据学院）；Institute of Trustworthy Embodied AI, Fudan University（复旦大学可信具身智能研究院）；Shanghai Key Laboratory of Multimodal Embodied AI（上海市多模态具身智能重点实验室）；Shanghai Innovation Institute（上海创新研究院）；ETH Zürich（苏黎世联邦理工学院）
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Heriot-Watt University（赫瑞-瓦特大学）
Agile control of mobile manipulator is challenging because of the high complexity coupled by the robotic system and the unstructured working environment. Tracking and grasping a dynamic object with a random trajectory is even harder. In this paper, a multi-task reinforcement learning-based mobile manipulation control framework is proposed to achieve general dynamic object tracking and grasping. Several basic types of dynamic trajectories are chosen as the task training set. To improve the policy generalization in practice, random noise and dynamics randomization are introduced during the train…
Osaka University（大阪大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
This paper uses a mobile manipulator with a collaborative robotic arm to manipulate objects beyond the robot's maximum payload. It proposes a single-shot probabilistic roadmap-based method to plan and optimize manipulation motion with environment support. The method uses an expanded object mesh model to examine contact and randomly explores object motion while keeping contact and securing affordable grasping force. It generates robotic motion trajectories after obtaining object motion using an optimization-based algorithm. With the proposed method's help, we can plan contact-rich manipulation …
Australian Centre for Robotic Vision (ACRV)（澳大利亚机器人视觉中心（ACRV））；Queensland University of Technology (QUT)（昆士兰科技大学（QUT））；University of Adelaide（阿德莱德大学）；Australian National University (ANU)（澳大利亚国立大学（ANU））
Robotic manipulation and grasping in cluttered and unstructured environments is a current challenge for robotics. Enabling robots to operate in these challenging environments have direct applications from automating warehouses to harvesting fruit in agriculture. One of the main challenges associated with these difficult robotic manipulation tasks is the motion planning and control problem for multi-DoF (Degree of Freedom) manipulators. This paper presents the design and performance evaluation of a low-cost Cartesian manipulator, Cartman who took first place in the Amazon Robotics Challenge 201…
We present a user-friendly interface to teleoperate a soft robot manipulator in a complex environment. Key components of the system include a manipulator with a grasping end-effector that grows via tip eversion, gesture-based control, and haptic display to the operator for feedback and guidance. In the initial work, the operator uses the soft robot to build a tower of blocks, and future works will extend this to shared autonomy scenarios in which the human operator and robot intelligence are both necessary for task completion.
Towards Autonomous Wood-Log Grasping with a Forestry Crane: Simulator and Benchmarking
Automation & Control Institute (ACIN), TU Wien（自动化与控制研究所（ACIN），维也纳技术大学）；AIT Austrian Institute of Technology GmbH（奥地利技术研究院）；Department of Computer Science, University of Liverpool（利物浦大学计算机科学系）
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所IPA）；Institute of Industrial Manufacturing and Management IFF, University of Stuttgart（斯图加特大学工业制造与管理研究所IFF）
This paper introduces a novel approach for the grasping and precise placement of various known rigid objects using multiple grippers within highly cluttered scenes. Using a single depth image of the scene, our method estimates multiple 6D object poses together with an object class, a pose distance for object pose estimation, and a pose distance from a target pose for object placement for each automatically obtained grasp pose with a single forward pass of a neural network. By incorporating model knowledge into the system, our approach has higher success rates for grasping than state-of-the-art…
Korea Institute of Science and Technology（韩国科学技术研究院）
This paper presents planning algorithms for a robotic manipulator with a fixed base in order to grasp a target object in cluttered environments. We consider a configuration of objects in a confined space with a high density so no collision-free path to the target exists. The robot must relocate some objects to retrieve the target while avoiding collisions. For fast completion of the retrieval task, the robot needs to compute a plan optimizing an appropriate objective value directly related to the execution time of the relocation plan. We propose planning algorithms that aim to minimize the num…
Autonomous Unmanned Aerial Manipulators (UAMs) have shown promising potentials to transform passive sensing missions into active 3-dimension interactive missions, but they still suffer from some difficulties impeding their wide applications, such as target detection and stabilization. This letter presents a vision-based autonomous UAM with a 3DoF robotic arm for rotational grasping, with a compensation on displacement for center of gravity. First, the hardware, software architecture and state estimation methods are detailed. All the mechanical designs are fully provided as open-source hardware…
Multi-Sensor Fusion-Based Mobile Manipulator Remote Control for Intelligent Smart Home Assistance
Dyson School of Design Engineering, Imperial College London（帝国理工学院伦敦设计工程学院）；School of Robotics, Xi’an Jiaotong-Liverpool University (XJTLU)（西安交通大学利物浦大学机器人学院）；Department of Engineering, University of Cambridge（剑桥大学工程学院）；School of Mechanical Engineering, Northwestern Polytechnical University（西北工业大学机械工程学院）；Department of Computer Science, University of Aberdeen（阿伯丁大学计算机科学学院）
