Robots-operating autonomous assembly applications in an unstructured environment require precise methods to locate the building components on site. However, the current available object detection systems are not well-optimised for construction applications, due to the tedious setups incorporated for referencing an object to a system and inability to cope with the elements imperfections. In this paper, we propose a flexible object pose estimation framework to enable robots to autonomously handle building components on-site with an error tolerance to build a specific design target without the need to sort or label them. We implemented an object recognition approach that uses the virtual representation model of all the objects found in a BIM model to autonomously search for the best-matched objects in a scene. The design layout is used to guide the robot to grasp and manipulate the found elements to build the desired structure. We verify our proposed framework by testing it in an automatic discrete wall assembly workflow. Although the precision is not as expected, we analyse the possible reasons that might cause this imprecision, which paves the path for future improvements.
We study the use of parametric building information modeling (BIM) to automatically generate training data for artificial neural networks (ANNs) to recognize building objects in photos. Teaching artificial intelligence (AI) machines to detect building objects in images is the foundation toward AI-assisted semantic 3D reconstruction of existing buildings. However, there exists the challenge of acquiring training data which is typically human-annotated, that is, unless a computer machine can generate high-quality data to train itself for a certain task. In that vein, we trained ANNs solely on re…
This paper presents a double jaw hand for industrial assembly. The hand comprises two orthogonal parallel grippers with different mechanisms. The inner gripper is made of a crank-slider mechanism which is compact and able to firmly hold objects like shafts. The outer gripper is made of a parallelogram that has large stroke to hold big objects like pulleys. The two grippers are connected by a prismatic joint along the hand's approaching vector. The hand is able to hold two objects and perform in-hand manipulation like pull-in (insertion) and push-out (ejection). This paper presents the detailed…
Fine robotic assembly, in which the parts to be assembled are small and fragile and lie in an unstructured environment, is still out of reach of today's industrial robots. The main difficulties arise in the precise localization of the parts in an unstructured environment and the control of contact interactions. Our contribution in this paper is twofold. First, we propose a taxonomy of the manipulation primitives that are specifically involved in fine assembly. Such a taxonomy is crucial for designing a scalable robotic system (both hardware and software) given the complexity of real-world asse…
Case Western Reserve University（凯斯西储大学）；Stanford University（斯坦福大学）
Effective force modulation during tissue manipulation is important for ensuring safe robot-assisted minimally invasive surgery (RMIS). Strict requirements for in-vivo distal force sensing have led to prior sensor designs that trade off ease of manufacture and integration against force measurement accuracy along the tool axis. These limitations have made collecting high-quality 3-degree-of-freedom (3-DoF) bimanual force data in RMIS inaccessible to researchers. We present a modular and manufacturable 3-DoF force sensor that integrates easily with an existing RMIS tool. We achieve this by relaxi…
Imperial College of Science, Technology and Medicine（帝国理工学院）
Traditional telesurgery relies on the surgeon's full control of the robot on the patient's side, which tends to increase surgeon fatigue and may reduce the efficiency of the operation. This paper introduces a Robotic Partner (RP) to facilitate intuitive bimanual telesurgery, aiming at reducing the surgeon workload and enhancing surgeon-assisted capability. An interval type-2 polynomial fuzzy-model-based learning algorithm is employed to extract expert domain knowledge from surgeons and reflect environmental interaction information. Based on this, a bimanual shared control is developed to inter…
An important application of haptic technology to digital product development is in virtual prototyping (VP), part of which deals with interactive planning, simulation, and verification of assembly-related activities, collectively called virtual assembly (VA). In spite of numerous research and development efforts over the last two decades, the industrial adoption of haptic-assisted VP/VA has been slower than expected. Putting hardware limitations aside, the main roadblocks faced in software development can be traced to the lack of effective and efficient computational models of haptic feedback.…
To effectively assist human workers in assembly tasks a robot must proactively offer support by inferring their preferences in sequencing the task actions. Previous work has focused on learning the dominant preferences of human workers for simple tasks largely based on their intended goal. However, people may have preferences at different resolutions: they may share the same high-level preference for the order of the sub-tasks but differ in the sequence of individual actions. We propose a two-stage approach for learning and inferring the preferences of human operators based on the sequence of …
Ochanomizu University（御茶水女子大学）；Sorbonne Université（索邦大学）；CNRS Institut des Systèmes Intelligents et de Robotique (ISIR)（法国国家科学研究中心智能系统与机器人研究所）
Swarms of molecular robots are a promising approach to create specific shapes at the microscopic scale through self-assembly. However, controlling their behavior is a challenging problem as it involves complex non-linear dynamics and high experimental variability. Hand-crafting a molecular controller will often be time-consuming and give sub-optimal results. Optimization methods, like the bioNEAT algorithm, were previously employed to partially overcome these difficulties, but they still had to cope with deceptive high-dimensional search spaces and computationally expensive simulations. Here, …
In this review we introduce the framework of reality-assisted evolution to summarize a growing trend towards combining model-based and model-free approaches to improve the design of physically embodied soft robots. In silico, data-driven models build, adapt and improve representations of the target system using real-world experimental data. By simulating huge numbers of virtual robots using these data-driven models, optimization algorithms can illuminate multiple design candidates for transference to the real world. In reality, large-scale physical experimentation facilitates the fabrication, …
Hierarchical Tri-manual Planning for Vision-assisted Fruit Harvesting with Quadrupedal Robots
Institute for Integrative & Innovative Research (I 3 R), University of Arkansas, USA（整合与创新研究所（I 3 R）、亚拉巴马大学）；Department of Electrical and Computer Engineering, University of California, Riverside, USA（电气与计算机工程系、加州大学河滨分校）
ALPHA-$α$ and Bi-ACT Are All You Need: Importance of Position and Force Information/Control for Imitation Learning of Unimanual and Bimanual Robotic Manipulation with Low-Cost System
Osaka University（大阪大学）；Graduate School of Information Science and Technology of Osaka University（大阪大学信息科学与技术研究生院）；School of Engineering Science, Osaka University（大阪大学工程科学学院）
ETSM: Automating Dissection Trajectory Suggestion and Confidence Map-Based Safety Margin Prediction for Robot-assisted Endoscopic Submucosal Dissection
The Chinese University of Hong Kong（香港中文大学）；Shenzhen Research Institute, The Chinese University of Hong Kong（香港中文大学深圳研究院）；Sun Yat-sen University（中山大学）；Qilu Hospital of Shandong University（山东大学齐鲁医院）
Department of XXX, University of YYY, Location, Country；School of ZZZ, Institute of WWW, Location, Country
Assembly of multi-part physical structures is both a valuable end product for autonomous robotics, as well as a valuable diagnostic task for open-ended training of embodied intelligent agents. We introduce a naturalistic physics-based environment with a set of connectable magnet blocks inspired by children's toy kits. The objective is to assemble blocks into a succession of target blueprints. Despite the simplicity of this objective, the compositional nature of building diverse blueprints from a set of blocks leads to an explosion of complexity in structures that agents encounter. Furthermore,…
Drilling a hole on a curved surface with a desired angle is prone to failure when done manually, due to the difficulties in drill alignment and also inherent instabilities of the task, potentially causing injury and fatigue to the workers. On the other hand, it can be impractical to fully automate such a task in real manufacturing environments because the parts arriving at an assembly line can have various complex shapes where drill point locations are not easily accessible, making automated path planning difficult. In this work, an adaptive admittance controller with 6 degrees of freedom is d…
Making Physical Objects with Generative AI and Robotic Assembly: Considering Fabrication Constraints, Sustainability, Time, Functionality, and Accessibility
