How to Integrate Digital Twin and Virtual Reality in Robotics Systems? Design and Implementation for Providing Robotics Maintenance Services in Data Centers
University of Twente（特文特大学）；Beijing Hanning Tech Co., Ltd（北京汉宁科技有限公司）
In the context of Industry 4.0, the physical and digital worlds are closely connected, and robots are widely used to achieve system automation. Digital twin solutions have contributed significantly to the growth of Industry 4.0. Combining various technologies is a trend that aims to improve system performance. For example, digital twinning can be combined with virtual reality in automated systems. This paper proposes a new concept to articulate this combination, which has mainly been implemented in engineering research projects. However, there are currently no guidelines, plans, or concepts to articulate this combination. The concept will be implemented in data centers, which are crucial for enabling virtual tasks in our daily lives. Due to the COVID-19 pandemic, there has been a surge in demand for services such as e-commerce and videoconferencing. Regular maintenance is necessary to ensure uninterrupted and reliable services. Manual maintenance strategies may not be sufficient to meet the current high demand, and innovative approaches are needed to address the problem. This paper presents a novel approach to data center maintenance: real-time monitoring by an autonomous robot. The robot is integrated with digital twins of assets and a virtual reality interface that allows human personnel to control it and respond to alarms. This methodology enables faster, more cost-effective, and higher quality data center maintenance. It has been validated in a real data centre and can be used for intelligent monitoring and management through joint data sources. The method has potential applications in other automated systems.
Digital Twin technology is playing a pivotal role in the modern industrial evolution. Especially, with the technological progress in the Internet-of-Things (IoT) and the increasing trend in autonomy, multi-sensor equipped robotics can create practical digital twin, which is particularly useful in the industrial applications for operations, maintenance and safety. Herein, we demonstrate a real-world digital twin of a safety-critical robotics applications with a Franka-Emika-Panda robotic arm. We develop and showcase an edge-assisted collaborative digital twin for dynamic obstacle avoidance whic…
University of Colorado Boulder（科罗拉多大学博尔德分校）；Center for Astrophysics（天体物理中心）；Space Astronomy（空间天文学）
Humans and robots will need to collaborate in order to create a sustainable human lunar presence by the end of the 2020s. This includes cases in which a human will be required to teleoperate an autonomous rover that has encountered an instrument assembly failure. To aid teleoperators in the troubleshooting process, we propose a virtual reality digital twin placed in a simulated environment. Here, the operator can virtually interact with a digital version of the rover and mechanical arm that uses the same controls and kinematic model. The user can also adopt the egocentric (a first person view …
Digital twin and extended reality for teleoperation of the electric vehicle battery disassembly
Knight Foundation School of Computing and Information Sciences, Florida International University（佛罗里达国际大学计算机与信息科学学院）；Centria University of Applied Sciences（应用科学大学）；Digital Engineering for Advanced Manufacturing Laboratory (DEAM Lab), Center for Precision Metrology, Department of Mechanical Engineering and Engineering Science, University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校数字工程高级制造实验室）；University of Oulu（奥卢大学）
XR-DT: Extended Reality-Enhanced Digital Twin for Safe Motion Planning via Human-Aware Model Predictive Path Integral Control
Department of Civil, Architectural, and Environmental Engineering, The University of Texas at Austin（德克萨斯大学奥斯汀分校土木、建筑与环境工程系）；School of Architecture, The University of Texas at Austin（德克萨斯大学奥斯汀分校建筑学院）；School of Civil and Construction Engineering, Purdue University（普渡大学土木与建设工程学院）；Department of Civil and Environmental Engineering, University of Wisconsin-Madison（威斯康星大学麦迪逊分校土木与环境工程系）
A Digital Twin for Robotic Post Mortem Tissue Sampling using Virtual Reality
Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（医学技术与智能系统研究所，汉堡技术大学）；Institute of Legal Medicine, University Medical Center Hamburg-Eppendorf（法医学研究所，汉堡艾本达大学医学中心）；Interdisciplinary Competence Center for Interface Research（界面研究跨学科能力建设中心）；SustAInLivWork Center of Excellence, Kaunas, Lithuania（可持续性与生活工作卓越中心，考纳斯，立陶宛）
How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence
Hubei University（湖北大学）；Worcester Polytechnic Institute（伍斯特理工学院）；Beijing Huairou Academy of Parallel Sensing（北京怀柔平行传感研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Waytous Inc.（驭势科技（北京）有限公司）
University of Augsburg（奥格斯堡大学）；XITASO GmbH（XITASO有限公司）
This paper introduces two case studies combining the field of industrial robotics with Mixed Reality (MR). The goal of those case studies is to get a better understanding of how MR can be useful and what are the limitations. The first case study describes an approach to visualize the digital twin of a robot arm. The second case study aims at facilitating the commissioning of industrial robots. Furthermore, this paper reports the experiences gained by implementing those two scenarios and discusses the limitations.
Institut de Recherche en Communications et Cybernétique de Nantes（南特通信与控制研究所）
Manufacturing is using Virtual Reality tools to enhance the product life cycle. Their definitions are still in flux and it is necessary to define their connections. Thus, firstly, we will introduce more closely some definitions where we will find that, if the Virtual manufacturing concepts originate from machining operations and evolve in this manufacturing area, there exist a lot of applications in different fields such as casting, forging, sheet metalworking and robotics (mechanisms). From the recent projects in Europe or in USA, we notice that the human perception or the simulation of manne…
Institut de Recherche en Communications et Cybernétique de Nantes（南特通信与控制研究所）；Laboratoire d'Informatique Industrielle, Ecole Nationale d’Ingénieurs de Brest（布雷斯特国立工程师学院工业计算机实验室）
This paper presents a new method, based on a multi-agent system and on digital mock-up technology, to assess an efficient path planner for a manikin for access and visibility task under ergonomic constraints. In order to solve this problem, the human operator is integrated in the process optimization to contribute to a global perception of the environment. This operator cooperates, in real-time, with several automatic local elementary agents. The result of this work validates solutions brought by digital mock-up and that can be applied to simulate maintenance task.
Digitizing physical objects into the virtual world has the potential to unlock new research and applications in embodied AI and mixed reality. This work focuses on recreating interactive digital twins of real-world articulated objects, which can be directly imported into virtual environments. We introduce Ditto to learn articulation model estimation and 3D geometry reconstruction of an articulated object through interactive perception. Given a pair of visual observations of an articulated object before and after interaction, Ditto reconstructs part-level geometry and estimates the articulation…
Autonomous warehouses with mobile, rack-carrying robots are starting to become commonplace, with systems such as Amazon's Kiva and Swisslog's CarryPick already implemented in functional warehouses. Such warehouses however still require human intervention for object picking and maintenance. In the European project SafeLog we are developing a safety-vest, used for safety-critical ranging and stopping of mobile robots, an improved planner that can handle large fleets of heterogeneous agents as well as an AR interaction system to navigate and support human workers in such automated environments. H…
Monash University（莫纳什大学）；University of Victoria（维多利亚大学）
When humans work together to complete a joint task, each person builds an internal model of the situation and how it will evolve. Efficient collaboration is dependent on how these individual models overlap to form a shared mental model among team members, which is important for collaborative processes in human-robot teams. The development and maintenance of an accurate shared mental model requires bidirectional communication of individual intent and the ability to interpret the intent of other team members. To enable effective human-robot collaboration, this paper presents a design and impleme…
Robotic Fruits with Tunable Stiffness and Sensing: Towards a Methodology for Developing Realistic Physical Twins of Fruits
School of Engineering and Materials Science, Queen Mary University of London（伦敦女王学院工程与材料科学学院）；Department of Robotics, Ritsumeikan University（立命馆大学机器人学系）
TwinAligner: Visual-Dynamic Alignment Empowers Physics-aware Real2Sim2Real for Robotic Manipulation
CFCS, School of Computer Science, Peking University（计算机学院，北京大学CFCS）；PKU-AgiBot Lab（北京大学AgiBot实验室）；State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）
We describe a software framework and a hardware platform used in tandem for the design and analysis of robot autonomy algorithms in simulation and reality. The software, which is open source, containerized, and operating system (OS) independent, has three main components: a ROS 2 interface to a C++ vehicle simulation framework (Chrono), which provides high-fidelity wheeled/tracked vehicle and sensor simulation; a basic ROS 2-based autonomy stack for algorithm design and testing; and, a development ecosystem which enables visualization, and hardware-in-the-loop experimentation in perception, st…
Digital Twins Below the Surface: Enhancing Underwater Teleoperation
School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）；School of Mathematics and Computer Sciences, Heriot-Watt University（赫瑞瓦特大学数学与计算机科学学院）
