Many of today's robot perception systems aim at accomplishing perception tasks that are too simplistic and too hard. They are too simplistic because they do not require the perception systems to provide all the information needed to accomplish manipulation tasks. Typically the perception results do not include information about the part structure of objects, articulation mechanisms and other attributes needed for adapting manipulation behavior. On the other hand, the perception problems stated are also too hard because -- unlike humans -- the perception systems cannot leverage the expectations about what they will see to their full potential. Therefore, we investigate a variation of robot perception tasks suitable for robots accomplishing everyday manipulation tasks, such as household robots or a robot in a retail store. In such settings it is reasonable to assume that robots know most objects and have detailed models of them.
We propose a perception system that maintains its beliefs about its environment as a scene graph with physics simulation and visual rendering. When detecting objects, the perception system retrieves the model of the object and places it at the corresponding place in a VR-based environment model. The physics simulation ensures that object detections that are physically not possible are rejected and scenes can be rendered to generate expectations at the image level. The result is a perception system that can provide useful information for manipulation tasks.
University of Bremen（不来梅大学）；Technical University of Munich（慕尼黑工业大学）；Fyuse Inc.（Fyuse公司）；German Aerospace Center (DLR)（德国航空航天中心（DLR））
A pressing question when designing intelligent autonomous systems is how to integrate the various subsystems concerned with complementary tasks. More specifically, robotic vision must provide task-relevant information about the environment and the objects in it to various planning related modules. In most implementations of the traditional Perception-Cognition-Action paradigm these tasks are treated as quasi-independent modules that function as black boxes for each other. It is our view that perception can benefit tremendously from a tight collaboration with cognition. We present RoboSherlock,…
SF-TIM: A Simple Framework for Enhancing Quadrupedal Robot Jumping Agility by Combining Terrain Imagination and Measurement
State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, China（浙江大学现代光学仪器国家重点实验室）；School of Artificial Intelligence and Robotics and National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China（湖南大学人工智能与机器人学院和机器人视觉感知与控制技术国家工程研究中心）；State Key Laboratory of Industrial Control Technology, Zhejiang University, China（浙江大学工业控制技术国家重点实验室）；DeepRobotics Co. Ltd., China（深睿科技有限公司）
SATac: A Thermoluminescence Enabled Tactile Sensor for Concurrent Perception of Temperature, Pressure, and Shear
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Shenzhen University（深圳大学）；RISC-V International Open Source Laboratory（RISC-V国际开源实验室）；Ryerson University（瑞尔森大学）
Autonomous robots in unstructured and dynamically changing retail environments have to master complex perception, knowledgeprocessing, and manipulation tasks. To enable them to act competently, we propose a framework based on three core components: (o) a knowledge-enabled perception system, capable of combining diverse information sources to cope with occlusions and stacked objects with a variety of textures and shapes, (o) knowledge processing methods produce strategies for tidying up supermarket racks, and (o) the necessary manipulation skills in confined spaces to arrange objects in semi-ac…
Deep-sea robot operations demand a high level of safety, efficiency and reliability. As a consequence, measures within the development stage have to be implemented to extensively evaluate and benchmark system components ranging from data acquisition, perception and localization to control. We present an approach based on high-fidelity simulation that embeds spatial and environmental conditions from recorded real-world data. This simulation in the loop (SIL) methodology allows for mitigating the discrepancy between simulation and real-world conditions, e.g. regarding sensor noise. As a result, …
A Semantic Communication Approach to Fiducial Marker Processing in 5G-Enabled Edge SLAM
Faculty of Technical Sciences, University of Novi Sad（诺威萨大学技术科学学院）；The Institute for Artificial Intelligence Research and Development of Serbia（塞尔维亚人工智能研究与发展研究所）
The University of Sydney（悉尼大学）；Cohda Wireless（科达无线公司）
The paper addresses the vehicle-to-X (V2X) data fusion for cooperative or collective perception (CP). This emerging and promising intelligent transportation systems (ITS) technology has enormous potential for improving efficiency and safety of road transportation. Recent advances in V2X communication primarily address the definition of V2X messages and data dissemination amongst ITS stations (ITS-Ss) in a traffic environment. Yet, a largely unsolved problem is how a connected vehicle (CV) can efficiently and consistently fuse its local perception information with the data received from other I…
From Prompt to Physical Actuation: Holistic Threat Modeling of LLM-Enabled Robotic Systems
School of Informatics, Computing, and Cyber Systems, Northern Arizona University（信息学、计算与网络系统学院，北亚利桑那大学）；Department of Software Science, Tallinn University of Technology（软件科学系，塔林技术大学）
Attaining animal-like legged locomotion on rough outdoor terrain with sparse foothold affordances -a primary use-case for legs vs other forms of locomotion- is a largely open problem. New advancements in control and perception have enabled bipeds to walk on flat and uneven indoor environments. But tasks that require reliable contact with unstructured world surfaces, for example walking on natural rocky terrain, need new perception and control algorithms. This thesis introduces 3D perception algorithms for contact tasks such as foot placement in rough terrain environments. We introduce a new me…
WaveMan: mmWave-Based Room-Scale Human Interaction Perception for Humanoid Robots
Key Laboratory for Information Science of Electromagnetic Waves, Ministry of Education, School of Information Science and Technology, Fudan University（电磁波信息科学重点实验室、教育部、信息科学与技术学院，复旦大学）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（机械与航空航天工程学院，南洋理工大学）
Recent years have witnessed increasing interest in the potential benefits of `intelligent' autonomous machines such as robots. Honda's Asimo humanoid robot, iRobot's Roomba robot vacuum cleaner and Google's driverless cars have fired the imagination of the general public, and social media buzz with speculation about a utopian world of helpful robot assistants or the coming robot apocalypse! However, there is a long way to go before autonomous systems reach the level of capabilities required for even the simplest of tasks involving human-robot interaction - especially if it involves communicati…
Built Different: Tactile Perception to Overcome Cross-Embodiment Capability Differences in Collaborative Manipulation
Mechanical Engineering Department at the University of Michigan（密歇根大学机械工程系）；Computer Science Department at the University of Michigan（密歇根大学计算机科学系）；Robotics Department at the University of Michigan（密歇根大学机器人学系）
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；Andy Thomas Centre for Space Resources（安迪·托马斯空间资源中心）；The University of Adelaide（阿德莱德大学）；SmartSat CRC（智能卫星合作研究中心）
Future Moon bases will likely be constructed using resources mined from the surface of the Moon. The difficulty of maintaining a human workforce on the Moon and communications lag with Earth means that mining will need to be conducted using collaborative robots with a high degree of autonomy. In this paper, we describe our solution for Phase 2 of the NASA Space Robotics Challenge, which provided a simulated lunar environment in which teams were tasked to develop software systems to achieve autonomous collaborative robots for mining on the Moon. Our 3rd place and innovation award winning soluti…
