The raise of collaborative robotics has led to wide range of sensor technologies to detect human-machine interactions: at short distances, proximity sensors detect nontactile gestures virtually occlusion-free, while at medium distances, active depth sensors are frequently used to infer human intentions. We describe an optical system for large workspaces to capture human pose based on a single panoramic color camera. Despite the two-dimensional input, our system is able to predict metric 3D pose information over larger field of views than would be possible with active depth measurement cameras. We merge posture context with proximity perception to reduce occlusions and improve accuracy at long distances. We demonstrate the capabilities of our system in two use cases involving multiple humans and robots.
Proximity3D: Shape from Capacitive Proximity on Sensing Manifold
Centre for Perceptual and Interactive Intelligence（感知与交互智能中心）；Xiamen University（厦门大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Manchester（曼彻斯特大学）
University of Leeds（利兹大学）；Istituto Italiano di Tecnologia（意大利理工学院）；University of Glasgow（格拉斯哥大学）
Touch sensing can help robots understand their sur- rounding environment, and in particular the objects they interact with. To this end, roboticists have, in the last few decades, developed several tactile sensing solutions, extensively reported in the literature. Research into interpreting the conveyed tactile information has also started to attract increasing attention in recent years. However, a comprehensive study on this topic is yet to be reported. In an effort to collect and summarize the major scientific achievements in the area, this survey extensively reviews current trends in robot …
Perceiving obstacles and avoiding collisions is fundamental to the safe operation of a robot system, particularly when the robot must operate in highly dynamic human environments. Proximity detection using on-robot sensors can be used to avoid or mitigate impending collisions. However, existing proximity sensing methods are orientation and placement dependent, resulting in blind spots even with large numbers of sensors. In this paper, we introduce the phenomenon of the Leaky Surface Wave (LSW), a novel sensing modality, and present AuraSense, a proximity detection system using the LSW. AuraSen…
Tactile perception is an essential ability of intelligent robots in interaction with their surrounding environments. This perception as an intermediate level acts between sensation and action and has to be defined properly to generate suitable action in response to sensed data. In this paper, we propose a feedback approach to address robot grasping task using force-torque tactile sensing. While visual perception is an essential part for gross reaching, constant utilization of this sensing modality can negatively affect the grasping process with overwhelming computation. In such case, human bei…
RS3DBench: A Comprehensive Benchmark for 3D Spatial Perception in Remote Sensing
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Software College of Zhejiang University（浙江大学软件学院）；Hangzhou City University（杭州市大学）
Proximity QA: Unleashing the Power of Multi-Modal Large Language Models for Spatial Proximity Analysis
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）；Intel Lab China（英特尔中国实验室）
Perception in 3D has become standard practice for a large part of robotics applications. High quality 3D perception is costly. Our previous work on a nodding 2D Lidar provides high quality 3D depth information with low cost, but the sparse data generated by this sensor poses challenges in understanding the characteristics of moving objects within an uncertain environment. This paper proposes a novel design of the nodding Lidar but provides dynamic reconfigurability in terms of limiting the field of view of the sensor using a set of optical mirrors. It not only provides denser scans, but it als…
Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）；Royal Brompton Hospital（皇家布朗普顿医院）；Imperial College London（伦敦帝国理工学院）
The long acquisition time has limited the accessibility of magnetic resonance imaging (MRI) because it leads to patient discomfort and motion artifacts. Although several MRI techniques have been proposed to reduce the acquisition time, compressed sensing in magnetic resonance imaging (CS-MRI) enables fast acquisition without compromising SNR and resolution. However, existing CS-MRI methods suffer from the challenge of aliasing artifacts. This challenge results in the noise-like textures and missing the fine details, thus leading to unsatisfactory reconstruction performance. To tackle this chal…
This manuscript describes the elements of a theory of information tailored to control and decision tasks and specifically to visual data. The concept of Actionable Information is described, that relates to a notion of information championed by J. Gibson, and a notion of "complete information" that relates to the minimal sufficient statistics of a complete representation. It is shown that the "actionable information gap" between the two can be reduced by exercising control on the sensing process. Thus, senging, control and information are inextricably tied. This has consequences in the so-calle…
Cross-View Urban Sensing: Mapping Subjective Streetscape Perception via AlphaEarth Embeddings and Urban Context
School of Geospatial Engineering and Science, Sun Yat-sen University（中山大学地理空间工程学院）；Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)（南方海洋科学与工程广东省实验室（珠海））；Key Laboratory of Comprehensive Observation of Polar Environment (Sun Yat-sen University), Ministry of Education（教育部极地环境综合观测重点实验室（中山大学））
Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University（武汉大学计算机学院国家多媒体软件工程技术研究中心）；Hubei Luojia Laboratory（湖北珞珈实验室）；School of Urban Design, Wuhan University（武汉大学城市设计学院）；School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）
Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；The University of Hong Kong（香港大学）；Global Innovation Exchange, Tsinghua University（清华大学全球创新交换）
University of Warwick（华威大学）；Jaguar Land Rover Ltd.（捷豹路虎有限公司）；University of Surrey（萨里大学）
3D object detection is a common function within the perception system of an autonomous vehicle and outputs a list of 3D bounding boxes around objects of interest. Various 3D object detection methods have relied on fusion of different sensor modalities to overcome limitations of individual sensors. However, occlusion, limited field-of-view and low-point density of the sensor data cannot be reliably and cost-effectively addressed by multi-modal sensing from a single point of view. Alternatively, cooperative perception incorporates information from spatially diverse sensors distributed around the…
Tongji University（同济大学）；Technical University of Munich（慕尼黑工业大学）；ETH Zurich（苏黎世联邦理工学院）；fortiss GmbH（福蒂斯有限公司）
The rapid growth of IoT era is shaping the future of mobile services. Advanced communication technology enables a heterogeneous connectivity where mobile devices broadcast information to everything. Mobile applications such as robotics and vehicles connecting to cloud and surroundings transfer the short-range on-board sensor perception system to long-range mobile-sensing perception system. However, the mobile sensing perception brings new challenges for how to efficiently analyze and intelligently interpret the deluge of IoT data in mission- critical services. In this article, we model the cha…
Lincoln Centre for Autonomous Systems（林肯自主系统中心）；University of Lincoln（林肯大学）
Automation and robotisation of the agricultural sector are seen as a viable solution to socio-economic challenges faced by this industry. This technology often relies on intelligent perception systems providing information about crops, plants and the entire environment. The challenges faced by traditional 2D vision systems can be addressed by modern 3D vision systems which enable straightforward localisation of objects, size and shape estimation, or handling of occlusions. So far, the use of 3D sensing was mainly limited to indoor or structured environments. In this paper, we evaluate modern s…
Vehicle-to-Everything Cooperative Perception for Autonomous Driving
College of Science and Engineering, James Cook University（科学与工程学院，詹姆斯库克大学）；Momoni AI；Department of Electrical and Computer Engineering, University of Alabama in Huntsville（电气与计算机工程系，阿拉巴马大学亨茨维尔分校）；Department of Automation and Intelligent Sensing, Shanghai Jiaotong University（自动化与智能感知系，上海交通大学）；School of Engineering, Swinburne University of Technology（工程学院，斯威本技术大学）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（信息与通信研究机构，科技研究局（A*STAR））
TriDP-PTM: a three-stage distortion-perception tradeoff guides the pre-training model for radar cardiac sensing
National Institute of Health Data Science, Peking University（北京大学国家健康数据科学研究院）；Institute of Medical Technology, Peking University（北京大学医学技术研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）
A Spatial Semantics and Continuity Perception Attention for Remote Sensing Water Body Change Detection
College of Information Science and Technology, Shihezi University（信息科学与技术学院，石河子大学）；Production and Construction Corps Engineering Research Center for Spatial Information Technology（空间信息技术工程研究中心）
Technische Universität Braunschweig（布伦瑞克工业大学）；Volkswagen Group Automation（大众汽车集团自动驾驶部门）
Enabling autonomous driving (AD) can be considered one of the biggest challenges in today's technology. AD is a complex task accomplished by several functionalities, with environment perception being one of its core functions. Environment perception is usually performed by combining the semantic information captured by several sensors, i.e., lidar or camera. The semantic information from the respective sensor can be extracted by using convolutional neural networks (CNNs) for dense prediction. In the past, CNNs constantly showed state-of-the-art performance on several vision-related tasks, such…
Queen Mary University of London（伦敦玛丽女王大学）；Onfido Research London（Onfido伦敦研究院）
WiFi-based sensing for human activity recognition (HAR) has recently become a hot topic as it brings great benefits when compared with video-based HAR, such as eliminating the demands of line-of-sight (LOS) and preserving privacy. Making the WiFi signals to 'see' the action, however, is quite coarse and thus still in its infancy. An end-to-end spatiotemporal WiFi signal neural network (STWNN) is proposed to enable WiFi-only sensing in both line-of-sight and non-line-of-sight scenarios. Especially, the 3D convolution module is able to explore the spatiotemporal continuity of WiFi signals, and t…
Toward Robust Single-Photon Perception for Robots: A Condition-Aware Active Learning Approach
School of Mechatronic Engineering and Automation, Shanghai University（机械电子工程与自动化学院，上海大学）；Australian Center for Robotics, School of Aerospace, Mechanical and Mechatronic Engineering, the University of Sydney（机器人研究中心，航空航天、机械与机电工程学院，悉尼大学）；Shanghai Institute of Technology（上海理工大学）
