3D scene graphs have recently emerged as a powerful high-level representation of 3D environments. A 3D scene graph describes the environment as a layered graph where nodes represent spatial concepts at multiple levels of abstraction and edges represent relations between concepts. While 3D scene graphs can serve as an advanced "mental model" for robots, how to build such a rich representation in real-time is still uncharted territory. This paper describes a real-time Spatial Perception System, a suite of algorithms to build a 3D scene graph from sensor data in real-time. Our first contribution is to develop real-time algorithms to incrementally construct the layers of a scene graph as the robot explores the environment; these algorithms build a local Euclidean Signed Distance Function (ESDF) around the current robot location, extract a topological map of places from the ESDF, and then segment the places into rooms using an approach inspired by community-detection techniques. Our second contribution is to investigate loop closure detection and optimization in 3D scene graphs. We show that 3D scene graphs allow defining hierarchical descriptors for loop closure detection; our descriptors capture statistics across layers in the scene graph, ranging from low-level visual appearance to summary statistics about objects and places. We then propose the first algorithm to optimize a 3D scene graph in response to loop closures; our approach relies on embedded deformation graphs to simultaneously correct all layers of the scene graph. We implement the proposed Spatial Perception System into a architecture named Hydra, that combines fast early and mid-level perception processes with slower high-level perception. We evaluate Hydra on simulated and real data and show it is able to reconstruct 3D scene graphs with an accuracy comparable with batch offline methods despite running online.
SSF-PAN: Semantic Scene Flow-Based Perception for Autonomous Navigation in Traffic Scenarios
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Shenzhen Deeproute.ai Co.,Ltd（深圳元戎启行科技有限公司）
Intelligent Spatial Perception by Building Hierarchical 3D Scene Graphs for Indoor Scenarios with the Help of LLMs
Shandong New Generation Information Industrial Technology Research Institute（山东省新一代信息技术产业技术研究院）；Inspur Intelligent Terminal Co., Ltd.（浪潮智能终端有限公司）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）
Project AutoVision: Localization and 3D Scene Perception for an Autonomous Vehicle with a Multi-Camera System
DSO National Laboratories（DSO国家实验室）；ETH Zürich（苏黎世联邦理工学院）；National University of Singapore（新加坡国立大学）；MPI-IS and University of Tübingen（马克斯·普朗克智能系统研究所和图宾根大学）；Microsoft, Switzerland（微软瑞士）；Chalmers University of Technology, Sweden（瑞典查尔姆斯理工大学）
V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
车联网通信技术的最新进展使自动驾驶汽车能够共享感知信息，以获得更好的感知性能。随着自动驾驶汽车和智能基础设施的快速发展，V2X感知系统将很快大规模部署，这就提出了一个安全关键问题：\textit{how can we evaluate and improve its performance under challenging traffic scenarios before the real-world deployment?}收集多样化的大规模真实世界测试场景似乎是最直接的解决方案，但这既昂贵又耗时，而且收集只能覆盖有限的场景。为此，我们提出了第一个开放的对抗场景生成器V2XP-ASG，它可以为现代基于激光雷达的多智能体感知系统生成逼真的、具有挑战性的场景。V2XP-ASG学习构建对抗协作图，并以对抗和合理的方式同时扰动多个代理的姿势。实验表明，V2XP-ASG可以有效地识别大范围V2X感知系统的挑战性场景。同时，通过对有限数量的生成的挑战场景进行训练，V2X感知系统的准确率可以进一步提高12.3%的挑战场景和4%的正常场景。我们的代码将在https://github.com/XHwind/V2XP-ASG上发布。
CoopScenes: Multi-Scene Infrastructure and Vehicle Data for Advancing Collective Perception in Autonomous Driving
Institute for Intelligent Systems（智能系统研究所）；Faculty of Computer Science and Engineering（计算机科学与工程学院）；University of Applied Sciences Esslingen（应用科学大学埃森堡）；Institut Pascal ISPR（帕斯卡尔研究所）；Universite Clermont Auvergne INP / CNRS（克勒蒙费朗大学 INP / CNRS）；Center for Artificial Intelligence（人工智能中心）
StixelNExT++: Lightweight Monocular Scene Segmentation and Representation for Collective Perception
Institut Pascal ISPR (Image, Systems of Perception, Robotics), Universite Clermont Auvergne INP / CNRS（帕斯卡尔研究所（图像、感知系统、机器人），克莱蒙特大学INP/CNRS）；Institute for Intelligent Systems at Esslingen University of Applied Sciences（埃斯林根应用科学大学智能系统研究所）
SEPT: Standard-Definition Map Enhanced Scene Perception and Topology Reasoning for Autonomous Driving
Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Zhuoyu Technology（卓予科技）；State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）
Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired
Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院人形机器人与智能系统研究所）；Hangzhou SurImage Technology Company Ltd.（杭州 SurImage 科技有限公司）；Hangzhou KrVision Technology Company Ltd.（杭州 KrVision 科技有限公司）
Open-Vocabulary Spatio-Temporal Scene Graph for Robot Perception and Teleoperation Planning
HI-Robot Lab, School of Automation Science and Engineering, South China University of Technology（华南理工大学自动化科学与工程学院）；Institute of AI Industries, Chinese Academy of Science（中国科学院人工智能产业研究所）；School of Computer Science, Central China Normal University（中央财经大学计算机科学学院）；School of Computer Science and Informatics, University of Liverpool（利物浦大学计算机科学与信息学院）
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning
Université de Montréal（蒙特利尔大学）；JHU APL（约翰斯·霍普金斯大学应用物理实验室）；JHU（约翰斯·霍普金斯大学）；UMass（马萨诸塞大学）；DEVCOM ARL（美国陆军作战能力发展司令部陆军研究实验室）；MIT（麻省理工学院）；University of Toronto（多伦多大学）
