We study active perception from first principles to argue that an autonomous agent performing active perception should maximize the mutual information that past observations posses about future ones. Doing so requires (a) a representation of the scene that summarizes past observations and the ability to update this representation to incorporate new observations (state estimation and mapping), (b) the ability to synthesize new observations of the scene (a generative model), and (c) the ability to select control trajectories that maximize predictive information (planning). This motivates a neural radiance field (NeRF)-like representation which captures photometric, geometric and semantic properties of the scene grounded. This representation is well-suited to synthesizing new observations from different viewpoints. And thereby, a sampling-based planner can be used to calculate the predictive information from synthetic observations along dynamically-feasible trajectories. We use active perception for exploring cluttered indoor environments and employ a notion of semantic uncertainty to check for the successful completion of an exploration task. We demonstrate these ideas via simulation in realistic 3D indoor environments.
Benchmarking Neural Radiance Fields for Autonomous Robots: An Overview
School of Computer Science, Hangzhou Dianzi University（杭州电子科技大学计算机学院）；High-speed Aerodynamics Institute, CARDC（中国空气动力研究与发展中心高速空气动力研究所）；Stevens Institute of Technology（史蒂文斯理工学院）；Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学勒迪信息、计算与工程学院）；City College of New York（纽约城市学院）；School of Computer Science, University of Bristol（布里斯托大学计算机学院）；State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）
FLAP: FOV-Constrained Active Perception Planning for Prior-Map-Free 3D Navigation
The State Key Laboratory of Industrial Control Technology, College of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院工业控制技术国家重点实验室）；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）；Huzhou Key Laboratory of Autonomous System（湖州市自动驾驶系统重点实验室）；Shanghai Institute of Special Equipment Inspection and Technical Research Co., Ltd（上海市特种设备监督检验技术研究院有限公司）
MAAP: Multi-Agent Active Perception for Collaborative Manipulation
Carnegie Mellon University（卡内基梅隆大学）；Shanghai Jiao Tong University（上海交通大学）；University of Science and Technology of China（中国科学技术大学）；The University of Hong Kong（香港大学）；Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
APREBot: Active Perception System for Reflexive Evasion Robot
School of Computing, National University of Singapore（计算学院，新加坡国立大学）；Department of Electrical and Computer Engineering, National University of Singapore（电气与计算机工程系，新加坡国立大学）；School of Automation, Beijing Institute of Technology（自动化学院，北京理工大学）；School of Computer Science and Engineering, Sun Yat-Sen University（计算机科学与工程学院，中山大学）
Perceptual Factors for Environmental Modeling in Robotic Active Perception
Instituto de Investigación en Ingeniería de Aragón, Universidad de Zaragoza（萨拉戈萨大学阿拉贡工程研究院）；Institute of Geodesy and Geoinformation, University of Bonn（波恩大学大地测量与地理信息研究所）
Apple: Toward General Active Perception via Reinforcement Learning
Department of Computer Science, TU Darmstadt, Germany（德国图林根大学计算机科学系）；LIRIS, CNRS UMR5205, École Centrale de Lyon, France（法国里里萨大学LIRIS实验室）；LASR Lab & CeTI, TU Dresden, Germany（德国德累斯顿技术大学LASR实验室）；DFKI, Hessian.AI, RIG, and Centre for Cognitive Science, TU Darmstadt, Germany（德国图林根大学DFKI、海德堡人工智能、RIG及认知科学中心）
Uncertainty-Informed Active Perception for Open Vocabulary Object Goal Navigation
Center for Robotics, University of Bonn（波恩大学机器人中心）；MAVLab, TU Delft（代尔夫特理工大学MAVLab）；Lamarr Institute for Machine Learning and Artificial Intelligence（机器学习与人工智能拉马尔研究所）
ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Beijing Key Laboratory of Research on Large Models and Intelligent Governance（北京大模型与智能治理重点实验室）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Beijing Jiaotong University（北京交通大学）；State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；AresoX
Commentaries on "Learning Sensorimotor Control with Neuromorphic Sensors: Toward Hyperdimensional Active Perception" [Science Robotics Vol. 4 Issue 30 (2019) 1-10
Research Institutes of Sweden（瑞典研究机构）；University of California, Berkeley（加州大学伯克利分校）；Luleå University of Technology（吕勒奥理工大学）
The Oxford Spires Dataset: Benchmarking Large-Scale LiDAR-Visual Localisation, Reconstruction and Radiance Field Methods
Oxford Robotics Inst., Dept. of Eng. Science, Univ. of Oxford, UK（牛津大学机器人研究所、工程科学系）；Group of Automation, Robotics and Computer Vision (AUROVA), University of Alicante, Spain（自动化、机器人与计算机视觉小组（AUROVA）、阿尔基兰特大学）
