LookOut：现实世界类人第一视角导航
LookOut: Real-World Humanoid Egocentric Navigation
AI总结：
针对类人第一视角导航中无碰撞轨迹预测的需求，提出基于时间聚合3D潜在特征推理的框架，构建含4小时现实场景数据的Aria导航数据集，模型可学习类人导航行为并泛化至新环境。
摘要：
从第一视角观测中预测无碰撞未来轨迹的能力在类人机器人、VR/AR及辅助导航等应用中至关重要。本研究提出了一个具有挑战性的任务：从第一视角视频中预测未来连续的6D头部姿态序列，具体包括预测头部平移和旋转以学习通过转头事件体现的主动信息收集行为。为解决该任务，我们提出一个对时间聚合3D潜在特征进行推理的框架，该框架对环境静态和动态部分的几何与语义约束进行建模。鉴于该领域缺乏训练数据，我们进一步贡献了一个使用Project Aria眼镜的数据采集管道，并展示了通过该方法采集的数据集。我们的数据集名为Aria导航数据集（AND），包含4小时用户在现实场景中导航的记录，涵盖多样情境与导航行为，为学习现实世界第一视角导航策略提供了宝贵资源。大量实验表明，我们的模型能学习类人导航行为（如等待/减速、重新规划路线、观察交通状况），并能泛化到未见过的环境。项目网页详见https://sites.google.com/stanford.edu/lookout。
英文摘要：
The ability to predict collision-free future trajectories from egocentric observations is crucial in applications such as humanoid robotics, VR / AR, and assistive navigation. In this work, we introduce the challenging problem of predicting a sequence of future 6D head poses from an egocentric video. In particular, we predict both head translations and rotations to learn the active information-gathering behavior expressed through head-turning events. To solve this task, we propose a framework that reasons over temporally aggregated 3D latent features, which models the geometric and semantic constraints for both the static and dynamic parts of the environment. Motivated by the lack of training data in this space, we further contribute a data collection pipeline using the Project Aria glasses, and present a dataset collected through this approach. Our dataset, dubbed Aria Navigation Dataset (AND), consists of 4 hours of recording of users navigating in real-world scenarios. It includes diverse situations and navigation behaviors, providing a valuable resource for learning real-world egocentric navigation policies. Extensive experiments show that our model learns human-like navigation behaviors such as waiting / slowing down, rerouting, and looking around for traffic while generalizing to unseen environments. Check out our project webpage at https://sites.google.com/stanford.edu/lookout.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
