Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user's intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, vision-language conditioned camera motion remains comparatively underexplored as a first-class action. We formulate vision-language conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. The challenge is that useful viewpoint changes depend on latent perceptual intent, ranging from coarse spatial moves to fine-grained inspection or occlusion revealing. To model this structure, we mine multi-intent camera-motion supervision from egocentric video via hindsight labeling, pairing each relative SE(3) target pose with plausible intents and observation-gain descriptions inferred from the reached view. We propose LIME, a vision-language camera-motion generator that combines an autoregressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across viewpoint-prediction experiments and downstream robotic tasks, we show that LIME learns intent-conditioned camera motion from passive egocentric video, turning ordinary human recordings into supervision for a reusable active-perception primitive that supports general-purpose viewpoint selection, manipulation, embodied question answering, and mid-distance navigation.
EgoPriMo: Egocentric Motion Generation for Interactive Humanoid Control
Tianjin University(天津大学);Zhongguancun Academy(中关村学院);Beihang University(北京航空航天大学);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院);DeepCybo
MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos
IRMV Lab, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University and State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Key Laboratory of Navigation and Location Based Services(IRMV实验室,自动化与智能感知学院,上海交通大学,航空集成与航空系统-of-Systems综合国家重点实验室,导航与定位服务重点实验室);College of Intelligence Science and Technology, National University of Defense Technology(智能科学与技术学院,国防科技大学);Department of Electronic Engineering, Shanghai Jiao Tong University(电子工程学院,上海交通大学)
Learning Robot Visual Navigation in Crowds via Intention-Aware Scene Representations
Shenzhen Key Laboratory of Robotics Perception and Intelligence, Department of Electronic and Electrical Engineering, SUSTech(南方科技大学电子与电气工程系深圳市机器人感知与智能重点实验室);Jiaxing Research Institute, SUSTech(南方科技大学嘉兴研究院);Research Institute of MA&EI, Peng Cheng Laboratory(鹏城实验室机器人与人工智能研究所)
IntentReact: Guiding Reactive Object-Centric Navigation via Topological Intent
School of Information Science and Engineering, Hangzhou Normal University(杭州师范大学信息科学与工程学院);State Key Laboratory of Industrial Control and Technology, Zhejiang University(浙江大学工业控制技术国家重点实验室);Neuromorphic Computing and Robotic Cognition Lab, Zhejiang University(浙江大学神经形态计算与机器人认知实验室);Department of Automation, Zhejiang University of Technology(浙江工业大学自动化系)
UC Berkeley(加州大学伯克利分校);Facebook AI Research(Facebook人工智能研究院);UIUC(伊利诺伊大学厄巴纳-香槟分校)
Planning at a higher level of abstraction instead of low level torques improves the sample efficiency in reinforcement learning, and computational efficiency in classical planning. We propose a method to learn such hierarchical abstractions, or subroutines from egocentric video data of experts performing tasks. We learn a self-supervised inverse model on small amounts of random interaction data to pseudo-label the expert egocentric videos with agent actions. Visuomotor subroutines are acquired from these pseudo-labeled videos by learning a latent intent-conditioned policy that predicts the inf…
Perspective-Shifted Neuro-Symbolic World Models: A Framework for Socially-Aware Robot Navigation
Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa(系统机器人研究所,理工学院,里斯本大学);LAAS-CNRS, Artificial and Natural Intelligence Toulouse Institute (ANITI)(LAAS-CNRS,图卢兹人工智能研究所(ANITI))
RoboMirror: Understand Before You Imitate for Video to Humanoid Locomotion
Nanyang Technological University(南洋理工大学);The Hong Kong University of Science and Technology(香港科技大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Xi’an Jiaotong University(西安交通大学);Chinese Academy of Sciences(中国科学院);Shanghai Jiao Tong University(上海交通大学);University of Sydney(悉尼大学);Peking University(北京大学)