Eye, Robot：通过VLM引导的Transformer-VAE学习富有表现力的注视动力学
Eye, Robot: Learning Expressive Gaze Dynamics via a VLM-Guided Transformer-VAE
AI总结：
该研究针对人形机器人注视建模问题，提出VLM引导的Transformer-VAE模型RGT，在基准测试中实现了更低的七关节MSE，验证了运动学保真度与实时推理能力。
摘要：
注视通过眼睛、头部和眼睑的耦合运动在人类交互中引导注意力。人形机器人必须选择注视目标，同时生成朝向该目标的合适运动；仅固定跟踪规则无法建模人类注视的时间结构。我们将这些任务分离：采用异步视觉语言模型（VLM）选择场景指代对象，采用反应式感知模块维持注视，以及采用现实注视Transformer（RGT）生成目标条件下的运动。RGT是在无标记人类记录数据上训练的Transformer-VAE，被重定向到机器人的11自由度（DoF）关节空间：自监督预训练编码近期完整头部历史，自回归微调在展开过程中让模型接触自身预测。推理时，四维双眼轨迹协调颈部与眼部运动；辅助眨眼预测器在回放时触发眼睑模板。在两秒基准测试中，所有控制器接收相同的真实目标，RGT-Base取得已报道的最低七关节均方误差（MSE），物理演示验证了集成的定位与控制流程。该证据确立了机器人空间的运动学保真度和RGT的实时推理能力；感知自然度与开放词汇目标选择精度仍需受控评估。
英文摘要：
Gaze coordinates attention in human interaction through coupled eye, head, and eyelid motion. A humanoid robot must choose where to look while generating a suitable movement toward that target; fixed tracking rules alone do not model the temporal structure of human gaze. We separate these tasks with an asynchronous vision-language model that selects a scene referent, a reactive perception module that maintains fixation, and a Realistic Gaze Transformer (RGT) that generates target-conditioned motion. RGT is a Transformer-VAE trained on markerless human recordings retargeted to the robot's 11-DoF joint space: self-supervised pretraining encodes recent full-head history, and autoregressive fine-tuning exposes the model to its own predictions during rollout. At inference, a four-dimensional binocular-eye trajectory conditions coordinated neck and eye motion; an auxiliary blink predictor triggers eyelid templates during playback. On a two-second benchmark in which all controllers receive the same ground-truth target, RGT-Base obtains the lowest reported overall seven-joint MSE, and physical demonstrations show the integrated targeting and control pipeline. The evidence establishes robot-space kinematic fidelity and real-time RGT inference; perceived naturalness and open-vocabulary target-selection accuracy remain questions for controlled evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
