HINT：面向长 horizon 机器人操作的人类意图 inception

中文摘要 AI 辅助

人类可基于简单意图指令完成复杂操作，同时持续适应变化的视觉观测。但当前视觉-语言动作（VLA）模型及其他动作策略，在密集、变化的视觉输入和稀疏语言引导下，难以实现这种高级智能行为，视觉关联会主导语义意图，导致动作遵循视觉捷径而非人类目标。我们提出 HINT（Human-INTent INcepTion），这是一个受人类操作原则启发的智能体框架：语义意图仅在操作模式转换时稀疏变化，连续控制主要依赖不断变化的手-物关系。HINT 仅在模式转换时调用语义推理以解决当前子任务和目标，随后通过多视图 grounding 和视觉跟踪维持该承诺。我们探索了两种视觉接口——图像空间语义高亮和注意力先验注入，以将跟踪的意图传递给动作策略，且不向基础动作模型引入额外可训练参数。在三个长 horizon 任务及分布外变体上的实验表明，HINT 在两种基础策略上显著提升了意图理解、任务进度和端到端成功率，同时保持低延迟控制。

英文摘要

Humans can perform complex manipulations given a simple intent through an overall instruction, while continuously adapting to evolving visual observations. However, current vision-language action (VLA) models and other action policies struggle to realize this high-level intelligent behavior under dense, evolving visual inputs and sparse language guidance. Visual correlations can then dominate semantic intent, leading actions to follow visual shortcuts rather than human goals. We present HINT (Human-INTent INcepTion), an agentic framework inspired by the human manipulation principles: semantic intent changes sparsely at manipulation-pattern transitions, whereas continuous control primarily depends on the evolving object-hand relationship. HINT invokes semantic reasoning only at pattern transitions to resolve the current subtask and target, then maintains this commitment through multi-view grounding and visual tracking. We explore two visual interfaces-image-space semantic highlighting and attention-prior injection-to communicate the tracked intent to the action policy without introducing additional trainable parameters into the foundation action model. Experiments across three long-horizon tasks and out-of-distribution variants show that HINT substantially improves intent understanding, task progress, and end-to-end success across two foundation policies while preserving low-latency control. Project page: https://robot-hint.github.io/

发表机构

• Zhejiang University（浙江大学）
• Shanghai Jiao Tong University（上海交通大学）
• Noematrix（诺埃马特里斯（Noematrix））
• EndlessAI（无尽人工智能（EndlessAI））

机构由 AI 辅助整理，请以论文原文为准。

相关深度报道

• 浙大让机器人少想多做：只在关键节点推理，也能守住长流程意图