TacPAC：面向接触丰富型操作的世界-动作模型中的触觉预测与实时动作校正

中文摘要 AI 辅助

世界-动作模型通过预测未来观测结果来指导动作生成，但以视觉为中心的预测会遗漏决定接触丰富型操作的局部接触线索。然而，在我们的实验中，仅将未来触觉观测作为额外视图进行朴素预测，仅能恢复可实现增益的三分之一。这一差距反映了时间不匹配：预测在执行前完成，而触觉反馈在执行过程中到达。我们提出TacPAC，它将触觉预测转化为实时动作校正。在基础模型规划完一个动作块后，TacPAC会缓存该规划所依赖的预测接触以及该规划自身的表示，随后一个触觉专家会将每一个新观测到的触觉图像与该缓存进行比对，以校正尚未执行的动作。因此，反馈是对照规划预期而非孤立地进行解读，且一次校正仅需对该缓存进行一次遍历，比重新生成动作块的成本低20.7倍。在涵盖精密插入、易碎物体处理、物体重新定向和长时程操作的五项真实机器人任务中，TacPAC在所有任务中均表现领先，将平均性能从其仅视觉的基础模型的22%提升至64%。代码可在该https URL获取。

英文摘要

World-action models guide action generation with predicted future observations, but vision-centric predictions miss the local contact cues that decide contact-rich manipulation. However, naively predicting future tactile observations as additional views recovers only a third of the achievable gain in our experiments. This gap reflects a timing mismatch: predictions precede execution, while tactile feedback arrives during it. We introduce TacPAC, which turns tactile prediction into real-time action correction. Once the base model has planned an action chunk, TacPAC caches the predicted contact that plan was conditioned on together with the plan's own representation, and a tactile expert reads each newly observed tactile image against that cache to correct the actions not yet executed. Feedback is thus interpreted against what the plan anticipated rather than in isolation, and one correction is a single pass over that cache, $20.7\times$ cheaper than regenerating the chunk. On five real-robot tasks spanning precision insertion, fragile-object handling, object reorientation, and long-horizon manipulation, TacPAC leads every task and raises the average from 22% for its vision-only base model to 64%. Code is available at https://github.com/LogosRoboticsGroup/TacPAC.

发表机构

• School of Data Science, Fudan University（复旦大学数据科学学院）
• Shanghai Innovation Institute（上海创新研究院）
• NeoteAI

机构由 AI 辅助整理，请以论文原文为准。