面向灵巧操作的预测力注意力自适应视觉-触觉融合
Adaptive Visuo-Tactile Fusion with Predictive Force Attention for Dexterous Manipulation
AI总结：
本文提出力引导注意力融合模块和自监督未来力预测辅助任务，实现视觉与触觉模态的自适应权重调整，在三个真实世界接触密集任务中取得93%平均成功率。
摘要：
有效利用多传感器数据对于机器人在多样化任务中的泛化能力至关重要。然而，这些模态的异构特性使得融合变得困难。现有方法提出了获取全面融合特征的策略，但往往忽略了每个模态在操作的不同阶段需要不同程度的注意力这一事实。为了解决这一问题，我们提出了一种力引导的注意力融合模块，该模块无需人工标注即可自适应地调整视觉和触觉特征的权重。我们还引入了一个自监督的未来力预测辅助任务，以强化触觉模态、改善数据不平衡并鼓励适当的调整。我们的方法在真实世界实验中，在三个细粒度、接触密集的任务上取得了93%的平均成功率。进一步分析表明，我们的策略在不同的操作阶段适当地调整了对每个模态的注意力。视频可在 https://adaptac-dex.github.io/ 查看。
英文摘要：
Effectively utilizing multi-sensory data is important for robots to generalize across diverse tasks. However, the heterogeneous nature of these modalities makes fusion challenging. Existing methods propose strategies to obtain comprehensively fused features but often ignore the fact that each modality requires different levels of attention at different manipulation stages. To address this, we propose a force-guided attention fusion module that adaptively adjusts the weights of visual and tactile features without human labeling. We also introduce a self-supervised future force prediction auxiliary task to reinforce the tactile modality, improve data imbalance, and encourage proper adjustment. Our method achieves an average success rate of 93% across three fine-grained, contactrich tasks in real-world experiments. Further analysis shows that our policy appropriately adjusts attention to each modality at different manipulation stages. The videos can be viewed at https://adaptac-dex.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
