面向机器人操作的以人为本表征学习
Human-oriented Representation Learning for Robotic Manipulation
AI总结：
该研究提出在预训练视觉编码器上进行以人为本的多任务感知技能微调，并用即插即用的 Task Fusion Decoder 融合技能关系，从而持续提升多种视觉编码器的机器人操作策略学习效果。
摘要：
人类天生具备可泛化的视觉表征，这使他们能够在操作任务中高效探索环境并与之交互。我们主张，这种表征会通过同时学习多种对日常场景至关重要的简单感知技能（例如手部检测、状态估计等）而自动产生；与当前纯粹基于自监督目标的最先进视觉表征相比，这种表征更适合用于学习机器人操作策略。我们通过在预训练视觉编码器之上开展以人为本的多任务微调来形式化这一思想，其中每个任务都是一项与人类—环境交互相关的感知技能。我们提出 Task Fusion Decoder（任务融合解码器），这是一种即插即用的嵌入转换器，它利用这些感知技能之间的潜在关系，引导表征学习去编码对所有感知技能都重要的有意义结构，最终增强下游机器人操作任务的学习。在仿真和真实环境中，跨一系列机器人任务与实施形态开展的大量实验表明，我们的 Task Fusion Decoder 持续改善了 R3M、MVP 和 EgoVLP 三种最先进视觉编码器用于下游操作策略学习的表征。项目页面：https://sites.google.com/view/human-oriented-robot-learning
英文摘要：
Humans inherently possess generalizable visual representations that empower them to efficiently explore and interact with the environments in manipulation tasks. We advocate that such a representation automatically arises from simultaneously learning about multiple simple perceptual skills that are critical for everyday scenarios (e.g., hand detection, state estimate, etc.) and is better suited for learning robot manipulation policies compared to current state-of-the-art visual representations purely based on self-supervised objectives. We formalize this idea through the lens of human-oriented multi-task fine-tuning on top of pre-trained visual encoders, where each task is a perceptual skill tied to human-environment interactions. We introduce Task Fusion Decoder as a plug-and-play embedding translator that utilizes the underlying relationships among these perceptual skills to guide the representation learning towards encoding meaningful structure for what's important for all perceptual skills, ultimately empowering learning of downstream robotic manipulation tasks. Extensive experiments across a range of robotic tasks and embodiments, in both simulations and real-world environments, show that our Task Fusion Decoder consistently improves the representation of three state-of-the-art visual encoders including R3M, MVP, and EgoVLP, for downstream manipulation policy-learning. Project page: https://sites.google.com/view/human-oriented-robot-learning
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
