PEEK：用于机器人操作策略零样本泛化的引导性最小化图像表示
PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies
AI总结：
针对机器人操作策略泛化失败问题，提出PEEK方法，将高层推理卸载至VLMs并微调其生成末端执行器路径与任务掩码的中间表示，提升零样本泛化性能，如仿真训练3D策略真实世界性能提升41.4倍。
摘要：
机器人操作策略常因需同时学习关注位置、动作选择与执行方式而泛化失败。我们认为，关于“何处”与“何事”的高层推理可卸载至视觉语言模型（VLMs），让策略专注于“如何”行动。我们提出PEEK（Policy-agnostic Extraction of Essential Keypoints，策略无关的关键关键点提取），通过微调VLMs预测统一的基于点的中间表示：1. 指定动作的末端执行器路径，2. 指示关注位置的任务相关掩码。这些标注直接叠加于机器人观测，使表示具备策略无关性且可跨架构迁移。为实现可扩展训练，我们引入自动标注流水线，在覆盖9种机器人形态的20余个数据集上生成标注数据。真实世界评估中，PEEK持续提升零样本泛化性能，包括仅在仿真中训练的3D策略实现41.4倍的真实世界性能提升，大型VLA和小型操作策略均获2-3.5倍增益。通过让VLMs吸收语义与视觉复杂性，PEEK为操作策略提供所需的最小线索——何处、何事与如何行动。项目网站：https://peek-robot.github.io/
英文摘要：
Robotic manipulation policies often fail to generalize because they must simultaneously learn where to attend, what actions to take, and how to execute them. We argue that high-level reasoning about where and what can be offloaded to vision-language models (VLMs), leaving policies to specialize in how to act. We present PEEK (Policy-agnostic Extraction of Essential Keypoints), which fine-tunes VLMs to predict a unified point-based intermediate representation: 1. end-effector paths specifying what actions to take, and 2. task-relevant masks indicating where to focus. These annotations are directly overlaid onto robot observations, making the representation policy-agnostic and transferable across architectures. To enable scalable training, we introduce an automatic annotation pipeline, generating labeled data across 20+ robot datasets spanning 9 embodiments. In real-world evaluations, PEEK consistently boosts zero-shot generalization, including a 41.4x real-world improvement for a 3D policy trained only in simulation, and 2-3.5x gains for both large VLAs and small manipulation policies. By letting VLMs absorb semantic and visual complexity, PEEK equips manipulation policies with the minimal cues they need--where, what, and how. Website at https://peek-robot.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
