聚焦分割：在干扰物存在下引导潜在动作模型
Segment to Focus: Guiding Latent Action Models in the Presence of Distractors
AI总结：
针对动作相关视觉干扰导致潜在动作模型失效的问题，提出MaskLAM方法，利用分割基础模型（如SAM）零样本获取智能体掩码，限制重建目标于智能体像素，迫使潜在动作编码内源动态，显著提升下游策略性能。
摘要：
潜在动作模型（LAMs）为在大规模无动作视频上预训练具身智能体提供了一条有前景的路径。它们推断连续观测之间的潜在动作，之后可以使用少量标签解码为真实动作。然而，近期工作表明，在真实世界视频中常见的动作相关视觉干扰物（如动态背景、相机抖动或其他移动物体）存在时，这一方法会失败。在这些场景中，标准重建目标会驱使潜在动作编码外源运动而非智能体控制的动态，导致微调后的策略性能不佳。然而，我们观察到内源和外源因素通常在像素空间中是空间分离的：控制相关的变化集中在智能体上，而干扰物运动发生在别处。我们利用这一观察，将重建目标限制在智能体像素上，迫使潜在动作解释智能体控制的动态而非外源动态。我们将该方法称为MaskLAM；它从现成的分割基础模型（如SAM）中零样本获取智能体掩码，并且在预训练期间不需要架构更改、辅助损失或动作标签。在两个连续控制基准（Distracting Control Suite、Distracting Meta-World）上，MaskLAM将归一化线性探针MSE降低了最多$3.51 imes$，并将归一化回报提高了最多$4.97 imes$，相比LAPO，同时缩小了与依赖真实动作监督的LAOM-Labels之间的差距。
英文摘要：
Latent action models (LAMs) offer a promising path to pre-training embodied agents on large amounts of action-free video. They infer latent actions between consecutive observations that can later be decoded to ground-truth actions using a small number of labels. However, recent work has shown that this recipe fails in the presence of action-correlated visual distractors common in real-world video, such as dynamic backgrounds, camera shake, or other moving objects. In these scenarios, the standard reconstruction objective drives latent actions to encode exogenous motion instead of agent-controlled dynamics, resulting in policies that underperform when fine-tuned. We observe, however, that endogenous and exogenous factors are typically spatially separated in pixel space: control-relevant change is concentrated on the agent, while distractor motion occurs elsewhere. We exploit this observation by restricting the reconstruction objective to agent pixels, forcing latent actions to explain agent-controlled dynamics rather than exogenous ones. We call this method MaskLAM; it obtains the agent mask zero-shot from off-the-shelf segmentation foundation models (e.g., SAM) and requires no architectural changes, auxiliary losses, or action labels during pre-training. Across two continuous-control benchmarks (Distracting Control Suite, Distracting Meta-World), MaskLAM reduces normalized linear-probe MSE by up to $3.51\times$ and improves normalized return by up to $4.97\times$ over LAPO, while narrowing the gap to LAOM-Labels, which relies on ground-truth action supervision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
