以代理为中心的观测适应用于动态扰动下的鲁棒视觉控制
Agent-Centric Observation Adaptation for Robust Visual Control under Dynamic Perturbations
AI总结：
本文提出ACO-MoE方法，通过结合路由的修复专家和前景掩码分支，解决动态扰动下的视觉控制问题，提升下游任务性能。
摘要：
现实中的视觉系统面临时间变化的扰动，包括天气、传感器噪声、压缩伪影和背景干扰。现有图像修复方法通常针对固定损坏类型设计，并优化像素级保真度，留下两个问题：修复在非平稳损坏切换下的表现如何，以及像素级保真度是否保留下游模型所需的信息。为此，我们引入了视觉退化控制套件（VDCS），在渲染场景中注入马尔可夫切换物理退化。我们进一步识别了基于重建的表示的基本失败模式：忠实重建受损观测迫使潜在状态编码特定于损坏的干扰信息，从而污染下游模型。从信息瓶颈的角度来看，将表示锚定在干净的前景上消除了这种污染。受此分析启发，我们提出了一种冻结、即插即用的观测适配器，即混合专家（ACO-MoE），结合路由的修复专家银行和前景掩码分支。ACO-MoE完全在合成渲染数据上预训练，使用自动生成功能退化对和模拟推导的前景掩码，无需手动注释。在推理时，它仅需损坏的RGB作为输入，无需损坏标签、干净参考帧或前景掩码。在VDCS、DMC-GB和RoboSuite上，ACO-MoE在模型无关和模型基于的回溯中一致提高下游控制性能，恢复95.3%的干净输入性能，在挑战性的马尔可夫切换退化下。它还泛化到未见的视觉退化，这些退化在适配器预训练中被排除。
英文摘要：
Real-world visual systems face time-varying perturbations, including weather, sensor noise, compression artifacts, and background distractions. Existing image restoration methods are typically designed for fixed corruption types and optimized for pixel-level fidelity, leaving open two questions: how restoration behaves under non-stationary corruption switching, and whether pixel-level fidelity preserves the task-relevant information needed by downstream models. To study this setting, we introduce the Visual Degraded Control Suite (VDCS), a benchmark that injects Markov-switching physical degradations into rendered scenes. We further identify a fundamental failure mode of reconstruction-based representations: faithfully reconstructing corrupted observations forces the latent state to encode corruption-specific nuisance information, thereby contaminating downstream models. From an information-bottleneck perspective, anchoring the representation to the clean foreground eliminates this contamination. Motivated by this analysis, we propose \emph{Agent-Centric Observations with Mixture-of-Experts} (ACO-MoE), a frozen, plug-and-play observation adapter that combines a routed bank of restoration experts with a foreground-mask branch. ACO-MoE is pretrained entirely offline on synthetic rendered data with automatically generated degradation pairs and simulation-derived foreground masks, requiring no manual annotation. At inference time, it takes only corrupted RGB as input without corruption labels, clean reference frames, or foreground masks. Across VDCS, DMC-GB, and RoboSuite, ACO-MoE consistently improves downstream control with both model-free and model-based backbones, recovering 95.3\% of clean-input performance under challenging Markov-switching corruptions. It also generalizes zero-shot to unseen visual perturbations excluded from adapter pretraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
