看见、行动、适应：通过个性化VLM引导代理的主动感知用于无监督跨域视觉适应
See, Act, Adapt: Active Perception for Unsupervised Cross-Domain Visual Adaptation via Personalized VLM-Guided Agent
AI总结：
Sea$^2$通过智能姿态控制代理实现无监督跨域视觉适应，利用现成感知模型提升视觉任务性能。
摘要：
预训练的感知模型在通用图像领域表现出色，但在新的环境中如室内场景中会显著退化。传统解决方法是在下游数据上微调，这会导致先前知识的灾难性遗忘，并且需要成本高昂、特定场景的标注。我们提出了一种范式转变，通过Sea$^2$（看见、行动、适应）：而不是适应感知模块本身，我们通过智能姿态控制代理来适应如何部署。Sea$^2$保持所有感知模块冻结，训练过程中不需要下游标签，并仅使用标量感知反馈来引导代理向信息丰富的视角导航。特别地，我们通过一个两阶段训练流程将视觉语言模型（VLM）转化为低层姿态控制器：首先在基于规则的探索轨迹上微调它，系统性地探测室内场景，然后通过无监督强化学习优化策略，从感知模块的输出和置信度构建奖励。不同于以往将探索与特定模型结合或收集数据重新训练这些模型的主动感知方法，Sea$^2$直接利用各种任务的现成感知模型，而无需重新训练。我们在三个视觉感知任务上进行了实验，包括视觉 grounding、分割和 3D 箱估计，在 ReplicaCAD 数据集上分别取得了 13.54%、15.92% 和 27.68% 的性能提升。
英文摘要：
Pre-trained perception models excel in generic image domains but degrade significantly in novel environments like indoor scenes. The conventional remedy is fine-tuning on downstream data which incurs catastrophic forgetting of prior knowledge and demands costly, scene-specific annotations. We propose a paradigm shift through Sea$^2$ (See, Act, Adapt): rather than adapting the perception modules themselves, we adapt how they are deployed through an intelligent pose-control agent. Sea$^2$ keeps all perception modules frozen, requiring no downstream labels during training, and uses only scalar perceptual feedback to navigate the agent toward informative viewpoints. Specially, we transform a vision-language model (VLM) into a low-level pose controller through a two-stage training pipeline: first fine-tuning it on rule-based exploration trajectories that systematically probe indoor scenes, and then refining the policy via unsupervised reinforcement learning that constructs rewards from the perception module's outputs and confidence. Unlike prior active perception methods that couple exploration with specific models or collect data for retraining them, Sea$^2$ directly leverages off-the-shelf perception models for various tasks without the need for retraining. We conducted experiments on three visual perception tasks, including visual grounding, segmentation and 3D box estimation, with performance improvements of 13.54%, 15.92% and 27.68% respectively on dataset ReplicaCAD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
