RegCL:用于多感官媒体视觉 grounding 的紧凑持续 SAM 适配方法
RegCL: Compact Continual SAM Adaptation for Visual Grounding in Multi-Sensorial Media
AI总结:
针对多感官媒体中 SAM 在动态领域性能下降的问题,提出非重放持续适配框架 RegCL,通过合并轻量适配模块实现紧凑部署,在五数据集实验中优于基线方法。
摘要:
包括 AR/VR、远程操作和具身智能在内的多感官媒体系统,需要视觉 grounding 模块在感知环境和应用领域演变时仍保持可靠性能。Segment Anything Model(SAM)为密集视觉分割提供了强大基础,但其在医学图像、伪装场景、阴影主导环境等专业及动态到达的领域上性能会下降。现有持续学习方法通常依赖重放数据或增长特定领域模块,限制了其在演变媒体流水线中的紧凑部署。为解决该问题,我们提出 RegCL,一种非重放持续适配框架,通过增量模型合并将多领域分割知识整合到单个 SAM 适配器中。RegCL 合并轻量适配模块(如 LoRA 风格的 AugModules),通过优化合并模型与特定领域适配器之间的预测一致性,同时保留紧凑的历史特征统计。在五个异构分割数据集上的实验表明,RegCL 在领域增量学习下实现了出色的保留与适配能力,优于竞争性的非重放持续学习及合并基线。这些结果表明,RegCL 可作为演变多感官媒体流水线的紧凑视觉适配组件,代码可在指定链接获取。
英文摘要:
Multi-sensorial media systems, including AR/VR, remote operation, and embodied AI, require visual grounding modules that remain reliable as sensing environments and application domains evolve. The Segment Anything Model (SAM) provides a strong foundation for dense visual segmentation, but its performance degrades on specialized and dynamically arriving domains such as medical imagery, camouflaged scenes, and shadow-dominant environments. Existing continual learning methods often rely on replay data or growing domain-specific modules, limiting compact deployment in evolving media pipelines. To address this issue, we propose RegCL, a non-replay continual adaptation framework that consolidates multi-domain segmentation knowledge into a single SAM adapter through incremental model merging. RegCL merges lightweight adaptation modules, e.g., LoRA-style AugModules, by optimizing prediction consistency between the merged model and domain-specific adapters while carrying forward compact historical feature statistics. Experiments across five heterogeneous segmentation datasets show that RegCL achieves strong retention and adaptation under domain-incremental learning, outperforming competitive non-replay continual learning and merging baselines. These results suggest that RegCL can serve as a compact visual adaptation component for evolving multi-sensorial media pipelines. The code is available at \href{https://github.com/Anderw-S/RegCL}{https://github.com/Anderw-S/RegCL}
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。