arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.01158cs.CVcs.RO
收藏

通过恢复损坏的视觉输入来提高视觉-语言-动作模型的鲁棒性

Improving Robustness of Vision-Language-Action Models by Restoring Corrupted Visual Inputs

Department of Electronics, Information, and Bioengineering(电子信息与生物工程系)

Daniel Yezid Guarnizo Orjuela, Leonardo Scappatura, Veronica Di Gennaro, Riccardo Andrea Izzo, Gianluca Bardaro, Matteo Matteucci

AI总结:

本研究提出CRT模块,通过对抗训练恢复损坏的视觉输入,提升VLA模型在现实环境中的鲁棒性与性能稳定性。

摘要:

视觉-语言-动作(VLA)模型已作为一种主导范式出现在通用机器人操作中,统一了感知和控制于单一端到端架构中。然而,尽管在受控环境中表现出色,可靠的现实世界部署却严重受制于其对视觉干扰的脆弱性。虽然现有文献广泛探讨了由场景几何引起的物理遮挡问题,但一种关键模式仍未被探索:图像损坏。这些传感器级的瑕疵,从电子噪声和死像素到镜头污渍,直接在解释之前破坏了视觉信号的完整性。在本工作中,我们量化了这种脆弱性,证明了最先进的VLA模型,如π₀.₅和SmolVLA,在常见的信号瑕疵下,性能会急剧下降,从90%的成功率降至最低2%。为缓解这一问题,我们引入了损坏恢复变压器(CRT),这是一种即插即用且模型无关的视觉变压器,旨在使VLA模型免受传感器干扰。利用对抗训练目标,CRT从损坏的输入中恢复干净的观测,无需对底层模型进行计算昂贵的微调。在LIBERO和Meta-World基准上的广泛实验表明,CRT有效恢复了失去的性能,使VLA能够在严重视觉损坏下保持接近基线的成功率。

英文摘要:

Vision-Language-Action (VLA) models have emerged as a dominant paradigm for generalist robotic manipulation, unifying perception and control within a single end-to-end architecture. However, despite their success in controlled environments, reliable real-world deployment is severely hindered by their fragility to visual disturbances. While existing literature extensively addresses physical occlusions caused by scene geometry, a critical mode remains largely unexplored: image corruptions. These sensor-level artifacts, ranging from electronic noise and dead pixels to lens contaminants, directly compromise the integrity of the visual signal prior to interpretation. In this work, we quantify this vulnerability, demonstrating that state-of-the-art VLAs such as $π_{0.5}$ and SmolVLA, suffer catastrophic performance degradation, dropping from 90\% success rates to as low as 2\%, under common signal artifacts. To mitigate this, we introduce the Corruption Restoration Transformer (CRT), a plug-and-play and model-agnostic vision transformer designed to immunize VLA models against sensor disturbances. Leveraging an adversarial training objective, CRT restores clean observations from corrupted inputs without requiring computationally expensive fine-tuning of the underlying model. Extensive experiments across the LIBERO and Meta-World benchmarks demonstrate that CRT effectively recovers lost performance, enabling VLAs to maintain near-baseline success rates, even under severe visual corruption.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑