V-Simba：释放强化学习在视觉连续控制中的架构潜力
V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
浏览论文内容
中文总结 AI 辅助
该研究针对视觉RL样本效率低的问题，提出受Simba架构启发的V-Simba视觉RL架构，基于SAC改进后在多个基准测试中性能优于或媲美现有方法，且计算效率更高。
中文摘要 AI 辅助
提升样本效率仍是强化学习（RL）的核心挑战，尤其在机器人等数据采集成本高昂的现实场景中；这一挑战在视觉RL领域更为突出，高维输入常掩盖学习信号。以往视觉RL研究多聚焦于算法层面的解决方案，如优化动力学模型或探索策略，但近期基于状态的RL进展表明，仅架构设计就能显著提升样本效率。这引出关键问题：这些架构原理能否迁移至视觉RL？为此，我们提出V-Simba——一种受基于状态RL的Simba架构启发的简单却高效的视觉RL架构。该架构构建于带数据增强的Soft Actor-Critic（SAC）之上，通过添加归一化层稳定训练，并使用逐点卷积减少计算量。尽管结构简单，V-Simba在DMC、Adroit、Meta-World基准测试中表现与现有最优方法相当或更优，且计算效率高于DrQ-v2。我们的代码已公开，链接为this https URL。
英文摘要
Improving sample efficiency remains a core challenge in reinforcement learning (RL), especially in real-world settings like robotics, where data collection is costly. This challenge is pronounced in visual RL, where high-dimensional inputs often obscure learning signals. While prior work in visual RL has focused on algorithmic solutions, such as better dynamics models or exploration strategies, recent advances in state-based RL show that architectural design alone can lead to significant gains in sample efficiency. This raises an important question: Can these architectural principles transfer to visual RL? In response, we introduce V-Simba, a simple yet effective visual RL architecture inspired by the Simba architecture from state-based RL. Built on top of Soft Actor-Critic (SAC) with data augmentation, V-Simba modifies the architecture by adding normalization layers to stabilize training and using pointwise convolutions to reduce computation. Despite its simplicity, V-Simba matches or outperforms the state-of-the-art methods across the DMC, Adroit, and Meta-World benchmarks, while being more computationally efficient than DrQ-v2. We make our code publicly available at https://github.com/DAVIAN-Robotics/V-Simba.
