CRL-VLA：持续视觉-语言-动作学习
CRL-VLA: Continual Vision-Language-Action Learning
AI总结：
CRL-VLA通过双批评者架构和目标条件价值公式，实现VLA模型在持续学习中的稳定与可塑性平衡，优于现有方法。
摘要：
终身学习对于在开放世界环境中具有身体的智能体至关重要，其中强化学习微调已成为一种重要的范式，使视觉-语言-动作（VLA）模型能够通过环境交互掌握灵巧操作。因此，持续强化学习（CRL）是部署VLA模型在终身机器人场景中的有希望的途径，但平衡稳定性（保留旧技能）和可塑性（学习新技能）仍然是现有方法面临的重大挑战。我们引入了CRL-VLA，一种用于VLA模型持续后训练的框架，具有严格的理论界限。我们推导了一个统一的性能界限，将稳定性-可塑性权衡与目标条件优势幅度联系起来，该幅度通过策略分歧进行缩放。CRL-VLA通过不对称调节解决这一难题：在先前任务上限制优势幅度，同时在新任务上允许受控增长。这通过一个简单但有效的双批评者架构实现，其中包含新的目标条件价值公式（GCVF），其中冻结的批评者锚定语义一致性，可训练的估计器驱动适应。在LIBERO基准测试中，CRL-VLA证明了其有效协调这些相互冲突的目标，优于基线方法在反遗忘和前向适应方面。
英文摘要：
Lifelong learning is critical for embodied agents in open-world environments, where reinforcement learning fine-tuning has emerged as an important paradigm to enable Vision-Language-Action (VLA) models to master dexterous manipulation through environmental interaction. Thus, Continual Reinforcement Learning (CRL) is a promising pathway for deploying VLA models in lifelong robotic scenarios, yet balancing stability (retaining old skills) and plasticity (learning new ones) remains a formidable challenge for existing methods. We introduce CRL-VLA, a framework for continual post-training of VLA models with rigorous theoretical bounds. We derive a unified performance bound linking the stability-plasticity trade-off to goal-conditioned advantage magnitude, scaled by policy divergence. CRL-VLA resolves this dilemma via asymmetric regulation: constraining advantage magnitudes on prior tasks while enabling controlled growth on new tasks. This is realized through a simple but effective dual-critic architecture with novel Goal-Conditioned Value Formulation (GCVF), where a frozen critic anchors semantic consistency and a trainable estimator drives adaptation. Experiments on the LIBERO benchmark demonstrate that CRL-VLA effectively harmonizes these conflicting objectives, outperforming baselines in both anti-forgetting and forward adaptation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
