FORCE: Efficient VLA Reinforcement Fine-Tuning via Value-Calibrated Warm-up and Self-Distillation
Institute of Automation, Chinese Academy of Sciences, Beijing, China(中国科学院自动化研究所);Beijing Academy of Artificial Intelligence, Beijing, China(北京人工智能研究院);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China(北京大学计算机科学学院多媒体信息处理国家重点实验室)
Vision-Language-Action (VLA) models are often constrained by the imitation ceiling imposed by sub-optimal data. While Reinforcement Learning (RL) fine-tuning can surpass this limit, it is notoriously sample inefficient. This challenge arises from two core issues: (1) catastrophic initial unlearning due to an unstable Q-function and (2) inefficient policy updates caused by low-quality exploration data, often forcing a reliance on costly human interventions. We introduce FORCE, a 3-stage framework that stabilizes fine-tuning by tackling both issues. FORCE first incorporates a Value-Calibrated Warm-Up phase, utilizing on-policy rollouts to mitigate the distributional shift of the Q-function. Subsequently, during the online stage, this calibrated Q-function acts as a filter for both the policy's own action proposals and expert data, ensuring only high-value actions are used for the policy update. We evaluate FORCE on various simulation and real-world tasks, and the result shows that FORCE achieves a 79% absolute improvement in success rates and outperform prior RL methods by 10%, while accelerating training by 32.5%. Critically, it mitigates the common success rate drop and achieves this robust performance without human intervention, marking a significant step towards deploying capable and autonomous robotic agents.
VLAConf: Calibrated Task-Success Confidence for Vision-Language-Action Models
Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China(南方科技大学电子与电气工程系,深圳,中国);Zhongguancun Academy, Beijing, China(中关村学院,北京,中国);National Cybersecurity Academy, Wuhan University, Wuhan, China(武汉大学国家网络安全学院,武汉,中国)
G$^3$VLA: Geometric Inductive Bias for Vision-Language-Action Models
New York University Shanghai(上海纽约大学);Technical University of Denmark(丹麦技术大学);MBZUAI - Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);New York University Abu Dhabi(纽约大学阿布扎比分校)
Trust-Calibrated VLA Policy Post-Training with Evolving Imagination
Sun Yat-sen University(中山大学);Shenzhen Loop Area Institute(深圳河套学院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shanghai AI Laboratory(上海人工智能实验室);University of Oxford(牛津大学);University of Science and Technology of China(中国科学技术大学)
VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电子与电气工程学院)
RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models
State Key Laboratory of Mechanical System and Vibration(机械系统与振动国家重点实验室);Shanghai Key Laboratory of Intelligent Robotics(上海智能机器人重点实验室);School of Mechanical Engineering, Shanghai Jiao Tong University(上海交通大学机械工程学院)
TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors
School of Computer Science, Wuhan University(武汉大学计算机科学学院);D-Robotics(D-机器人公司);HUST(华中科技大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Tracking
Autel Robotics(大疆创新科技有限公司);Northeast Normal University(东北师范大学);Southern University of Science and Technology(南方科技大学);Peking University(北京大学);University of Hong Kong(香港大学)
PAMAE: Phase-Aware-MoE Action Experts Towards Reliable Flow-Matching Vision-Language-Action Policies
Department of Artificial Intelligence, School of Informatics, Xiamen University(厦门大学信息学院人工智能系);Department of Computer Science, Aberystwyth University(阿伯里斯特威斯大学计算机科学系)
Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models
Tsinghua University(清华大学);Harbin Institute of Technology(哈尔滨工业大学);Peking University(北京大学);Carnegie Mellon University(卡内基梅隆大学);Shanghai AI Laboratory(上海人工智能实验室);Zhongguancun Academy(中关村学院)
TacCoRL: Integrating Tactile Feedback into VLA via Simulation
University of California, Los Angeles(加利福尼亚大学洛杉矶分校);University of California, San Diego(加利福尼亚大学圣迭戈分校);University of Electronic Science and Technology of China(电子科技大学);Peking University(北京大学);University of Utah(犹他大学)
World-Calibrated Proposal-to-Action Flow for Vision-Language-Action Models
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE, China(中兴通讯移动网络和移动多媒体技术国家重点实验室);Sun Yat-sen University(中山大学)
Bit-Flip Attacks on Vision-Language-Action Models: Action-Decoding Architecture Shapes the Vulnerability
The Hong Kong University of Science and Technology(香港科技大学);Adelaide University(阿德莱德大学);Wuhan University(武汉大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);ETH Zurich(苏黎世联邦理工学院);China University of Petroleum (East China)(中国石油大学(华东))
TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(信息处理国家重点实验室,计算机学院,北京大学);Simplexity Robotics(Simplexity机器人);Tsinghua University(清华大学);Hong Kong University of Science and Technology(香港科技大学)
TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Zhejiang University(浙江大学);Simon Fraser University(西蒙菲莎大学);AgiBot(云圣智能科技(上海)有限公司)