StateSkill-Agent: An Effective and Efficient Embodied Agent via State-conditioned Skill
The Hong Kong Polytechnic University(香港理工大学);ACE Robotics(ACE机器人公司);Nanyang Technological University(南洋理工大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);Fudan University(复旦大学)
In this work, we aim to develop an embodied MLLM agent that understands and solves embodied tasks by learning, abstracting, and reusing transferrable and generalizable behavioral patterns. To this end, we propose StateSkill, a new skill representation for embodied agents, which introduces the concept of "finite-state machines" into skill modeling for effective and efficient skill construction and learning. The core idea of StateSkill is to model embodied interaction experiences as a finite-state machine, where a skill is organized around a set of task-progress states, state transitions, and state-conditioned guidance, enabling embodied agents to dynamically adjust their skills as the task progresses while keeping skill contexts compact and stage-relevant. We further develop StateSkill-Evolve, an experience-driven evolution mechanism that iteratively refines StateSkill through retrospective analysis of new successful and failed state-guided trajectories. Upon this, we introduce StateGRPO, which integrates StateSkill evolution and model optimization via state-conditioned GRPO into a unified iterative learning loop, enabling state-conditioned skills and model capabilities to co-evolve and reinforce each other. Extensive experiments demonstrate the effectiveness of our proposed methods and trained model (i.e., StateSkill-Agent) across various embodied tasks.
Embodied Skill Bench: Benchmarking How Well Agents with Embodied Skills Perform on Physical Tasks
Shanghai Jiao Tong University(上海交通大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Sun Yat-Sen University(中山大学);Jilin University(吉林大学);Beijing University of Aeronautics and Astronautics(北京航空航天大学)
SkillMemo: Expert-guided Skill Memory Framework for Compositional Embodied Manipulation
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Department of Automation, Tsinghua University(清华大学自动化系);Nanyang Technological University(南洋理工大学);Beijing Normal University(北京师范大学)
Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis
Northeastern University, Boston, MA, USA;The Chinese University of Hong Kong, Hong Kong, China;Peking University, Beijing, China;Westlake University, Hangzhou, China;Harvard University, Cambridge, MA, USA;Purdue University, West Lafayette, IN, USA;University of Oxford, Oxford, United Kingdom
ManipulationNet: An Infrastructure for Benchmarking Real-World Robot Manipulation with Physical Skill Challenges and Embodied Multimodal Reasoning
Rice University(里士大学);U.S. National Institute of Standards and Technology(美国国家标准与技术研究院);Massachusetts Institute of Technology(麻省理工学院);Karlsruhe Institute of Technology(卡尔斯鲁厄技术大学);Autodesk Research(Autodesk研究);University of California, Berkeley(加州大学伯克利分校);KTH Royal Institute of Technology(皇家理工学院);Tsinghua University(清华大学);Columbia University(哥伦比亚大学);ASTM International(美国材料与试验协会);Carnegie Mellon University(卡内基梅隆大学);German Aerospace Center (DLR)(德国航空航天中心);University of Texas at Dallas(德克萨斯大学达拉斯分校);University of Texas at Austin(德克萨斯大学奥斯汀分校);NVIDIA Research(NVIDIA研究)
RoboOS: A Hierarchical Embodied Framework for Cross-Embodiment and Multi-Agent Collaboration
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院)
EmbodiedSkills: World Action Model-augmented Cascaded Skills for Vision-Language-Action Agent
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Cornell University(康奈尔大学);Universal Ubiquitous AI Co., Ltd.(Universal Ubiquitous AI有限公司);Hangzhou DEEP Robotics Technology Co., Ltd.(杭州深度机器人科技有限公司);National University of Singapore(新加坡国立大学)
UAVs Meet Embodied Intelligence: Bridging Human Intents and Flying Dynamics Via Harnessing Physical-Digital AI Agents
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beihang University(北京航空航天大学);Shanxi Datong University(山西大同大学);Johns Hopkins University(约翰斯·霍普金斯大学);University of Sanya(三亚学院);Wenzhou-Kean University(温州肯恩大学);Macau University of Science and Technology(澳门科技大学);KOKONI 3D, Moxin Technology(魔芯科技KOKONI 3D);University of Science and Technology of China(中国科学技术大学)
Never-Ending Behavior-Cloning Agent for Robotic Manipulation
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences(机器人国家重点实验室,沈阳自动化研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);School of Automation Science and Engineering, South China University of Technology(自动化科学与工程学院,华南理工大学);Mohamed bin Zayed University of Artificial Intelligence(马斯达尔大学人工智能学院)
SpaceMind: A Modular and Self-Evolving Embodied Vision-Language Agent Framework for Autonomous On-orbit Servicing
University of Chinese Academy of Sciences(中国科学院大学);Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences(中国科学院空间利用技术与工程中心)
State Key Laboratory of Robotics and Intelligent Systems(机器人与智能系统国家重点实验室);Shenyang Institute of Automation(沈阳自动化研究所);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)