Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents
Qwen Large Model Application Team, Alibaba(阿里云大模型应用团队);Alibaba Zhejiang University(阿里巴巴浙江大学);University of Waterloo(多伦多大学);Vector Institute(向量研究所);Nanjing University(南京大学);Binjiang Institute of Zhejiang University(浙江大学滨江学院)
Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools. We present Dynamo, a training-free framework that adapts a frozen VLM without any weight updates. On a small labeled training subset, the agent inspects its own correct and incorrect attempts and evolves two complementary capabilities: reusable reasoning skills for cognitive bottlenecks, and executable visual tools for perceptual ones. Each generated tool is paired with a skill that specifies when to invoke it, and both capability types accumulate in a persistent library. Across four visual reasoning benchmarks and five VLM backbones, Dynamo improves direct inference on all 20 model--benchmark settings (avg. +5.6 acc). When the tool set is given in advance, the framework learns when to call each tool, and per-step tool choice improves on every tested backbone. Against task-specific RL (VTool-R1, DeepEyes), Dynamo closes 65--99% of the RL gap at a fraction of the compute, and combines additively with RL when available.
PerspAct: Enhancing LLM Situated Collaboration Skills through Perspective Taking and Active Vision
University of Milan-Bicocca(米兰-比科卡大学);University of Naples Federico II(那不勒斯费德里科二世大学);Oversonic Robotics(Oversonic机器人公司);University of Essex(埃塞克斯大学);TUM School of Social Sciences and Technology(慕尼黑技术大学社会科学与技术学院)
Discovering Failure Modes in Vision-Language Models using RL
Mila – Québec AI Institute(魁北克AI研究所);Université de Montréal(蒙特利尔大学);Michigan State University(密歇根州立大学);McGill University(麦吉尔大学);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
Investigating Cross-Modal Skill Injection: Scenarios, Methods, and Hyperparameters
State Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机科学学院,北京大学);WeChat AI, Tencent Inc., China(腾讯公司,中国);The University of Hong Kong(香港大学)
EmbodiedSkills: World Action Model-augmented Cascaded Skills for Vision-Language-Action Agent
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Cornell University(康奈尔大学);Universal Ubiquitous AI Co., Ltd.(Universal Ubiquitous AI有限公司);Hangzhou DEEP Robotics Technology Co., Ltd.(杭州深度机器人科技有限公司);National University of Singapore(新加坡国立大学)
Sample-Efficient Robot Skill Learning for Construction Tasks: Benchmarking Hierarchical Reinforcement Learning and Vision-Language-Action VLA Model
Department of Civil Engineering, Stony Brook University(土木工程系,石溪大学);Department of Civil and Environmental Engineering, Virginia Tech(土木与环境工程系,弗吉尼亚理工大学);Department of Electrical and Computer Engineering, Virginia Tech(电气与计算机工程系,弗吉尼亚理工大学)
CLASP: Language-Driven Robot Skill Selection and Composition using Task-Parameterized Learning
German Aerospace Center (DLR), Institute of Robotics and Mechatronics (RMC)(德国航空航天中心(DLR),机器人与机电一体化研究所(RMC));Technical University of Munich (TUM)(慕尼黑工业大学(TUM))
OSS: Open Suturing Skills Vision-Based Assessment Challenge 2024-2025
Department of Translational Surgical Oncology, National Center for Tumor Diseases (NCT/UCC) Dresden(转化外科肿瘤学部,肿瘤疾病国家中心(NCT/UCC)德累斯顿);The Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden University of Technology(具有人环路触觉互联网中心(CeTI),德累斯顿技术大学);Department of Oral and Maxillofacial Surgery, University Hospital RWTH Aachen(口腔和颌面外科部,亚琛大学医院);Center for Tooth-, Mouth- and Jaw Medicine, University Göttingen(牙科、口科和颌科医学中心,哥廷根大学);Institute of Medical Informatics, University Hospital RWTH Aachen(医学信息学研究所,亚琛大学医院);Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology(医学系和卡尔·戈斯塔·卡鲁斯大学医院,德累斯顿技术大学);German Cancer Research Center (DKFZ)(德国癌症研究中心(DKFZ));Muroran Institute of Technology(牟然技术学院);Niigata University of Health and Welfare(北九州市保健福利大学);Konica Minolta, Inc.(柯尼卡美能达公司);Jmees, Inc.(Jmees公司);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(计算机科学与工程部,香港科学与技术大学);Center Algoritmi/LASI, University of Minho(算法中心/ALASI,米尼奥大学);Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho(生命与健康科学研究院(ICVS),医学院,米尼奥大学);ICVS/3B's - PT Government Associate Laboratory(ICVS/3B's - PT政府附属实验室);Institute for AI in Medicine (IKIM), University Medicine Essen(医学人工智能研究所(IKIM),埃森大学医学部);The Faculty of Data and Decisions Science, Technion - Israel Institute of Technology(数据与决策科学系,技术学院-以色列理工学院);UCL Hawkes Institute, University College London(UCL Hawkes研究所,伦敦大学学院);School of Computing, Queen's University(计算学院,皇后大学);Department of Transdisciplinary Medicine, Seoul National University Hospital(跨学科医学部,首尔国立大学医院);Interdisciplinary Program in Medical Informatics, Seoul National University(医学信息学跨学科项目,首尔国立大学);Department of Clinical Medical Sciences, Seoul National University(临床医学科学部,首尔国立大学);Institute of Convergence Medicine with Innovative Technology, Seoul National University Hospital(融合医学与创新技术研究所,首尔国立大学医院);Department of Surgery, Seoul National University College of Medicine and Seoul National University Hospital(外科部,首尔国立大学医学院和首尔国立大学医院)
SpaceMind: A Modular and Self-Evolving Embodied Vision-Language Agent Framework for Autonomous On-orbit Servicing
University of Chinese Academy of Sciences(中国科学院大学);Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences(中国科学院空间利用技术与工程中心)