视觉大模型 / VLM

Galaxea Open-World Dataset and G0 Dual-System VLA Model

Tao Jiang, Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Jianning Cui, Xiao Liu, Shuiqi Cheng, Jiyang Gao, Huazhe Xu, Hang Zhao

机构 * Galaxea Team（Galaxea 团队） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments https://opengalaxea.github.io/G0/

SafeBimanual: Diffusion-based Trajectory Optimization for Safe Bimanual Manipulation

Haoyuan Deng, Wenkai Guo, Qianzhun Wang, Zhenyu Wu, Ziwei Wang

机构 * Nanyang Technological University（南洋理工大学） ； Beijing University of Posts and Telecommunications（北京邮电大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments Project website is at: https://denghaoyuan123.github.io/SafeBimanip/

AeroDuo: Aerial Duo for UAV-based Vision and Language Navigation

Ruipu Wu, Yige Zhang, Jinyu Chen, Linjiang Huang, Shifeng Zhang, Xu Zhou, Liang Wang, Si Liu

机构 * Beihang University（北京航空航天大学） ； Sangfor Technologies Inc.（深信服科技有限公司） ； Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所） 纠错

专题命中 GUI与屏幕智能体 ：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

Comments Accepted by ACM MM 2025

UI-Venus Technical Report: Building High-performance UI Agents with RFT

Zhangxuan Gu, Zhengwen Zeng, Zhenyu Xu, Xingran Zhou, Shuheng Shen, Yunfei Liu, Beitong Zhou, Changhua Meng, Tianyu Xia, Weizhi Chen, Yue Wen, Jingya Dou, Fei Tang, Jinzhen Lin, Yulin Liu, Zhenlin Guo, Yichen Gong, Heng Jia, Changlong Gao, Yuan Guo, Yong Deng, Zhenyu Guo, Liang Chen, Weiqiang Wang

专题命中 GUI与屏幕智能体 ：grounding(abstract)；multimodal large language model(abstract)；分类 cs.CV

Rhetorical Text-to-Image Generation via Two-layer Diffusion Policy Optimization

Yuxi Zhang, Yueting Li, Xinyu Du, Sibo Wang

机构 * The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）） ； University of California, Berkeley（加州大学伯克利分校） 纠错

专题命中 GUI与屏幕智能体 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Cross-Modal Dual-Causal Learning for Long-Term Action Recognition

Xu Shaowu, Jia Xibin, Gao Junyu, Sun Qianmei, Chang Jing, Fan Chao

机构 * Beijing University of Technology（北京理工大学） ； Chinese Academy of Sciences（中国科学院） ； Capital Medical University（首都医科大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

ERMV: Editing 4D Robotic Multi-view images to enhance embodied agents

Chang Nie, Guangming Wang, Zhe Lie, Hesheng Wang

机构 * School of Automation and Intelligent Sensing, Shanghai Jiao Tong University and Key Laboratory of System Control and Information Processing, Ministry of Education of China（自动化与智能感知学院，上海交通大学；系统控制与信息处理重点实验室，中华人民共和国教育部） 纠错

专题命中 GUI与屏幕智能体 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Tactile-VLA: Unlocking Vision-Language-Action Model's Physical Knowledge for Tactile Generalization

Jialei Huang, Shuo Wang, Fanqi Lin, Yihang Hu, Chuan Wen, Yang Gao

专题命中 GUI与屏幕智能体 ：VLM(abstract)；grounding(abstract)；分类 cs.LG

Adaptive Framework for Ambient Intelligence in Rehabilitation Assistance

Gábor Baranyi, Zsolt Csibi, Kristian Fenech, Áron Fóthi, Zsófia Gaál, Joul Skaf, András Lőrincz

机构 * Emineo Private Hospital（埃米尼欧私人医院） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments The paper has been submitted to a journal and waiting for review

RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models

Jacky Kwok, Christopher Agia, Rohan Sinha, Matt Foutter, Shulu Li, Ion Stoica, Azalia Mirhoseini, Marco Pavone

机构 * Stanford University（斯坦福大学） ； UC Berkeley（加州大学伯克利分校） ； NVIDIA Research（NVIDIA研究） 纠错

专题命中 GUI与屏幕智能体 ：vision language model(abstract)；VLM(abstract)；分类 cs.AI

Balancing Performance and Efficiency in Zero-shot Robotic Navigation

Dmytro Kuzmenko, Nadiya Shvai

机构 * Department of Multimedia Systems, National University of Kyiv-Mohyla Academy, Kyiv, Ukraine（多媒体系统系，基辅-莫希拉学院国家大学，乌克兰基辅） ； Department of Mathematics, National University of Kyiv-Mohyla Academy, Kyiv, Ukraine（数学系，基辅-莫希拉学院国家大学，乌克兰基辅） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Submitted to ICTERI 2024 Posters Track

Journal ref ICTERI 2024: Communications in Computer and Information Science, vol. 2020, pp. 370-381, Springer, 2025

HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model

Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, Chengkai Hou, Mengdi Zhao, KC alex Zhou, Pheng-Ann Heng, Shanghang Zhang

机构 * State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学） ； Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院） ； CUHK（香港中文大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

From Intention to Execution: Probing the Generalization Boundaries of Vision-Language-Action Models

Irving Fang, Juexiao Zhang, Shengbang Tong, Chen Feng

机构 * New York University（纽约大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Comments Under review

Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights

Mathieu Andreux, Breno Baldas Skuk, Hamza Benchekroun, Emilien Biré, Antoine Bonnet, Riaz Bordie, Nathan Bout, Matthias Brunel, Pierre-Louis Cedoz, Antoine Chassang, Mickaël Chen, Alexandra D. Constantinou, Antoine d'Andigné, Hubert de La Jonquière, Aurélien Delfosse, Ludovic Denoyer, Alexis Deprez, Augustin Derupti, Michael Eickenberg, Mathïs Federico, Charles Kantor, Xavier Koegler, Yann Labbé, Matthew C. H. Lee, Erwan Le Jumeau de Kergaradec, Amir Mahla, Avshalom Manevich, Adrien Maret, Charles Masson, Rafaël Maurin, Arturo Mena, Philippe Modard, Axel Moyal, Axel Nguyen Kerbel, Julien Revelle, Mats L. Richter, María Santos, Laurent Sifre, Maxime Theillard, Marc Thibault, Louis Thiry, Léo Tronchon, Nicolas Usunier, Tony Wu

机构 * H Company（H公司） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments Alphabetical order

LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks

Yi Yang, Jiaxuan Sun, Siqi Kou, Yihan Wang, Zhijie Deng

机构 * Fudan University（复旦大学） ； ShanghaiTech University（上海科技大学） ； Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 GUI与屏幕智能体 ：vision language model(abstract)；VLM(abstract)；分类 cs.AI

Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better

Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, Sergey Levine

机构 * Physical Intelligence 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.LG

AdInject: Real-World Black-Box Attacks on Web Agents via Advertising Delivery

Haowei Wang, Junjie Wang, Xiaojun Jia, Rupeng Zhang, Mingyang Li, Zhe Liu, Yang Liu, Qing Wang

机构 * State Key Laboratory of Intelligent Game（智能游戏国家重点实验室） ； Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； Nanyang Technological University（南洋理工大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigation

Jiaping Xiao, Cheng Wen Tsao, Yuhang Zhang, Mir Feroskhan

机构 * School of Mechanical and Aerospace Engineering, Nanyang Technological University（机械与航空航天工程学院，南洋理工大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments This work has been submitted for possible publication

QUART-Online: Latency-Free Large Multimodal Language Model for Quadruped Robot Learning

Xinyang Tong, Pengxiang Ding, Yiguo Fan, Donglin Wang, Wenjie Zhang, Can Cui, Mingyang Sun, Han Zhao, Hongyin Zhang, Yonghao Dang, Siteng Huang, Shangke Lyu

机构 * MiLAB, Westlake University, Hangzhou, 310030, China（西交利物浦大学微实验室，杭州，310030，中国） ； Zhejiang University, Hangzhou, 310027, China（浙江大学，杭州，310027，中国） ； Beijing University of Posts and Telecommunications, Beijing, 100876, China（北京邮电大学，北京，100876，中国） 纠错

专题命中 GUI与屏幕智能体 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted to ICRA 2025; Github page: https://quart-online.github.io

UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learning

Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, Hongsheng Li

机构 * vivo AI Lab（vivo人工智能实验室） 纠错

专题命中 GUI与屏幕智能体 ：grounding(abstract)；multimodal large language model(abstract)；分类 cs.AI

Comments Updated UI-R1-E-3B

TinyClick: Single-Turn Agent for Empowering GUI Automation

Pawel Pawlowski, Krystian Zawistowski, Wojciech Lapacz, Adam Wiacek, Marcin Skorupa, Sebastien Postansque, Jakub Hoscilowicz

机构 * Warsaw University of Technology（华沙技术大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments Accepted to Interspeech 2025

Building a Stable Planner: An Extended Finite State Machine Based Planning Module for Mobile GUI Agent

Fanglin Mo, Junzhe Chen, Haoxuan Zhu, Xuming Hu

机构 * Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）） ； School of Computer Science & Engineering, South China University of Technology（华南理工大学计算机科学与工程学院） 纠错

专题命中 GUI与屏幕智能体 ：vision language model(abstract)；VLM(abstract)；分类 cs.AI

Comments 10 pages. Submitted to EMNLP 2025

Vision and Intention Boost Large Language Model in Long-Term Action Anticipation

Congqi Cao, Lanshu Hu, Yating Yu, Yanning Zhang

机构 * Northwestern Polytechnical University（西北工业大学） 纠错

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV

Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach

Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, Michael R. Lyu

机构 * The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 GUI与屏幕智能体 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments Accepted by FSE 2025

SpiritSight Agent: Advanced GUI Agent with One Look

Zhiyuan Huang, Ziming Cheng, Junting Pan, Zhaohui Hou, Mingjie Zhan

专题命中 GUI与屏幕智能体 ：vision language model(abstract)；grounding(abstract)；分类 cs.CV

Comments Paper accepted to CVPR 2025

SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation

Jingxuan Chen, Derek Yuen, Bin Xie, Yuhao Yang, Gongwei Chen, Zhihao Wu, Li Yixing, Xurui Zhou, Weiwen Liu, Shuai Wang, Kaiwen Zhou, Rui Shao, Liqiang Nie, Yasheng Wang, Jianye Hao, Jun Wang, Kun Shao

专题命中 GUI与屏幕智能体 ：grounding(abstract)；MLLM(abstract)；分类 cs.AI

Comments ICLR 2025 Spotlight

Does Chain-of-Thought Reasoning Help Mobile GUI Agent? An Empirical Study

Li Zhang, Longxi Gao, Mengwei Xu

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；VLM(abstract)；分类 cs.AI

World Modeling Makes a Better Planner: Dual Preference Optimization for Embodied Task Planning

Siyin Wang, Zhaoye Fei, Qinyuan Cheng, Shiduo Zhang, Panpan Cai, Jinlan Fu, Xipeng Qiu

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV

TLA: Tactile-Language-Action Model for Contact-Rich Manipulation

Peng Hao, Chaofan Zhang, Dingzhe Li, Xiaoge Cao, Xiaoshuai Hao, Shaowei Cui, Shuo Wang

专题命中 GUI与屏幕智能体 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV

Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy

Zaijing Li, Yuquan Xie, Rui Shao, Gongwei Chen, Dongmei Jiang, Liqiang Nie

专题命中 GUI与屏幕智能体 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI

Comments Accept to CVPR 2025, Project page: https://cybertronagent.github.io/Optimus-2.github.io/