视觉大模型 / VLM - arXivDaily 专题

2511.14499 2025-11-19 cs.CV cs.RO 83%

Enhancing End-to-End Autonomous Driving with Risk Semantic Distillaion from VLM

Jack Qin, Zhitao Wang, Yinan Zheng, Keyu Chen, Yang Zhou, Yuanxin Zhong, Siyuan Cheng

机构 * Tsinghua University（清华大学）； Laboratories, Huawei Technologies（华为技术有限公司2012实验室）

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2511.13269 2025-11-18 cs.CV 83%

Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation

Lingfeng Zhang, Yuchen Zhang, Hongsheng Li, Haoxiang Fu, Yingbo Tang, Hangjun Ye, Long Chen, Xiaojun Liang, Xiaoshuai Hao, Wenbo Ding

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision-language model(abstract)；分类 cs.CV

2511.08942 2025-11-13 cs.RO cs.AI 83%

Think, Remember, Navigate: Zero-Shot Object-Goal Navigation with VLM-Powered Reasoning

Mobin Habibpour, Fatemeh Afghah

机构 * Holcombe Department of Electrical and Computer Engineering（霍尔科姆电气与计算机工程系）； Clemson University（克莱姆森大学）

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision-language model(abstract)；分类 cs.AI

2409.15922 2025-11-11 cs.LG cs.RO 83%

The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards

Sukai Huang, Shu-Wei Liu, Nir Lipovetzky, Trevor Cohn

机构 * Google DeepMind（谷歌DeepMind）

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision-language model(abstract)；分类 cs.LG

Comments accepted by PRL Workshop Series @ ICAPS 2025. 11 main body pages, 21 appendix pages

2510.21722 2025-10-28 cs.HC cs.AI 83%

AquaVLM: Improving Underwater Situation Awareness with Mobile Vision Language Models

Beitong Tian, Lingzhi Zhao, Bo Chen, Haozhen Zheng, Jingcheng Yang, Mingyuan Wu, Deepak Vasisht, Klara Nahrstedt

机构 * University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）

专题命中 GUI与屏幕智能体：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.AI

Comments 12 pages, 10 figures, under review

2509.24917 2025-09-30 cs.RO cs.LG 83%

From Code to Action: Hierarchical Learning of Diffusion-VLM Policies

Markus Peschl, Pietro Mazzaglia, Daniel Dijkman

机构 * Qualcomm AI Research（高通人工智能研究）

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision-language model(abstract)；分类 cs.LG

Comments 19 pages including references, 6 figures. Accepted to CoRL LEAP 2025

2506.13205 2025-09-08 cs.CR cs.AI 83%

Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents

Xuan Wang, Siyuan Liang, Zhe Liu, Yi Yu, Aishan Liu, Yuliang Lu, Xitong Gao, Ee-Chien Chang

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision-language model(abstract)；分类 cs.AI

Comments 10 pages

2410.02730 2025-09-03 cs.CV cs.CL cs.RO 83%

DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes

Zhaowei Wang, Hongming Zhang, Tianqing Fang, Ye Tian, Yue Yang, Kaixin Ma, Xiaoman Pan, Yangqiu Song, Dong Yu

机构 * CSE Department, HKUST（香港科技大学计算机科学与工程系）； Tencent AI Lab（腾讯AI实验室）； Robotics X, Tencent（腾讯机器人实验室）； University of Pennsylvania（宾夕法尼亚大学）

专题命中 GUI与屏幕智能体：vision language model(title)；vision-language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments EMNLP 2025

2508.18040 2025-08-26 cs.AI 83%

PerPilot: Personalizing VLM-based Mobile Agents via Memory and Exploration

Xin Wang, Zhiyao Cui, Hao Li, Ya Zeng, Chenxu Wang, Ruiqi Song, Yihang Chen, Kun Shao, Qiaosheng Zhang, Jinzhuo Liu, Siyue Ren, Shuyue Hu, Zhen Wang

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision language model(abstract)；分类 cs.AI

2508.11093 2025-08-18 cs.RO cs.AI cs.HC 83%

Utilizing Vision-Language Models as Action Models for Intent Recognition and Assistance

Cesar Alan Contreras, Manolis Chiou, Alireza Rastegarpanah, Michal Szulik, Rustam Stolkin

机构 * University of Birmingham（伯明翰大学）； Queen Mary University of London（伦敦大学女王学院）； Aston University（阿斯顿大学）； United Kingdom National Nuclear Laboratory Ltd.（英国国家核实验室有限公司）

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments Accepted at Human-Centered Robot Autonomy for Human-Robot Teams (HuRoboT) at IEEE RO-MAN 2025, Eindhoven, the Netherlands

2508.09789 2025-08-14 cs.IR cs.CV 83%

Describe What You See with Multimodal Large Language Models to Enhance Video Recommendations

Marco De Nadai, Andreas Damianou, Mounia Lalmas

机构 * Spotify Denmark（Spotify丹麦分公司）； Spotify United Kingdom（Spotify英国分公司）

专题命中 GUI与屏幕智能体：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

2502.05855 2025-08-12 cs.RO cs.CV 83%

DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control

Junjie Wen, Yichen Zhu, Jinming Li, Zhibin Tang, Chaomin Shen, Feifei Feng

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments The webpage is at https://dex-vla.github.io/. DexVLA is accepted by CoRL 2025

2504.11257 2025-07-31 cs.HC cs.CL cs.CV 83%

UI-E2I-Synth: Advancing GUI Grounding with Large-Scale Instruction Synthesis

Xinyi Liu, Xiaoyi Zhang, Ziyun Zhang, Yan Lu

专题命中 GUI与屏幕智能体：grounding(title,abstract)；vision-language model(abstract)；分类 cs.CV

2503.02247 2025-07-22 cs.CV cs.RO 83%

WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation

Dujun Nie, Xianda Guo, Yiqun Duan, Ruijun Zhang, Long Chen

机构 * Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）； School of Computer Science, Wuhan University（武汉大学计算机学院）； School of Computer Science, University of Technology Sydney（悉尼大学计算机学院）； IAIR, Xi’an Jiaotong University（西安交通大学IAIR）

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments 8 pages, 5 figures

Journal ref IROS 2025

2507.08024 2025-07-14 cs.CV 83%

Self-Consistency in Vision-Language Models for Precision Agriculture: Multi-Response Consensus for Crop Disease Management

Mihir Gupta, Abhay Mangla, Ross Greer, Pratik Desai

机构 * The Harker School, USA（哈克尔学校）； Dougherty Valley High School, USA（道格拉斯谷高中）； Kissan.ai, USA（加州大学梅尔德分校）； University of California, Merced, USA

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

2506.16623 2025-06-23 cs.RO cs.AI 83%

History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigation

Mobin Habibpour, Fatemeh Afghah

机构 * Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA（霍尔科姆电气与计算机工程系，克莱姆森大学）

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

2506.07196 2025-06-16 cs.CV cs.CL 83%

SAP-Bench: Benchmarking Multimodal Large Language Models in Surgical Action Planning

Mengya Xu, Zhongzhen Huang, Dillan Imans, Yiru Ye, Xiaofan Zhang, Qi Dou

专题命中 GUI与屏幕智能体：multimodal large language model(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments The authors could not reach a consensus on the final version of this paper, necessitating its withdrawal

2502.14254 2025-06-12 cs.RO cs.AI 83%

Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation

Lingfeng Zhang, Yuecheng Liu, Zhanguang Zhang, Matin Aghaei, Yaochen Hu, Hongjian Gu, Mohammad Ali Alomrani, David Gamaliel Arcos Bravo, Raika Karimi, Atia Hamidizadeh, Haoping Xu, Guowei Huang, Zhanpeng Zhang, Tongtong Cao, Weichao Qiu, Xingyue Quan, Jianye Hao, Yuzheng Zhuang, Yingxue Zhang

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

2506.02354 2025-06-04 cs.CV 83%

RATE-Nav: Region-Aware Termination Enhancement for Zero-shot Object Navigation with Vision-Language Models

Junjie Li, Nan Zhang, Xiaoyang Qu, Kai Lu, Guokuan Li, Jiguang Wan, Jianzong Wang

机构 * Huazhong University of Science and Technology（华中科技大学）； Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）

专题命中 GUI与屏幕智能体：vision-language model(title)；visual language model(abstract)；visual question answering(abstract)；分类 cs.CV

Comments Accepted by the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025)

2505.19095 2025-05-27 cs.AI 83%

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

Runliang Niu, Jinglong Ji, Yi Chang, Qi Wang

机构 * School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

2310.07937 2025-05-07 cs.RO cs.AI 83%

Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models

Bangguo Yu, Qihao Yuan, Kailai Li, Hamidreza Kasaei, Ming Cao

机构 * Faculty of Science and Engineering, University of Groningen（工程学院，格罗宁根大学）

专题命中 GUI与屏幕智能体：vision language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments 8 pages, 4 figures

2503.02950 2025-05-07 cs.AI cs.CL cs.MA 83%

LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applications

Danqing Zhang, Balaji Rama, Jingyi Ni, Shiying He, Fu Zhao, Kunyu Chen, Arnold Chen, Junyu Cao

机构 * Rutgers University（罗格斯大学）； The University of Texas at Austin（德克萨斯大学奥斯汀分校）

专题命中 GUI与屏幕智能体：VLM(title,abstract)；grounding(abstract)；分类 cs.AI

2503.15491 2025-03-21 cs.HC cs.CL cs.LG cs.RO 83%

Agreeing to Interact in Human-Robot Interaction using Large Language Models and Vision Language Models

Kazuhiro Sasabuchi, Naoki Wake, Atsushi Kanehira, Jun Takamatsu, Katsushi Ikeuchi

专题命中 GUI与屏幕智能体：vision language model(title,abstract)；VLM(abstract)；分类 cs.LG

2502.06395 2025-02-11 cs.AI 83%

AppVLM: A Lightweight Vision Language Model for Online App Control

Georgios Papoudakis, Thomas Coste, Zhihao Wu, Jianye Hao, Jun Wang, Kun Shao

专题命中 GUI与屏幕智能体：vision language model(title)；vision-language model(abstract)；VLM(abstract)；分类 cs.AI

2312.08914 2024-12-30 cs.CV 83%

CogAgent: A Visual Language Model for GUI Agents

Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, Jie Tang

专题命中 GUI与屏幕智能体：visual language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments CVPR 2024 (Highlight), 27 pages, 19 figures

2406.19263 2024-10-29 cs.CL cs.CV 83%

Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding

Yue Fan, Lei Ding, Ching-Chen Kuo, Shan Jiang, Yang Zhao, Xinze Guan, Jie Yang, Yi Zhang, Xin Eric Wang

专题命中 GUI与屏幕智能体：grounding(title,abstract)；multimodal large language model(abstract)；分类 cs.CV

2410.16232 2024-10-22 cs.CL cs.AI 83%

Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping

Ryan Li, Yanzhe Zhang, Diyi Yang

专题命中 GUI与屏幕智能体：vision-language model(title)；vision language model(abstract)；VLM(abstract)；分类 cs.AI

Comments preprint, 9 pages

2410.06237 2024-10-10 cs.RO cs.AI 83%

BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulation

Rutav Shah, Albert Yu, Yifeng Zhu, Yuke Zhu, Roberto Martín-Martín

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.AI

Comments 7 Figures, 2 Tables, 11 Pages

2407.13368 2024-07-19 cs.CV 83%

Affordance Perception by a Knowledge-Guided Vision-Language Model with Efficient Error Correction

Gertjan Burghouts, Marianne Schaaphok, Michael van Bekkum, Wouter Meijer, Fieke Hillerström, Jelle van Mil

专题命中 GUI与屏幕智能体：vision-language model(title,abstract)；VLM(abstract)；分类 cs.CV

Comments 15 pages

Journal ref International Conference on Pattern Recognition and Artificial Intelligence (ICPRAI) 2024

2402.15852 2024-07-02 cs.CV cs.RO 83%

NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation

Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, He Wang

专题命中 GUI与屏幕智能体：VLM(title,abstract)；vision language model(abstract)；分类 cs.CV

Comments Accepted by Robotics: Science and Systems (RSS 2024)

1. GUI与屏幕智能体 1570 篇

Enhancing End-to-End Autonomous Driving with Risk Semantic Distillaion from VLM

Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation

Think, Remember, Navigate: Zero-Shot Object-Goal Navigation with VLM-Powered Reasoning

The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards

AquaVLM: Improving Underwater Situation Awareness with Mobile Vision Language Models

From Code to Action: Hierarchical Learning of Diffusion-VLM Policies

Poison Once, Control Anywhere: Clean-Text Visual Backdoors in VLM-based Mobile Agents

DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes

PerPilot: Personalizing VLM-based Mobile Agents via Memory and Exploration

Utilizing Vision-Language Models as Action Models for Intent Recognition and Assistance

Describe What You See with Multimodal Large Language Models to Enhance Video Recommendations

DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control

UI-E2I-Synth: Advancing GUI Grounding with Large-Scale Instruction Synthesis

WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation

Self-Consistency in Vision-Language Models for Precision Agriculture: Multi-Response Consensus for Crop Disease Management

History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigation

SAP-Bench: Benchmarking Multimodal Large Language Models in Surgical Action Planning

Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation

RATE-Nav: Region-Aware Termination Enhancement for Zero-shot Object Navigation with Vision-Language Models

ScreenExplorer: Training a Vision-Language Model for Diverse Exploration in Open GUI World

Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models

LiteWebAgent: The Open-Source Suite for VLM-Based Web-Agent Applications

Agreeing to Interact in Human-Robot Interaction using Large Language Models and Vision Language Models

AppVLM: A Lightweight Vision Language Model for Online App Control

CogAgent: A Visual Language Model for GUI Agents

Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding

Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping

BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulation

Affordance Perception by a Knowledge-Guided Vision-Language Model with Efficient Error Correction

NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation