视觉大模型 / VLM - arXivDaily 专题

2409.18060 2024-10-10 cs.HC cs.SE 50%

Inferring Alt-text For UI Icons With Large Language Models During App Development

Sabrina Haque, Christoph Csallner

专题命中 GUI与屏幕智能体：vision language model(abstract)

2410.03603 2024-10-07 cs.RO 50%

LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos

Noriaki Hirose, Catherine Glossop, Ajay Sridhar, Dhruv Shah, Oier Mees, Sergey Levine

专题命中 GUI与屏幕智能体：vision-language model(abstract)

Comments 23 pages, 9 figures, 5 tables, Conference on Robot Learning 2024

2409.16484 2024-10-04 cs.RO 50%

BehAV: Behavioral Rule Guided Autonomy Using VLMs for Robot Navigation in Outdoor Scenes

Kasun Weerakoon, Mohamed Elnoor, Gershom Seneviratne, Vignesh Rajagopal, Senthil Hariharan Arul, Jing Liang, Mohamed Khalid M Jaffar, Dinesh Manocha

专题命中 GUI与屏幕智能体：vision language model(abstract)

2405.03806 2024-10-03 cs.HC 50%

In Situ AI Prototyping: Infusing Multimodal Prompts into Mobile Settings with MobileMaker

Savvas Petridis, Michael Xieyang Liu, Alexander J. Fiannaca, Vivian Tsai, Michael Terry, Carrie J. Cai

专题命中 GUI与屏幕智能体：multimodal large language model(abstract)

2409.13393 2024-09-23 cs.RO 50%

Hey Robot! Personalizing Robot Navigation through Model Predictive Control with a Large Language Model

Diego Martinez-Baselga, Oscar de Groot, Luzia Knoedler, Javier Alonso-Mora, Luis Riazuelo, Luis Montano

专题命中 GUI与屏幕智能体：visual language model(abstract)

2409.11667 2024-09-19 cs.SE 50%

Bridging Design and Development with Automated Declarative UI Code Generation

Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, Haoyu Wang

专题命中 GUI与屏幕智能体：multimodal large language model(abstract)

2408.10578 2024-08-21 cs.RO 50%

Where to Fetch: Extracting Visual Scene Representation from Large Pre-Trained Models for Robotic Goal Navigation

Yu Li, Dayou Li, Chenkun Zhao, Ruifeng Wang, Ran Song, Wei Zhang

专题命中 GUI与屏幕智能体：visual language model(abstract)

2407.14146 2024-07-22 cs.MM 50%

Fine-grained Knowledge Graph-driven Video-Language Learning for Action Recognition

Rui Zhang, Yafen Lu, Pengli Ji, Junxiao Xue, Xiaoran Yan

专题命中 GUI与屏幕智能体：vision-language model(abstract)

2402.00956 2024-05-28 cs.CL 50%

Exploring Spatial Schema Intuitions in Large Language and Vision Models

Philipp Wicke, Lennart Wachowiak

专题命中 GUI与屏幕智能体：vision language model(abstract)

Comments ACL Findings 2024

2405.04497 2024-05-09 cs.HC 50%

Unveiling Disparities in Web Task Handling Between Human and Web Agent

Kihoon Son, Jinhyeon Kwon, DaEun Choi, Tae Soo Kim, Young-Ho Kim, Sangdoo Yun, Juho Kim

专题命中 GUI与屏幕智能体：vision-language model(abstract)

2404.03049 2024-04-05 cs.CL 50%

Language, Environment, and Robotic Navigation

Johnathan E. Avery

专题命中 GUI与屏幕智能体：grounding(abstract)

2403.19028 2024-03-29 eess.SY cs.SY 50%

Nonlinear Model Predictive Control for Enhanced Navigation of Autonomous Surface Vessels

Daniel Menges, Trym Tengesdal, Adil Rasheed

专题命中 GUI与屏幕智能体：grounding(abstract)

2402.10670 2024-03-26 cs.CL cs.RO 50%

OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models

Yuxuan Kuang, Hai Lin, Meng Jiang

专题命中 GUI与屏幕智能体：vision language model(abstract)

Comments NAACL 2024 Findings

2402.02559 2024-02-06 cs.CL 50%

NavHint: Vision and Language Navigation Agent with a Hint Generator

Yue Zhang, Quan Guo, Parisa Kordjamshidi

专题命中 GUI与屏幕智能体：grounding(abstract)

2305.15003 2023-11-14 cs.MA 50%

Feasible Action-Space Reduction as a Metric of Causal Responsibility in Multi-Agent Spatial Interactions

Ashwin George, Luciano Cavalcante Siebert, David Abbink, Arkady Zgonnikov

专题命中 GUI与屏幕智能体：grounding(abstract)

2310.02324 2023-10-05 cs.RO 50%

ALT-Pilot: Autonomous navigation with Language augmented Topometric maps

Mohammad Omama, Pranav Inani, Pranjal Paul, Sarat Chandra Yellapragada, Krishna Murthy Jatavallabhula, Sandeep Chinchali, Madhava Krishna

专题命中 GUI与屏幕智能体：vision-language model(abstract)

2309.10309 2023-09-22 cs.RO 50%

Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill

Wenzhe Cai, Siyuan Huang, Guangran Cheng, Yuxing Long, Peng Gao, Changyin Sun, Hao Dong

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments 8 pages, 5 figures

2211.07615 2023-05-24 cs.CL 50%

UGIF: UI Grounded Instruction Following

Sagar Gubbi Venkatesh, Partha Talukdar, Srini Narayanan

专题命中 GUI与屏幕智能体：grounding(abstract)

2109.11011 2022-08-03 cs.RO 50%

SOCIALGYM: A Framework for Benchmarking Social Robot Navigation

Jarrett Holtz, Joydeep Biswas

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments Published in IROS2022

2202.10667 2022-02-25 cs.RO 50%

Visually Grounded Task and Motion Planning for Mobile Manipulation

Xiaohan Zhang, Yifeng Zhu, Yan Ding, Yuke Zhu, Peter Stone, Shiqi Zhang

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments To be published in IEEE International Conference on Robotics and Automation (ICRA), May 23-27, 2022

2105.10396 2021-12-23 cs.RO cs.CL 50%

Language Understanding for Field and Service Robots in a Priori Unknown Environments

Matthew R. Walter, Siddharth Patki, Andrea F. Daniele, Ethan Fahnestock, Felix Duvallet, Sachithra Hemachandra, Jean Oh, Anthony Stentz, Nicholas Roy, Thomas M. Howard

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments Field Robotics (accepted, to appear)

2011.00642 2021-03-25 cs.RO 50%

Technical Report: Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments

Vasileios Vasilopoulos, Yiannis Kantaros, George J. Pappas, Daniel E. Koditschek

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments Technical Report accompanying the paper "Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments" at ICRA 2021 (12 pages, 7 figures)

2008.01281 2020-08-05 cs.RO 50%

Stochastic Grounded Action Transformation for Robot Learning in Simulation

Siddharth Desai, Haresh Karnan, Josiah P. Hanna, Garrett Warnell, Peter Stone

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments Accepted at 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2020

2008.01279 2020-08-05 cs.RO 50%

Reinforced Grounded Action Transformation for Sim-to-Real Transfer

Haresh Karnan, Siddharth Desai, Josiah P. Hanna, Garrett Warnell, Peter Stone

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments Accepted at International Conference on Intelligent Robots and Systems (IROS) 2020

0905.0200 2016-09-05 cs.HC 50%

A Vehicle for Research: Using Street Sweepers to Explore the Landscape of Environmental Community Action

Paul M. Aoki, R. J. Honicky, Alan Mainwaring, Chris Myers, Eric Paulos, Sushmita Subramanian, Allison Woodruff

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments 10 pages

Journal ref Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, Boston, MA, Apr. 2009, 375-384. ACM Press

1606.00892 2016-06-06 cs.CY 50%

Developing a Methodology for Online Service Failure Prevention: Reporting on an Action Design Research Project-in-Progress

Jacques Louis Du Preez, Mary Tate, Alireza Nili

专题命中 GUI与屏幕智能体：grounding(abstract)

Comments ISBN# 978-0-646-95337-3 Presented at the Australasian Conference on Information Systems 2015 (arXiv:1605.01032)

1. GUI与屏幕智能体 1586 篇

Inferring Alt-text For UI Icons With Large Language Models During App Development

LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos

BehAV: Behavioral Rule Guided Autonomy Using VLMs for Robot Navigation in Outdoor Scenes

In Situ AI Prototyping: Infusing Multimodal Prompts into Mobile Settings with MobileMaker

Hey Robot! Personalizing Robot Navigation through Model Predictive Control with a Large Language Model

Bridging Design and Development with Automated Declarative UI Code Generation

Where to Fetch: Extracting Visual Scene Representation from Large Pre-Trained Models for Robotic Goal Navigation

Fine-grained Knowledge Graph-driven Video-Language Learning for Action Recognition

Exploring Spatial Schema Intuitions in Large Language and Vision Models

Unveiling Disparities in Web Task Handling Between Human and Web Agent

Language, Environment, and Robotic Navigation

Nonlinear Model Predictive Control for Enhanced Navigation of Autonomous Surface Vessels

OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models

NavHint: Vision and Language Navigation Agent with a Hint Generator

Feasible Action-Space Reduction as a Metric of Causal Responsibility in Multi-Agent Spatial Interactions

ALT-Pilot: Autonomous navigation with Language augmented Topometric maps

Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill

UGIF: UI Grounded Instruction Following

SOCIALGYM: A Framework for Benchmarking Social Robot Navigation

Visually Grounded Task and Motion Planning for Mobile Manipulation

Language Understanding for Field and Service Robots in a Priori Unknown Environments

Technical Report: Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments

Stochastic Grounded Action Transformation for Robot Learning in Simulation

Reinforced Grounded Action Transformation for Sim-to-Real Transfer

A Vehicle for Research: Using Street Sweepers to Explore the Landscape of Environmental Community Action

Developing a Methodology for Online Service Failure Prevention: Reporting on an Action Design Research Project-in-Progress