VIALM：基于大型模型的视觉障碍辅助综述与基准
VIALM: A Survey and Benchmark of Visually Impaired Assistance with Large Models
AI总结：
本文针对基于大型模型的视觉障碍辅助任务（VIALM）进行了综述与基准实验，探究大型模型在多模态环境理解下的逐步指导能力，发现其仍存在环境结合度不足与缺乏细粒度指导的局限性。
摘要：
视觉障碍辅助（VIA）旨在自动帮助视觉障碍者（VI）处理日常活动。VIA的发展主要依赖于计算机视觉（CV）和自然语言处理（NLP）的进步，这两者均展现出基于大型模型（LMs）的前沿范式。此外，LMs还表现出卓越的多模态能力，能够应对具身机器人等具有挑战性的物理基础任务。为了探究最先进（SOTA）LMs在VIA应用中的潜力与局限性，我们针对基于LMs的VIA任务（VIALM）进行了广泛的研究。在该任务中，给定一张描绘物理环境的图像和来自VI用户的语言请求，VIALM旨在输出逐步指导，以协助VI用户在环境基础上完成该请求。本研究包含一篇回顾近期LM研究的综述，以及检验所选LMs在VIA中能力的基准实验。结果表明，尽管LMs可能有益于VIA，但其输出无法很好地与环境结合（即25.7%的GPT-4响应），且缺乏细粒度指导（即32.1%的GPT-4响应）。
英文摘要：
Visually Impaired Assistance (VIA) aims to automatically help the visually impaired (VI) handle daily activities. The advancement of VIA primarily depends on developments in Computer Vision (CV) and Natural Language Processing (NLP), both of which exhibit cutting-edge paradigms with large models (LMs). Furthermore, LMs have shown exceptional multimodal abilities to tackle challenging physically-grounded tasks such as embodied robots. To investigate the potential and limitations of state-of-the-art (SOTA) LMs' capabilities in VIA applications, we present an extensive study for the task of VIA with LMs (VIALM). In this task, given an image illustrating the physical environments and a linguistic request from a VI user, VIALM aims to output step-by-step guidance to assist the VI user in fulfilling the request grounded in the environment. The study consists of a survey reviewing recent LM research and benchmark experiments examining selected LMs' capabilities in VIA. The results indicate that while LMs can potentially benefit VIA, their output cannot be well environment-grounded (i.e., 25.7% GPT-4's responses) and lacks fine-grained guidance (i.e., 32.1% GPT-4's responses).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
