@Bench：面向以人为本辅助技术的视觉-语言模型基准评测
@Bench: Benchmarking Vision-Language Models for Human-centered Assistive Technology
AI总结：
本文提出面向视障辅助技术的基准@Bench和统一模型@Model，覆盖五项关键视觉-语言任务，通过多模态信息整合实现多任务同时处理并展现优异性能。
摘要：
随着视觉-语言模型（VLMs）的进步，用于帮助视障人士（PVIs）的以人为本辅助技术（ATs）正在演变为通才型系统，能够同时执行多项任务。然而，针对辅助技术的视觉-语言模型基准评测仍未被充分探索。为弥补这一空白，我们首先创建了一个新颖的辅助技术基准（@Bench）。在事先与视障人士开展的设计用户研究的指导下，我们的基准包含五项最关键的视觉-语言任务：全景分割、深度估计、光学字符识别（OCR）、图像描述和视觉问答（VQA）。此外，我们提出了一种新颖的辅助技术模型（@Model），该模型可同时处理所有任务，并可扩展至更多辅助功能以帮助视障人士。我们的框架通过整合多模态信息，在各任务上展现出卓越性能，并为视障人士提供更全面的辅助。大量实验证明了该框架的有效性和泛化能力。
英文摘要：
As Vision-Language Models (VLMs) advance, human-centered Assistive Technologies (ATs) for helping People with Visual Impairments (PVIs) are evolving into generalists, capable of performing multiple tasks simultaneously. However, benchmarking VLMs for ATs remains under-explored. To bridge this gap, we first create a novel AT benchmark (@Bench). Guided by a pre-design user study with PVIs, our benchmark includes the five most crucial vision-language tasks: Panoptic Segmentation, Depth Estimation, Optical Character Recognition (OCR), Image Captioning, and Visual Question Answering (VQA). Besides, we propose a novel AT model (@Model) that addresses all tasks simultaneously and can be expanded to more assistive functions for helping PVIs. Our framework exhibits outstanding performance across tasks by integrating multi-modal information, and it offers PVIs a more comprehensive assistance. Extensive experiments prove the effectiveness and generalizability of our framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
