评估多模态语言模型作为视障用户视觉助手的效果
Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users
AI总结：
本研究通过用户调查和五项任务评估十二个多模态大语言模型，发现其在文化、多语言、盲文识别及防幻觉方面存在局限，为无障碍视觉辅助技术发展指明方向。
摘要：
本文探讨了多模态大语言模型（MLLMs）作为视障人士辅助技术的有效性。我们开展了一项用户调查，以识别用户在使用此类技术时的采用模式及面临的关键挑战。尽管这些模型的采用率很高，但我们的研究结果突显了在上下文理解、文化敏感性和复杂场景理解方面的担忧，尤其是对于那些可能完全依赖它们进行视觉解读的个体。基于这些结果，我们整理了五项以用户为中心的任务，涉及图像和视频输入，其中包括一项关于光学盲文识别的新任务。我们对十二个MLLMs的系统评估表明，需要进一步的发展来克服与文化背景、多语言支持、盲文阅读理解、辅助物体识别及幻觉相关的局限性。这项工作为多模态AI在无障碍领域的未来方向提供了关键见解，强调了开发更具包容性、更稳健且更值得信赖的视觉辅助技术的必要性。
英文摘要：
This paper explores the effectiveness of Multimodal Large Language models (MLLMs) as assistive technologies for visually impaired individuals. We conduct a user survey to identify adoption patterns and key challenges users face with such technologies. Despite a high adoption rate of these models, our findings highlight concerns related to contextual understanding, cultural sensitivity, and complex scene understanding, particularly for individuals who may rely solely on them for visual interpretation. Informed by these results, we collate five user-centred tasks with image and video inputs, including a novel task on Optical Braille Recognition. Our systematic evaluation of twelve MLLMs reveals that further advancements are necessary to overcome limitations related to cultural context, multilingual support, Braille reading comprehension, assistive object recognition, and hallucinations. This work provides critical insights into the future direction of multimodal AI for accessibility, underscoring the need for more inclusive, robust, and trustworthy visual assistance technologies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
