arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.03054cs.HC
收藏

面向网络可访问性的上下文感知图像描述

Context-Aware Image Descriptions for Web Accessibility

Ananya Gubbi Mohanbabu, Amy Pavel

AI总结:

本文针对盲人和低视力用户对图像描述上下文相关性的需求,设计了一个自动提取网页上下文的 Chrome 扩展来指导 GPT-4V 生成上下文感知图像描述,并通过用户研究和技术评估验证了该方法在质量、可想象性、相关性和合理性上均优于无上下文描述。

摘要:

盲人和低视力(BLV)互联网用户通过文本描述访问网页上的图像。GPT-V、Gemini 和 LLaVa 等新型视觉到语言模型现在可以按需提供详细的图像描述。尽管先前的研究和指南指出,BLV 受众的信息偏好取决于图像的上下文,但现有的视觉到语言模型访问工具仅通过单独生成图像描述而不考虑周围网页上下文来提供无上下文的图像描述。为了探索如何将图像上下文整合到图像描述中,我们设计了一个 Chrome 扩展程序,该扩展程序自动提取网页上下文以指导 GPT-4V 生成的图像描述。我们在一项用户研究中获得了 12 名 BLV 参与者的反馈,该研究比较了典型的无上下文图像描述与上下文感知图像描述。随后,我们通过技术评估进一步评价了我们的上下文信息图像描述。我们的用户评估表明,BLV 参与者通常更喜欢上下文感知描述而非无上下文描述。BLV 参与者还在质量、可想象性、相关性和合理性方面对上下文感知描述给予了显著更高的评分。所有参与者都表示希望在未来使用上下文感知描述,并强调了其在在线购物、社交媒体、新闻和个人兴趣博客中的应用潜力。

英文摘要:

Blind and low vision (BLV) internet users access images on the web via text descriptions. New vision-to-language models such as GPT-V, Gemini, and LLaVa can now provide detailed image descriptions on-demand. While prior research and guidelines state that BLV audiences' information preferences depend on the context of the image, existing tools for accessing vision-to-language models provide only context-free image descriptions by generating descriptions for the image alone without considering the surrounding webpage context. To explore how to integrate image context into image descriptions, we designed a Chrome Extension that automatically extracts webpage context to inform GPT-4V-generated image descriptions. We gained feedback from 12 BLV participants in a user study comparing typical context-free image descriptions to context-aware image descriptions. We then further evaluated our context-informed image descriptions with a technical evaluation. Our user evaluation demonstrated that BLV participants frequently prefer context-aware descriptions to context-free descriptions. BLV participants also rated context-aware descriptions significantly higher in quality, imaginability, relevance, and plausibility. All participants shared that they wanted to use context-aware descriptions in the future and highlighted the potential for use in online shopping, social media, news, and personal interest blogs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑