EgoBlind：面向盲人的以自我为中心的视觉辅助
EgoBlind: Towards Egocentric Visual Assistance for the Blind
AI总结：
本文提出首个从盲人收集的以自我为中心的VideoQA数据集EgoBlind，包含1,392个视频和5,311个问题，评估16个先进MLLMs发现其准确率远低于人类，并总结现有局限性及启发式改进方案，为开发有效AI视觉辅助奠定基础。
摘要：
我们提出了EgoBlind，这是首个从盲人个体中收集的以自我为中心的VideoQA数据集，旨在评估当代多模态大语言模型（MLLMs）的辅助能力。EgoBlind包含来自盲人和视障人士日常生活的1,392个第一人称视频。它还包含5,311个由盲人直接提出或验证的问题，以反映他们对视觉辅助的情境需求。每个问题平均有3个手动标注的参考答案，以减少主观性。使用EgoBlind，我们全面评估了16个先进的MLLMs，发现所有模型都表现不佳。表现最好的模型准确率接近60%，远低于人类87.4%的表现。为了指导未来的进展，我们识别并总结了现有MLLMs在盲人以自我为中心的视觉辅助中的主要局限性，并探索了启发式改进方案。通过这些努力，我们希望EgoBlind能成为开发有效AI助手的基础，以增强盲人和视障人士的独立性。数据和代码可在https://github.com/doc-doc/EgoBlind获取。
英文摘要：
We present EgoBlind, the first egocentric VideoQA dataset collected from blind individuals to evaluate the assistive capabilities of contemporary multimodal large language models (MLLMs). EgoBlind comprises 1,392 first-person videos from the daily lives of blind and visually impaired individuals. It also features 5,311 questions directly posed or verified by the blind to reflect their in-situation needs for visual assistance. Each question has an average of 3 manually annotated reference answers to reduce subjectiveness. Using EgoBlind, we comprehensively evaluate 16 advanced MLLMs and find that all models struggle. The best performers achieve an accuracy near 60\%, which is far behind human performance of 87.4\%. To guide future advancements, we identify and summarize major limitations of existing MLLMs in egocentric visual assistance for the blind and explore heuristic solutions for improvement. With these efforts, we hope that EgoBlind will serve as a foundation for developing effective AI assistants to enhance the independence of the blind and visually impaired. Data and code are available at https://github.com/doc-doc/EgoBlind.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
