TEXT2TASTE：一种基于大语言模型的多功能第一视角智能阅读辅助视觉系统
TEXT2TASTE: A Versatile Egocentric Vision System for Intelligent Reading Assistance Using Large Language Model
AI总结：
针对部分视力障碍人群的阅读困难问题，提出基于智能眼镜和LLM的第一视角阅读助手TEXT2TASTE，经真实场景测试验证了其文本检索准确率与用户满意度，展现了辅助特殊需求群体的潜力。
摘要：
从书面文本中阅读、理解并提取重要信息的能力，是我们日常生活中保障独立性、舒适度与安全性的关键技能。然而，社会中有相当一部分群体受部分视力障碍的影响，在日常活动中会产生不适感并依赖他人帮助。为解决这一群体面临的限制，我们提出了一款基于内置RGB摄像头的智能眼镜与大语言模型（LLM）的智能阅读助手，其功能远超矫正镜片。系统会对佩戴眼镜者的第一视角拍摄的视频进行处理，通过目标检测与光学字符识别方法定位文本信息。LLM处理相关数据，支持用户与文本交互并响应特定查询，从而在矫正镜片的基础上拓展了从文本中查找、总结知识的功能。为评估我们的方法，我们开发了一款聊天式应用，支持用户与系统交互。评估在餐厅读菜单等真实场景中开展，共招募4名参与者。结果显示系统在文本检索方面具备稳定的准确率，不仅能提供准确的餐食建议，还获得了很高的用户满意度，凸显了智能眼镜与LLM在辅助特殊需求群体方面的潜力。
英文摘要：
The ability to read, understand and find important information from written text is a critical skill in our daily lives for our independence, comfort and safety. However, a significant part of our society is affected by partial vision impairment, which leads to discomfort and dependency in daily activities. To address the limitations of this part of society, we propose an intelligent reading assistant based on smart glasses with embedded RGB cameras and a Large Language Model (LLM), whose functionality goes beyond corrective lenses. The video recorded from the egocentric perspective of a person wearing the glasses is processed to localise text information using object detection and optical character recognition methods. The LLM processes the data and allows the user to interact with the text and responds to a given query, thus extending the functionality of corrective lenses with the ability to find and summarize knowledge from the text. To evaluate our method, we create a chat-based application that allows the user to interact with the system. The evaluation is conducted in a real-world setting, such as reading menus in a restaurant, and involves four participants. The results show robust accuracy in text retrieval. The system not only provides accurate meal suggestions but also achieves high user satisfaction, highlighting the potential of smart glasses and LLMs in assisting people with special needs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
