arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2607.01115cs.CLcs.AI

面向大学利益相关者的多模态聊天助手开发:基于RAG的方法

Towards Developing a Multimodal Chat Assistant for University Stakeholders: RAG-based Approach

Institute of Information and Communication Technology, Khulna University of Engineering & Technology(信息与通信技术研究所,库尔纳工程技术大学)

Md Abu Hanif Shaikh, Abdullah Al Shafi

AI总结:

针对大学利益相关者获取信息困难的问题,提出基于检索增强生成的多模态聊天系统,结合大语言模型与语义检索,支持文本和图像查询,将幻觉率从31.7%降至6.6%。

摘要:

大学利益相关者常常难以获取及时可靠的信息,尤其是在发展中国家,智能支持系统非常少。现有的基于规则的聊天机器人无法处理复杂的、特定领域的查询,也不擅长适应不断变化的机构政策。作为填补空白的解决方案,我们提出了具有检索增强生成的多模态大学聊天机器人。该系统将大语言模型与语义检索相结合,从机构中心资源(如大学手册)生成基于上下文的响应。系统通过视觉语言模型接受文本和图像查询,并应用量化推理以在受限硬件上快速部署。基于FastAPI构建的可扩展后端,配合使用此http URL开发的响应式前端,确保了实时可用性。我们的多模态评估表明,尽管视觉输入响应时间增加,系统在文本和图像查询中均保持了较高的满意度分数。此外,定量评估显示,在我们提出的基于RAG的系统中,幻觉从31.7%降低到6.6%,证实了检索基础的有效性。

英文摘要:

University stakeholders often face difficulties in accessing timely and reliable information, especially in developing countries, where there are very few intelligent support systems. Existing rule-based chatbots are unable to handle complex, domain-specific queries and are not well-equipped to adapt to evolving institutional policies. As a fill-in-the-gap solution, we present the multimodal university chatbot with retrieval-augmented generation. The system combines the large language model with semantic retrieval to produce context-based responses from institution-centric resources, such as the university handbook. The system accepts text and image queries through the vision-language model and applies quantized inference for rapid deployment on constrained hardware. A scalable backend built with FastAPI, adjoined with a responsive frontend developed with Next.js, ensures real-time usability. Our multimodal evaluation demonstrates that the system maintains strong satisfaction scores across both text and image queries, despite increased response time for visual inputs. Furthermore, quantitative evaluation shows that hallucination is reduced from 31.7% to 6.6% in our proposed RAG-based system, confirming the effectiveness of retrieval grounding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑