EmoAssist：视障群体的情感助手
EmoAssist: Emotional Assistant for Visual Impairment Community
AI总结：
针对现有视障辅助LMMs忽视情感需求、缺乏情感评估基准的问题，本文构建EmoAssist基准并提出EmoAssist模型，显著提升对视障用户情感需求的响应能力，性能优于现有模型。
摘要：
大型多模态模型（LMMs）的快速发展极大推动了人工智能与实际应用的融合。视觉问答（VQA）系统可处理视觉、文本和音频等多模态数据，在帮助视障（VI）群体应对复杂动态的现实环境方面潜力巨大。然而，现有面向视障群体的辅助性LMMs忽视了视障人士的情感需求，当前基准也缺乏对这些LMMs的情感评估。为填补这些空白，本文引入EmoAssist基准，这是一个评估视障群体辅助性LMMs性能的综合基准，据我们所知，这是首个将情商纳入关键考量的基准。此外，我们提出EmoAssist模型，这是专为视障群体设计的情感辅助LMM。该模型利用直接偏好优化（DPO）使输出与人类情感偏好对齐。实验结果表明，EmoAssist模型显著提升了对视障用户隐含情绪和意图的识别能力，能提供共情回应和可操作指导；在EmoAssist基准的共情和建议指标上，与预训练LMM相比分别提升147.8%和89.7%，甚至优于GPT-4o等最先进LLMs。
英文摘要：
The rapid advancement of large multi-modality models (LMMs) has significantly propelled the integration of artificial intelligence into practical applications. Visual Question Answering (VQA) systems, which can process multi-modal data including vision, text, and audio, hold great potential for assisting the Visual Impairment (VI) community in navigating complex and dynamic real-world environments. However, existing VI assistive LMMs overlook the emotional needs of VI individuals, and current benchmarks lack emotional evaluation of these LMMs. To address these gaps, this paper introduces the EmoAssist Benchmark, a comprehensive benchmark designed to evaluate the assistive performance of LMMs for the VI community. To the best of our knowledge, this is the first benchmark that incorporates emotional intelligence as a key consideration. Furthermore, we propose the EmoAssist Model, an Emotion-Assistive LMM specifically designed for the VI community. The EmoAssist Model utilizes Direct Preference Optimization (DPO) to align outputs with human emotional preferences. Experiment results demonstrate that the EmoAssist Model significantly enhances the recognition of implicit emotions and intentions of VI users, delivers empathetic responses, and provides actionable guidance. Specifically, it shows respective improvements of 147.8% and 89.7% in the Empathy and Suggestion metrics on the EmoAssist Benchmark, compared to the pre-tuning LMM, and even outperforms state-of-the-art LLMs such as GPT-4o.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
