arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.02244cs.CV
收藏

利用大语言模型中的视觉提示增强情感识别

Visual Prompting in LLMs for Enhancing Emotion Recognition

Australian National University(澳大利亚国立大学);Quriosity Pty Ltd(Quriosity私人有限公司);Webumate Pty Ltd(Webumate私人有限公司);Curtin University(科廷大学);Yale University(耶鲁大学)

Qixuan Zhang, Zhifeng Wang, Dylan Zhang, Wenjia Niu, Sabrina Caldwell, Tom Gedeon, Yang Liu, Zhenyue Qin

AI总结:

本文提出SoV视觉提示方法,利用边界框和面部关键点等空间信息增强VLLMs的零样本情感识别能力,实验证明其能有效提升人脸计数与情感分类性能。

摘要:

视觉大语言模型(VLLMs)正在改变计算机视觉与自然语言处理的交叉领域。然而,在这些模型中使用视觉提示进行情感识别的潜力在很大程度上仍未被探索和利用。VLLMs中的传统方法在空间定位方面存在困难,并且常常丢弃有价值的全局上下文。为了解决这一问题,我们提出了一种视觉提示集合(Set-of-Vision prompting, SoV)方法,该方法通过利用空间信息(如边界框和面部关键点)来精确标记目标,从而增强零样本情感识别能力。SoV在保持丰富图像上下文的同时,提高了人脸计数和情感分类的准确性。通过对近期商业或开源VLLMs的大量实验和分析,我们评估了SoV模型在自然环境中理解面部表情的能力。我们的研究结果表明,将空间视觉提示集成到VLLMs中可以有效提升情感识别性能。

英文摘要:

Vision Large Language Models (VLLMs) are transforming the intersection of computer vision and natural language processing. Nonetheless, the potential of using visual prompts for emotion recognition in these models remains largely unexplored and untapped. Traditional methods in VLLMs struggle with spatial localization and often discard valuable global context. To address this problem, we propose a Set-of-Vision prompting (SoV) approach that enhances zero-shot emotion recognition by using spatial information, such as bounding boxes and facial landmarks, to mark targets precisely. SoV improves accuracy in face count and emotion categorization while preserving the enriched image context. Through a battery of experimentation and analysis of recent commercial or open-source VLLMs, we evaluate the SoV model's ability to comprehend facial expressions in natural environments. Our findings demonstrate the effectiveness of integrating spatial visual prompts into VLLMs for improving emotion recognition performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑