arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04235cs.CV
收藏

LLDif:用于低光情绪识别的扩散模型

LLDif: Diffusion Models for Low-light Emotion Recognition

College of Engineering and Computer Science, Australian National University(澳大利亚国立大学工程与计算机科学学院);School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机科学与技术学院)

Zhifeng Wang, Kaihao Zhang, Ramesh Sankaranarayana

AI总结:

本文提出 LLDif 框架,结合 LA-CLIP、嵌入先验网络和扩散模型,通过两阶段训练提升极低光环境下的面部表情识别性能。

摘要:

本文提出 LLDif,一种面向极低光(LL)环境的新型基于扩散的面部表情识别(FER)框架。在此类条件下拍摄的图像通常亮度低且对比度显著下降,给传统方法带来挑战,包括图像质量差会显著降低情绪识别准确率。LLDif 通过一种新颖的两阶段训练过程解决这些问题,该过程结合了标签感知 CLIP(LA-CLIP)、嵌入先验网络(PNET)以及一个擅长处理低光图像噪声的基于 Transformer 的网络。第一阶段由 LA-CLIP 生成联合嵌入先验分布(EPD),以引导 LLformer 进行标签恢复。第二阶段,扩散模型(DM)利用 EPD 的紧凑性细化 EPD 推断,从而实现精确预测。在多个 LL-FER 数据集上的实验评估表明,LLDif 取得了具有竞争力的性能,凸显了其在挑战性光照条件下增强 FER 应用的潜力。

英文摘要:

This paper introduces LLDif, a novel diffusion-based facial expression recognition (FER) framework tailored for extremely low-light (LL) environments. Images captured under such conditions often suffer from low brightness and significantly reduced contrast, presenting challenges to conventional methods. These challenges include poor image quality that can significantly reduce the accuracy of emotion recognition. LLDif addresses these issues with a novel two-stage training process that combines a Label-aware CLIP (LA-CLIP), an embedding prior network (PNET), and a transformer-based network adept at handling the noise of low-light images. The first stage involves LA-CLIP generating a joint embedding prior distribution (EPD) to guide the LLformer in label recovery. In the second stage, the diffusion model (DM) refines the EPD inference, ultilising the compactness of EPD for precise predictions. Experimental evaluations on various LL-FER datasets have shown that LLDif achieves competitive performance, underscoring its potential to enhance FER applications in challenging lighting conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑