arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.08972cs.CV
收藏

一对一对话中基于面部表情的听力损失检测

Hearing Loss Detection from Facial Expressions in One-on-one Conversations

Meta Reality Labs Research(Meta现实实验室)

Yufeng Yin, Ishwarya Ananthabhotla, Vamsi Krishna Ithapu, Stavros Petridis, Yu-Hsiang Wu, Christi Miller

AI总结:

本文提出从一对一对话中的面部表情检测听力损失,通过自监督预训练建模表情变化,并利用对抗性表示学习消除年龄偏差,在大规模真实对话数据集上取得优于基线的效果。

摘要:

听力受损者在对话中会遇到困难,尤其是在嘈杂环境中。这种困难通常表现为行为变化,并可能通过面部表情被捕捉到,例如不适或疲劳的表情。在本工作中,我们基于这一思路,提出了从个体在对话期间的面部表情中检测听力损失的问题。构建能够表示与听力相关的面部表情变化的机器学习模型是一项挑战。此外,模型需要将与年龄相关的虚假关联从听力驱动的表情中解耦出来。为此,我们提出了一种针对表情变化建模而定制的自监督预训练策略。我们还使用对抗性表示学习来缓解年龄偏差。我们在一个大规模第一人称数据集上评估了我们的方法,该数据集包含涉及听力损失受试者的真实世界对话场景,并表明我们的听力损失检测方法相比基线取得了更优越的性能。

英文摘要:

Individuals with impaired hearing experience difficulty in conversations, especially in noisy environments. This difficulty often manifests as a change in behavior and may be captured via facial expressions, such as the expression of discomfort or fatigue. In this work, we build on this idea and introduce the problem of detecting hearing loss from an individual's facial expressions during a conversation. Building machine learning models that can represent hearing-related facial expression changes is a challenge. In addition, models need to disentangle spurious age-related correlations from hearing-driven expressions. To this end, we propose a self-supervised pre-training strategy tailored for the modeling of expression variations. We also use adversarial representation learning to mitigate the age bias. We evaluate our approach on a large-scale egocentric dataset with real-world conversational scenarios involving subjects with hearing loss and show that our method for hearing loss detection achieves superior performance over baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑