arXiv 2407.198092024-07-29cs.CVcs.AI

Twins-PainViT：面向使用面部视频和fNIRS的多模态自动疼痛评估的模态无关视觉Transformer框架

Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS

Computational Biomedicine Laboratory of the Foundation for Research and Technology (FORTH)（希腊国家研究与技术基金会计算生物医学实验室）；Department of Electrical & Computer Engineering Hellenic Mediterranean University（希腊地中海大学电气与计算机工程系）

Stefanos Gkikas, Manolis Tsiknakis

发表信息：: 2024 International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)
DOI：: 10.1109/ACIIW63320.2024.00007

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Twins-PainViT框架，通过双ViT配置和波形表示融合面部视频与fNIRS，实现模态无关的多模态自动疼痛评估，在AI4PAIN挑战中达到46.76%准确率。

摘要：

自动疼痛评估在推进医疗保健和优化疼痛管理策略方面发挥着关键作用。本研究已提交至第一届下一代疼痛评估多模态感知大挑战（AI4PAIN）。所提出的多模态框架利用面部视频和fNIRS，并提出了一种模态无关的方法，从而减轻了对特定领域模型的需求。采用双ViT配置，并对fNIRS以及从两种模态中提取的嵌入采用波形表示，证明了所提方法的有效性，在多级疼痛评估任务中达到了46.76%的准确率。

英文摘要：

Automatic pain assessment plays a critical role for advancing healthcare and optimizing pain management strategies. This study has been submitted to the First Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed multimodal framework utilizes facial videos and fNIRS and presents a modality-agnostic approach, alleviating the need for domain-specific models. Employing a dual ViT configuration and adopting waveform representations for the fNIRS, as well as for the extracted embeddings from the two modalities, demonstrate the efficacy of the proposed method, achieving an accuracy of 46.76% in the multilevel pain assessment task.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。