多模态喉镜视频分析用于辅助声带麻痹诊断
Multimodal Laryngoscopic Video Analysis for Assisted Diagnosis of Vocal Fold Paralysis
AI总结：
本文提出多模态喉镜视频分析系统，结合音频和视频数据自动提取关键片段，用于辅助诊断声带麻痹。
摘要：
本文提出了多模态喉镜视频分析系统（MLVAS），一种利用音频和视频数据自动提取关键视频片段和指标的新型系统，用于辅助临床评估。该系统集成了基于视频的声带检测与音频关键词检测方法，分析视频和音频数据，识别患者发音并优化视频高亮部分，以确保最佳的声带运动检查。除了从原始喉镜视频中提取关键视频片段外，MLVAS还能生成有效的音频和视觉特征用于声带麻痹（VFP）检测。预训练的音频编码器用于编码患者声音以获取音频特征。视觉特征通过测量左右声带相对于估计的声带中线的角度偏差来生成，该中线基于分割的声带掩膜估计。为了获得更好的掩膜，我们引入了基于扩散的精修方法，该方法遵循传统的U-Net分割流程以减少假阳性。我们进行了几次消融研究以证明所提出MLVAS中每个模块和模态的有效性。在公共分割数据集上的实验结果展示了我们提出分割模块的有效性。此外，单侧VFP分类结果在真实世界临床数据集上展示了MLVAS提供可靠和客观指标以及可视化辅助临床诊断的能力。
英文摘要：
This paper presents the Multimodal Laryngoscopic Video Analyzing System (MLVAS), a novel system that leverages both audio and video data to automatically extract key video segments and metrics from raw laryngeal videostroboscopic videos for assisted clinical assessment. The system integrates video-based glottis detection with an audio keyword spotting method to analyze both video and audio data, identifying patient vocalizations and refining video highlights to ensure optimal inspection of vocal fold movements. Beyond key video segment extraction from the raw laryngeal videos, MLVAS is able to generate effective audio and visual features for Vocal Fold Paralysis (VFP) detection. Pre-trained audio encoders are utilized to encode the patient voice to get the audio features. Visual features are generated by measuring the angle deviation of both the left and right vocal folds to the estimated glottal midline on the segmented glottis masks. To get better masks, we introduce a diffusion-based refinement that follows traditional U-Net segmentation to reduce false positives. We conducted several ablation studies to demonstrate the effectiveness of each module and modalities in the proposed MLVAS. The experimental results on a public segmentation dataset show the effectiveness of our proposed segmentation module. In addition, unilateral VFP classification results on a real-world clinic dataset demonstrate MLVAS's ability of providing reliable and objective metrics as well as visualization for assisted clinical diagnosis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
