发表机构
University of Sheffield（谢菲尔德大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出CCMAN跨模态注意力网络，融合语义、声学和语言信息，利用时间言语不稳定性作为可解释生物标志物，在843人言语数据上实现优于基线的认知衰退早期检测。
AI 中文摘要
基于言语的认知衰退早期检测提供了一种可扩展且非侵入性的传统临床评估替代方案。言语流畅性任务尤其具有信息量，但大多数自动化方法会聚合整段录音的特征，忽视了言语的时间动态。我们提出了认知不稳定性感知跨模态注意力网络（CCMAN），这是一种迁移学习框架，先在多个记忆探测任务上学习与任务无关的认知言语表征，再在持续一分钟的语义和音位言语流畅性任务上进行微调。CCMAN通过双向交叉注意力、门控多模态融合和基于Transformer的时间建模，整合语义、声学和语言信息，以推导可解释的认知衰退生物标志物。实验在843名参与者（498名健康对照、245名轻度认知障碍、100名痴呆）的165.44小时言语数据上进行。CCMAN在二元和多元语义流畅性分类中分别取得了0.81和0.77的Macro-F1分数，在音位流畅性分类中分别取得了0.77和0.53的Macro-F1分数，持续优于强静态和时间基线。统计分析显示，与健康对照相比，轻度认知障碍和痴呆组的语义漂移方差和停顿方差显著升高，而平均语义漂移无显著差异；停顿持续时间在任务过程中逐渐增加，其中痴呆组斜率最陡，这支持全局和进行性时间言语不稳定性作为可解释的生物标志物。在独立PD-2基准上的评估进一步证明了所提出框架的泛化能力，将基线Macro-F1提高了最多9%。这些发现支持时间言语不稳定性作为动态言语生物标志物，用于稳健、可解释且可泛化的认知衰退早期检测。
英文摘要
Early detection of cognitive decline from speech offers a scalable and non-invasive alternative to conventional clinical assessment. Verbal fluency tasks are particularly informative, but most automated approaches aggregate features across an entire recording, overlooking temporal speech dynamics. We propose the Cognitive Instability-Aware Cross-Modal Attention Network (CCMAN), a transfer learning framework that learns task-agnostic cognitive speech representations from multiple memory-probing tasks before fine-tuning on a minute-long semantic and phonemic verbal fluency task. CCMAN integrates semantic, acoustic, and linguistic information through bidirectional cross-attention, gated multimodal fusion, and transformer-based temporal modelling to derive interpretable biomarkers of cognitive decline. Experiments were conducted on 165.44 hours of speech from 843 participants (498 healthy controls, 245 with mild cognitive impairment, and 100 with dementia). CCMAN achieved Macro-F1 scores of 0.81 and 0.59 for binary and multiclass semantic fluency classification, and 0.77 and 0.53 for phonemic fluency, consistently outperforming strong static and temporal baselines. Statistical analyses showed that semantic drift variance and pause variance, but not mean semantic drift, were significantly elevated in both MCI and dementia relative to healthy controls, while pause duration increased progressively over the task with the steepest slope in dementia, supporting global and progressive temporal speech instability as interpretable biomarkers. Evaluation on the independent PROCESS-2 benchmark further demonstrated the generalisability of the proposed framework, improving the baseline Macro-F1 by up to 9%. These findings support temporal speech instability as a dynamic speech biomarker for robust, interpretable, and generalisable early detection of cognitive decline.
