面向分析老年人语言能力的多语言访谈中的说话人角色与语言 diarization（说话人 diarization）
Speaker Role and Language Diarization for Analyzing Multilingual Interviews for Language Proficiency of Older Adults
浏览论文内容
中文总结 AI 辅助
本研究开发了基于 Whisper 的说话人角色与语言 diarization 系统，探究其衍生特征用于老年人多语言访谈语言能力评估的可行性，发现该系统可实现高效且准确的评估。
中文摘要 AI 辅助
在基于多语言访谈的场景中，自动语言能力评估的研究仍未得到充分探索。本研究开发了基于 Whisper 的说话人角色与语言 diarization（说话人 diarization）系统，以自动提取多语言访谈中老年人的应答者语音并刻画其语言使用特征。我们进一步探究由 diarization 衍生的会话与语言使用行为是否能支持下游的语言能力评估。结果表明，适配语言的 Whisper 模型显著提升了低资源及语言相关的印度语言的语言 diarization 性能。统计分析显示，应答者语音占比与目标语言使用是能力等级的强预测因子。此外，简单的 diarization 衍生行为特征在能力预测上取得了与基于 Whisper 的语音嵌入相当的性能，而将两者结合则获得最佳结果。重要的是，当使用完全自动的 diarization 输出时，语音与语言使用统计分析及语言能力预测性能均基本保持，证明了以应答者为中心的会话分析在可扩展语言能力评估中的潜力。
英文摘要
Automatic language proficiency assessment in the context of multilingual interview-based settings remains underexplored. In this work, we develop Whisper-based speaker-role and language diarization systems to automatically extract respondent speech and characterize language usage in multilingual interviews with older adults. We further investigate whether diarization-derived conversational and language-use behaviors can support downstream language proficiency assessment. Results show that language-adapted Whisper models substantially improve language diarization performance for lower-resource and linguistically related Indian languages. Statistical analyses reveal that respondent speech ratio and intended language usage are strong predictors of proficiency ratings. Furthermore, simple diarization-derived behavioral features achieve performance comparable to Whisper-based speech embeddings for proficiency prediction, while combining both yields the best results. Importantly, both the speech and language use statistical analyses and language proficiency prediction performance remain largely preserved when using fully automatic diarization outputs, demonstrating the potential of respondent-centric conversational analysis for scalable language proficiency assessment.
