2026 IEEE SLT智能眼镜挑战赛：基于音频-语言模型的自我中心多说话人语音识别与理解基准测试
The SLT 2026 SmartGlasses Challenge: Benchmarking Egocentric Multi-Talker Speech Recognition and Understanding with Audio-Language Models
浏览论文内容
中文总结 AI 辅助
本文介绍IEEE SLT 2026智能眼镜挑战赛，构建含714个真实场景会话的106小时四通道自我中心语音数据集，评估TSA-ASR与SLU，发现说话人重叠影响TSA-ASR性能、音频-语言模型难理解副语言声学。
中文摘要 AI 辅助
大型语言模型（LLMs）和多模态大型语言模型（MLLMs）的最新进展为可穿戴语音界面创造了新机遇，智能眼镜作为自我中心平台可实现连续音频感知与辅助。但该场景下的语音识别与理解仍面临挑战，原因在于动态声学条件、说话人重叠，以及佩戴者中心录制几何结构带来的空间歧义。为支持该场景下的系统评估，我们推出IEEE SLT 2026智能眼镜挑战赛，聚焦自我中心多说话人语音处理。挑战赛包含双轮对话理解和多方会议理解两个赛道，联合评估带时间戳的说话人归属自动语音识别（TSA-ASR）和口语语言理解（SLU）。该挑战赛基于106小时的四通道自我中心语音数据集构建，该数据集包含714个在真实场景中采集的会话。本文描述了挑战赛任务、数据集构建、提交要求，并总结了共享评估的主要发现。结果显示，严重的说话人重叠仍是影响TSA-ASR性能的主要因素，而在复杂SLU场景下，当前音频-语言模型对副语言声学的理解仍存在困难。更多详情可在挑战赛官方网站查询。
英文摘要
Recent advances in large language models (LLMs) and multimodal LLMs (MLLMs) have created new opportunities for wearable speech interfaces, with smart glasses providing an egocentric platform for continuous audio sensing and assistance. However, speech recognition and understanding in this setting remain challenging because of dynamic acoustic conditions, speaker overlap, and the spatial ambiguity introduced by wearer-centered recording geometry. To support systematic evaluation in this setting, we introduce the IEEE SLT 2026 SmartGlasses Challenge for egocentric multi-speaker speech processing. The challenge consists of two tracks, Dyadic Dialogue Understanding and Multi-party Meeting Understanding, and jointly evaluates Time-Stamped Speaker-Attributed Automatic Speech Recognition (TSA-ASR) and Spoken Language Understanding (SLU). It is built on a 106-hour four-channel egocentric speech dataset containing 714 sessions collected in real-world scenarios. This paper describes challenge tasks, dataset construction, submissions, and summarizes the main findings from the shared evaluation. The results show that heavy speaker overlap remains a major factor affecting TSA-ASR performance, while paralinguistic acoustic understanding continues to be difficult for current audio-language models in complex SLU settings. Further details can be found on the official challenge website.
发表机构
- Northwestern Polytechnical University（西北工业大学）
- Huawei（华为）
- AIShell Inc（AIShell公司）
- Shanghai Jiao Tong University（上海交通大学）
- Nanjing University（南京大学）
- University of Science and Technology of China（中国科学技术大学）
- Nanyang Technological University（南洋理工大学）
- Rokid
机构由 AI 辅助整理，请以论文原文为准。
