音频用于体育精彩片段检测：一项比较实证研究
Audio for Sports Highlight Detection: A Comparative Empirical Study
浏览论文内容
中文总结 AI 辅助
本研究通过构建轻量级仅音频基线，在SV-Highlights基准上证明音频在体育精彩片段检测中具有强效性，且与视觉融合效果最佳，应作为主要信号而非辅助线索。
中文摘要 AI 辅助
体育精彩片段检测旨在从长体育视频中识别最激动人心和有意义的时刻。虽然现有方法通常强调视觉或视觉-语言表示，但体育视频包含丰富的音频线索，包括解说员语音、观众反应、哨声、球击声和裁判判罚声。在这项工作中，我们重新审视音频在体育精彩片段检测中的作用，并提出一个简单的问题：仅凭音频能走多远？我们使用预训练的音频表示构建了轻量级的仅音频基线，并将其与仅视觉和音频-视觉方法在SV-Highlights基准上进行比较。令人惊讶的是，在我们的监督评估设置下，我们的仅音频GRU基线取得了强劲性能，并优于几种现有的音频-视觉方法。此外，一个简单的音频-视觉融合基线在所有指标上取得了最佳性能，表明音频和视觉线索提供了互补信息。为了更好地理解音频的贡献，我们进行了源分离分析，并表明人声/解说音频比仅背景音频更具信息量，而它们的组合表现最佳。我们还分析了可解释的音频线索，发现精彩片段剪辑比非精彩片段剪辑具有更高的RMS响度、峰值响度和中频能量，尽管显著的分布重叠表明仅响度是不够的。我们的发现表明，音频是体育精彩片段检测中一个未被充分探索但高度信息丰富的模态，应被视为主要信号而非仅仅是辅助线索。
英文摘要
Sports highlight detection aims to identify the most exciting and meaningful moments from long sports videos. While existing methods often emphasize visual or visual-language representations, sports videos contain rich audio cues, including commentator speech, crowd reactions, whistles, ball impacts, and referee calls. In this work, we revisit the role of audio in sports highlight detection and ask a simple question: how far can audio alone go? We construct lightweight audio-only baselines using pretrained audio representations and compare them with visual-only and audio-visual methods on the SV-Highlights benchmark. Surprisingly, our audio-only GRU baseline achieves strong performance and outperforms several existing audio-visual methods under our supervised evaluation setting. Furthermore, a simple audio-visual fusion baseline achieves the best performance across all metrics, indicating that audio and visual cues provide complementary information. To better understand the contribution of audio, we conduct source-separated analysis and show that vocal/commentary audio is more informative than background-only audio, while their combination performs best. We also analyze interpretable audio cues and find that highlight clips exhibit higher RMS loudness, peak loudness, and mid-frequency energy than non-highlight clips, although substantial distribution overlap indicates that loudness alone is insufficient. Our findings suggest that audio is an underexplored but highly informative modality for sports highlight detection and should be treated as a primary signal rather than merely an auxiliary cue.
发表机构
- Deakin University（迪肯大学）
- Dolby Laboratories Inc.（杜比实验室公司）
机构由 AI 辅助整理，请以论文原文为准。
