利用声音事件检测增强音频字幕生成中的时间关系
Enhance Temporal Relations in Audio Captioning with Sound Event Detection
AI总结：
本文利用声音事件检测（SED）提供的时间戳信息，通过提出时间标签系统，增强音频字幕生成中的时间关系表达，实验证明显著提升了时间关系生成效果。
摘要：
自动音频字幕生成旨在为给定的音频片段生成自然语言描述，不仅需要检测和分类声音，还要总结音频事件之间的关系。近年来音频字幕生成的研究进展引入了额外的指导，以提高生成句子中音频事件的准确性。然而，音频事件之间的时间关系很少受到关注，而揭示复杂关系是总结音频内容的关键组成部分。因此，本文旨在利用声音事件检测（SED）——一项定位事件时间戳的任务——更好地捕捉字幕生成中的时间关系。我们研究了在字幕模型中整合时间信息的最佳方法，并提出了一种时间标签系统，将时间戳转换为可理解的关系。由所提出的时间指标评估的结果表明，在时间关系生成方面取得了显著改进。
英文摘要：
Automated audio captioning aims at generating natural language descriptions for given audio clips, not only detecting and classifying sounds, but also summarizing the relationships between audio events. Recent research advances in audio captioning have introduced additional guidance to improve the accuracy of audio events in generated sentences. However, temporal relations between audio events have received little attention while revealing complex relations is a key component in summarizing audio content. Therefore, this paper aims to better capture temporal relationships in caption generation with sound event detection (SED), a task that locates events' timestamps. We investigate the best approach to integrate temporal information in a captioning model and propose a temporal tag system to transform the timestamps into comprehensible relations. Results evaluated by the proposed temporal metrics suggest that great improvement is achieved in terms of temporal relation generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
