跨模态检索训练中的数据泄露：一项案例研究
Data leakage in cross-modal retrieval training: A case study
AI总结：
本研究针对自动采集的SoundDesc基准数据集存在训练数据泄露至评估集的问题，提出了按录制设置分组的新数据集划分方案，构建了更具挑战性的跨模态检索基准。
摘要：
基于文本的音频检索领域的近期进展很大程度上得益于合适数据集的发布。由于人工构建这类数据集耗时耗力，从在线资源获取数据是创建大规模数据集的低成本方案。我们研究了近期提出的SoundDesc基准数据集（该数据集自动从BBC音效网页采集）。分析发现，SoundDesc存在多个重复样本，导致训练数据泄露到评估数据中。这种数据泄露最终使得此前基准测试中的检索性能估计过于乐观。我们为该数据集提出了新的训练集、验证集和测试集划分，并已在线发布。为避免测试数据受到轻微污染，我们将具有相似录制设置的音频文件归为一组。实验表明，新的划分方案构成了更具挑战性的基准。
英文摘要：
The recent progress in text-based audio retrieval was largely propelled by the release of suitable datasets. Since the manual creation of such datasets is a laborious task, obtaining data from online resources can be a cheap solution to create large-scale datasets. We study the recently proposed SoundDesc benchmark dataset, which was automatically sourced from the BBC Sound Effects web page. In our analysis, we find that SoundDesc contains several duplicates that cause leakage of training data to the evaluation data. This data leakage ultimately leads to overly optimistic retrieval performance estimates in previous benchmarks. We propose new training, validation, and testing splits for the dataset that we make available online. To avoid weak contamination of the test data, we pool audio files that share similar recording setups. In our experiments, we find that the new splits serve as a more challenging benchmark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
