Synth-AC：利用合成监督增强音频描述生成
Synth-AC: Enhancing Audio Captioning with Synthetic Supervision
AI总结：
本文提出SynthAC框架，利用AudioLDM文本到音频生成模型和图像描述文本语料库创建合成文本-音频对，将文本-视觉领域的标注数据扩展到音频描述生成，缓解数据稀缺问题并显著提升现有音频描述生成模型的性能。
摘要：
数据驱动方法在音频描述生成领域具有广阔前景。然而，由于文本-音频数据的可用性和质量有限，音频描述生成方法的发展可能存在偏差。本文提出了SynthAC框架，该框架利用音频生成模型的最新进展和普遍可用的文本语料库来创建合成的文本-音频对，从而增强文本-音频表示。具体而言，使用文本到音频生成模型（即AudioLDM）根据图像描述数据集中的描述生成合成音频信号。我们的SynthAC将标注良好的描述从文本-视觉领域扩展到音频描述生成，从而通过学习合成文本-音频对中的关系来增强文本-音频表示。实验表明，我们的SynthAC框架可以通过引入文本-视觉领域中标注良好的文本语料库使音频描述生成模型受益，为数据稀缺带来的挑战提供了一种有前景的解决方案。此外，SynthAC可以轻松适配各种最先进的方法，带来显著的性能提升。
英文摘要：
Data-driven approaches hold promise for audio captioning. However, the development of audio captioning methods can be biased due to the limited availability and quality of text-audio data. This paper proposes a SynthAC framework, which leverages recent advances in audio generative models and commonly available text corpus to create synthetic text-audio pairs, thereby enhancing text-audio representation. Specifically, the text-to-audio generation model, i.e., AudioLDM, is used to generate synthetic audio signals with captions from an image captioning dataset. Our SynthAC expands the availability of well-annotated captions from the text-vision domain to audio captioning, thus enhancing text-audio representation by learning relations within synthetic text-audio pairs. Experiments demonstrate that our SynthAC framework can benefit audio captioning models by incorporating well-annotated text corpus from the text-vision domain, offering a promising solution to the challenge caused by data scarcity. Furthermore, SynthAC can be easily adapted to various state-of-the-art methods, leading to substantial performance improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
