EnCLAP：结合神经音频编解码器与音文联合嵌入的自动音频字幕生成方法
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning
AI总结：
本研究提出自动音频字幕生成框架EnCLAP，结合EnCodec、CLAP与预训练模型BART，通过掩码编解码器建模提升声学感知，在AudioCaps和Clotho数据集上性能优于基线，并开源代码与演示。
摘要：
我们提出EnCLAP，一种用于自动音频字幕生成的新型框架。EnCLAP采用两种声学表征模型——EnCodec和CLAP，以及预训练语言模型BART。我们还引入了名为掩码编解码器建模的新训练目标，以提升预训练语言模型的声学感知能力。在AudioCaps和Clotho数据集上的实验结果表明，该模型性能优于基线模型。源代码将发布于https://github.com/jaeyeonkim99/EnCLAP，在线演示可访问https://huggingface.co/spaces/enclap-team/enclap。
英文摘要：
We propose EnCLAP, a novel framework for automated audio captioning. EnCLAP employs two acoustic representation models, EnCodec and CLAP, along with a pretrained language model, BART. We also introduce a new training objective called masked codec modeling that improves acoustic awareness of the pretrained language model. Experimental results on AudioCaps and Clotho demonstrate that our model surpasses the performance of baseline models. Source code will be available at https://github.com/jaeyeonkim99/EnCLAP . An online demo is available at https://huggingface.co/spaces/enclap-team/enclap .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
