重新思考基于音乐元数据的音乐描述生成
Rethinking Music Captioning with Music Metadata LLMs
AI总结：
本文提出基于元数据的音乐描述生成方法，通过训练元数据预测模型和预训练LLMs，实现更灵活的描述生成和元数据填补。
摘要：
音乐描述生成，即生成音乐的自然语言描述，对音乐理解和可控音乐生成都有帮助。然而，训练描述生成模型通常需要高质量的音乐描述数据，而这种数据比元数据（例如类型、情绪等）更稀缺。因此，通常使用大型语言模型（LLMs）从元数据合成描述以生成描述生成模型的训练数据，尽管这一过程会施加固定的风格化并使事实信息与自然语言风格纠缠。作为更直接的方法，我们提出了基于元数据的描述生成。我们训练了一个元数据预测模型，从音频中推断详细的音乐元数据，然后在推理时通过预训练的LLMs将其转换为具有表现力的描述。与一个在LLM生成的描述上训练的强端到端基线相比，我们的方法：（1）在较少的训练时间内实现了与端到端描述器相当的性能；（2）提供了灵活性，可以轻松地在训练后更改风格化，使输出描述能够根据特定的风格和质量要求进行定制；（3）可以提示音频和部分元数据，以实现强大的元数据填补或填充——这是整理音乐数据的常见任务。
英文摘要：
Music captioning, or the task of generating a natural language description of music, is useful for both music understanding and controllable music generation. Training captioning models, however, typically requires high-quality music caption data which is scarce compared to metadata (e.g., genre, mood, etc.). As a result, it is common to use large language models (LLMs) to synthesize captions from metadata to generate training data for captioning models, though this process imposes a fixed stylization and entangles factual information with natural language style. As a more direct approach, we propose metadata-based captioning. We train a metadata prediction model to infer detailed music metadata from audio and then convert it into expressive captions via pre-trained LLMs at inference time. Compared to a strong end-to-end baseline trained on LLM-generated captions derived from metadata, our method: (1) achieves comparable performance in less training time over end-to-end captioners, (2) offers flexibility to easily change stylization post-training, enabling output captions to be tailored to specific stylistic and quality requirements, and (3) can be prompted with audio and partial metadata to enable powerful metadata imputation or in-filling--a common task for organizing music data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
