TMU系统用于XACLE挑战:利用CLAP伪标签训练大型音频语言模型
The TMU System for the XACLE Challenge: Training Large Audio Language Models with CLAP Pseudo-Labels
AI总结:
TMU系统通过CLAP伪标签预训练,在XACLE挑战中实现0.632的SRCC成绩,优于基线系统并获得第三名。
摘要:
在本文中,我们提出了对XACLE挑战的提交。目标是预测给定的一般音频和文本对的语义对齐。所提出的系统基于大型音频语言模型(LALM)架构。我们采用三阶段训练流水线:自动音频字幕预训练、使用CLAP伪标签的预训练以及在XACLE数据集上的微调。我们的实验表明,使用CLAP伪标签的预训练是性能的主要驱动因素。在XACLE测试集上,我们的系统达到了SRCC为0.632的性能,显著优于基线系统(0.334),并在挑战团队排名中位列第三。代码和模型可在https://github.com/shiotalab-tmu/tmu-xacle2026找到。
英文摘要:
In this paper, we propose a submission to the x-to-audio alignment (XACLE) challenge. The goal is to predict semantic alignment of a given general audio and text pair. The proposed system is based on a large audio language model (LALM) architecture. We employ a three-stage training pipeline: automated audio captioning pretraining, pretraining with CLAP pseudo-labels, and fine-tuning on the XACLE dataset. Our experiments show that pretraining with CLAP pseudo-labels is the primary performance driver. On the XACLE test set, our system reaches an SRCC of 0.632, significantly outperforming the baseline system (0.334) and securing third place in the challenge team ranking. Code and models can be found at https://github.com/shiotalab-tmu/tmu-xacle2026
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。