利用相似性-差异性解耦的音频差异字幕生成
Audio Difference Captioning Utilizing Similarity-Discrepancy Disentanglement
AI总结：
本研究提出音频差异字幕生成新任务ADC，设计聚焦交叉注意力的Transformer编码器与相似性-差异性解耦方法，构建AudioDiffCaps数据集，实验验证方法可有效提取音频差异并完成字幕生成。
摘要：
我们提出音频差异字幕生成（Audio Difference Captioning，ADC）作为音频字幕的新拓展任务，用于描述输入的一对相似但存在细微差异的音频片段之间的语义差异。ADC解决了传统音频字幕有时会为相似音频片段生成相似字幕、无法描述内容差异的问题。我们还提出了一种聚焦交叉注意力的Transformer编码器，通过对比一对音频片段提取差异，并提出相似性-差异性解耦方法，在潜在空间中强化差异特征。为评估所提方法，我们构建了AudioDiffCaps数据集，包含多组相似但有细微差异的音频片段对，并配有人类标注的差异描述。在AudioDiffCaps数据集上的实验表明，所提方法可有效完成ADC任务，且通过对Transformer编码器中的注意力权重可视化，证实该方法提升了用于提取差异的注意力权重。
英文摘要：
We proposed Audio Difference Captioning (ADC) as a new extension task of audio captioning for describing the semantic differences between input pairs of similar but slightly different audio clips. The ADC solves the problem that conventional audio captioning sometimes generates similar captions for similar audio clips, failing to describe the difference in content. We also propose a cross-attention-concentrated transformer encoder to extract differences by comparing a pair of audio clips and a similarity-discrepancy disentanglement to emphasize the difference in the latent space. To evaluate the proposed methods, we built an AudioDiffCaps dataset consisting of pairs of similar but slightly different audio clips with human-annotated descriptions of their differences. The experiment with the AudioDiffCaps dataset showed that the proposed methods solve the ADC task effectively and improve the attention weights to extract the difference by visualizing them in the transformer encoder.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
