面向音频编辑的自动评估与高质量伪平行数据集构建：一种人机协同方法
Towards Automatic Evaluation and High-Quality Pseudo-Parallel Dataset Construction for Audio Editing: A Human-in-the-Loop Method
AI总结：
本文提出一种人机协同方法，通过建立高质量数据集和自动评估系统，提升音频编辑任务的评估与数据质量。
摘要：
音频编辑旨在基于文本描述操纵音频内容，支持添加、删除或替换音频事件等任务。尽管近年来取得了一定进展，但缺乏高质量基准数据集和全面的评估指标仍然是评估音频编辑质量及提升该任务本身的主要挑战。在本文中，我们提出了一种新的音频编辑方法，通过将专家知识融入评估和数据集构建过程：1）首先，我们建立了AuditScore，第一个用于音频编辑主观评估的全面数据集，包含来自7个代表性音频编辑框架和23种系统配置生成的超过6300个编辑样本。每个样本由专业评分者在音频编辑质量的三个关键方面进行标注：整体质量、与编辑意图的相关性和对原始特征的忠实性。2）基于此数据集，我们系统地提出AuditEval，一个专门针对音频编辑的自动MOS型评估器家族，涵盖基于SSL和基于LLM的方法。它解决了该领域缺乏有效客观指标和主观评估成本高昂的问题。3）我们进一步利用AuditEval来评估和过滤大量合成混合的编辑对，通过选择最可能的样本挖掘出高质量的伪平行子集。全面实验验证了我们的专家指导过滤策略能有效产生更高质量的数据，同时揭示了传统客观指标的局限性和AuditEval的优势。数据集、代码和工具可在：https://github.com/NKU-HLT/AuditEval获取。
英文摘要：
Audio editing aims to manipulate audio content based on textual descriptions, supporting tasks such as adding, removing, or replacing audio events. Despite recent progress, the lack of high-quality benchmark datasets and comprehensive evaluation metrics remains a major challenge for both assessing audio editing quality and improving the task itself. In this work, we propose a novel approach for audio editing task by incorporating expert knowledge into both the evaluation and dataset construction processes: 1) First, we establish AuditScore, the first comprehensive dataset for subjective evaluation of audio editing, consisting of over 6,300 edited samples generated from 7 representative audio editing frameworks and 23 system configurations. Each sample is annotated by professional raters on three key aspects of audio editing quality: overall Quality, Relevance to editing intent, and Faithfulness to original features. 2) Based on this dataset, we systematically propose AuditEval, a family of automatic MOS-style evaluators tailored for audio editing, covering both SSL-based and LLM-based approaches. It addresses the lack of effective objective metrics and the prohibitive cost of subjective evaluation in this field. 3) We further leverage AuditEval to evaluate and filter a large amount of synthetically mixed editing pairs, mining a high-quality pseudo-parallel subset by selecting the most plausible samples. Comprehensive experiments validate that our expert-informed filtering strategy effectively yields higher-quality data, while also exposing the limitations of traditional objective metrics and the advantages of AuditEval. The dataset, codes and tools can be found at: https://github.com/NKU-HLT/AuditEval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
