AuditoryHuM: 利用人机协同生成和聚类听觉场景标签
AuditoryHuM: Auditory Scene Label Generation and Clustering using Human-MLLM Collaboration
AI总结：
AuditoryHuM通过人机协同方法实现听觉场景标签的自动生成与聚类，提供了一种高效且低成本的标准化分类解决方案，适用于边缘设备部署。
摘要：
手动标注音频数据集耗时费力，难以在标签粒度与声学分离性之间取得平衡。我们介绍了AuditoryHuM，这是一种利用人机协同方法进行无监督发现和聚类听觉场景标签的新型框架。通过利用MLLM（Gemma和Qwen），该框架为音频数据生成上下文相关的标签。为确保标签质量并缓解幻觉，我们采用零样本学习技术（Human-CLAP）来量化生成文本标签与原始音频内容之间的对齐程度。然后通过战略性的人工干预来优化最不匹配的配对。发现的标签通过调整的轮廓系数进行聚类，该系数结合了惩罚参数以平衡聚类凝聚性和主题粒度。在三个多样化的听觉场景数据集（ADVANCE、AHEAD-DS和TAU 2019）上评估，AuditoryHuM提供了一种可扩展、低成本的创建标准化分类体系的解决方案。该解决方案促进了轻量级场景识别模型的训练，这些模型可以部署到边缘设备，如助听器和智能家庭助手。项目页面和代码：https://github.com/Australian-Future-Hearing-Initiative
英文摘要：
Manual annotation and clustering of audio datasets is labour intensive. We introduce AuditoryHuM, a training-free framework for the unsupervised discovery and clustering of auditory scene labels using human-Multimodal Large Language Model (MLLM) collaboration. Leveraging MLLMs, our framework generates contextually relevant labels for audio data. To ensure label quality and mitigate hallucinations, zero-shot learning (Human-CLAP) quantifies the alignment between generated text labels and raw audio. A targeted human-in-the-loop intervention, refines only the lowest aligned pairs. The discovered labels form an interpretable alignment vector to group audio into cohesive clusters. The framework was evaluated across three auditory scene datasets (ADVANCE, AHEAD-DS, and TAU 2019), achieving a 96.1\% reduction in human labour for ADVANCE during testing. Downstream models trained on our clusters exhibit enhanced classification accuracy vs the baseline (0.79 to 0.84) in clean acoustic environments, though performance scales down in dense, complex soundscapes. The project page: \url{https://github.com/Australian-Future-Hearing-Initiative}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
