面向噪声的上下文学习用于ALLMs中的幻觉抑制
Noise-Aware In-Context Learning for Hallucination Mitigation in ALLMs
AI总结：
本文提出噪声感知上下文学习方法，通过构建噪声先验库和引入噪声示例作为上下文先验，减少生成模型的幻觉问题，并建立音频描述任务的幻觉基准，实验表明该方法有效降低幻觉率。
摘要：
本文提出噪声感知上下文学习方法，通过构建噪声先验库和引入噪声示例作为上下文先验，减少生成模型的幻觉问题，并建立音频描述任务的幻觉基准，实验表明该方法有效降低幻觉率。
英文摘要：
Auditory large language models (ALLMs) have demonstrated strong general capabilities in audio understanding and reasoning tasks. However, their reliability is still undermined by hallucination issues. Existing hallucination evaluation methods are formulated as binary classification tasks, which are insufficient to characterize the more complex hallucination patterns that arise in generative tasks. Moreover, current hallucination mitigation strategies rely on fine-tuning, resulting in high computational costs. To address the above limitations, we propose a plug-and-play Noise-Aware In-Context Learning (NAICL) method. Specifically, we construct a noise prior library, retrieve noise examples relevant to the input audio, and incorporate them as contextual priors, thereby guiding the model to reduce speculative associations when acoustic evidence is insufficient and to adopt a more conservative generation strategy. In addition, we establish a hallucination benchmark for audio caption tasks including the construction of the Clotho-1K multi-event benchmark dataset, the definition of four types of auditory hallucinations, and the introduction of metrics such as hallucination type distribution to support fine-grained analysis. Experimental results show that all evaluated ALLMs exhibit same hallucination behaviors. Moreover, the proposed NAICL method reduces the overall hallucination rate from 26.53% to 16.98%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
