Audio-DeepThinker：渐进式推理感知强化学习用于音频语言模型中高质量推理链涌现
Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
AI总结：
本文提出Audio-DeepThinker框架，通过混合奖励和渐进式课程学习提升音频推理质量，实现高质量推理链涌现，取得多项评测冠军。
摘要：
大型音频-语言模型（LALMs）在音频理解上取得显著进展，但主要作为感知和回答系统，缺乏显式推理过程。现有方法通过监督式推理链（CoT）微调或强化学习（RL）提升音频推理，但受限于训练数据质量和粗略奖励。本文提出Audio-DeepThinker框架，结合混合推理相似度奖励和渐进式双阶段课程学习，通过纯RL探索无需监督推理微调，从无先验推理能力的指令微调模型出发，实现高质量CoT推理。
英文摘要：
Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems without explicit reasoning processes. Existing methods for enhancing audio reasoning rely either on supervised chain-of-thought (CoT) fine-tuning, which is limited by training data quality, or on reinforcement learning (RL) with coarse rewards that do not directly evaluate reasoning quality. As a result, the generated reasoning chains often appear well-structured yet lack specific acoustic grounding. We propose Audio-DeepThinker, a framework built on two core ideas. First, we introduce a hybrid reasoning similarity reward that directly supervises the quality of generated reasoning chains by combining an LLM evaluator assessing logical path alignment, key step coverage, and analytical depth with an embedding similarity component enforcing semantic alignment with reference reasoning chains. Second, we propose a progressive two-stage curriculum that enables high-quality CoT reasoning to emerge through pure RL exploration, without any supervised reasoning fine-tuning, from an instruction-tuned model that possesses no prior chain-of-thought capability. Stage 1 trains on foundational audio QA with the hybrid reward to foster basic reasoning patterns, while Stage 2 shifts to acoustically challenging boundary cases with an LLM-only reward for greater reasoning diversity. Audio-DeepThinker achieves state-of-the-art results on MMAR (74.0%), MMAU-test-mini (78.5%), and MMSU (77.26%), winning 1st Place in the Interspeech 2026 Audio Reasoning Challenge (Single Model Track). Interpretability analyses further reveal that RL training primarily reshapes upper-layer MoE gating mechanisms and that reasoning tokens crystallize progressively in the upper transformer layers, offering mechanistic insights into how audio reasoning emerges through exploration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
