Echo：通过音频交错推理实现高级音频理解
Echo: Towards Advanced Audio Comprehension via Audio-Interleaved Reasoning
AI总结：
Echo通过音频交错推理方法，提升音频理解能力，在基准测试中表现出色。
摘要：
大型音频语言模型（LALMs）的成熟促使人们对其理解复杂音频的能力寄予厚望。当前的努力主要通过一次编码将音频内容上下文化，从而复制文本推理，这引入了关键的信息瓶颈。受人类认知启发，我们提出音频交错推理以突破这一瓶颈。它将音频视为一个主动推理组件，使音频能够持续参与并基于感知进行分析。为实现这一目标，我们引入了两阶段训练框架，首先通过监督微调教LALMs定位显著的音频片段，然后通过强化学习激励熟练的重新聆听。同时，开发了结构化的数据生成管道以生成高质量的训练数据。因此，我们提出了Echo，一种能够动态重新聆听音频以满足推理需求的LALM。在音频理解基准测试中，Echo在具有挑战性的专家级和通用任务中均实现了总体优势。全面分析进一步证实了音频交错推理的效率和通用性，确立了其作为推进音频理解的有前途方向。项目页面：https://github.com/wdqqdw/Echo。
英文摘要：
The maturation of Large Audio Language Models (LALMs) has raised growing expectations for them to comprehend complex audio much like humans. Current efforts primarily replicate text-based reasoning by contextualizing audio content through a one-time encoding, which introduces a critical information bottleneck. Drawing inspiration from human cognition, we propose audio-interleaved reasoning to break through this bottleneck. It treats audio as an active reasoning component, enabling sustained audio engagement and perception-grounded analysis. To instantiate it, we introduce a two-stage training framework, first teaching LALMs to localize salient audio segments through supervised fine-tuning, and then incentivizing proficient re-listening via reinforcement learning. In parallel, a structured data generation pipeline is developed to produce high-quality training data. Consequently, we present Echo, a LALM capable of dynamically re-listening to audio in demand during reasoning. On audio comprehension benchmarks, Echo achieves overall superiority in both challenging expert-level and general-purpose tasks. Comprehensive analysis further confirms the efficiency and generalizability of audio-interleaved reasoning, establishing it as a promising direction for advancing audio comprehension. Project page: https://github.com/wdqqdw/Echo.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
