基于结构化检索的长音频事件问答
Event-Grounded Question Answering over Long Audio via Structured Retrieval
AI总结：
提出LA-RAG框架，通过音频基础模型将长音频转为带时间戳的事件记录并存入SQL数据库，结合意图感知检索和LLM生成，实现低延迟高精度的长音频问答，在多个基准上提升时间定位F1达11-17%。
摘要：
回答跨多小时的音频自然语言问题需要事件识别和时间定位。当前大型音频语言模型在短片段上表现良好，但受限于上下文长度、查询时间成本和较弱的时间定位能力。我们提出LA-RAG（长音频检索增强生成），一种结构化框架，使用开放词汇音频基础模型（AGM）将连续音频转换为带时间戳的事件记录，存储在SQL事件数据库中，并通过意图感知检索和基于LLM的生成来回答查询。LA-RAG支持离线基础模式（对长录音进行预索引以实现低延迟问答）和推理时基础模式（对较短开放片段进行查询条件基础）。我们创建了24小时的Home-IoT和Industrial-IoT音频基准，并扩展了CASTELLA（一个带有问答对的真实世界音频时刻检索数据集）。在离线基础模式下，LA-RAG在Home-IoT上达到76.88%的整体准确率，在Industrial-IoT上达到71.10%，平均查询延迟低于0.6秒。在推理时基础模式下，最先进的LALM在CASTELLA-QA上实现了有竞争力的事件检测准确率，但时间检测F1较低。我们进一步表明，使用我们的结构化检索元数据增强的LALM在时间检测上持续改进，基线模型的F1提升11-17%，同时延迟改善。这些结果表明，显式的时间戳基础和结构化检索为面向部署的长音频问答提供了对生成式音频语言模型的实际补充。
英文摘要：
Answering natural-language questions over multi-hour audio requires reliable event recognition, temporal grounding, and efficient retrieval. We present LA-RAG (Long Audio Retrieval-Augmented Generation), a structured framework that converts audio into timestamped event records, stores them in an event database, and answers questions using intent-aware retrieval and LLM-based generation. LA-RAG supports two deployment settings: offline grounding mode, in which long recordings are pre-indexed for low-latency querying, and inference-time grounding mode, which performs query-conditioned grounding over shorter, open-ended clips. We evaluate LA-RAG on controlled 24-hour Home-IoT and Industrial-IoT benchmarks and on CASTELLA-QA, derived from real-world audio recordings. The results demonstrate effective long-audio question answering with low query-time latency after grounding and indexing. They also reveal a substantial gap between event detection and temporal localization in current large audio-language models, while showing that explicit timestamped grounding consistently improves temporal reasoning. These findings establish structured grounding and retrieval as a practical complement to generative audio-language models for deployment-oriented long-audio understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
