NAAQA：一种用于声学问答的神经架构
NAAQA: A Neural Architecture for Acoustic Question Answering
AI总结：
本文提出声学问答新基准 CLEAR2 和神经架构 NAAQA，利用一维卷积与时间坐标图提升声学场景问答性能，以更少参数达到 79.5% 准确率。
摘要：
声学问答（AQA）任务的目标是回答关于声学场景内容的自由形式文本问题，该任务受视觉问答（VQA）任务启发。本文基于此前提出的 CLEAR 数据集，提出了一个新的 AQA 基准，即 CLEAR2，以突出声学输入的特定挑战，包括处理可变时长的场景，以及由训练集和测试集之间不同的基本声音构建的场景。我们还提出了 NAAQA，一种利用声学输入特定性质的神经架构。使用时间和频率上的一维卷积处理声学内容的二维谱时表示，取得了有前景的结果，并能够降低模型复杂度。我们表明，时间坐标图增强了时间定位能力，使网络性能提升约 17 个百分点；相比之下，频率坐标图对该任务影响很小。NAAQA 在 AQA 任务上达到 79.5% 的准确率，参数数量比此前探索的 VQA 模型少约 4 倍。我们在由 DAQA 重建的独立数据集上评估了 NAAQA 的性能，并在 CLEAR2 和 DAQA 上测试了在模型中添加 MALiMo 模块的效果。我们对不同问题类型的结果进行了详细分析。我们发布了生成 CLEAR2 以及 NAAQA 的代码，以促进这一新兴机器学习任务的研究。
英文摘要：
The goal of the Acoustic Question Answering (AQA) task is to answer a free-form text question about the content of an acoustic scene. It was inspired by the Visual Question Answering (VQA) task. In this paper, based on the previously introduced CLEAR dataset, we propose a new benchmark for AQA, namely CLEAR2, that emphasizes the specific challenges of acoustic inputs. These include handling of variable duration scenes, and scenes built with elementary sounds that differ between training and test set. We also introduce NAAQA, a neural architecture that leverages specific properties of acoustic inputs. The use of 1D convolutions in time and frequency to process 2D spectro-temporal representations of acoustic content shows promising results and enables reductions in model complexity. We show that time coordinate maps augment temporal localization capabilities which enhance performance of the network by ~17 percentage points. On the other hand, frequency coordinate maps have little influence on this task. NAAQA achieves 79.5% of accuracy on the AQA task with ~4 times fewer parameters than the previously explored VQA model. We evaluate the perfomance of NAAQA on an independent data set reconstructed from DAQA. We also test the addition of a MALiMo module in our model on both CLEAR2 and DAQA. We provide a detailed analysis of the results for the different question types. We release the code to produce CLEAR2 as well as NAAQA to foster research in this newly emerging machine learning task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
