ReAct：基于关系映射的时间动作检测
ReAct: Temporal Action Detection with Relational Queries
摘要：
这项工作的目的是在推进时间动作检测（ESTA）使用的编码器-解码器框架与动作查询，类似于DETR，这已经显示出巨大的成功，在目标检测。然而，该框架遭受的几个问题，如果直接应用到搜索引擎：在解码器中的查询间关系的探索不足，不充分的分类训练，由于有限数量的训练样本，和不可靠的分类分数在推理。为此，我们首先在解码器中提出了一种关系注意力机制，它根据查询之间的关系来引导查询之间的注意力。此外，我们提出了两个损失，以促进和稳定的动作分类的训练。最后，我们建议在推理时预测每个动作查询的本地化质量，以区分高质量的查询。所提出的方法，命名为ReAct，实现了最先进的性能在THUMOS 14上，与以前的方法相比，具有更低的计算成本。此外，进行了广泛的消融研究，以验证每个拟议组件的有效性。该代码可在https://github.com/sssste/React上获得。
英文摘要：
This work aims at advancing temporal action detection (TAD) using an encoder-decoder framework with action queries, similar to DETR, which has shown great success in object detection. However, the framework suffers from several problems if directly applied to TAD: the insufficient exploration of inter-query relation in the decoder, the inadequate classification training due to a limited number of training samples, and the unreliable classification scores at inference. To this end, we first propose a relational attention mechanism in the decoder, which guides the attention among queries based on their relations. Moreover, we propose two losses to facilitate and stabilize the training of action classification. Lastly, we propose to predict the localization quality of each action query at inference in order to distinguish high-quality queries. The proposed method, named ReAct, achieves the state-of-the-art performance on THUMOS14, with much lower computational costs than previous methods. Besides, extensive ablation studies are conducted to verify the effectiveness of each proposed component. The code is available at https://github.com/sssste/React.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
