arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.02549cs.CVcs.CL
收藏

DemaFormer: 带能量建模的阻尼指数移动平均变换器用于时序语言定位

DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Grounding

National University of Singapore(国立新加坡大学);Nanyang Technological University(南洋理工大学);Carnegie Mellon University(卡内基梅隆大学)

Thong Nguyen, Xiaobao Wu, Xinshuai Dong, Cong-Duy Nguyen, See-Kiong Ng, Luu Anh Tuan

AI总结:

本文提出DemaFormer,通过阻尼指数移动平均机制改进时序语言定位任务中的注意力机制,有效分离目标视频片段与其余片段。

摘要:

时序语言定位旨在定位与自然语言查询语义对应的视频时刻。近期进展利用注意力机制学习视频时刻与文本查询之间的关系。然而,朴素的注意力机制可能无法恰当捕捉此类关系,导致目标视频时刻难以与其他片段区分。为了解决这一问题,我们提出了一种基于能量模型的框架,以显式学习时刻-查询分布。此外,我们提出DemaFormer,一种新的基于Transformer的架构,利用带有可学习阻尼因子的指数移动平均来有效编码时刻-查询输入。在四个公开的时序语言定位数据集上的全面实验展示了我们方法优于最新基线的优越性。

英文摘要:

Temporal Language Grounding seeks to localize video moments that semantically correspond to a natural language query. Recent advances employ the attention mechanism to learn the relations between video moments and the text query. However, naive attention might not be able to appropriately capture such relations, resulting in ineffective distributions where target video moments are difficult to separate from the remaining ones. To resolve the issue, we propose an energy-based model framework to explicitly learn moment-query distributions. Moreover, we propose DemaFormer, a novel Transformer-based architecture that utilizes exponential moving average with a learnable damping factor to effectively encode moment-query inputs. Comprehensive experiments on four public temporal language grounding datasets showcase the superiority of our methods over the state-of-the-art baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑