Video grounding aims to localize the corresponding video moment in an untrimmed video given a language query. Existing methods often address this task in an indirect way, by casting it as a proposal-and-match or fusion-and-detection problem. Solving these surrogate problems often requires sophisticated label assignment during training and hand-crafted removal of near-duplicate results. Meanwhile, existing works typically focus on sparse video grounding with a single sentence as input, which could result in ambiguous localization due to its unclear description. In this paper, we tackle a new problem of dense video grounding, by simultaneously localizing multiple moments with a paragraph as input. From a perspective on video grounding as language conditioned regression, we present an end-to-end parallel decoding paradigm by re-purposing a Transformer-alike architecture (PRVG). The key design in our PRVG is to use languages as queries, and directly regress the moment boundaries based on language-modulated visual representations. Thanks to its simplicity in design, our PRVG framework can be applied in different testing schemes (sparse or dense grounding) and allows for efficient inference without any post-processing technique. In addition, we devise a robust proposal-level attention loss to guide the training of PRVG, which is invariant to moment duration and contributes to model convergence. We perform experiments on two video grounding benchmarks of ActivityNet Captions and TACoS, demonstrating that our PRVG can significantly outperform previous methods. We also perform in-depth studies to investigate the effectiveness of parallel regression paradigm on video grounding.
Siamese Learning with Joint Alignment and Regression for Weakly-Supervised Video Paragraph Grounding
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
ScreenParse: Moving Beyond Sparse Grounding with Complete Screen Parsing Supervision
IBM Research Zurich, Zurich, Switzerland（IBM苏黎世研究实验室，瑞士苏黎世）；ETH Zurich, Computer Vision（苏黎世联邦理工学院，计算机视觉）；ETH AI Center, Switzerland（苏黎世联邦理工学院人工智能中心，瑞士）；ETH Zurich, Photogrammetry（苏黎世联邦理工学院，摄影测量学）；Microsoft, Switzerland（微软公司，瑞士）
OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding
Wangxuan Institute of Computer Technology, Peking University（北京大学王轩计算机技术研究所）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；Central Media Technology Institute, Huawei Technologies Ltd.（华为技术有限公司中央媒体技术研究所）；PKU-WUHAN Institute for Artificial Intelligence, Peking University（北京大学武汉人工智能研究所）
ViGT: Proposal-free Video Grounding with Learnable Token in Transformer
Hefei University of Technology（合肥工业大学）；Key Laboratory of Knowledge Engineering with Big Data, Ministry of Education（教育部大数据知识工程重点实验室）；Intelligent Interconnected Systems Laboratory of Anhui Province（安徽省智能互联系统实验室）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding
Zhejiang University（浙江大学）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院）；Hikvision Research Institute（海康威视研究院）；Northwestern Polytechnical University（西北工业大学）；Shanghai AI Laboratory（上海人工智能实验室）
SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
Beihang University（北京航空航天大学）；State Key Laboratory of Space Information System and Integrated Application（空间信息系统与集成应用国家重点实验室）；Nanjing University（南京大学）；Harbin Engineering University（哈尔滨工程大学）；Beijing Zhongguancun Academy（北京中关村科学城）
What, when, and where? -- Self-Supervised Spatio-Temporal Grounding in Untrimmed Multi-Action Videos from Narrated Instructions
Columbia University（哥伦比亚大学）；Goethe University Frankfurt（法兰克福大学）；University of Bonn（波恩大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Quality Match GmbH（Quality Match有限公司）；IBM Research AI（IBM研究院人工智能部门）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Australian National University（澳大利亚国立大学）；Aalto University（阿尔托大学）；Carnegie Mellon University（卡内基梅隆大学）；University of California - Merced（加州大学默塞德分校）；Linköping University（林雪平大学）；Google Research（谷歌研究院）
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding
Nanyang Technological University（南洋理工大学）；University of Reading（阅读大学）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）；SenseTime Research（商汤科技研究院）
