多模态大模型

Visually-aware Acoustic Event Detection using Heterogeneous Graphs

Amir Shirian, Krishna Somandepalli, Victor Sanchez, Tanaya Guha

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.MM、eess.AS

Technical Report for CVPR 2022 LOVEU AQTC Challenge

Hyeonyu Kim, Jongeun Kim, Jeonghun Kang, Sanguk Park, Dongchan Park, Taehwan Kim

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 4 pages, 3 figures, technical report for track3 of CVPR 2022 LOVEU challenge

The Second Place Solution for The 4th Large-scale Video Object Segmentation Challenge--Track 3: Referring Video Object Segmentation

Leilei Cao, Zhuang Li, Bo Yan, Feng Zhang, Fengliang Qi, Yuchen Hu, Hongbin Wang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments 4 pages, 2 figures

TubeDETR: Spatio-Temporal Video Grounding with Transformers

Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, Cordelia Schmid

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Updated vIoU results compared to the CVPR'22 camera-ready version; 17 pages; 8 figures

Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation

Zihan Ding, Tianrui Hui, Junshi Huang, Xiaoming Wei, Jizhong Han, Si Liu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by CVPR 2022

Entity-aware and Motion-aware Transformers for Language-driven Action Localization in Videos

Shuo Yang, Xinxiao Wu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments accepted by IJCAI-22, Codes are available at https://github.com/shuoyang129/EAMAT

Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering

Min Peng, Chongyang Wang, Yuan Gao, Yu Shi, Xiang-Dong Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IJCAI 2022. arXiv admin note: text overlap with arXiv:2109.04735

Learning Commonsense-aware Moment-Text Alignment for Fast Video Temporal Grounding

Ziyue Wu, Junyu Gao, Shucheng Huang, Changsheng Xu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Submitted to IEEE TMM; Code is available at https://github.com/ZiyueWu59/CCA

Spatio-Temporal SAR-Optical Data Fusion for Cloud Removal via a Deep Hierarchical Model

Alessandro Sebastianelli, Artur Nowakowski, Erika Puglisi, Maria Pia Del Rosso, Jamila Mifdal, Fiora Pirri, Pierre Philippe Mathieu, Silvia Liberata Ullo

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Ego4D: Around the World in 3,000 Hours of Egocentric Video

Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Cartillier, Sean Crane, Tien Do, Morrie Doulaty, Akshay Erapalli, Christoph Feichtenhofer, Adriano Fragomeni, Qichen Fu, Abrham Gebreselasie, Cristina Gonzalez, James Hillis, Xuhua Huang, Yifei Huang, Wenqi Jia, Weslie Khoo, Jachym Kolar, Satwik Kottur, Anurag Kumar, Federico Landini, Chao Li, Yanghao Li, Zhenqiang Li, Karttikeya Mangalam, Raghava Modhugu, Jonathan Munro, Tullie Murrell, Takumi Nishiyasu, Will Price, Paola Ruiz Puentes, Merey Ramazanova, Leda Sari, Kiran Somasundaram, Audrey Southerland, Yusuke Sugano, Ruijie Tao, Minh Vo, Yuchen Wang, Xindi Wu, Takuma Yagi, Ziwei Zhao, Yunyi Zhu, Pablo Arbelaez, David Crandall, Dima Damen, Giovanni Maria Farinella, Christian Fuegen, Bernard Ghanem, Vamsi Krishna Ithapu, C. V. Jawahar, Hanbyul Joo, Kris Kitani, Haizhou Li, Richard Newcombe, Aude Oliva, Hyun Soo Park, James M. Rehg, Yoichi Sato, Jianbo Shi, Mike Zheng Shou, Antonio Torralba, Lorenzo Torresani, Mingfei Yan, Jitendra Malik

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI

Comments To appear in the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. This version updates the baseline result numbers for the Hands and Objects benchmark (appendix)

Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video

Jinseok Bae, Hojun Jang, Cheol-Hui Min, Hyungun Choi, Young Min Kim

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 7 pages (main), 10 pages (appendix) and to be appeared in AAAI2022

NEWSKVQA: Knowledge-Aware News Video Question Answering

Pranay Gupta, Manish Gupta

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Deep Video Prior for Video Consistency and Propagation

Chenyang Lei, Yazhou Xing, Hao Ouyang, Qifeng Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by TPAMI in Dec 2021; extension of NeurIPS2020 Blind Video Temporal Consistency via Deep Video Prior. arXiv admin note: substantial text overlap with arXiv:2010.11838

Transformers in Vision: A Survey

Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 30 pages (Accepted in ACM Computing Surveys December 2021)

Negative Sample Matters: A Renaissance of Metric Learning for Temporal Grounding

Zhenzhi Wang, Limin Wang, Tao Wu, Tianhao Li, Gangshan Wu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments AAAI 2022 Camera Ready Version

Relation-aware Video Reading Comprehension for Temporal Language Grounding

Jialin Gao, Xin Sun, Mengmeng Xu, Xi Zhou, Bernard Ghanem

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by EMNLP-21

Fully Automatic Page Turning on Real Scores

Florian Henkel, Stephanie Schwaiger, Gerhard Widmer

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、eess.AS

Comments ISMIR 2021 Late Breaking/Demo

Towards Debiasing Temporal Sentence Grounding in Video

Hao Zhang, Aixin Sun, Wei Jing, Joey Tianyi Zhou

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 13 pages, 6 figures, 11 tables

Will You Ever Become Popular? Learning to Predict Virality of Dance Clips

Jiahao Wang, Yunhong Wang, Nina Weng, Tianrui Chai, Annan Li, Faxi Zhang, Sansi Yu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by TOMM

Visual Keyword Spotting with Attention

K R Prajwal, Liliane Momeni, Triantafyllos Afouras, Andrew Zisserman

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Appears in: British Machine Vision Conference 2021 (BMVC 2021)

GraphTMT: Unsupervised Graph-based Topic Modeling from Video Transcripts

Lukas Stappen, Jason Thies, Gerhard Hagerer, Björn W. Schuller, Georg Groh

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.MM

Comments JT and LS contributed equally to this work

AttendAffectNet: Self-Attention based Networks for Predicting Affective Responses from Movies

Ha Thi Phuong Thao, Balamurali B. T., Dorien Herremans, Gemma Roig

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments 8 pages, 6 figures

Journal ref Proceedings of the International Conference on Pattern Recognition (ICPR2020)

A CLIP-Enhanced Method for Video-Language Understanding

Guohao Li, Feng He, Zhifan Feng

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments 3 pages, 1 figure, 2 tables. Technical report

Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble

Songyao Jiang, Bin Sun, Lichen Wang, Yue Bai, Kunpeng Li, Yun Fu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Grounding Spatio-Temporal Language with Transformers

Tristan Karch, Laetitia Teodorescu, Katja Hofmann, Clément Moulin-Frier, Pierre-Yves Oudeyer

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Contains main article and supplementaries

Journal ref Neurips 2021

Contrastive Video-Language Segmentation

Chen Liang, Yawei Luo, Yu Wu, Yi Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Joint Multimedia Event Extraction from Video and Article

Brian Chen, Xudong Lin, Christopher Thomas, Manling Li, Shoya Yoshida, Lovish Chum, Heng Ji, Shih-Fu Chang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments To be presented at EMNLP 2021 findings

Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding

Daizong Liu, Xiaoye Qu, Pan Zhou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted as a long paper in the main conference of EMNLP 2021

WhyAct: Identifying Action Reasons in Lifestyle Vlogs

Oana Ignat, Santiago Castro, Hanwen Miao, Weiji Li, Rada Mihalcea

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at EMNLP 2021

SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videos

Nada Osman, Guglielmo Camporese, Pasquale Coscia, Lamberto Ballan

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to EPIC@ICCV 2021