多模态大模型

Navigating Open Set Scenarios for Skeleton-based Action Recognition

Kunyu Peng, Cheng Yin, Junwei Zheng, Ruiping Liu, David Schneider, Jiaming Zhang, Kailun Yang, M. Saquib Sarfraz, Rainer Stiefelhagen, Alina Roitberg

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to AAAI 2024. The benchmark, code, and models will be released at https://github.com/KPeng9510/OS-SAR

SPOT! Revisiting Video-Language Models for Event Understanding

Gengyuan Zhang, Jinhe Bi, Jindong Gu, Yanyu Chen, Volker Tresp

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challenges

Tongtong Yuan, Xuange Zhang, Kun Liu, Bo Liu, Chen Chen, Jian Jin, Zhenzhen Jiao

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

LLVMs4Protest: Harnessing the Power of Large Language and Vision Models for Deciphering Protests in the News

Yongjun Zhang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

FFINet: Future Feedback Interaction Network for Motion Forecasting

Miao Kang, Shengqi Wang, Sanping Zhou, Ke Ye, Jingjing Jiang, Nanning Zheng

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 8 figures, 12 tables

Temporal Shift -- Multi-Objective Loss Function for Improved Anomaly Fall Detection

Stefan Denkovski, Shehroz S. Khan, Alex Mihailidis

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Long Story Short: a Summarize-then-Search Method for Long Video Question Answering

Jiwan Chung, Youngjae Yu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Published in BMVC 2023

Video Referring Expression Comprehension via Transformer with Content-conditioned Query

Ji Jiang, Meng Cao, Tengtao Song, Long Chen, Yi Wang, Yuexian Zou

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACM International Conference on Multimedia Workshop (ACM MM), 2023. arXiv admin note: substantial text overlap with arXiv:2210.02953

Paxion: Patching Action Knowledge in Video-Language Foundation Models

Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, Heng Ji

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2023 spotlight

Information-Theoretic Text Hallucination Reduction for Video-grounded Dialogue

Sunjae Yoon, Eunseop Yoon, Hee Suk Yoon, Junyeong Kim, Chang D. Yoo

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 12 pages, Accepted in EMNLP 2022

Learning to Locate Visual Answer in Video Corpus Using Question

Bin Li, Yixuan Weng, Bin Sun, Shutao Li

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ICASSP 2023

DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population

Ningyu Zhang, Xin Xu, Liankuan Tao, Haiyang Yu, Hongbin Ye, Shuofei Qiao, Xin Xie, Xiang Chen, Zhoubo Li, Lei Li, Xiaozhuan Liang, Yunzhi Yao, Shumin Deng, Peng Wang, Wen Zhang, Zhenru Zhang, Chuanqi Tan, Qiang Chen, Feiyu Xiong, Fei Huang, Guozhou Zheng, Huajun Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted by EMNLP 2022 System Demonstrations and the project website is http://deepke.zjukg.cn/

Pretrained Language Models as Visual Planners for Human Assistance

Dhruvesh Patel, Hamid Eghbalzadeh, Nitin Kamra, Michael Louis Iuzzolino, Unnat Jain, Ruta Desai

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at ICCV 2023

Towards Privacy-Supporting Fall Detection via Deep Unsupervised RGB2Depth Adaptation

Hejun Xiao, Kunyu Peng, Xiangsheng Huang, Alina Roitberg1, Hao Li, Zhaohui Wang, Rainer Stiefelhagen

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Video OWL-ViT: Temporally-consistent open-world localization in video

Georg Heigold, Matthias Minderer, Alexey Gritsenko, Alex Bewley, Daniel Keysers, Mario Lučić, Fisher Yu, Thomas Kipf

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023

Orthogonal Temporal Interpolation for Zero-Shot Video Recognition

Yan Zhu, Junbao Zhuo, Bin Ma, Jiajia Geng, Xiaoming Wei, Xiaolin Wei, Shuhui Wang

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.MM

Journal ref Proceedings of the 31st ACM International Conference on Multimedia (MM '23), October 29-November 3, 2023

ModelScope Text-to-Video Technical Report

Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, Shiwei Zhang

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Technical report. Project page: \url{https://modelscope.cn/models/damo/text-to-video-synthesis/summary}

Training Multimedia Event Extraction With Generated Images and Captions

Zilin Du, Yunxin Li, Xu Guo, Yidan Sun, Boyang Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

SODFormer: Streaming Object Detection with Transformer Using Events and Frames

Dianze Li, Jianing Li, Yonghong Tian

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 18 pages, 15 figures, in IEEE Transactions on Pattern Analysis and Machine Intelligence

ViLP: Knowledge Exploration using Vision, Language, and Pose Embeddings for Video Action Recognition

Soumyabrata Chaudhuri, Saumik Bhattacharya

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 3 figures, 2 Tables

Redundancy-aware Transformer for Video Question Answering

Yicong Li, Xun Yang, An Zhang, Chun Feng, Xiang Wang, Tat-Seng Chua

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACM MM23

SBNet: Segmentation-based Network for Natural Language-based Vehicle Search

Sangrok Lee, Taekang Woo, Sang Hun Lee

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 4 figures, CVPR Workshop Paper

Journal ref 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 4049-4055

Robustness Analysis of Video-Language Models Against Visual and Language Perturbations

Madeline C. Schiappa, Shruti Vyas, Hamid Palangi, Yogesh S. Rawat, Vibhav Vineet

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments NeurIPS 2022 Datasets and Benchmarks Track. This projects webpage is located at https://bit.ly/3CNOly4

Journal ref Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (2022)

MultiVENT: Multilingual Videos of Events with Aligned Natural Text

Kate Sanders, David Etter, Reno Kriz, Benjamin Van Durme

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Causal Video Summarizer for Video Exploration

Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Andrew Brown, Marcel Worring

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments This paper is accepted by IEEE International Conference on Multimedia and Expo (ICME), 2022

A CNN regression model to estimate buildings height maps using Sentinel-1 SAR and Sentinel-2 MSI time series

Ritu Yadav, Andrea Nascetti, Yifang Ban

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

GroundNLQ @ Ego4D Natural Language Queries Challenge 2023

Zhijian Hou, Lei Ji, Difei Gao, Wanjun Zhong, Kun Yan, Chao Li, Wing-Kwong Chan, Chong-Wah Ngo, Nan Duan, Mike Zheng Shou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 5 pages, 2 figures, 4 tables, the champion solution for Ego4D Natural Language Queries Challenge in CVPR 2023

A Solution to CVPR'2023 AQTC Challenge: Video Alignment for Multi-Step Inference

Chao Zhang, Shiwei Wu, Sirui Zhao, Tong Xu, Enhong Chen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 5 pages, 1 figure, technical report for track3 of CVPR 2023 LOVEU challenge

Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting

Xinli Yu, Zheng Chen, Yuan Ling, Shujing Dong, Zongyi Liu, Yanbin Lu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Video-to-Music Recommendation using Temporal Alignment of Segments

Laure Prétet, Gaël Richard, Clément Souchier, Geoffroy Peeters

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Journal ref IEEE Transactions on Multimedia, 18 February 2022