多模态大模型

Augmented Commonsense Knowledge for Remote Object Grounding

Bahram Mohammadi, Yicong Hong, Yuankai Qi, Qi Wu, Shirui Pan, Javen Qinfeng Shi

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Video sentence grounding with temporally global textual knowledge

Cai Chen, Runzhong Zhang, Jianjun Gao, Kejun Wu, Kim-Hui Yap, Yi Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Context-Enhanced Video Moment Retrieval with Large Language Models

Weijia Liu, Bo Miao, Jiuxin Cao, Xuelin Zhu, Bo Liu, Mehwish Nasim, Ajmal Mian

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

MADRL-Based Rate Adaptation for 360° Video Streaming with Multi-Viewpoint Prediction

Haopeng Wang, Zijian Long, Haiwei Dong, Abdulmotaleb El Saddik

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.AI、cs.MM

Comments Accepted by IEEE Internet of Things Journal

Mamba-FETrack: Frame-Event Tracking via State Space Model

Ju Huang, Shiao Wang, Shuai Wang, Zhe Wu, Xiao Wang, Bo Jiang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments In Peer Review

HDBN: A Novel Hybrid Dual-branch Network for Robust Skeleton-based Action Recognition

Jinfu Liu, Baiqiao Yin, Jiaying Lin, Jiajun Wen, Yue Li, Mengyuan Liu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commenting

Fengyi Fu, Shancheng Fang, Weidong Chen, Zhendong Mao

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 27 pages, 10 figures, ACM Transactions on Multimedia Computing, Communications and Applications, 2024

Comment-aided Video-Language Alignment via Contrastive Pre-training for Short-form Video Humor Detection

Yang Liu, Tongfei Shen, Dong Zhang, Qingying Sun, Shoushan Li, Guodong Zhou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICMR 2024

LITA: Language Instructed Temporal-Localization Assistant

De-An Huang, Shijia Liao, Subhashree Radhakrishnan, Hongxu Yin, Pavlo Molchanov, Zhiding Yu, Jan Kautz

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

A Number Sense as an Emergent Property of the Manipulating Brain

Neehar Kondapaneni, Pietro Perona

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, 5 figures, 15 supplemental figures

Journal ref Scientific reports, 14(6858) 2024

VidLA: Video-Language Alignment at Scale

Mamshad Nayeem Rizve, Fan Fei, Jayakrishnan Unnikrishnan, Son Tran, Benjamin Z. Yao, Belinda Zeng, Mubarak Shah, Trishul Chilimbi

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2024

MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators

Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, Wanli Ouyang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 18 pages, 8 figures, accepted by AAAI 2024

WorldGPT: A Sora-Inspired Video AI Agent as Rich World Models from Text and Image Inputs

Deshun Yang, Luhui Hu, Yu Tian, Zihao Li, Chris Kelly, Bang Yang, Cindy Yang, Yuexian Zou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 2 figures, 2 tables

Towards Video Anomaly Retrieval from Video Anomaly Detection: New Benchmarks and Model

Peng Wu, Jing Liu, Xiangteng He, Yuxin Peng, Peng Wang, Yanning Zhang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments This work has been accepted to the IEEE TIP. Copyright has been transferred

LVCHAT: Facilitating Long Video Comprehension

Yu Wang, Zeyuan Zhang, Julian McAuley, Zexue He

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 17 pages; 8 figures

Perceptual Video Quality Assessment: A Survey

Xiongkuo Min, Huiyu Duan, Wei Sun, Yucheng Zhu, Guangtao Zhai

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM

YTCommentQA: Video Question Answerability in Instructional Videos

Saelyne Yang, Sunghyun Park, Yunseok Jang, Moontae Lee

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments AAAI 2024

An Open Software Suite for Event-Based Video

Andrew C. Freeman

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Memory-Inspired Temporal Prompt Interaction for Text-Image Classification

Xinyao Yu, Hao Sun, Ziwei Niu, Rui Qin, Zhenjia Bai, Yen-Wei Chen, Lanfen Lin

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge

Dimitrios Psychogyios, Emanuele Colleoni, Beatrice Van Amsterdam, Chih-Yang Li, Shu-Yu Huang, Yuchong Li, Fucang Jia, Baosheng Zou, Guotai Wang, Yang Liu, Maxence Boels, Jiayu Huo, Rachel Sparks, Prokar Dasgupta, Alejandro Granados, Sebastien Ourselin, Mengya Xu, An Wang, Yanan Wu, Long Bai, Hongliang Ren, Atsushi Yamada, Yuriko Harai, Yuto Ishikawa, Kazuyuki Hayashi, Jente Simoens, Pieter DeBacker, Francesco Cisternino, Gabriele Furnari, Alex Mottrie, Federica Ferraguti, Satoshi Kondo, Satoshi Kasai, Kousuke Hirasawa, Soohee Kim, Seung Hyun Lee, Kyu Eun Lee, Hyoun-Joong Kong, Kui Fu, Chao Li, Shan An, Stefanie Krell, Sebastian Bodenstedt, Nicolas Ayobi, Alejandra Perez, Santiago Rodriguez, Juanita Puentes, Pablo Arbelaez, Omid Mohareri, Danail Stoyanov

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, Wancai Zhang, Zhifeng Li, Wei Liu, Li Yuan

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICLR 2024

Affective Video Content Analysis: Decade Review and New Perspectives

Junxiao Xue, Jie Wang, Xuecheng Wu, Qian Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VKIE: The Application of Key Information Extraction on Video Text

Siyu An, Ye Liu, Haoyuan Peng, Di Yin

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

STAIR: Spatial-Temporal Reasoning with Auditable Intermediate Results for Video Question Answering

Yueqian Wang, Yuxuan Wang, Kai Chen, Dongyan Zhao

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments To appear in AAAI 2024

SkateboardAI: The Coolest Video Action Recognition for Skateboarding

Hanxiao Chen

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments The original first-author work has been accepted and presented by CVPR 2022 WiCV Workshop (This is the long-version paper)

EvPlug: Learn a Plug-and-Play Module for Event and Image Fusion

Jianping Jiang, Xinyu Zhou, Peiqi Duan, Boxin Shi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities

Hammad A. Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Long Chen, Yulei Niu, Xudong Lin, Xuande Feng, Jaywon Koo, Sounak Ray, Shih-Fu Chang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments AAAI 2024

TF-CLIP: Learning Text-free CLIP for Video-based Person Re-Identification

Chenyang Yu, Xuehu Liu, Yingquan Wang, Pingping Zhang, Huchuan Lu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments This work is accepted by AAAI2024

Not all Fake News is Written: A Dataset and Analysis of Misleading Video Headlines

Yoo Yeon Sung, Jordan Boyd-Graber, Naeemul Hassan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2023 Main Paper

PG-Video-LLaVA: Pixel Grounding Large Video-Language Models

Shehan Munasinghe, Rusiru Thushara, Muhammad Maaz, Hanoona Abdul Rasheed, Salman Khan, Mubarak Shah, Fahad Khan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Technical Report