多模态大模型

Kangaroo: A Powerful Video-Language Model Supporting Long-context Video Input

Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, Jie Hu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baseline

Qi Jia, Baoyu Fan, Cong Xu, Lu Liu, Liang Jin, Guoguang Du, Zhenhua Guo, Yaqian Zhao, Xuanjing Huang, Rengang Li

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Towards Event-oriented Long Video Understanding

Yifan Du, Kun Zhou, Yuqi Huo, Yifan Li, Wayne Xin Zhao, Haoyu Lu, Zijia Zhao, Bingning Wang, Weipeng Chen, Ji-Rong Wen

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments Work on progress

VideoVista: A Versatile Benchmark for Video Understanding and Reasoning

Yunxin Li, Xinyu Chen, Baotian Hu, Longyue Wang, Haoyuan Shi, Min Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 38 pages, 44 figures

TrafficVLM: A Controllable Visual Language Model for Traffic Video Captioning

Quang Minh Dinh, Minh Khoi Ho, Anh Quan Dang, Hung Phong Tran

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 7134-7143

An Empirical Study of Excitation and Aggregation Design Adaptions in CLIP4Clip for Video-Text Retrieval

Xiaolun Jing, Genke Yang, Jian Chu

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 20 pages

Vision Mamba: A Comprehensive Survey and Taxonomy

Xiao Liu, Chenxu Zhang, Lei Zhang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments https://github.com/lx6c78/Vision-Mamba-A-Comprehensive-Survey-and-Taxonomy

Spatio-Temporal Side Tuning Pre-trained Foundation Models for Video-based Pedestrian Attribute Recognition

Xiao Wang, Qian Zhu, Jiandong Jin, Jun Zhu, Futian Wang, Bo Jiang, Yaowei Wang, Yonghong Tian

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Parameter Efficient Fine-Tuning Strategy for Video-based Pedestrian Attribute Recognition

Can I Trust Your Answer? Visually Grounded Video Question Answering

Junbin Xiao, Angela Yao, Yicong Li, Tat Seng Chua

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to CVPR'24. (Compared with preprint version, we mainly improve the presentation, discuss more related works, and extend experiments in Appendix.)

HawkEye: Training Video-Text LLMs for Grounding Text in Videos

Yueqian Wang, Xiaojun Meng, Jianxin Liang, Yuxuan Wang, Qun Liu, Dongyan Zhao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

MuLTI: Efficient Video-and-Language Understanding with Text-Guided MultiWay-Sampler and Multiple Choice Modeling

Jiaqi Xu, Bo Liu, Yunkuo Chen, Mengli Cheng, Xing Shi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.MM

Change Detection Between Optical Remote Sensing Imagery and Map Data via Segment Anything Model (SAM)

Hongruixuan Chen, Jian Song, Naoto Yokoya

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Panel Transitions for Genre Analysis in Visual Narratives

Yi-Chun Chen, Arnav Jhala

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation

Shilin Yan, Renrui Zhang, Ziyu Guo, Wenchao Chen, Wei Zhang, Hongyang Li, Yu Qiao, Hao Dong, Zhongjiang He, Peng Gao

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by AAAI 2024. Code is released at https://github.com/OpenGVLab/MUTR

Modular Blended Attention Network for Video Question Answering

Mingjie Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments I will not add others' names since this work has not been published

Conditional Modeling Based Automatic Video Summarization

Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hung Chen, Marcel Worring

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments This work has been submitted to the IEEE for possible publication. arXiv admin note: substantial text overlap with arXiv:2305.00455

Generative Action Description Prompts for Skeleton-based Action Recognition

Wangmeng Xiang, Chao Li, Yuxuan Zhou, Biao Wang, Lei Zhang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by ICCV23

Exploiting Spatial-temporal Data for Sleep Stage Classification via Hypergraph Learning

Yuze Liu, Ziming Zhao, Tiehua Zhang, Kang Wang, Xin Chen, Xiaowei Huang, Jun Yin, Zhishu Shen

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)

Improving Social Media Popularity Prediction with Multiple Post Dependencies

Zhizhen Zhang, Xiaohui Xie, Mengyu Yang, Ye Tian, Yong Jiang, Yong Cui

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI、cs.MM

Verbs in Action: Improving verb understanding in video-language models

Liliane Momeni, Mathilde Caron, Arsha Nagrani, Andrew Zisserman, Cordelia Schmid

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning

Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, Cordelia Schmid

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2023 Camera-Ready; Project Webpage: https://antoyang.github.io/vid2seq.html ; 18 pages; 6 figures

COVID-VTS: Fact Extraction and Verification on Short Video Platforms

Fuxiao Liu, Yaser Yacoob, Abhinav Shrivastava

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments 11 pages, 5 figures, accepted to EACL2023

LGDN: Language-Guided Denoising Network for Video-Language Modeling

Haoyu Lu, Mingyu Ding, Nanyi Fei, Yuqi Huo, Zhiwu Lu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by NeurIPS2022

Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition

Bulat Khaertdinov, Stylianos Asteriadis

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted to IJCB 2022

Extending Compositional Attention Networks for Social Reasoning in Videos

Christina Sartzetaki, Georgios Paraskevopoulos, Alexandros Potamianos

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref Proc. Interspeech 2022, 1116-1120

What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data

Oier Mees, Lukas Hermann, Wolfram Burgard

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted for publication at IEEE Robotics and Automation Letters (RAL). Codebase and trained models available at http://hulc.cs.uni-freiburg.de

Learning to Retrieve Videos by Asking Questions

Avinash Madasu, Junier Oliva, Gedas Bertasius

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Journal ref ACM Multimedia 2022

Revisiting the "Video" in Video-Language Understanding

Shyamal Buch, Cristóbal Eyzaguirre, Adrien Gaidon, Jiajun Wu, Li Fei-Fei, Juan Carlos Niebles

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2022 (Oral)

Video as Conditional Graph Hierarchy for Multi-Granular Question Answering

Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, Tat-Seng Chua

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments AAAI'22 (Oral)

Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval

Jianfeng Dong, Yabing Wang, Xianke Chen, Xiaoye Qu, Xirong Li, Yuan He, Xun Wang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI、cs.MM

Comments Accepted by IEEE Transactions on Circuits and Systems for Video Technology. Code is available at https://github.com/LiJiaBei-7/rivrl