多模态大模型

Hierarchical Banzhaf Interaction for General Video-Language Representation Learning

Peng Jin, Hao Li, Li Yuan, Shuicheng Yan, Jie Chen

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). arXiv admin note: substantial text overlap with arXiv:2303.14369

Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Study

Yulin Fei, Yuhui Gao, Xingyuan Xian, Xiaojin Zhang, Tao Wu, Wei Chen

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by CoLing 2025 (The 31st International Conference on Computational Linguistics)

SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations

Zhaorun Chen, Francesco Pinto, Minzhou Pan, Bo Li

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments 43 pages, 20 figures

SimBase: A Simple Baseline for Temporal Video Grounding

Peijun Bao, Alex C. Kot

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Technical report

ActionAtlas: A VideoQA Benchmark for Domain-specialized Action Recognition

Mohammadreza Salehi, Jae Sung Park, Tanush Yadav, Aditya Kusupati, Ranjay Krishna, Yejin Choi, Hannaneh Hajishirzi, Ali Farhadi

专题命中 视频多模态 ：multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV

Journal ref NeurIPS 2024 Track Datasets and Benchmarks

Spatio-temporal Transformers for Action Unit Classification with Event Cameras

Luca Cultrera, Federico Becattini, Lorenzo Berlincioni, Claudio Ferrari, Alberto Del Bimbo

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Under review at CVIU. arXiv admin note: substantial text overlap with arXiv:2409.10213

LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding

Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed Elhoseiny, Vikas Chandra

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Project page: https://vision-cair.github.io/LongVU

CPFD: Confidence-aware Privileged Feature Distillation for Short Video Classification

Jinghao Shi, Xiang Shen, Kaili Zhao, Xuedong Wang, Vera Wen, Zixuan Wang, Yifan Wu, Zhixin Zhang

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Camera ready for CIKM 2024

Uncertainty-Guided Enhancement on Driving Perception System via Foundation Models

Yunhao Yang, Yuxin Hu, Mao Ye, Zaiwei Zhang, Zhichao Lu, Yi Xu, Ufuk Topcu, Ben Snyder

专题命中 视频多模态 ：multimodal(abstract)；multimodal foundation model(abstract)；分类 cs.CV

T2VIndexer: A Generative Video Indexer for Efficient Text-Video Retrieval

Yili Li, Jing Yu, Keke Gai, Bang Liu, Gang Xiong, Qi Wu

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Infusing Environmental Captions for Long-Form Video Language Grounding

Hyogun Lee, Soyeon Hong, Mujeen Sung, Jinwoo Choi

专题命中 视频多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments 7 pages, 3 figures

Adversarial Robustness in RGB-Skeleton Action Recognition: Leveraging Attention Modality Reweighter

Chao Liu, Xin Liu, Zitong Yu, Yonghong Hou, Huanjing Yue, Jingyu Yang

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted by IJCB 2024

Masked Video and Body-worn IMU Autoencoder for Egocentric Action Recognition

Mingfang Zhang, Yifei Huang, Ruicong Liu, Yoichi Sato

专题命中 视频多模态 ：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

Comments ECCV 2024

MultiIoT: Benchmarking Machine Learning for the Internet of Things

Shentong Mo, Louis-Philippe Morency, Russ Salakhutdinov, Paul Pu Liang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams

Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, Xiaojie Jin

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

3rd Place Solution for MeViS Track in CVPR 2024 PVUW workshop: Motion Expression guided Video Segmentation

Feiyu Pan, Hao Fang, Xiankai Lu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Hybrid-Learning Video Moment Retrieval across Multi-Domain Labels

Weitong Cai, Jiabo Huang, Shaogang Gong

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by BMVC2022

Human-Centered Automation

Carlos Toxtli

专题命中 视频多模态 ：multimodal(abstract)；MLLM(abstract)；分类 cs.AI

Comments 12 pages, 0 figures

Unified Video-Language Pre-training with Synchronized Audio

Shentong Mo, Haofan Wang, Huaxia Li, Xu Tang

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.AI、cs.MM

Pegasus-v1 Technical Report

Raehyuk Jung, Hyojun Go, Jaehyuk Yi, Jiho Jang, Daniel Kim, Jay Suh, Aiden Lee, Cooper Han, Jae Lee, Jeff Kim, Jin-Young Kim, Junwan Kim, Kyle Park, Lucas Lee, Mars Ha, Minjoon Seo, Abraham Jo, Ed Park, Hassan Kianinejad, SJ Kim, Tony Moon, Wade Jeong, Andrei Popescu, Esther Kim, EK Yoon, Genie Heo, Henry Choi, Jenna Kang, Kevin Han, Noah Seo, Sunny Nguyen, Ryan Won, Yeonhoo Park, Anthony Giuliani, Dave Chung, Hans Yoon, James Le, Jenny Ahn, June Lee, Maninder Saini, Meredith Sanders, Soyoung Lee, Sue Kim, Travis Couture

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

Self-Adaptive Sampling for Efficient Video Question-Answering on Image--Text Models

Wei Han, Hui Chen, Min-Yen Kan, Soujanya Poria

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 13 pages, 7 figures, accepted to Findings of NAACL 2024

AUD-TGN: Advancing Action Unit Detection with Temporal Convolution and GPT-2 in Wild Audiovisual Contexts

Jun Yu, Zerui Zhang, Zhihong Wei, Gongpeng Zhao, Zhongpeng Cai, Yongqi Wang, Guochen Xie, Jichao Zhu, Wangyuan Zhu

专题命中 视频多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV

Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow

Hanyu Zhou, Yi Chang, Zhiwei Shi, Luxin Yan

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Unmasked Teacher: Towards Training-Efficient Video Foundation Models

Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, Yu Qiao

专题命中 视频多模态 ：multimodal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments ICCV2023

From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities

Chaochao Lu, Chen Qian, Guodong Zheng, Hongxing Fan, Hongzhi Gao, Jie Zhang, Jing Shao, Jingyi Deng, Jinlan Fu, Kexin Huang, Kunchang Li, Lijun Li, Limin Wang, Lu Sheng, Meiqi Chen, Ming Zhang, Qibing Ren, Sirui Chen, Tao Gui, Wanli Ouyang, Yali Wang, Yan Teng, Yaru Wang, Yi Wang, Yinan He, Yingchun Wang, Yixu Wang, Yongting Zhang, Yu Qiao, Yujiong Shen, Yurong Mou, Yuxi Chen, Zaibin Zhang, Zhelun Shi, Zhenfei Yin, Zhipin Wang

专题命中 视频多模态 ：multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Beyond the Frame: Single and mutilple video summarization method with user-defined length

Vahid Ahmadi Kalkhorani, Qingquan Zhang, Guanqun Song, Ting Zhu

专题命中 视频多模态 ：multimodal(abstract)；audio-visual(abstract)；分类 cs.CV

Temporal Adaptive RGBT Tracking with Modality Prompt

Hongyu Wang, Xiaotao Liu, Yifan Li, Meng Sun, Dian Yuan, Jing Liu

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

MapFormer: Boosting Change Detection by Using Pre-change Information

Maximilian Bernhard, Niklas Strauß, Matthias Schubert

专题命中 视频多模态 ：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments accepted at ICCV 2023

Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognition

Chengyou Jia, Minnan Luo, Xiaojun Chang, Zhuohang Dang, Mingfei Han, Mengmeng Wang, Guang Dai, Sizhe Dang, Jingdong Wang

专题命中 视频多模态 ：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

VTimeLLM: Empower LLM to Grasp Video Moments

Bin Huang, Xin Wang, Hong Chen, Zihan Song, Wenwu Zhu

专题命中 视频多模态 ：cross-modal(abstract)；image-text(abstract)；分类 cs.CV