多模态大模型

CinePile: A Long Video Question Answering Dataset and Benchmark

Ruchit Rawal, Khalid Saifullah, Miquel Farré, Ronen Basri, David Jacobs, Gowthami Somepalli, Tom Goldstein

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Project page with all the artifacts - https://ruchitrawal.github.io/cinepile/. Updated version with adversarial refinement pipeline and more model evaluations

Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligence

Andres Karjus

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Generating Signed Language Instructions in Large-Scale Dialogue Systems

Mert İnan, Katherine Atwell, Anthony Sicilia, Lorna Quandt, Malihe Alikhani

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2024) Industry Track

SatSwinMAE: Efficient Autoencoding for Multiscale Time-series Satellite Imagery

Yohei Nakayama, Jiawei Su, Luis M. Pazos-Outón

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Sparse Prototype Network for Explainable Pedestrian Behavior Prediction

Yan Feng, Alexander Carballo, Kazuya Takeda

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Free Video-LLM: Prompt-guided Visual Perception for Efficient Training-free Video LLMs

Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, Yunhe Wang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Tech report

InterACT: Inter-dependency Aware Action Chunking with Hierarchical Attention Transformers for Bimanual Manipulation

Andrew Lee, Ian Chuang, Ling-Yuan Chen, Iman Soltani

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at Conference on Robot Learning (CoRL) 2024

VidCompress: Memory-Enhanced Temporal Compression for Video Understanding in Large Language Models

Xiaohan Lan, Yitian Yuan, Zequn Jie, Lin Ma

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 9 pages, 4 figures

The Solution for Temporal Action Localisation Task of Perception Test Challenge 2024

Yinan Han, Qingyuan Jiang, Hongming Mei, Yang Yang, Jinhui Tang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning

Thong Nguyen, Yi Bin, Xiaobao Wu, Xinshuai Dong, Zhiyuan Hu, Khoi Le, Cong-Duy Nguyen, See-Kiong Ng, Luu Anh Tuan

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ECCV 2024

Temporal Reasoning Transfer from Text to Video

Lei Li, Yuanxin Liu, Linli Yao, Peiyuan Zhang, Chenxin An, Lean Wang, Xu Sun, Lingpeng Kong, Qi Liu

专题命中 视频多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://video-t3.github.io

RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation

Hanxiao Jiang, Binghao Huang, Ruihai Wu, Zhuoran Li, Shubham Garg, Hooshang Nayyeri, Shenlong Wang, Yunzhu Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://jianghanxiao.github.io/roboexp-web/

AIM 2024 Challenge on Video Saliency Prediction: Methods and Results

Andrey Moskalenko, Alexey Bryncev, Dmitry Vatolin, Radu Timofte, Gen Zhan, Li Yang, Yunlong Tang, Yiting Liao, Jiongzhi Lin, Baitao Huang, Morteza Moradi, Mohammad Moradi, Francesco Rundo, Concetto Spampinato, Ali Borji, Simone Palazzo, Yuxin Zhu, Yinan Sun, Huiyu Duan, Yuqin Cao, Ziheng Jia, Qiang Hu, Xiongkuo Min, Guangtao Zhai, Hao Fang, Runmin Cong, Xiankai Lu, Xiaofei Zhou, Wei Zhang, Chunyu Zhao, Wentao Mu, Tao Deng, Hamed R. Tavakoli

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.MM

Comments ECCVW 2024

ViT LoS V2X: Vision Transformers for Environment-aware LoS Blockage Prediction for 6G Vehicular Networks

Ghazi Gharsallah, Georges Kaddoum

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Action-Based ADHD Diagnosis in Video

Yichun Li, Yuxing Yang, Syed Nohsen Naqvi

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 31st European Symposium on Artificial Neural Networks

RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation

Yonglin Li, Jing Zhang, Xiao Teng, Long Lan, Xinwang Liu

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Beyond Uncertainty: Evidential Deep Learning for Robust Video Temporal Grounding

Kaijing Ma, Haojian Huang, Jin Chen, Haodong Chen, Pengliang Ji, Xianghao Zang, Han Fang, Chao Ban, Hao Sun, Mulin Chen, Xuelong Li

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Ongoing work: 28pages, 19 figures, 7 tables. Code is available at: https://kaijing.space/SRAM/

On the Efficacy of Text-Based Input Modalities for Action Anticipation

Apoorva Beedu, Harish Haresamudram, Karan Samel, Irfan Essa

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopses

Chaolei Tan, Zihang Lin, Junfu Pu, Zhongang Qi, Wei-Yi Pei, Zhi Qu, Yexin Wang, Ying Shan, Wei-Shi Zheng, Jian-Fang Hu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM MM 2024. Project page: https://synopground.github.io/

DeVAn: Dense Video Annotation for Video-Language Models

Tingkai Liu, Yunzhe Tao, Haogeng Liu, Qihang Fan, Ding Zhou, Huaibo Huang, Ran He, Hongxia Yang

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Published in 62nd ACL (2024) Main Track

Edit As You Wish: Video Caption Editing with Multi-grained User Control

Linli Yao, Yuanmeng Zhang, Ziheng Wang, Xinglin Hou, Tiezheng Ge, Yuning Jiang, Xu Sun, Qin Jin

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by ACM MM 2024

Comparing Optical Flow and Deep Learning to Enable Computationally Efficient Traffic Event Detection with Space-Filling Curves

Tayssir Bouraffa, Elias Kjellberg Carlson, Erik Wessman, Ali Nouri, Pierre Lamart, Christian Berger

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 27th IEEE International Conference on Intelligent Transportation Systems (IEEE ITSC 2024)

LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding

Haoning Wu, Dongxu Li, Bei Chen, Junnan Li

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 29 pages

Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMs

Jinmin Li, Kuofeng Gao, Yang Bai, Jingyun Zhang, Shu-Tao Xia

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments arXiv admin note: substantial text overlap with arXiv:2403.13507

Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment

Hao Fei, Shengqiong Wu, Meishan Zhang, Min Zhang, Tat-Seng Chua, Shuicheng Yan

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by IEEE TPAMI 2024

Journal ref [J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

Unified and Dynamic Graph for Temporal Character Grouping in Long Videos

Xiujun Shu, Wei Wen, Liangsheng Xu, Ruizhi Qiao, Taian Guo, Hanjun Li, Bei Gan, Xiao Wang, Xing Sun

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Towards Holistic Language-video Representation: the language model-enhanced MSR-Video to Text Dataset

Yuchen Yang, Yingxuan Duan

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies

Hung-Ting Su, Chun-Tong Chao, Ya-Ching Hsu, Xudong Lin, Yulei Niu, Hung-Yi Lee, Winston H. Hsu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://ander1119.github.io/TiM

A Survey of Video Datasets for Grounded Event Understanding

Kate Sanders, Benjamin Van Durme

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models

Qiang Sun, Yuanyi Luo, Wenxiao Zhang, Sirui Li, Jichunyang Li, Kai Niu, Xiangrui Kong, Wei Liu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI