多模态大模型

Do You Remember? Dense Video Captioning with Cross-Modal Memory Retrieval

Minkuk Kim, Hyeon Bae Kim, Jinyoung Moon, Jinwoo Choi, Seong Tae Kim

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments CVPR 2024

Tensor-based Multimodal Learning for Prediction of Pulmonary Arterial Wedge Pressure from Cardiac MRI

Prasun C. Tripathi, Mohammod N. I. Suvon, Lawrence Schobs, Shuo Zhou, Samer Alabed, Andrew J. Swift, Haiping Lu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities

AJ Piergiovanni, Isaac Noble, Dahun Kim, Michael S. Ryoo, Victor Gomes, Anelia Angelova

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments CVPR 2024

X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalization

Anna Kukleva, Fadime Sener, Edoardo Remelli, Bugra Tekin, Eric Sauser, Bernt Schiele, Shugao Ma

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments CVPR 2024

A survey on knowledge-enhanced multimodal learning

Maria Lymperaiou, Giorgos Stamou

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model

Zheng Zhang, Yeyao Ma, Enming Zhang, Xiang Bai

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

EventDance: Unsupervised Source-free Cross-modal Adaptation for Event-based Object Recognition

Xu Zheng, Lin Wang

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted to CVPR2024

FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs

Jinmin Li, Kuofeng Gao, Yang Bai, Jingyun Zhang, Shu-tao Xia, Yisen Wang

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Multi-modal learning for geospatial vegetation forecasting

Vitus Benson, Claire Robin, Christian Requena-Mesa, Lazaro Alonso, Nuno Carvalhais, José Cortés, Zhihan Gao, Nora Linscheid, Mélanie Weynants, Markus Reichstein

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at CVPR 2024. We provide open source code and pre-trained weights to reproduce our experimental results under https://github.com/vitusbenson/greenearthnet

EtC: Temporal Boundary Expand then Clarify for Weakly Supervised Video Grounding with Multimodal Large Language Model

Guozhang Li, Xinpeng Ding, De Cheng, Jie Li, Nannan Wang, Xinbo Gao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

An Empirical Evaluation of Neural and Neuro-symbolic Approaches to Real-time Multimodal Complex Event Detection

Liying Han, Mani B. Srivastava

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Exploring the dynamic interplay of cognitive load and emotional arousal by using multimodal measurements: Correlation of pupil diameter and emotional arousal in emotionally engaging tasks

C. Kosel, S. Michel, T. Seidel, M. Foerster

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments The first two authors contributed equally to the manuscript

Global Tropical Cyclone Intensity Forecasting with Multi-modal Multi-scale Causal Autoregressive Model

Xinyu Wang, Kang Chen, Lei Liu, Tao Han, Bin Li, Lei Bai

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog

Adnen Abdessaied, Manuel von Hochmeister, Andreas Bulling

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments COLING 2024

MORE-3S:Multimodal-based Offline Reinforcement Learning with Shared Semantic Spaces

Tianyu Zheng, Ge Zhang, Xingwei Qu, Ming Kuang, Stephen W. Huang, Zhaofeng He

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generation

Hongcheng Liu, Pingjie Wang, Yu Wang, Yanfeng Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Monitoring of Urban Changes with multi-modal Sentinel 1 and 2 Data in Mariupol, Ukraine, in 2022/23

Georg Zitzlsberger, Michal Podhoranyi

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted for publication in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

Multi-Modal Machine Learning Framework for Automated Seizure Detection in Laboratory Rats

Aaron Mullen, Samuel E. Armstrong, Jasmine Perdeh, Bjorn Bauer, Jeffrey Talbert, V. K. Cody Bumgardner

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding

Yunze Liu, Changxi Chen, Zifan Wang, Li Yi

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Journal ref ICRA2024

M2-CLIP: A Multimodal, Multi-task Adapting Framework for Video Action Recognition

Mengmeng Wang, Jiazheng Xing, Boyuan Jiang, Jun Chen, Jianbiao Mei, Xingxing Zuo, Guang Dai, Jingdong Wang, Yong Liu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Journal ref AAAI2024

Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation

Chen Liang, Yu Wu, Tianfei Zhou, Wenguan Wang, Zongxin Yang, Yunchao Wei, Yi Yang

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Champion solution in YouTube-VOS 2021 Track 3. Extended version published in https://ieeexplore.ieee.org/abstract/document/10083244

Multi-view Distillation based on Multi-modal Fusion for Few-shot Action Recognition(CLIP-$\mathrm{M^2}$DF)

Fei Guo, YiKang Wang, Han Qi, WenPing Jin, Li Zhu

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Cross-Modal Semi-Dense 6-DoF Tracking of an Event Camera in Challenging Conditions

Yi-Fan Zuo, Wanting Xu, Xia Wang, Yifu Wang, Laurent Kneip

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by IEEE Transactions on Robotics (T-RO). arXiv admin note: text overlap with arXiv:2202.02556

Self-supervised Event-based Monocular Depth Estimation using Cross-modal Consistency

Junyu Zhu, Lina Liu, Bofeng Jiang, Feng Wen, Hongbo Zhang, Wanlong Li, Yong Liu

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by IROS2023

InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, Conghui He, Ping Luo, Ziwei Liu, Yali Wang, Limin Wang, Yu Qiao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Data and Code: https://github.com/OpenGVLab/InternVideo/tree/main/Data/InternVid

Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models

Xinpeng Ding, Jinahua Han, Hang Xu, Xiaodan Liang, Wei Zhang, Xiaomeng Li

专题命中 视频多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

A graph-based multimodal framework to predict gentrification

Javad Eshtiyagh, Baotong Zhang, Yujing Sun, Linhui Wu, Zhao Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Journal ref International Conference on Urban Informatics 2023 - Best Paper Award 3rd Place

Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding

Haifeng Huang, Yang Zhao, Zehan Wang, Yan Xia, Zhou Zhao

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Parameterized Decision-making with Multi-modal Perception for Autonomous Driving

Yuyang Xia, Shuncheng Liu, Quanlin Yu, Liwei Deng, You Zhang, Han Su, Kai Zheng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments IEEE International Conference on Data Engineering (ICDE2024)

MOISST: Multimodal Optimization of Implicit Scene for SpatioTemporal calibration

Quentin Herau, Nathan Piasco, Moussab Bennehar, Luis Roldão, Dzmitry Tsishkou, Cyrille Migniot, Pascal Vasseur, Cédric Demonceaux

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at IROS2023 Project site: https://qherau.github.io/MOISST/