多模态大模型

X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfer

Linglin Jing, Ying Xue, Xu Yan, Chaoda Zheng, Dong Wang, Ruimao Zhang, Zhigang Wang, Hui Fang, Bin Zhao, Zhen Li

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

A multi-modal table tennis robot system

Andreas Ziegler, Thomas Gossard, Karl Vetter, Jonas Tebbe, Andreas Zell

专题命中 视频多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Comments Accepted for RoboLetics: Workshop on Robot Learning in Athletics @CoRL 2023

Modality Mixer Exploiting Complementary Information for Multi-modal Action Recognition

Sumin Lee, Sangmin Woo, Muhammad Adi Nugroho, Changick Kim

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2208.11314

MuST: Multimodal Spatiotemporal Graph-Transformer for Hospital Readmission Prediction

Yan Miao, Lequan Yu

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Video-Helpful Multimodal Machine Translation

Yihang Li, Shuichiro Shimizu, Chenhui Chu, Sadao Kurohashi, Wei Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by EMNLP 2023 Main Conference (long paper)

$\mathbb{VD}$-$\mathbb{GR}$: Boosting $\mathbb{V}$isual $\mathbb{D}$ialog with Cascaded Spatial-Temporal Multi-Modal $\mathbb{GR}$aphs

Adnen Abdessaied, Lei Shi, Andreas Bulling

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments WACV 2024

Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detection

Linyan Huang, Zhiqi Li, Chonghao Sima, Wenhai Wang, Jingdong Wang, Yu Qiao, Hongyang Li

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2023

MST-GAT: A Multimodal Spatial-Temporal Graph Attention Network for Time Series Anomaly Detection

Chaoyue Ding, Shiliang Sun, Jing Zhao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments Information Fusion 2023 accepted

MMTSA: Multimodal Temporal Segment Attention Network for Efficient Human Activity Recognition

Ziqi Gao, Yuntao Wang, Jianguo Chen, Junliang Xing, Shwetak Patel, Xin Liu, Yuanchun Shi

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling

Haogeng Liu, Qihang Fan, Tingkai Liu, Linjie Yang, Yunzhe Tao, Huaibo Huang, Ran He, Hongxia Yang

专题命中 视频多模态 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV

MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer

Fudong Lin, Summer Crawford, Kaleb Guillot, Yihe Zhang, Yan Chen, Xu Yuan, Li Chen, Shelby Williams, Robert Minvielle, Xiangming Xiao, Drew Gholson, Nicolas Ashwell, Tri Setiyono, Brenda Tubana, Lu Peng, Magdy Bayoumi, Nian-Feng Tzeng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref ICCV 2023

SeisCLIP: A seismology foundation model pre-trained by multi-modal data for multi-purpose seismic feature extraction

Xu Si, Xinming Wu, Hanlin Sheng, Jun Zhu, Zefeng Li

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments 27 pages, 9 figures, 4 tables

Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers

Zhiyu Zhu, Junhui Hou, Dapeng Oliver Wu

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by ICCV

Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor

Xinyang Liu, Yijin Li, Yanbin Teng, Hujun Bao, Guofeng Zhang, Yinda Zhang, Zhaopeng Cui

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to ICCV 2023 (Oral). Project Page: https://zju3dv.github.io/tof_slam/

Multimodal Motion Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection

Alessandro Flaborea, Luca Collorone, Guido D'Amely, Stefano D'Arrigo, Bardh Prenkaj, Fabio Galasso

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at ICCV2023

MEGA: Multimodal Alignment Aggregation and Distillation For Cinematic Video Segmentation

Najmeh Sadoughi, Xinyu Li, Avijit Vajpayee, David Fan, Bing Shuai, Hector Santos-Villalobos, Vimal Bhat, Rohith MV

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments ICCV 2023 accepted

Cross-Modal Learning with 3D Deformable Attention for Action Recognition

Sangwon Kim, Dasom Ahn, Byoung Chul Ko

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by ICCV2023

Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration

Juan Del Aguila Ferrandis, João Moura, Sethu Vijayakumar

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Cross-modal Manifold Cutmix for Self-supervised Video Representation Learning

Srijan Das, Michael S. Ryoo

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at MVA 2023

Estudio de la Experiencia de Usuario mediante un Sistema de Dashboards de Análisis de Aprendizaje Multimodal

Álvaro Becerra, Roberto Daza, Ruth Cobos, Aythami Morales, Julian Fierrez

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.MM

Comments Accepted in "XXIII CONGRESO INTERNACIONAL DE INTERACCIÓN PERSONA-ORDENADOR 2023". Article in Spanish language. The abstract in English and Spanish. There is an extended abstract of 2 pages in English

MUVF-YOLOX: A Multi-modal Ultrasound Video Fusion Network for Renal Tumor Diagnosis

Junyu Li, Han Huang, Dong Ni, Wufeng Xue, Dongmei Zhu, Jun Cheng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments MICCAI 2023

Cross-modal Place Recognition in Image Databases using Event-based Sensors

Xiang Ji, Jiaxin Wei, Yifu Wang, Huiliang Shang, Laurent Kneip

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

S-Omninet: Structured Data Enhanced Universal Multimodal Learning Architecture

Ye Xue, Diego Klabjan, Jean Utke

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

ContentCTR: Frame-level Live Streaming Click-Through Rate Prediction with Multimodal Transformer

Jiaxin Deng, Dong Shen, Shiyao Wang, Xiangyu Wu, Fan Yang, Guorui Zhou, Gaofeng Meng

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

Improving Medical Predictions by Irregular Multimodal Electronic Health Records Modeling

Xinlu Zhang, Shiyang Li, Zhiyu Chen, Xifeng Yan, Linda Petzold

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Interpretable Multimodal Emotion Recognition using Facial Features and Physiological Signals

Puneet Kumar, Xiaobai Li

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted for Oral Presentation in DAI 2023 (https://rbcdsai.iitm.ac.in/DAI-2023/program.html)

Multi-Modal Deep Learning for Multi-Temporal Urban Mapping With a Partly Missing Optical Modality

Sebastian Hafner, Yifang Ban

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments 4 pages, 2 figures, accepted for publication in the IGARSS 2023 Proceedings

A Multi-Modal Transformer Network for Action Detection

Matthew Korban, Scott T. Acton, Peter Youngs

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref Pattern Recognition 2023

MHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization

Wujiang Xu, Runzhong Wang, Xiaobo Guo, Shaoshuai Li, Qiongxu Ma, Yunan Zhao, Sheng Guo, Zhenfeng Zhu, Junchi Yan

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV

CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation

Yunyao Mao, Wengang Zhou, Zhenbo Lu, Jiajun Deng, Houqiang Li

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments To appear in ECCV 2022 (Oral)