多模态大模型

MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformer

Yifang Xu, Yunzhuo Sun, Yang Li, Yilei Shi, Xiaoxiang Zhu, Sidan Du

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Deep Learning Based Multimodal with Two-phase Training Strategy for Daily Life Video Classification

Lam Pham, Trang Le, Cam Le, Dat Ngo, Weissenfeld Axel, Alexander Schindler

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.MM、eess.AS

Robust Cross-Modal Knowledge Distillation for Unconstrained Videos

Wenke Xia, Xingjian Li, Andong Deng, Haoyi Xiong, Dejing Dou, Di Hu

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning

Peng Jin, Jinfa Huang, Pengfei Xiong, Shangxuan Tian, Chang Liu, Xiangyang Ji, Li Yuan, Jie Chen

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments CVPR 2023 Highlight

MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action

Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, Lijuan Wang

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

ParkPredict+: Multimodal Intent and Motion Prediction for Vehicles in Parking Lots with CNN and Transformer

Xu Shen, Matthew Lacayo, Nidhir Guggilla, Francesco Borrelli

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Published at IEEE ITSC 2022

TAMFormer: Multi-Modal Transformer with Learned Attention Mask for Early Intent Prediction

Nada Osman, Guglielmo Camporese, Lamberto Ballan

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Progressive Video Summarization via Multimodal Self-supervised Learning

Li Haopeng, Ke Qiuhong, Gong Mingming, Tom Drummond

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Multi-modal Video Chapter Generation

Xiao Cao, Zitan Chen, Canyu Le, Lei Meng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering

Jiong Wang, Zhou Zhao, Weike Jin

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

A Multi-Modal Wildfire Prediction and Personalized Early-Warning System Based on a Novel Machine Learning Framework

Rohan Tan Bhowmik

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency

Medhini Narasimhan, Arsha Nagrani, Chen Sun, Michael Rubinstein, Trevor Darrell, Anna Rohrbach, Cordelia Schmid

专题命中 视频多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted to ECCV 2022. Website: https://medhini.github.io/ivsum/

Egocentric Scene Understanding via Multimodal Spatial Rectifier

Tien Do, Khiem Vuong, Hyun Soo Park

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Appearing in the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation

Wangbo Zhao, Kai Wang, Xiangxiang Chu, Fuzhao Xue, Xinchao Wang, Yang You

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR2022

End-to-End Referring Video Object Segmentation with Multimodal Transformers

Adam Botach, Evgenii Zheltonozhskii, Chaim Baskin

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2022

M-SENA: An Integrated Platform for Multimodal Sentiment Analysis

Huisheng Mao, Ziqi Yuan, Hua Xu, Wenmeng Yu, Yihe Liu, Kai Gao

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.AI、cs.MM

Comments 11 pages, 4 figures, to be published in ACL 2022 System Demonstration Track

Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioning

Ligong Han, Jian Ren, Hsin-Ying Lee, Francesco Barbieri, Kyle Olszewski, Shervin Minaee, Dimitris Metaxas, Sergey Tulyakov

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to CVPR 2022

Predicting the Popularity of Micro-videos with Multimodal Variational Encoder-Decoder Framework

Yaochen Zhu, Jiayi Xie, Zhenzhong Chen

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Unsupervised Multimodal Language Representations using Convolutional Autoencoders

Panagiotis Koromilas, Theodoros Giannakopoulos

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments 5 pages

Fine-grained Multi-Modal Self-Supervised Learning

Duo Wang, Salah Karout

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted at BMVC 2021

Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition

Jianyuan Ni, Raunak Sarbajna, Yang Liu, Anne H. H. Ngu, Yan Yan

专题命中 视频多模态 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.MM

Comments 5 pages, 2 figures, submitted to ICASSP2022

With a Little Help from my Temporal Context: Multimodal Egocentric Action Recognition

Evangelos Kazakos, Jaesung Huh, Arsha Nagrani, Andrew Zisserman, Dima Damen

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted at BMVC 2021

MERLOT: Multimodal Neural Script Knowledge Models

Rowan Zellers, Ximing Lu, Jack Hessel, Youngjae Yu, Jae Sung Park, Jize Cao, Ali Farhadi, Yejin Choi

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments project page at https://rowanzellers.com/merlot; NeurIPS 2021 camera ready

Don't Judge Me by My Face : An Indirect Adversarial Approach to Remove Sensitive Information From Multimodal Neural Representation in Asynchronous Job Video Interviews

Léo Hemamou, Arthur Guillon, Jean-Claude Martin, Chloé Clavel

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments published in ACII 2021

Multi-modal Representation Learning for Video Advertisement Content Structuring

Daya Guo, Zhaoyang Zeng

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

The Multi-Modal Video Reasoning and Analyzing Competition

Haoran Peng, He Huang, Li Xu, Tianjiao Li, Jun Liu, Hossein Rahmani, Qiuhong Ke, Zhicheng Guo, Cong Wu, Rongchang Li, Mang Ye, Jiahao Wang, Jiaxu Zhang, Yuanzhong Liu, Tao He, Fuwei Zhang, Xianbin Liu, Tao Lin

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ICCV 2021 Workshops

HetEmotionNet: Two-Stream Heterogeneous Graph Recurrent Neural Network for Multi-modal Emotion Recognition

Ziyu Jia, Youfang Lin, Jing Wang, Zhiyang Feng, Xiangheng Xie, Caijie Chen

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.AI、cs.MM

Comments Accepted by ACM MM 2021. The SOLE copyright holder is ACM Multimedia, all rights reserved

Toward Automated Classroom Observation: Multimodal Machine Learning to Estimate CLASS Positive Climate and Negative Climate

Anand Ramakrishnan, Brian Zylich, Erin Ottmar, Jennifer LoCasale-Crouch, Jacob Whitehill

专题命中 视频多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、eess.AS

Comments The authors discovered that the results are not reproducible

Journal ref IEEE Transactions on Affective Computing, 2021

AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition

Rameswar Panda, Chun-Fu Chen, Quanfu Fan, Ximeng Sun, Kate Saenko, Aude Oliva, Rogerio Feris

专题命中 视频多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

A multimodal movie review corpus for fine-grained opinion mining

Alexandre Garcia, Slim Essid, Florence d'Alché-Buc, Chloé Clavel

专题命中 视频多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM