多模态大模型

SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environments

Muhammad Zubair Irshad, Niluthpol Chowdhury Mithun, Zachary Seymour, Han-Pang Chiu, Supun Samarasekera, Rakesh Kumar

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 4 figures

Spatio-Temporal Interaction Graph Parsing Networks for Human-Object Interaction Recognition

Ning Wang, Guangming Zhu, Liang Zhang, Peiyi Shen, Hongsheng Li, Cong Hua

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments ACM MM Oral paper

Mounting Video Metadata on Transformer-based Language Model for Open-ended Video Question Answering

Donggeon Lee, Seongho Choi, Youwon Jang, Byoung-Tak Zhang

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 1 figure

Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading

Shahd Elashmawy, Marian Ramsis, Hesham M. Eraqi, Farah Eldeshnawy, Hadeel Mabrouk, Omar Abugabal, Nourhan Sakr

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI

Hybrid Reasoning Network for Video-based Commonsense Captioning

Weijiang Yu, Jian Liang, Lei Ji, Lu Li, Yuejian Fang, Nong Xiao, Nan Duan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 11 pages, 6 figures

Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery

Yonghao Long, Jie Ying Wu, Bo Lu, Yueming Jin, Mathias Unberath, Yun-Hui Liu, Pheng Ann Heng, Qi Dou

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted for ICRA 2021

Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering

Ahjeong Seo, Gi-Cheon Kang, Joonhan Park, Byoung-Tak Zhang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments ACL 2021

PyKale: Knowledge-Aware Machine Learning from Multiple Sources in Python

Haiping Lu, Xianyuan Liu, Robert Turner, Peizhen Bai, Raivo E Koot, Shuo Zhou, Mustafa Chasmai, Lawrence Schobs

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments This library is available at https://github.com/pykale/pykale

Relation-aware Hierarchical Attention Framework for Video Question Answering

Fangtao Li, Ting Bai, Chenyu Cao, Zihe Liu, Chenghao Yan, Bin Wu

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 9 pages, This paper is accepted by ICMR 2021

Video-aided Unsupervised Grammar Induction

Songyang Zhang, Linfeng Song, Lifeng Jin, Kun Xu, Dong Yu, Jiebo Luo

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments This paper is accepted by NAACL'21

Towards Natural Language Question Answering over Earth Observation Linked Data using Attention-based Neural Machine Translation

Abhishek V. Potnis, Rajat C. Shinde, Surya S. Durbha

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted at IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2020

Dual Encoding for Video Retrieval by Text

Jianfeng Dong, Xirong Li, Chaoxi Xu, Xun Yang, Gang Yang, Xun Wang, Meng Wang

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence. Code and data will be available at https://github.com/danieljf24/hybrid_space. Conference version: arXiv:1809.06181

Rethinking movie genre classification with fine-grained semantic clustering

Edward Fish, Jon Weinbren, Andrew Gilbert

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Associated Spatio-Temporal Capsule Network for Gait Recognition

Aite Zhao, Junyu Dong, Jianbo Li, Lin Qi, Huiyu Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Trying Bilinear Pooling in Video-QA

Thomas Winterbottom, Sarah Xiao, Alistair McLean, Noura Al Moubayed

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 16 Pages, 8 Figures, 4 Tables, +Supp Mats

Movie Summarization via Sparse Graph Construction

Pinelopi Papalampidi, Frank Keller, Mirella Lapata

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at AAAI 2021

Deep learning for video game genre classification

Yuhang Jiang, Lukun Zheng

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 21 pages, 6 figures, 3 tables. arXiv admin note: substantial text overlap with arXiv:2011.07658

Language guided machine action

Feng Qi

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 4 figures

Scaling Laws for Autoregressive Generative Modeling

Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B. Brown, Prafulla Dhariwal, Scott Gray, Chris Hallacy, Benjamin Mann, Alec Radford, Aditya Ramesh, Nick Ryder, Daniel M. Ziegler, John Schulman, Dario Amodei, Sam McCandlish

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 20+17 pages, 33 figures; added appendix with additional language results

CNN based Multistage Gated Average Fusion (MGAF) for Human Action Recognition Using Depth and Inertial Sensors

Zeeshan Ahmad, Naimul khan

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments arXiv admin note: text overlap with arXiv:1910.11482

What Gives the Answer Away? Question Answering Bias Analysis on Video QA Datasets

Jianing Yang, Yuying Zhu, Yongxin Wang, Ruitao Yi, Amir Zadeh, Louis-Philippe Morency

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Video-Grounded Dialogues with Pretrained Generation Language Models

Hung Le, Steven C. H. Hoi

专题命中 视频多模态 ：audio-visual(abstract)；分类 cs.CV、cs.CL

Comments Accepted at ACL 2020 (Short Paper)

Journal ref Association for Computational Linguistics (2020) 5842-5848

Span-based Localizing Network for Natural Language Video Localization

Hao Zhang, Aixin Sun, Wei Jing, Joey Tianyi Zhou

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments To appear at ACL 2020

Retrieving and Highlighting Action with Spatiotemporal Reference

Seito Kasai, Yuchi Ishikawa, Masaki Hayashi, Yoshimitsu Aoki, Kensho Hara, Hirokatsu Kataoka

专题命中 视频多模态 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ICIP 2020

Video Storytelling: Textual Summaries for Events

Junnan Li, Yongkang Wong, Qi Zhao, Mohan S. Kankanhalli

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Published in IEEE Transactions on Multimedia

Journal ref J. Li, Y. Wong, Q. Zhao and M. S. Kankanhalli, "Video Storytelling: Textual Summaries for Events," in IEEE Transactions on Multimedia, 2019

Video Object Grounding using Semantic Roles in Language Description

Arka Sadhu, Kan Chen, Ram Nevatia

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments CVPR20 camera-ready including appendix

Zero-Shot Activity Recognition with Videos

Evin Pinar Ornek

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments This is a research report done during master's studies

Language2Pose: Natural Language Grounded Pose Forecasting

Chaitanya Ahuja, Louis-Philippe Morency

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Seeing and Hearing Egocentric Actions: How Much Can We Learn?

Alejandro Cartas, Jordi Luque, Petia Radeva, Carlos Segura, Mariella Dimiccoli

专题命中 视频多模态 ：multimodal(abstract)；分类 cs.CV、eess.AS

Comments Accepted for the Fifth International Workshop on Egocentric Perception, Interaction and Computing (EPIC) at the International Conference on Computer Vision (ICCV) 2019

Posture and sequence recognition for Bharatanatyam dance performances using machine learning approach

Tanwi Mallick, Partha Pratim Das, Arun Kumar Majumdar

专题命中 视频多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM