多模态大模型 - arXivDaily 专题

2103.07659 2021-03-16 cs.CL cs.MM 81%

Targeted aspect based multimodal sentiment analysis:an attention capsule extraction and multi-head fusion network

Jiaqian Wang, Donghong Gu, Chi Yang, Yun Xue, Zhengxin Song, Haoliang Zhao, Luwei Xiao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.MM

2103.00466 2021-03-02 cs.CV cs.CL 81%

NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes

Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 3rd rank DravidianLangTech workshop shared task, EACL-2021, 7 pages

2102.01998 2021-02-04 cs.AI cs.CV cs.IT cs.LG math.IT 81%

Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond

Guang Yang, Qinghao Ye, Jun Xia

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 68 pages, 19 figures, submitted to the Information Fusion journal

2011.04592 2020-11-10 cs.CL cs.CV 81%

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

Ece Takmaz, Sandro Pezzelle, Lisa Beinborn, Raquel Fernández

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)

2010.11685 2020-10-23 cs.CV cs.AI 81%

DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding

Zilong Wang, Mingjie Zhan, Xuebo Liu, Ding Liang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to EMNLP 2020 Findings

2010.08708 2020-10-20 cs.CV cs.CL 81%

Answer-checking in Context: A Multi-modal FullyAttention Network for Visual Question Answering

Hantao Huang, Tao Han, Wei Han, Deep Yap, Cheng-Ming Chiang

专题命中多模态训练与对齐：multi-modal(title)；cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted in ICPR2020

2010.05379 2020-10-13 cs.CL cs.CV cs.LG 81%

MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding

Qinxin Wang, Hao Tan, Sheng Shen, Michael W. Mahoney, Zhewei Yao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

2010.01082 2020-10-05 cs.CL cs.AI 81%

Multi-Modal Open-Domain Dialogue

Kurt Shuster, Eric Michael Smith, Da Ju, Jason Weston

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CL、cs.AI

2009.09926 2020-09-24 cs.CV cs.AI cs.IR cs.LG 81%

Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation

Po Li, Lei Li, Yan Fu, Jun Rong, Yu Zhang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV、cs.AI

2004.11838 2020-04-27 cs.CV cs.CY cs.LG cs.MM 81%

Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response

Ferda Ofli, Firoj Alam, Muhammad Imran

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted in ISCRAM 2020

1911.06258 2020-03-26 cs.CV cs.CL 81%

Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA

Ronghang Hu, Amanpreet Singh, Trevor Darrell, Marcus Rohrbach

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments CVPR 2020

2003.04981 2020-03-12 cs.CL cs.CV cs.LG stat.ML 81%

SAFE: Similarity-Aware Multi-Modal Fake News Detection

Xinyi Zhou, Jindi Wu, Reza Zafarani

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments To be published in The 24th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2020)

1910.04964 2020-01-07 cs.MM cs.CV cs.IR cs.LG 81%

Multi-modal Deep Analysis for Multimedia

Wenwu Zhu, Xin Wang, Hongzhi Li

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments 25 pages, 39 figures, IEEE Transactions on Circuits and Systems for Video Technology

1908.11216 2019-09-11 cs.CL cs.AI cs.IR 81%

From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining

Alexandre Garcia, Pierre Colombo, Slim Essid, Florence d'Alché-Buc, Chloé Clavel

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP) and 9th International Joint Conference on Natural Language Processing (IJCNLP)

1908.06008 2019-08-19 cs.LG cs.AI cs.CL stat.ML 81%

Variational Fusion for Multimodal Sentiment Analysis

Navonil Majumder, Soujanya Poria, Gangeshwar Krishnamurthy, Niyati Chhaya, Rada Mihalcea, Alexander Gelbukh

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

1908.05067 2019-08-15 cs.CL cs.CV 81%

Reactive Multi-Stage Feature Fusion for Multimodal Dialogue Modeling

Yi-Ting Yeh, Tzu-Chuan Lin, Hsiao-Hua Cheng, Yu-Hsuan Deng, Shang-Yu Su, Yun-Nung Chen

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted for a poster session at the DSTC7 workshop at AAAI 2019

1907.03196 2019-07-09 cs.CV eess.AS eess.IV 81%

Multimodal Fusion with Deep Neural Networks for Audio-Video Emotion Recognition

Juan D. S. Ortega, Mohammed Senoussaoui, Eric Granger, Marco Pedersoli, Patrick Cardinal, Alessandro L. Koerich

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、eess.AS

1906.00378 2019-06-04 cs.CL cs.CV 81%

Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data

Shizhe Chen, Qin Jin, Alexander Hauptmann

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by AAAI 2019

1811.02627 2018-11-08 cs.CV cs.AI 81%

Vehicle Tracking Using Surveillance with Multimodal Data Fusion

Yue Zhang, Bin Song, Xiaojiang Du, Mohsen Guizani

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 8 pages,6 figures,33 conferences

1810.10565 2018-10-26 cs.CV cs.AI cs.HC 81%

Multimodal Polynomial Fusion for Detecting Driver Distraction

Yulun Du, Chirag Raman, Alan W Black, Louis-Philippe Morency, Maxine Eskenazi

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments INTERSPEECH 2018

1710.09779 2018-10-15 cs.CV cs.AI cs.LG q-bio.QM q-bio.TO 81%

Deep Multi-Modal Classification of Intraductal Papillary Mucinous Neoplasms (IPMN) with Canonical Correlation Analysis

Sarfaraz Hussein, Pujan Kandel, Juan E. Corral, Candice W. Bolan, Michael B. Wallace, Ulas Bagci

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication in IEEE International Symposium on Biomedical Imaging (ISBI) 2018

1809.06225 2018-09-18 cs.CV cs.AI 81%

Investigation of Multimodal Features, Classifiers and Fusion Methods for Emotion Recognition

Zheng Lian, Ya Li, Jianhua Tao, Jian Huang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 11 figures and 4 Tables. EmotiW2018 challenge

1808.04456 2018-09-17 cs.LG cs.AI cs.CV stat.ML 81%

Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction

Garrett B. Goh, Khushmeen Sakloth, Charles Siegel, Abhinav Vishnu, Jim Pfaendtner

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Submitted to a peer-reviewed ML conference

1806.06228 2018-06-19 cs.CL cs.CV 81%

Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling

N. Majumder, D. Hazarika, A. Gelbukh, E. Cambria, S. Poria

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted for publication at Knowledge Based Systems

1805.11264 2018-05-30 stat.ML cs.CL cs.LG cs.SD eess.AS 81%

Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data

Wei-Ning Hsu, James Glass

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、eess.AS

1710.05106 2018-04-27 cs.MM cs.CV cs.LG 81%

CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning

Yuxin Peng, Jinwei Qi, Yuxin Yuan

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV、cs.MM

1802.09816 2018-02-28 cs.CV cs.AI cs.LG cs.NE stat.ML 81%

Coarse to fine non-rigid registration: a chain of scale-specific neural networks for multimodal image alignment with application to remote sensing

Armand Zampieri, Guillaume Charpiat, Yuliya Tarabalka

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

1802.00924 2018-02-06 cs.LG cs.AI cs.CL stat.ML 81%

Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning

Minghai Chen, Sen Wang, Paul Pu Liang, Tadas Baltrušaitis, Amir Zadeh, Louis-Philippe Morency

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments ICMI 2017 Oral Presentation, Honorable Mention Award

1712.06228 2017-12-19 cs.CV cs.AI cs.LG 81%

Visual Explanations from Hadamard Product in Multimodal Deep Networks

Jin-Hwa Kim, Byoung-Tak Zhang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 8 pages, 5 figures, including appendix, NIPS 2017 Workshop on Visually-Grounded Interaction and Language (ViGIL)

1709.02251 2017-09-08 cs.CV cs.LG cs.MM 81%

Multi-modal Conditional Attention Fusion for Dimensional Emotion Prediction

Shizhe Chen, Qin Jin

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments Appeared at ACM Multimedia 2016

1. 多模态训练与对齐 6918 篇

Targeted aspect based multimodal sentiment analysis:an attention capsule extraction and multi-head fusion network

NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes

Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

DocStruct: A Multimodal Method to Extract Hierarchy Structure in Document for General Form Understanding

Answer-checking in Context: A Multi-modal FullyAttention Network for Visual Question Answering

MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding

Multi-Modal Open-Domain Dialogue

Cross-Modal Alignment with Mixture Experts Neural Network for Intral-City Retail Recommendation

Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response

Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA

SAFE: Similarity-Aware Multi-Modal Fake News Detection

Multi-modal Deep Analysis for Multimedia

From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining

Variational Fusion for Multimodal Sentiment Analysis

Reactive Multi-Stage Feature Fusion for Multimodal Dialogue Modeling

Multimodal Fusion with Deep Neural Networks for Audio-Video Emotion Recognition

Unsupervised Bilingual Lexicon Induction from Mono-lingual Multimodal Data

Vehicle Tracking Using Surveillance with Multimodal Data Fusion

Multimodal Polynomial Fusion for Detecting Driver Distraction

Deep Multi-Modal Classification of Intraductal Papillary Mucinous Neoplasms (IPMN) with Canonical Correlation Analysis

Investigation of Multimodal Features, Classifiers and Fusion Methods for Emotion Recognition

Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction

Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling

Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Data

CM-GANs: Cross-modal Generative Adversarial Networks for Common Representation Learning

Coarse to fine non-rigid registration: a chain of scale-specific neural networks for multimodal image alignment with application to remote sensing

Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning

Visual Explanations from Hadamard Product in Multimodal Deep Networks

Multi-modal Conditional Attention Fusion for Dimensional Emotion Prediction