Multi-Modality Collaborative Learning for Sentiment Analysis
Shanmin Wang, Chengguang Liu, Qingshan Liu
备注:
The method has flaws, especially with the decoupling module. During the decoupling process, the heterogeneity of the three modal data and the differences in distribution were not taken into account
Multimodal sentiment analysis (MSA) identifies individuals' sentiment states in videos by integrating visual, audio, and text modalities. Despite progress in existing methods, the inherent modality heterogeneity limits the effective capture of interactive sentiment features across modalities. In this paper, by introducing a Multi-Modality Collaborative Learning (MMCL) framework, we facilitate cross-modal interactions and capture enhanced and complementary features from modality-common and modality-specific representations, respectively. Specifically, we design a parameter-free decoupling module and separate uni-modality into modality-common and modality-specific components through semantics assessment of cross-modal elements. For modality-specific representations, inspired by the act-reward mechanism in reinforcement learning, we design policy models to adaptively mine complementary sentiment features under the guidance of a joint reward. For modality-common representations, intra-modal attention is employed to highlight crucial components, playing enhanced roles among modalities. Experimental results, including superiority evaluations on four databases, effectiveness verification of each module, and assessment of complementary features, demonstrate that MMCL successfully learns collaborative features across modalities and significantly improves performance. The code can be available at https://github.com/smwanghhh/MMCL.
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach
College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算机与数据科学学院);Singapore University of Social Sciences, Singapore(新加坡社会科学研究大学)
Enhancing Collaborative Filtering Recommender with Prompt-Based Sentiment Analysis
New York University(纽约大学)
协同过滤推荐系统是在线市场和电子商务中的一个重要应用。然而,CF推荐已被证明遭受持续的问题,与稀疏的用户评级,这将进一步导致冷启动问题。现有方法通过应用标记级情感分析来解决数据稀疏问题,该标记级情感分析将文本评论转换为情感分数作为用户评级的补充。在本文中,我们试图优化情感分析与先进的NLP模型,包括BERT和RoBERTa,并实验是否CF推荐已进一步增强。我们在Amazon US Reviews数据集上构建了训练器,并使用传统的微调范式以及新的基于自适应的学习范式来调整预训练的BERT和RoBERTa。实验结果表明,基于RoBERTa预测的情感评分的推荐系统性能最优,与基线推荐系统相比,MAP、NDCG和K点的精确度提高了30.7%.基于情感分析的学习范式虽然在纯情感分析方面优于传统的微调范式,但不能进一步改进CF推荐系统。
Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis
University of Macau(澳门大学);Zhejiang University(浙江大学);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);Juntendo University(立命馆大学);Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学)
FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis
School of Future Science and Engineering, Soochow University(苏州大学未来科学与工程学院);School of Advanced Manufacturing Engineering, Hefei University(合肥大学先进制造工程学院);College of Global Talents, BITZH, Beijing Institute of Technology(北京理工大学珠海学院全球人才学院);Academy for Engineering and Technology, Fudan University(复旦大学工程与技术学院)
Uncertainty-Aware Collaborative System of Large and Small Models for Multimodal Sentiment Analysis
School of Computer Science, South China Normal University(计算机科学学院,华南师范大学);School of Artificial Intelligence, South China Normal University(人工智能学院,华南师范大学);School of Electronics and Information Technology, Sun Yat-sen University(电子与信息工程学院,中山大学)
A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios
Institute of Computational Perception, Johannes Kepler University Linz(约翰开普勒林茨大学计算感知研究所);Human-centered AI Group, AI Lab, Linz Institute of Technology(林茨理工大学人工智能实验室以人为中心的人工智能组)
Multi-Modal Opinion Integration for Financial Sentiment Analysis using Cross-Modal Attention
College of Computing Georgia Institute of Technology Atlanta, USA(计算学院 佐治亚理工学院 美国亚特兰大);College of Engineering University of Pennsylvania Philadelphia, USA(工程学院 宾夕法尼亚大学 美国费城)
ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors
Southwest Jiaotong University(西南交通大学);Microsoft Research Asia(微软亚洲研究院);Southwestern University of Finance and Economics(西南财经大学);Ocean University of China(中国海洋大学);Tencent(腾讯)
Modality Unified Attack for Omni-Modality Person Re-Identification
College of Electrical and Information Engineering at Hunan University(湖南大学电气与信息工程学院);National Engineering Research Center of Robot Visual Perception and Control Technology(机器人视觉感知与控制技术国家工程研究中心);College of Information Science and Engineering at Hunan Normal University(湖南师范大学信息科学与工程学院);Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing(湖南省智能计算与语言信息处理重点实验室)
Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection
University of Macau(澳门大学);Zhejiang University(浙江大学);Tsinghua University(清华大学);Tongji University(同济大学);University of Southampton(南安普顿大学);University of Bologna(博洛尼亚大学);Minzu University of China(中央民族大学)