SPP-SCL:半推拉监督对比学习用于图像-文本情感分析及更广泛的领域
SPP-SCL: Semi-Push-Pull Supervised Contrastive Learning for Image-Text Sentiment Analysis and Beyond
AI总结:
SPP-SCL通过半推拉监督对比学习平衡图像-文本情感关系,提升跨模态情感分析的性能。
摘要:
现有的图像-文本情感分析(ITSA)方法可能面临内在模态和跨模态情感关系不一致的问题。因此,我们开发了一种在融合前进行平衡的方法,以解决视觉-语言不平衡的内在模态和跨模态情感关系问题;即提出了一种半推拉监督对比学习(SPP-SCL)方法。具体来说,该方法采用了一种新的两步策略,即首先使用所提出的内在模态监督对比学习来拉近内在模态之间的关系,然后执行一个精心设计的条件执行语句。如果语句结果为假,我们的方法将执行第二步,即跨模态监督对比学习,以推远跨模态之间的关系。两步策略将平衡内在模态和跨模态的关系,以实现关系的一致性,并最终进行跨模态特征融合进行情感分析和检测。在三个公开的图像-文本情感和讽刺检测数据集上的实验研究表明,SPP-SCL在很大程度上优于最先进的方法,并且在情感上更具辨别性。
英文摘要:
Existing Image-Text Sentiment Analysis (ITSA) methods may suffer from inconsistent intra-modal and inter-modal sentiment relationships. Therefore, we develop a method that balances before fusing to solve the issue of vision-language imbalance intra-modal and inter-modal sentiment relationships; that is, a Semi-Push-Pull Supervised Contrastive Learning (SPP-SCL) method is proposed. Specifically, the method is implemented using a novel two-step strategy, namely first using the proposed intra-modal supervised contrastive learning to pull the relationships between the intra-modal and then performing a well-designed conditional execution statement. If the statement result is false, our method will perform the second step, which is inter-modal supervised contrastive learning to push away the relationships between inter-modal. The two-step strategy will balance the intra-modal and inter-modal relationships to achieve the purpose of relationship consistency and finally perform cross-modal feature fusion for sentiment analysis and detection. Experimental studies on three public image-text sentiment and sarcasm detection datasets demonstrate that SPP-SCL significantly outperforms state-of-the-art methods by a large margin and is more discriminative in sentiment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。