arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.19981cs.MM
收藏

用于连续情感预测的冻结多模态模型共享情感证据

Shared Sentiment Evidence from Frozen Multimodal Models for Continuous Sentiment Prediction

UNSW(新南威尔士大学)

Shuobo Wang, Ruiqi Chen, Shaobo Qiao, Shixu Yang, Feng Xu

AI总结:

提出共享情感证据(SSE)输出空间接口,在冻结多模态骨干上实现连续情感预测,在CH-SIMS-v2、CMU-MOSI等数据集上优于基线方法,验证了该接口的通用性与有效性。

摘要:

连续多模态情感分析从可能存在分歧的语言、声学和视觉线索中估计情感极性与强度。硬判断压缩情感选项,而学习到的融合接口将任务适配与模型表示关联。我们提出共享情感证据(Shared Sentiment Evidence, SSE),这是一种输出空间接口,在冻结骨干网络间具有固定的视图和情感选项语义。四个查询生成五选项分布,其概率、熵及跨视图关系构成28维向量用于CatBoost回归。在CH-SIMS-v2和CMU-MOSI数据集上,基于Qwen的接口优于仅监督联合读出的结果;匹配的Ridge和CatBoost对照实验评估额外视图、确定性摘要及非线性读出的贡献。在相同接口规格与固定读出设置下,分别拟合的头部也在三个额外冻结骨干网络上优于仅联合预测的结果,这些结果支持从可复用输出坐标学习连续情感。

英文摘要:

Continuous multimodal sentiment analysis estimates polarity and intensity from linguistic, acoustic, and visual cues that may disagree. Hard judgments compress sentiment alternatives, while learned fusion interfaces tie task adaptation to model representations. We introduce Shared Sentiment Evidence (SSE), an output-space interface with fixed view and sentiment-option semantics across frozen backbones. Four queries yield five-option distributions whose probabilities, entropy, and cross-view relations form a 28-dimensional vector for CatBoost regression. On CH-SIMS-v2 and CMU-MOSI, the Qwen-based interface improves on supervised joint-only readout. Matched Ridge and CatBoost controls assess the contributions of additional views, deterministic summaries, and nonlinear readout. With the same interface specification and fixed readout settings, separately fitted heads also improve on joint-only prediction for three additional frozen backbones. These results support learning continuous sentiment from reusable output coordinates.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑