arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.07113cs.SDcs.AIcs.HCeess.AS
收藏

一种基于理论的可解释音乐情感深度学习架构

A Theory-Based Explainable Deep Learning Architecture for Music Emotion

Columbia Business School(哥伦比亚商学院);Yale School of Management(耶鲁管理学院)

Hortense Fong, Vineet Kumar, K. Sudhir

AI总结:

本研究提出一种基于声学谐波理论的可解释CNN音乐情绪预测模型,性能优异且可解释性强,还验证了其在广告精准插放中提升参与度的应用价值。

摘要:

本文开发了一种基于理论的可解释深度学习卷积神经网络(CNN)分类器,用于预测音乐引发的时变情绪反应。我们设计了新型CNN滤波器,利用声学物理学中已知会影响音乐特征感知的频率谐波结构。我们的基于理论的模型更为精简,但预测性能与非理论深度学习模型相当,且优于使用手工特征的模型。该模型可辅以手工特征,但性能提升幅度很小。重要的是,CNN滤波器上设置的基于谐波的结构能更好地解释模型如何预测情绪反应(效价和唤醒度),因为情绪与协和音程——一种由谐波对齐定义的感知特征——密切相关。最后,我们通过一个数字广告应用展示了该模型的实用性。受YouTube中插广告启发,我们开展了一项实验室实验,在视频的不同时间点外源插入广告。我们发现,放置在情绪相似语境中的广告能提升广告参与度(更低的跳过率、更高的品牌回忆率)。基于我们的可解释理论模型预测的情绪相似度指标插入广告,所产生的参与度与非理论模型相当或更优。

英文摘要:

This paper paper develops a theory-based, explainable deep learning convolutional neural network (CNN) classifier to predict the time-varying emotional response to music. We design novel CNN filters that leverage the frequency harmonics structure from acoustic physics known to impact the perception of musical features. Our theory-based model is more parsimonious, but provides comparable predictive performance to atheoretical deep learning models, while performing better than models using handcrafted features. Our model can be complemented with handcrafted features, but the performance improvement is marginal. Importantly, the harmonics-based structure placed on the CNN filters provides better explainability for how the model predicts emotional response (valence and arousal), because emotion is closely related to consonance--a perceptual feature defined by the alignment of harmonics. Finally, we illustrate the utility of our model with an application involving digital advertising. Motivated by YouTube mid-roll ads, we conduct a lab experiment in which we exogenously insert ads at different times within videos. We find that ads placed in emotionally similar contexts increase ad engagement (lower skip rates, higher brand recall rates). Ad insertion based on emotional similarity metrics predicted by our theory-based, explainable model produces comparable or better engagement relative to atheoretical models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑