This paper paper develops a theory-based, explainable deep learning convolutional neural network (CNN) classifier to predict the time-varying emotional response to music. We design novel CNN filters that leverage the frequency harmonics structure from acoustic physics known to impact the perception of musical features. Our theory-based model is more parsimonious, but provides comparable predictive performance to atheoretical deep learning models, while performing better than models using handcrafted features. Our model can be complemented with handcrafted features, but the performance improvement is marginal. Importantly, the harmonics-based structure placed on the CNN filters provides better explainability for how the model predicts emotional response (valence and arousal), because emotion is closely related to consonance--a perceptual feature defined by the alignment of harmonics. Finally, we illustrate the utility of our model with an application involving digital advertising. Motivated by YouTube mid-roll ads, we conduct a lab experiment in which we exogenously insert ads at different times within videos. We find that ads placed in emotionally similar contexts increase ad engagement (lower skip rates, higher brand recall rates). Ad insertion based on emotional similarity metrics predicted by our theory-based, explainable model produces comparable or better engagement relative to atheoretical models.
EmoGen: Eliminating Subjective Bias in Emotional Music Generation
Shanghai Jiao Tong University(上海交通大学);Microsoft Research Asia(微软亚洲研究院);Nanjing University(南京大学);National Engineering Research Center for Software Engineering, Peking University(北京大学软件工程国家工程研究中心)
Emotion-aware Personalized Music Recommendation with a Heterogeneity-aware Deep Bayesian Network
School of Management, Hefei University of Technology(合肥工业大学管理学院);Rawls College of Business, Texas Tech University(德克萨斯理工大学罗尔斯商学院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院)
这项工作提出了一个初步的概念证明,音乐情感识别(MER)系统可以故意偏向音乐引起的情绪在政治背景下的注释。具体而言,我们分析了传统的哥伦比亚音乐包含两种类型的政治性歌词:(1)vallenatos和社会歌曲从“左翼”游击队Fuerzas Armadas Revolucionarias de Colombia(FARC)和(2)corridos从“右翼”准军事组织哥伦比亚联合自卫军(AUC)。我们训练个性化的机器学习模型来预测三个具有不同政治观点的用户的诱导情绪-我们的目标是识别可能引起特定用户负面情绪的歌曲,例如愤怒和恐惧。在这种程度上,用户的情感判断可以被解释为问题化数据-主观情感判断可以反过来用于在以人为中心的机器学习环境中影响用户。简而言之,高度期望的“情绪调节”应用可能会偏离“情绪操纵”-最近对情绪识别技术的怀疑可能超越多样性和包容性的伦理问题。
EmotionBox: a music-element-driven emotional music generation system using Recurrent Neural Network
Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所噪声与振动研究重点实验室);University of Chinese Academy of Sciences(中国科学院大学);State Key Laboratory of Media Convergence and Communication, Communication University of China(中国传媒大学媒体融合与传播国家重点实验室);School of Electronics and Communication Engineering, Guangzhou University(广州大学电子与通信工程学院);National Environmental Protection Engineering and Technology Center for Road Traffic Noise Control(国家环境保护道路交通噪声控制工程技术中心)
EMelodyGen: Emotion-Conditioned Melody Generation in ABC Notation with the Musical Feature Template
Department of Music AI and Information Technology, Central Conservatory of Music(中央音乐学院音乐人工智能与信息科技系);School of Computer Science and Technology, Fudan University(复旦大学计算机科学与技术学院);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学智能信息处理上海重点实验室)
Are Expressions for Music Emotions the Same Across Cultures?
Max Planck Institute for Empirical Aesthetics(马克斯·普朗克实证美学研究所);Max Planck Institute for Human Cognitive and Brain Sciences(马克斯·普朗克人类认知与脑科学研究所);Cornell University(康奈尔大学)
A New Multilabel System for Automatic Music Emotion Recognition
University of Rome Tor Vergata(罗马第二大学)
在自动识别音乐所能诱发的情感方面取得进展需要考虑情感的多样性和复杂性。比较不同的机器学习算法执行多标签和多类分类是我们工作的核心。该研究分析了Geneva Emotional Music Scale 9在Quantify音乐数据集中的实施情况,并从机器学习的角度调查了其采用情况。我们的方法的情况下,通过音乐的情感表达/感应作为一个多标签和多类的问题,其中多个情感标签可以通过相同的音乐曲目的每个注释器(多标签),和每个情感可以识别或不音乐(多类)。其目的是通过音乐自动识别诱导的情感。
Semi-Supervised Self-Learning Enhanced Music Emotion Recognition
School of Computer Science and Technology, Fudan University(复旦大学计算机科学与技术学院);Ping An Technology Co., Ltd.(平安科技有限公司);Department of Music AI and Information Technology, Central Conservatory of Music(中央音乐学院音乐人工智能与信息技术系);Shanghai Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学上海智能信息处理重点实验室)
Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI
Creative Computing Institute, University of the Arts London(伦敦艺术大学创意计算学院);School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽皇后学院电子工程与计算机科学学院);Carleton College(卡尔顿学院)
Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
University of Copenhagen(哥本哈根大学)
本文旨在测试多模态方法的音乐情感识别(MER)是否比单模态的高层次的歌曲特征和歌词。我们使用从Spotify API检索的11个歌曲特征,组合歌词特征,包括情感,TF-IDF和Anew,以预测Deezer情绪检测数据集(DMDD)(Delbouys et al.,2018),具有4种不同的回归模型。我们发现,在11个高级歌曲特征中,主要有5个对性能有贡献,多模态特征在预测效价时比单独的音频做得更好。我们使我们的代码可供使用。
LARA-Gen: Enabling Continuous Emotion Control for Music Generation Models via Latent Affective Representation Alignment
X-LANCE Lab, Shanghai Jiao Tong University, China(上海交通大学X-LANCE实验室);Shanghai AI Lab, China(上海人工智能实验室);Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, China(上海交通大学医学院精神卫生中心)