Marmoset monkeys encode vital information in their calls and serve as a surrogate model for neuro-biologists to understand the evolutionary origins of human vocal communication. Traditionally analyzed with signal processing-based features, recent approaches have utilized self-supervised models pre-trained on human speech for feature extraction, capitalizing on their ability to learn a signal's intrinsic structure independently of its acoustic domain. However, the utility of such foundation models remains unclear for marmoset call analysis in terms of multi-class classification, bandwidth, and pre-training domain. This study assesses feature representations derived from speech and general audio domains, across pre-training bandwidths of 4, 8, and 16 kHz for marmoset call-type and caller classification tasks. Results show that models with higher bandwidth improve performance, and pre-training on speech or general audio yields comparable results, improving over a spectral baseline.
Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale
Modality.ai, Inc.(Modality.ai公司);University of California, San Francisco(加州大学旧金山分校);EverythingALS, Peter Cohen Foundation(EverythingALS, Peter Cohen基金会);Pr3vent, Inc.(Pr3vent公司);IBM Thomas J. Watson Research Center(IBM托马斯·J·沃森研究中心);Massachusetts Institute of Technology(麻省理工学院);MGH Institute Health Professions(MGH健康职业学院);Harvard University(哈佛大学);University of Nebraska(内布拉斯加大学);University of Texas at Austin(德克萨斯大学奥斯汀分校)
An Analysis of Classification Approaches for Hit Song Prediction using Engineered Metadata Features with Lyrics and Audio Features
The University of Sheffield(谢菲尔德大学);Universität Koblenz(科布伦茨大学)
热门歌曲预测是音乐信息检索中的一个新兴领域,也是一个相当大的挑战。能够理解是什么让一首给定的歌曲成为热门显然对整个音乐行业都有好处。先前的热门歌曲预测方法集中于使用唱片的音频特征。本研究旨在使用更多的替代元数据,包括Spotify提供的歌曲音频特征、歌曲歌词和基于新元数据的特征(标题主题、流行度连续性和流派类别),来改善Billboard Hot 100歌曲中前10名热门歌曲的预测结果。应用了五种机器学习方法,包括:k-最近邻、朴素贝叶斯、随机森林、逻辑回归和多层感知器。我们的结果表明,随机森林(RF)和逻辑回归(LR)的所有功能(包括小说特征,歌曲音频特征和歌词特征)优于其他模型,分别达到89.1%和87.2%的准确率,和0.91和0.93 AUC。我们的研究结果还证明了我们的新音乐元数据特征的实用性,这对模型的区分性能贡献最大。
Towards Fine-grained Temporal Perception: Post-Training Large Audio-Language Models with Audio-Side Time Prompt
National Engineering Research Center of Speech and Language Information Processing, University of Science and Technology of China, Hefei, China(语音与语言信息处理国家级工程研究中心,中国科学技术大学,合肥,中国);ICT Cluster, Singapore Institute of Technology, Singapore(新加坡理工学院ICT集群,新加坡)
本文提出Audio-Side Time Prompt方法,结合强化学习改进大音频-语言模型的时间感知能力,在音频定位、事件检测等任务中取得显著提升。
Enhancing Crowdsourced Audio for Text-to-Speech Models
Barcelona Supercomputing Center (BSC)(巴塞罗那超级计算中心);Centre de Llenguatge i Computació (CLiC), Universitat de Barcelona(巴塞罗那大学语言与计算中心);Universität Zürich(苏黎世大学)
本文针对众包音频质量不足的问题,在 Common Voice 加泰罗尼亚语子集上提出音频增强加 NISQA 自动过滤的去噪流水线,使扩散 TTS 模型的 UTMOS 分数提升 0.4。
Department of Brain and Cognitive Sciences, MIT, USA(脑科学与认知科学系,麻省理工学院);McGovern Institute for Brain Research, MIT, USA(脑研究麦戈文研究所,麻省理工学院);Center for Brains, Minds and Machines, MIT, USA(脑、心智与机器中心,麻省理工学院);MIT-IBM Watson AI Lab, IBM Research, USA(麻省理工-IBM沃森人工智能实验室,IBM研究院)
WavInWav: Time-domain Speech Hiding via Invertible Neural Network
CAS Key Laboratory of Electromagnetic Space Information, School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学与技术学院电磁空间信息关键实验室)