首页/论文/2012.00337

arXiv 2012.003372020-12cs.SDcs.HCeess.AS

NHSS: A Speech and Singing Parallel Database

National University of Singapore（新加坡国立大学）

Bidisha Sharma, Xiaoxue Gao, Karthika Vijayan, Xiaohai Tian, Haizhou Li

备注：: Accepted to Speech Communication

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

We present a database of parallel recordings of speech and singing, collected and released by the Human Language Technology (HLT) laboratory at the National University of Singapore (NUS), that is called NUS-HLT Speak-Sing (NHSS) database. We release this database to the public to support research activities, that include, but not limited to comparative studies of acoustic attributes of speech and singing signals, cooperative synthesis of speech and singing voices, and speech-to-singing conversion. This database consists of recordings of sung vocals of English pop songs, the spoken counterpart of lyrics of the songs read by the singers in their natural reading manner, and manually prepared utterance-level and word-level annotations. The audio recordings in the NHSS database correspond to 100 songs sung and spoken by 10 singers, resulting in a total of 7 hours of audio data. There are 5 male and 5 female singers, singing and reading the lyrics of 10 songs each. In this paper, we discuss the design methodology of the database, analyse the similarities and dissimilarities in characteristics of speech and singing voices, and provide some strategies to address relationships between these characteristics for converting one to another. We develop benchmark systems, which can be used as reference for speech-to-singing alignment, spectral mapping, and conversion using the NHSS database.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Leveraging Symmetrical Convolutional Transformer Networks for Speech to Singing Voice Style Transfer

Indian Institute of Science（印度科学学院）；Sony Group Corporation（索尼集团公司）

In this paper, we propose a model to perform style transfer of speech to singing voice. Contrary to the previous signal processing-based methods, which require high-quality singing templates or phoneme synchronization, we explore a data-driven approach for the problem of converting natural speech to singing voice. We develop a novel neural network architecture, called SymNet, which models the alignment of the input speech with the target melody while preserving the speaker identity and naturalness. The proposed SymNet model is comprised of symmetrical stack of three types of layers - convoluti…

2022-08-26cs.LGcs.SDeess.AS

A Comparative Study of Pitch Extraction Algorithms on a Large Variety of Singing Sounds

UMONS（蒙斯大学）；Acapela Group S.A.（Acapela集团股份有限公司）；GIPSA-lab（GIPSA实验室）；CNRS（法国国家科学研究中心）；INPG（格勒诺布尔国立理工学院）；Univ. Stendhal（司汤达大学）；UJF（约瑟夫·傅里叶大学）

The problem of pitch tracking has been extensively studied in the speech research community. The goal of this paper is to investigate how these techniques should be adapted to singing voice analysis, and to provide a comparative evaluation of the most representative state-of-the-art approaches. This study is carried out on a large database of annotated singing sounds with aligned EGG recordings, comprising a variety of singer categories and singing exercises. The algorithmic performance is assessed according to the ability to detect voicing boundaries and to accurately estimate pitch contour. …

2019-12-29cs.SDeess.AS

Modeling Singing F0 With Neural Network Driven Transition-Sustain Models

University of Illinois（伊利诺伊大学）

This study focuses on generating fundamental frequency (F0) curves of singing voice from musical scores stored in a midi-like notation. Current statistical parametric approaches to singing F0 modeling meet difficulties in reproducing vibratos and the temporal details at note boundaries due to the oversmoothing tendency of statistical models. This paper presents a neural network based solution that models a pair of neighboring notes at a time (the transition model) and uses a separate network for generating vibratos (the sustain model). Predictions from the two models are combined by summation …

2018-03-11cs.SDeess.AS

SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy

Renmin University of China（中国人民大学）；Carnegie Mellon University（卡内基梅隆大学）

Deep learning based singing voice synthesis (SVS) systems have been demonstrated to flexibly generate singing with better qualities, compared to conventional statistical parametric based methods. However, neural systems are generally data-hungry and have difficulty to reach reasonable singing quality with limited public available training data. In this work, we explore different data augmentation methods to boost the training of SVS systems, including several strategies customized to SVS based on pitch augmentation and mix-up augmentation. To further stabilize the training, we introduce the cy…

2022-07-06cs.LGcs.SDeess.AS

Sequence-to-sequence Singing Voice Synthesis with Perceptual Entropy Loss

Johns Hopkins University（约翰霍普金斯大学）；Renmin University of China（中国人民大学）

The neural network (NN) based singing voice synthesis (SVS) systems require sufficient data to train well and are prone to over-fitting due to data scarcity. However, we often encounter data limitation problem in building SVS systems because of high data acquisition and annotation costs. In this work, we propose a Perceptual Entropy (PE) loss derived from a psycho-acoustic hearing model to regularize the network. With a one-hour open-source singing voice database, we explore the impact of the PE loss on various mainstream sequence-to-sequence models, including the RNN-based, transformer-based,…

2021-02-26cs.LGcs.SDeess.AS

Learning a Joint Embedding Space of Monophonic and Mixed Music Signals for Singing Voice

KAIST（韩国科学技术院）

Previous approaches in singer identification have used one of monophonic vocal tracks or mixed tracks containing multiple instruments, leaving a semantic gap between these two domains of audio. In this paper, we present a system to learn a joint embedding space of monophonic and mixed tracks for singing voice. We use a metric learning method, which ensures that tracks from both domains of the same singer are mapped closer to each other than those of different singers. We train the system on a large synthetic dataset generated by music mashup to reflect real-world music recordings. Our approach…

2019-06-26cs.SDeess.AS

更多相关论文收起更多论文

Opencpop: A High-Quality Open Source Chinese Popular Song Corpus for Singing Voice Synthesis

Tongji University（同济大学）；Shanghai Institute of Visual Arts（上海视觉艺术学院）；Northwestern Polytechnical University（西北工业大学）；NetEase Inc.（网易公司）

This paper introduces Opencpop, a publicly available high-quality Mandarin singing corpus designed for singing voice synthesis (SVS). The corpus consists of 100 popular Mandarin songs performed by a female professional singer. Audio files are recorded with studio quality at a sampling rate of 44,100 Hz and the corresponding lyrics and musical scores are provided. All singing recordings have been phonetically annotated with phoneme boundaries and syllable (note) boundaries. To demonstrate the reliability of the released data and to provide a baseline for future research, we built baseline deep …

2022-01-20cs.DBcs.SDeess.AS

An Extensive Analysis of Query by Singing/Humming System Through Query Proportion

S.J.College of Engineering（S.J.工程学院）

Query by Singing/Humming (QBSH) is a Music Information Retrieval (MIR) system with small audio excerpt as query. The rising availability of digital music stipulates effective music retrieval methods. Further, MIR systems support content based searching for music and requires no musical acquaintance. Current work on QBSH focuses mainly on melody features such as pitch, rhythm, note etc., size of databases, response time, score matching and search algorithms. Even though a variety of QBSH techniques are proposed, there is a dearth of work to analyze QBSH through query excerption. Here, we presen…

2013-01-09cs.IRcs.MMcs.SD

Adversarial Semi-Supervised Audio Source Separation applied to Singing Voice Extraction

Queen Mary University of London（伦敦玛丽女王大学）

The state of the art in music source separation employs neural networks trained in a supervised fashion on multi-track databases to estimate the sources from a given mixture. With only few datasets available, often extensive data augmentation is used to combat overfitting. Mixing random tracks, however, can even reduce separation performance as instruments in real music are strongly correlated. The key concept in our approach is that source estimates of an optimal separator should be indistinguishable from real source signals. Based on this idea, we drive the separator towards outputs deemed a…

2018-04-06cs.LGcs.SD

DurIAN-SC: Duration Informed Attention Network based Singing Voice Conversion System

Tencent AI Lab（腾讯人工智能实验室）

Singing voice conversion is converting the timbre in the source singing to the target speaker's voice while keeping singing content the same. However, singing data for target speaker is much more difficult to collect compared with normal speech data.In this paper, we introduce a singing voice conversion algorithm that is capable of generating high quality target speaker's singing using only his/her normal speech data. First, we manage to integrate the training and conversion process of speech and singing into one framework by unifying the features used in standard speech synthesis system and s…

2020-08-07cs.SDeess.AS

歌唱声音转换挑战2025评估结果的深入分析

An Extensive Analysis of the Singing Voice Conversion Challenge 2025 Evaluation Results

Graduate School of Informatics, Nagoya University, Japan（名古屋大学信息学研究科，日本）；Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳））；Carnegie Mellon University, USA（卡内基梅隆大学，美国）；National Institute of Informatics, Japan（日本信息处理学会）；Information Technology Center, Nagoya University, Japan（名古屋大学信息技术中心，日本）

本文对2025年歌唱声音转换挑战赛的评估结果进行了深入分析，通过新数据库、两个任务、开源基线和大规模众包测试，比较了33个系统在歌手身份和歌唱风格转换上的表现，发现顶级系统在身份相似性上接近真实样本，但风格建模（如气息、滑音、颤音）仍具挑战，且现有客观指标无法完全替代主观评分。

2026-05-28cs.SDeess.AS