首页/论文/1908.01919

arXiv 1908.019192019-08-06cs.SDeess.AS

Adversarially Trained End-to-end Korean Singing Voice Synthesis System

Seoul National University（首尔大学）

Juheon Lee, Hyeong-Seok Choi, Chang-Bin Jeon, Junghyun Koo, Kyogu Lee

备注：: 5 pages, 3 figures, INTERSPEECH 2019 (oral presentation)

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

In this paper, we propose an end-to-end Korean singing voice synthesis system from lyrics and a symbolic melody using the following three novel approaches: 1) phonetic enhancement masking, 2) local conditioning of text and pitch to the super-resolution network, and 3) conditional adversarial training. The proposed system consists of two main modules; a mel-synthesis network that generates a mel-spectrogram from the given input information, and a super-resolution network that upsamples the generated mel-spectrogram into a linear-spectrogram. In the mel-synthesis network, phonetic enhancement masking is applied to generate implicit formant masks solely from the input text, which enables a more accurate phonetic control of singing voice. In addition, we show that two other proposed methods -- local conditioning of text and pitch, and conditional adversarial training -- are crucial for a realistic generation of the human singing voice in the super-resolution process. Finally, both quantitative and qualitative evaluations are conducted, confirming the validity of all proposed methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

N-Singer: A Non-Autoregressive Korean Singing Voice Synthesis System for Pronunciation Enhancement

NCSOFT（网石游戏公司）

Recently, end-to-end Korean singing voice systems have been designed to generate realistic singing voices. However, these systems still suffer from a lack of robustness in terms of pronunciation accuracy. In this paper, we propose N-Singer, a non-autoregressive Korean singing voice system, to synthesize accurate and pronounced Korean singing voices in parallel. N-Singer consists of a Transformer-based mel-generator, a convolutional network-based postnet, and voicing-aware discriminators. It can contribute in the following ways. First, for accurate pronunciation, N-Singer separately models ling…

2022-02-21cs.SDeess.AS

MLP Singer: Towards Rapid Parallel Korean Singing Voice Synthesis

Neosapience

Recent developments in deep learning have significantly improved the quality of synthesized singing voice audio. However, prominent neural singing voice synthesis systems suffer from slow inference speed due to their autoregressive design. Inspired by MLP-Mixer, a novel architecture introduced in the vision literature for attention-free image classification, we propose MLP Singer, a parallel Korean singing voice synthesis system. To the best of our knowledge, this is the first work that uses an entirely MLP-based architecture for voice synthesis. Listening tests demonstrate that MLP Singer out…

2021-11-20cs.SD

Adversarially Trained Multi-Singer Sequence-To-Sequence Singing Synthesizer

Xiaoice, Software Technology Center Asia, Microsoft（微软亚洲软件技术中心小冰团队）

This paper presents a high quality singing synthesizer that is able to model a voice with limited available recordings. Based on the sequence-to-sequence singing model, we design a multi-singer framework to leverage all the existing singing data of different singers. To attenuate the issue of musical score unbalance among singers, we incorporate an adversarial task of singer classification to make encoder output less singer dependent. Furthermore, we apply multiple random window discriminators (MRWDs) on the generated acoustic features to make the network be a GAN. Both objective and subjectiv…

2020-06-18cs.LGcs.SDeess.AS

Period Singer：融合周期性与非周期性变分自编码器的自然感端到端歌声合成

Period Singer: Integrating Periodic and Aperiodic Variational Autoencoders for Natural-Sounding End-to-End Singing Voice Synthesis

Korea Electronics Technology Institute（韩国电子技术研究院）

针对现有端到端歌声合成模型确定性音高条件导致的一对多问题，提出融合周期与非周期变分自编码器的Period Singer模型，通过音符边界内单调对齐搜索估计音素对齐，在中、韩数据集上表现优于现有模型。

2024-09-11cs.SDeess.AS

Speech-to-Singing Conversion based on Boundary Equilibrium GAN

National Taiwan University（国立台湾大学）

This paper investigates the use of generative adversarial network (GAN)-based models for converting the spectrogram of a speech signal into that of a singing one, without reference to the phoneme sequence underlying the speech. This is achieved by viewing speech-to-singing conversion as a style transfer problem. Specifically, given a speech input, and optionally the F0 contour of the target singing, the proposed model generates as the output a singing signal with a progressive-growing encoder/decoder architecture and boundary equilibrium GAN loss functions. Our quantitative and qualitative ana…

2020-08-05cs.LGcs.SDeess.AS

盘索里演唱的计算分析研究

Towards Computational Analysis of Pansori Singing

Georgia Institute of Technology（佐治亚理工学院）；KAIST（韩国科学技术院）；Sogang University（西江大学）

本文提出基于音频与转录的盘索里计算分析方法，利用现代音乐信息检索任务分析传统音乐并揭示其独特音频特征。

2024-10-16cs.IRcs.SDeess.AS

更多相关论文收起更多论文

通过流匹配缓解基于cVAE的歌唱语音合成中的潜在不匹配

Mitigating Latent Mismatch in cVAE-Based Singing Voice Synthesis via Flow Matching

本文提出FM-Singer框架，通过在潜在空间中学习连续向量场来减少训练与推理时的潜在不匹配，提升合成语音的表达性与质量。

2026-03-13cs.AIcs.SDeess.AS

阐明歌声转写中的性别公平性

Elucidate Gender Fairness in Singing Voice Transcription

National University of Singapore（新加坡国立大学）

本文揭示了歌声转写中由音高分布差异导致的女性优势性别偏差，并提出基于属性预测器对抗训练与音符事件条件对齐的方法，在保持SVT性能的同时显著降低性别偏差。

2023-08-05cs.LGcs.SDeess.AS

Pitch Preservation In Singing Voice Synthesis

College of Geophysics, Chengdu University of Technology（成都理工大学地球物理学院）；Changhong AI Lab (CHAIR), Sichuan Changhong Electronics Holding Group Co., Ltd.（四川长虹电子控股集团有限公司长虹人工智能实验室）

Suffering from limited singing voice corpus, existing singing voice synthesis (SVS) methods that build encoder-decoder neural networks to directly generate spectrogram could lead to out-of-tune issues during the inference phase. To attenuate these issues, this paper presents a novel acoustic model with independent pitch encoder and phoneme encoder, which disentangles the phoneme and pitch information from music score to fully utilize the corpus. Specifically, according to equal temperament theory, the pitch encoder is constrained by a pitch metric loss that maps distances between adjacent inpu…

2021-10-12cs.AIcs.SDeess.AS

Gugak-VocalSet：带有Sigimsae标注的多流派韩国传统声乐录音室级数据集

Gugak-VocalSet: A Multi-Genre Studio-Quality Dataset of Korean Traditional Singing with Sigimsae Annotations

National Gugak Center（国立国乐中心）；Neutune（纽 Tune（音译））

该研究发布首个带Sigimsae标注的韩国传统声乐数据集Gugak-VocalSet，经实验验证其标注与F0表示可用于Sigimsae分类和事件检测，性能优于通用音乐基础模型的冻结探测。

2026-10-14cs.AIcs.DBcs.DL

PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network

Tencent AI Lab（腾讯人工智能实验室）

Singing voice conversion is to convert a singer's voice to another one's voice without changing singing content. Recent work shows that unsupervised singing voice conversion can be achieved with an autoencoder-based approach [1]. However, the converted singing voice can be easily out of key, showing that the existing approach cannot model the pitch information precisely. In this paper, we propose to advance the existing unsupervised singing voice conversion method proposed in [1] to achieve more accurate pitch translation and flexible pitch manipulation. Specifically, the proposed PitchNet add…

2020-02-18cs.CLcs.SDeess.AS