In most cases, bilingual TTS needs to handle three types of input scripts: first language only, second language only, and second language embedded in the first language. In the latter two situations, the pronunciation and intonation of the second language are usually quite different due to the influence of the first language. Therefore, it is a big challenge to accurately model the pronunciation and intonation of the second language in different contexts without mutual interference. This paper builds a Mandarin-English TTS system to acquire more standard spoken English speech from a monolingual Chinese speaker. We introduce phonology embedding to capture the English differences between different phonology. Embedding mask is applied to language embedding for distinguishing information between different languages and to phonology embedding for focusing on English expression. We specially design an embedding strength modulator to capture the dynamic strength of language and phonology. Experiments show that our approach can produce significantly more natural and standard spoken English speech of the monolingual Chinese speaker. From analysis, we find that suitable phonology control contributes to better performance in different scenarios.
Zapr Media Labs (Red Brick Lane Marketing Solutions Pvt. Ltd.)（Zapr媒体实验室（红砖巷营销解决方案私人有限公司））
Bilingual English speakers speak English as one of their languages. Their English is of a non-native kind, and their conversations are of a code-mixed fashion. The intelligibility of a bilingual text-to-speech (TTS) system for such non-native English speakers depends on a lexicon that captures the phoneme sequence used by non-native speakers. However, due to the lack of non-native English lexicon, existing bilingual TTS systems employ native English lexicons that are widely available, in addition to their native language lexicon. Due to the inconsistency between the non-native English pronunci…
Idiap Research Institute（伊迪亚普研究所）；Constantine the Philosopher University in Nitra（尼特拉康斯坦丁哲学家大学）；Institute of Informatics, Slovak Academy of Sciences（斯洛伐克科学院信息学研究所）
Using phonological speech vocoding, we propose a platform for exploring relations between phonology and speech processing, and in broader terms, for exploring relations between the abstract and physical structures of a speech signal. Our goal is to make a step towards bridging phonology and speech processing and to contribute to the program of Laboratory Phonology. We show three application examples for laboratory phonology: compositional phonological speech modelling, a comparison of phonological systems and an experimental phonological parametric text-to-speech (TTS) system. The featural rep…
Recent state-of-the-art neural text-to-speech (TTS) synthesis models have dramatically improved intelligibility and naturalness of generated speech from text. However, building a good bilingual or code-switched TTS for a particular voice is still a challenge. The main reason is that it is not easy to obtain a bilingual corpus from a speaker who achieves native-level fluency in both languages. In this paper, we explore the use of Mandarin speech recordings from a Mandarin speaker, and English speech recordings from another English speaker to build high-quality bilingual and code-switched TTS fo…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；University of Science and Technology of China（中国科学技术大学）
Current end-to-end code-switching Text-to-Speech (TTS) can already generate high quality two languages speech in the same utterance with single speaker bilingual corpora. When the speakers of the bilingual corpora are different, the naturalness and consistency of the code-switching TTS will be poor. The cross-lingual embedding layers structure we proposed makes similar syllables in different languages relevant, thus improving the naturalness and consistency of generated speech. In the end-to-end code-switching TTS, there exists problem of prosody instability when synthesizing paragraph text. T…
We work to create a multilingual speech synthesis system which can generate speech with the proper accent while retaining the characteristics of an individual voice. This is challenging to do because it is expensive to obtain bilingual training data in multiple languages, and the lack of such data results in strong correlations that entangle speaker, language, and accent, resulting in poor transfer capabilities. To overcome this, we present a multilingual, multiaccented, multispeaker speech synthesis model based on RADTTS with explicit control over accent, language, speaker and fine-grained $F…
SyncVoice: Simple and Effective Automatic Video Dubbing with Vision-Augmented TTS
School of Informatics, Xiamen University, China（厦门大学信息学院）；MiLM Plus, Xiaomi Inc., China（小米公司）；School of Electronic Science and Engineering, Xiamen University, China（厦门大学电子科学与技术学院）
Brainekt AI Lab（Brainekt AI实验室）；Khulna University of Engineering and Technology（库尔纳工程技术大学）
Speech synthesis is one of the challenging tasks to automate by deep learning, also being a low-resource language there are very few attempts at Bangla speech synthesis. Most of the existing works can't work with anything other than simple Bangla characters script, very short sentences, etc. This work attempts to solve these problems by introducing Byakta, the first-ever open-source deep learning-based bilingual (Bangla and English) text to a speech synthesis system. A speech recognition model-based automated scoring metric was also proposed to evaluate the performance of a TTS model. We also …
WordVoice: Explicit and Decoupled Multi-Dimensional Word-Level Control for LLM-Based TTS
South China University of Technology（南方科技大学）；Huya Inc.（Huya公司）；Tongji University（同济大学）；The Hongkong polytechnic university（香港理工大学）；Foshan University（佛山大学）
COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning
The Hong Kong University of Science and Technology（香港科技大学）；JIUTIAN Research, China Mobile（中国移动九天研究院）；The State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；Nanjing University（南京大学）；China Mobile (Hong Kong) Innovation Research Institute（中国移动（香港）创新研究院）
SceneTTS-Bench: A Benchmark for Scene-Level TTS in Drama Dubbing
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing DeepLogic Intelligence Technology Co., Ltd.（北京深逻辑智能科技有限公司）；University of California（加利福尼亚大学）
ISCSLP 2026 CoT-TTS Challenge: Chain-of-Thought Reasoning for Context-Aware Text-to-Speech
The Hong Kong University of Science and Technology（香港科技大学）；China Mobile（中国移动）；Jiutian Artificial Intelligence Technology (Beijing) Co., Ltd., China Mobile（九天人工智能技术（北京）有限公司，中国移动）；China Mobile (Hong Kong) Innovation Research Institute（中国移动（香港）创新研究院）；Nanjing University（南京大学）
We present a multispeaker, multilingual text-to-speech (TTS) synthesis model based on Tacotron that is able to produce high quality speech in multiple languages. Moreover, the model is able to transfer voices across languages, e.g. synthesize fluent Spanish speech using an English speaker's voice, without training on any bilingual or parallel examples. Such transfer works across distantly related languages, e.g. English and Mandarin. Critical to achieving this result are: 1. using a phonemic input representation to encourage sharing of model capacity across languages, and 2. incorporating an a…
BENYO-S2ST-Corpus-1: A Bilingual English-to-Yoruba Direct Speech-to-Speech Translation Corpus
Covenant University（圣约大学）；Durban University of Technology（德班理工大学）；Walter Sisulu University（沃尔特·西苏鲁大学）；Summit University（萨米特大学）；International University of Grand-Bassam（大巴萨姆国际大学）
