合成说话儿童——为何需要以及如何制作
Synthetic Speaking Children -- Why We Need Them and How to Make Them
AI总结：
针对儿童语音交互中真实数据稀缺和隐私限制，本研究提出利用StyleGAN2和生成式TTS构建可控合成说话儿童视频数据集，为边缘AI平台提供训练数据。
摘要：
当代人机交互（HCI）研究主要依赖神经网络模型进行系统用户的机器视觉和语音理解。此类模型需要大量带注释的训练数据集才能达到最佳性能，而在为幼儿等弱势群体构建界面时，GDPR在数据收集、管理和处理方面引入了显著的复杂性。受边缘AI智能玩具平台训练需求的驱动，本研究探索了生成式神经技术的最新进展，并提供了一种可控数据生成管道的可行概念验证，用于大规模生成语音驱动的面部训练数据。在此背景下，我们展示了如何微调StyleGAN2以创建性别平衡的儿童面部数据集。该数据集包含多种可控因素，如面部表情、年龄变化、面部姿态，甚至具有逼真唇部同步的语音驱动动画。通过结合用于儿童语音合成的生成式文本到语音模型和基于3D地标的说话头部管道，我们可以生成高度逼真、完全合成的说话儿童视频片段。这些视频片段可以为神经网络模型提供有价值且可控的合成训练数据，在真实数据稀缺或因隐私法规受限时弥合差距。
英文摘要：
Contemporary Human Computer Interaction (HCI) research relies primarily on neural network models for machine vision and speech understanding of a system user. Such models require extensively annotated training datasets for optimal performance and when building interfaces for users from a vulnerable population such as young children, GDPR introduces significant complexities in data collection, management, and processing. Motivated by the training needs of an Edge AI smart toy platform this research explores the latest advances in generative neural technologies and provides a working proof of concept of a controllable data generation pipeline for speech driven facial training data at scale. In this context, we demonstrate how StyleGAN2 can be finetuned to create a gender balanced dataset of children's faces. This dataset includes a variety of controllable factors such as facial expressions, age variations, facial poses, and even speech-driven animations with realistic lip synchronization. By combining generative text to speech models for child voice synthesis and a 3D landmark based talking heads pipeline, we can generate highly realistic, entirely synthetic, talking child video clips. These video clips can provide valuable, and controllable, synthetic training data for neural network models, bridging the gap when real data is scarce or restricted due to privacy regulations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
