重建无约束语音的动态指向性
Reconstructing the Dynamic Directivity of Unconstrained Speech
AI总结：
本文提出一种两阶段方法，利用虚拟阵列和机器学习模型，从有限语音信号重建自然无约束语音的动态指向性模式，以增强虚拟通信中的声音真实感。
摘要：
本文提出了一种估计和重建自然语音空间能量分布模式的方法，这对于在虚拟通信环境中实现逼真的声音存在感至关重要。该方法包括两个阶段。首先，利用真实静态麦克风阵列捕获的语音录音，创建一个随说话者随时间移动而跟踪其运动的自我中心虚拟阵列。该虚拟阵列用于测量和编码语音信号的高分辨率指向性模式，该模式随自然语音和运动动态演变。在第二阶段，利用编码后的指向性表示来训练机器学习模型，该模型能够根据有限的语音信号（例如使用头戴式显示器上的麦克风录制的信号）估计完整的动态指向性模式。我们的结果表明，神经网络能够从有限信息中准确估计自然、无约束语音的完整指向性模式。所提出的估计和重建自然语音空间能量分布模式的方法，以及对各种机器学习模型和训练范式的评估，为在虚拟通信环境中实现逼真的声音存在感的发展做出了重要贡献。
英文摘要：
This article presents a method for estimating and reconstructing the spatial energy distribution pattern of natural speech, which is crucial for achieving realistic vocal presence in virtual communication settings. The method comprises two stages. First, recordings of speech captured by a real, static microphone array are used to create an egocentric virtual array that tracks the movement of the speaker over time. This virtual array is used to measure and encode the high-resolution directivity pattern of the speech signal as it evolves dynamically with natural speech and movement. In the second stage, the encoded directivity representation is utilized to train a machine learning model that can estimate the full, dynamic directivity pattern given a limited set of speech signals, such as those recorded using the microphones on a head-mounted display. Our results show that neural networks can accurately estimate the full directivity pattern of natural, unconstrained speech from limited information. The proposed method for estimating and reconstructing the spatial energy distribution pattern of natural speech, along with the evaluation of various machine learning models and training paradigms, provides an important contribution to the development of realistic vocal presence in virtual communication settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
