打破文本依赖与音频匮乏的人工智能壁垒
Breaking the Barriers of Text-Hungry and Audio-Deficient AI
AI总结:
本文提出完全无文本的音频到音频机器智能框架,核心为多尺度音频语义变换MAST与分数扩散生成,使系统能直接从原始音频学习,服务无文字语言及偏好音频的人群。
摘要:
尽管全球语言多样性涵盖超过7164种被认可的语言,当前机器智能的主导架构仍从根本上偏向书面文本。这种偏见将超过7亿人——尤其是农村和偏远地区具备音频读写能力的人群——排除在外。在本工作中,我们引入了一个完全无文本、音频到音频的机器智能框架,旨在服务这一服务不足的人群,以及所有偏好音频效率的用户。我们的贡献包括新颖的音频到音频翻译架构,完全绕过文本,涵盖基于频谱图、尺度图、小波和单元(unit)的模型。我们方法的核心是多尺度音频语义变换(Multiscale Audio-Semantic Transform,MAST),这是一种编码音调、韵律、说话人和表达特征的表示。我们进一步将MAST集成到由分数布朗运动驱动的平均场型分数扩散框架中。它能够在无需文本监督的情况下生成高保真、语义一致的语音。其成果是一个稳健且可扩展的系统,能够直接从原始音频中学习,即使是对于无文字或很少被数字化的语言也是如此。这项工作代表了向音频原生机器智能系统的根本性转变,为历史上被排除在当前机器智能生态系统之外的社区扩展了语言技术的可及性。
英文摘要:
While global linguistic diversity spans more than 7164 recognized languages, the current dominant architecture of machine intelligence remains fundamentally biased toward written text. This bias excludes over 700 million people particularly in rural and remote regions who are audio-literate. In this work, we introduce a fully textless, audio-to-audio machine intelligence framework designed to serve this underserved population, and all the people who prefer audio-efficiency. Our contributions include novel Audio-to-Audio translation architectures that bypass text entirely, including spectrogram-, scalogram-, wavelet-, and unit-based models. Central to our approach is the Multiscale Audio-Semantic Transform (MAST), a representation that encodes tonal, prosodic, speaker, and expressive features. We further integrate MAST into a fractional diffusion of mean-field-type framework powered by fractional Brownian motion. It enables the generation of high-fidelity, semantically consistent speech without reliance on textual supervision. The result is a robust and scalable system capable of learning directly from raw audio, even in languages that are unwritten or rarely digitized. This work represents a fundamental shift toward audio-native machine intelligence systems, expanding access to language technologies for communities historically left out of the current machine intelligence ecosystem.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。