发表机构
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
ARIA提出两阶段音频驱动框架，直接从歌唱录音预测声调序列并生成声调一致的粤语歌词，构建大规模数据集，实验验证其有效性。
AI 中文摘要
粤语歌词创作要求歌词的声调与旋律音高紧密对齐。现有的旋律引导歌词生成方法通常依赖符号化旋律来生成歌词。然而，在实际歌曲创作场景中，旋律常以原始歌唱音频或哼唱录音的形式呈现，其中音高隐含、嘈杂且结构不清晰，使得这些方法难以直接应用。为解决这一局限，我们提出ARIA，一个两阶段的音频驱动旋律-声调关系建模框架，用于粤语歌词创作，能够从带有字符级时间戳的歌唱录音中生成粤语歌词。具体而言，我们首先设计了一个三流关系感知声调估计器（TRATE），通过建模多流声学线索和关系性声调结构，从带时间戳的歌唱音频中预测0243序列。然后，我们提出一个解耦的检索增强声调条件歌词生成器（DRA-TCLG），在检索增强的词汇引导下，根据预测的声调计划生成流畅的歌词。此外，我们从真实粤语歌唱录音中构建了一个大规模的对齐音频-粤拼-0243数据集，以支持这一新任务。实验结果表明，ARIA在0243预测和声调一致的歌词生成方面均取得了强劲性能，验证了所提出框架的有效性。
英文摘要
Cantonese lyric writing requires close alignment between lexical tones and melodic pitch. Existing melody-guided lyric generation methods typically rely on symbolic melody to generate lyrics. However, in real songwriting scenarios, melodies are often expressed as raw singing audio or hummed recordings, where pitch is implicit, noisy, and unstructured, making these methods difficult to apply directly. To address this limitation, we propose ARIA, a two-stage audio-driven melody-tone relation modeling framework for Cantonese lyric authoring that generates Cantonese lyrics from singing recordings with provided character-level timestamps. Specifically, we first design a Tri-Stream Relation-Aware Tone Estimator (TRATE) to predict 0243 sequences from timestamped singing audio by modeling multi-stream acoustic cues and relational tonal structure. We then propose a Decoupled Retrieval-Augmented Tone-Conditioned Lyric Generator (DRA-TCLG) to generate fluent lyrics conditioned on predicted tonal plans with retrieval-enhanced lexical guidance. Moreover, we construct a large-scale aligned audio-Jyutping-0243 dataset from real Cantonese singing recordings to support this new task. Experimental results demonstrate that ARIA achieves strong performance in both 0243 prediction and tone-consistent lyric generation, validating the effectiveness of the proposed framework.
CommentsAccepted for publication in Findings of EMNLP 2026. 24 pages, including references and appendices. Author-prepared version
