参考引导的全双工口语语言模型话轮转换风格迁移
Reference-Guided Turn-Taking Style Transfer for Full-Duplex Spoken Language Models
AI总结：
本研究提出从模型对话中提取 initiative vector，可预判语音起始点并调整语音占比，通过参考录音实现说话者话轮转换风格向全双工口语语言模型的迁移。
摘要：
全双工口语语言模型会在每一帧决定是否获得话语权，以此定义对话个性——但每组权重仅对应一种个性，人类说话者会区分不同个性而非共享。现有控制方法大多从外部作用，未找到决定模型说话量的关键量。我们确定了该关键量：一个从模型自身对话中提取的、无需额外训练或标注的 late-layer residual-stream 方向，即 initiative vector（主动性向量）。该方向可读取决策，提前预判最多 800 毫秒的语音起始点，并能调整决策：在生成过程中对其缩放，可将语音占比从无引导时的 21-30% 提升至 42-61%，且在不同对话、权重和语料库中表现一致。仅需一段简短的参考录音，即可将说话者的话轮转换风格迁移至模型，且该迁移具有参考特异性：替换为其他说话者的参考时，93-98% 的案例匹配度会下降，而最佳统一剂量与无引导无差异。话轮转换风格并非固化在权重中，而是可针对每个人读取、调整和校准的方向。音频示例可在指定 URL 获取。
英文摘要：
Full-duplex spoken language models decide at every frame whether to take the floor, defining a conversational personality -- yet each set of weights encodes exactly one, which human speakers bracket rather than share. Existing controls mostly act from outside; none identifies the quantity that sets how much the model speaks. We identify it: a single late-layer residual-stream direction, the initiative vector, extracted from the model's own conversations without training or annotation. The same direction reads the decision, anticipating onsets up to 800 ms ahead, and shifts it: scaled during generation, it moves speech ratio from 6-12% to 42-61% (no steering: 21-30%), consistently across conversations, weights, and corpora. A short reference recording suffices to transfer a speaker's turn-taking style to the model, and the transfer is reference-specific: swapping in another speaker's reference worsens the match in 93-98% of cases, whereas the best uniform dose is indistinguishable from no steering. Turn-taking style is not frozen into the weights but a direction that can be read, shifted, and calibrated per person. Audio examples are available at this https URL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
