arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.17036cs.SDeess.AS
收藏

PitchIn:基于非结构化社区反馈的自然语言声乐指导

PitchIn: Natural-Language Vocal Coaching from Unstructured Community Feedback

KAIST(韩国科学技术院);The University of Tokyo(东京大学);ETRI(韩国电子通信研究院)

Hayeon Bang, Hounsu Kim, Joonyong Park, Jung-Jae Yu, Juhan Nam

AI总结:

PitchIn基于在线歌唱社区非结构化反馈构建音频-语言模型,结合PeerCoach数据集训练后,在VocalCoachBench上大幅提升声乐指导的诊断-纠正对匹配比例,实用性获专家与人类偏好认可。

摘要:

有效的声乐指导需将演唱问题与纠正建议关联,但带有专家诊断和建议的录音十分稀缺。在线歌唱社区提供了潜在的监督来源,但其观察结果和建议分散在非结构化评论中。我们推出PitchIn(用于自然语言声乐指导的音频-语言模型)和PeerCoach数据集,该数据集包含14445条指令示例,涵盖聚焦问答、整体反馈和识别问答三类任务。我们在Qwen2.5-Omni和MOSS-Music上对比目标组合,并使用聚焦问答和整体反馈训练PitchIn。在VocalCoachBench上,Qwen2.5-Omni的请求诊断-纠正对匹配专家问题及有效建议的比例从21.3%提升至61.0%,MOSS-Music则从31.4%提升至51.2%;直接在源评论上训练的Qwen2.5-Omni仅达19.8%。对于Qwen,仅聚焦问答就达到65.6%,但加入整体反馈可提升与专家评论及人类偏好的一致性。两种模型均比预训练基线提升了与专家评论的一致性,且在实用性上分别获得92.0%和67.3%的显著偏好。演示示例可在该https URL获取。

英文摘要:

Effective vocal coaching connects performance issues with corrective advice, but recordings paired with expert diagnoses and advice are scarce. Online singing communities provide a potential source of supervision, but observations and advice are scattered across unstructured comments. We introduce PitchIn, audio-language models for natural-language vocal coaching, and the PeerCoach Dataset: 14,445 instruction examples covering Focused QA, Overall Feedback, and Recognition QA. We compare target combinations on Qwen2.5-Omni and train PitchIn on Focused QA and Overall Feedback using Qwen2.5-Omni and MOSS-Music. On VocalCoachBench, the proportion of requested diagnosis-correction pairs matching an expert issue with valid advice rises from 21.3% to 61.0% for Qwen2.5-Omni and from 31.4% to 51.2% for MOSS-Music. Direct training on source comments yields only 19.8% for Qwen2.5-Omni. For Qwen, Focused QA alone reaches 65.6%, but adding Overall Feedback improves alignment with expert reviews and human preference. Both models improve expert-review alignment over pretrained baselines and are significantly preferred for usefulness (92.0% and 67.3%, respectively). Demo examples are available at https://hayeonbang.github.io/PitchIn.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑