发表机构
Linköping University; University of Iceland（林雪平大学; 冰岛大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出SAGA框架，用依存句法分析器监督取代人类偏好标注，在丹麦语、冰岛语等低资源北欧语言上提升了GPT-SW3-1.3B的语法质量，为相关语言的语法对齐提供了实用替代方案。
AI 中文摘要
偏好优化已被证明对提升大语言模型有效，但通常依赖成本高昂的人类偏好标注。将这些方法扩展到形态丰富的低资源语言仍具挑战性，因为此类标注稀缺。我们提出SAGA（Score-weighted Adaptive Generation Alignment，分数加权自适应生成对齐），这是一种由解析器引导的偏好优化框架，它用依存句法分析器的监督取代人类标签。SAGA将解析器判断转换为delta-DPO的偏好对，在复合奖励中结合解析器质量与词汇多样性，利用奖励差距标准过滤低信息对，并监控奖励黑客行为以维持可靠的监督。在使用GPT-SW3-1.3B的丹麦语、冰岛语和挪威书面语（Bokmål）上，SAGA无需人类偏好标注即可持续提升语法质量：丹麦语解析成功率从69.0%升至93.8%；冰岛语在独立Stanza评估中提升4.5个百分点（三次运行均值提升3.3个百分点），且母语者在80%的成对比较中偏好SAGA的输出；挪威书面语提升28个百分点。这些结果表明，在可获得高质量依存句法分析器的低资源语言中，解析器衍生的监督是人类偏好标注的实用替代方案，可用于语法对齐。
英文摘要
Preference optimisation has proven effective for improving large language models but typically relies on costly human preference annotations. Extending these methods to morphologically rich, low-resource languages remains challenging because such annotations are scarce. We present SAGA (Score-weighted Adaptive Generation Alignment), a parser-guided preference optimisation framework that replaces human labels with dependency-parser supervision. SAGA converts parser judgements into preference pairs for delta-DPO, combines parser quality with lexical diversity in a composite reward, filters low-information pairs using a reward-gap criterion, and monitors reward hacking to maintain reliable supervision. Across Danish, Icelandic, and Norwegian Bokmål using GPT-SW3-1.3B, SAGA consistently improves grammatical quality without requiring human preference labels. Danish parse success increases from 69.0% to 93.8%, Icelandic achieves a +4.5 percentage-point improvement on an independent Stanza evaluation (three-run mean +3.3 percentage points) while native speakers prefer SAGA outputs in 80% of pairwise comparisons, and Norwegian Bokmål improves by +28 percentage points. These results demonstrate that parser-derived supervision is a practical alternative to human preference annotation for grammatical alignment in low-resource languages where high-quality dependency parsers are available.
Comments18 pages, 7 figures
