arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.03380cs.CL
收藏

噪声引导用于受控文本生成:在阿拉伯语教育故事生成中提高多样性与阅读水平的保真度

Noise Steering for Controlled Text Generation: Improving Diversity and Reading-Level Fidelity in Arabic Educational Story Generation

American University of Sharjah(阿联酋沙迦美国大学)

Haziq Mohammad Khalid, Salsabeel Shapsough, Imran Zualkernan

AI总结:

本文探讨了噪声引导方法在生成阿拉伯语早期阅读评估故事中的应用,通过对比不同策略发现内部表示扰动比输出层面的随机性更适用于受约束的教育内容生成。

摘要:

生成多样且具有教育价值的阿拉伯语早期阅读评估故事需要在词汇、阅读水平和叙事结构的严格限制与避免重复情节以保持评估有效性之间取得平衡。我们研究了噪声引导,即在推断过程中向Transformer模型的内部表示注入校准的高斯扰动,作为一种无需训练的多样性方法,在五个小型阿拉伯语中心语言模型(7-9B参数)上进行评估。我们比较了四种注入策略与高温采样基线,测量多样性、质量、约束遵守和阅读等级。残差流噪声在提高叙事多样性的同时,对质量和约束成本影响最小,并在所有阿拉伯语中心模型中保持早期阅读水平。注意力熵噪声注入(AENI)稳定了原本不可靠的注意力-logit噪声并恢复了质量。高温采样会提高阅读等级并在多个模型上导致灾难性崩溃。我们发现内部表示层面的扰动比输出层面的随机性更适合受约束的教育内容生成。

英文摘要:

Generating diverse, pedagogically valid stories for Arabic early-grade reading assessments requires balancing tight constraints on vocabulary, reading level, and narrative structure against the need to avoid repetitive plots that undermine assessment validity. We investigate noise steering, injecting calibrated Gaussian perturbations into the internal representations of transformer models at inference time, as a training-free diversity method evaluated across five small Arabic-centric language models (7-9B parameters). We compare four injection strategies against high-temperature sampling baselines, measuring diversity, quality, constraint adherence, and reading grade level. Residual stream noise consistently improves narrative diversity with minimal quality or constraint cost and preserves early-grade reading level across all Arabic-centric models. Attention entropy noise injection (AENI) stabilizes the otherwise unreliable attention-logit noise while recovering quality. High-temperature sampling inflates reading grade level and causes catastrophic collapse on several models. We find internal representation-level perturbation to be a more suitable diversity strategy than output-level stochasticity for constrained educational content generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑