arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.12593cs.CL
收藏

Standardize:使语言模型与专家定义的内容生成标准对齐

Standardize: Aligning Language Models with Expert-Defined Standards for Content Generation

University of Bath(巴斯大学);National University, Philippines(菲律宾国立大学)

Joseph Marvin Imperial, Gail Forey, Harish Tayyar Madabushi

AI总结:

针对现有可控文本生成未利用专家标准的问题,提出基于检索式上下文学习的Standardize框架,引导大模型对齐教育领域英语标准,实验显示模型精确准确率提升45%至100%。

摘要:

工程、医疗保健和教育领域的领域专家在生成技术手册、用药说明和儿童阅读材料等高质量内容时会遵循严格的标准。然而,当前可控文本生成领域的研究尚未探索将这些标准用作控制参考。为此,我们提出了Standardize,这是一种基于检索式上下文学习的框架,用于引导大语言模型与专家定义的标准对齐。我们以教育领域的英语语言标准为用例,针对开放式内容生成任务,采用了欧洲语言共同参考框架(Common European Framework of Reference for Languages, CEFR)和共同核心标准(Common Core Standards, CCS)。研究结果表明,在接受评估的开源和商业大语言模型中,模型的精确准确率可提升45%至100%,这证明使用从标准中提取的知识构件并将其整合到生成过程中,能够有效引导模型生成更符合标准的内容。

英文摘要:

Domain experts across engineering, healthcare, and education follow strict standards for producing quality content such as technical manuals, medication instructions, and children's reading materials. However, current works in controllable text generation have yet to explore using these standards as references for control. Towards this end, we introduce Standardize, a retrieval-style in-context learning-based framework to guide large language models to align with expert-defined standards. Focusing on English language standards in the education domain as a use case, we consider the Common European Framework of Reference for Languages (CEFR) and Common Core Standards (CCS) for the task of open-ended content generation. Our findings show that models can gain a 45% to 100% increase in precise accuracy across open and commercial LLMs evaluated, demonstrating that the use of knowledge artifacts extracted from standards and integrating them in the generation process can effectively guide models to produce better standard-aligned content.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑