arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2308.06259cs.CL
收藏

基于指令回译的自对齐

Self-Alignment with Instruction Backtranslation

Meta

Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, Mike Lewis

AI总结:

本研究提出指令回译的自对齐方法,通过种子模型自增强、自筛选构建训练数据迭代微调LLaMa,在不依赖蒸馏数据的Alpaca榜单上超越同类模型。

摘要:

我们提出一种可扩展的方法,通过为人类撰写的文本自动标注对应指令,构建高质量的指令跟随语言模型。我们的方法名为instruction backtranslation(指令回译),以在少量种子数据上微调的语言模型和给定的网络语料库为起点。利用种子模型为网络文档生成指令提示以构建训练样本(自增强),再从这些候选样本中筛选高质量样本(自筛选)。随后用这些数据微调出更强的模型。经两轮该方法微调的LLaMa模型,在不依赖蒸馏数据的Alpaca排行榜上优于所有其他基于LLaMa的模型,证明了自对齐的高效性。

英文摘要:

We present a scalable method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions. Our approach, named instruction backtranslation, starts with a language model finetuned on a small amount of seed data, and a given web corpus. The seed model is used to construct training examples by generating instruction prompts for web documents (self-augmentation), and then selecting high quality examples from among these candidates (self-curation). This data is then used to finetune a stronger model. Finetuning LLaMa on two iterations of our approach yields a model that outperforms all other LLaMa-based models on the Alpaca leaderboard not relying on distillation data, demonstrating highly effective self-alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑