Self-Tuning：指导大语言模型通过自教学有效获取新知识
Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching
AI总结：
针对大语言模型知识更新难、新文档知识提取效果差的问题，提出Self-Tuning自教学框架，通过自监督生成三类知识密集型任务增强文档，结合新构建的数据集验证，该方法在多类知识获取任务上性能更优且能保留原有知识。
摘要：
大语言模型（LLM）由于采用一次性训练方式，且世界处于不断变化之中，往往难以提供最新信息。为使LLM保持知识时效性，现有方法通常会基于新文档开展持续预训练，但这类方法常面临存储知识提取困难的问题。受费曼技巧在人类高效学习中取得显著成功的启发，我们提出Self-Tuning学习框架，旨在提升LLM通过自教学从未见过的原始文档中有效获取新知识的能力。具体而言，我们设计了自教学策略，通过自监督方式生成一组知识密集型任务对文档进行增强，重点覆盖记忆、理解和自我反思三个关键维度。此外，我们构建了三个Wiki-Newpages-2023-QA数据集，用于深入分析LLM在记忆、提取和推理方面的知识获取能力。在Llama2-7B等多种模型上开展的大量实验结果表明，Self-Tuning在所有知识获取任务中始终表现出更优性能，且在保留原有知识方面表现出色。
英文摘要：
Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training and the constantly evolving nature of the world. To keep LLMs current, existing approaches typically involve continued pre-training on new documents. However, they frequently face difficulties in extracting stored knowledge. Motivated by the remarkable success of the Feynman Technique in efficient human learning, we introduce Self-Tuning, a learning framework aimed at improving an LLM's ability to effectively acquire new knowledge from unseen raw documents through self-teaching. Specifically, we develop a Self-Teaching strategy that augments the documents with a set of knowledge-intensive tasks created in a self-supervised manner, focusing on three crucial aspects: memorization, comprehension, and self-reflection. Additionally, we introduce three Wiki-Newpages-2023-QA datasets to facilitate an in-depth analysis of an LLM's knowledge acquisition ability concerning memorization, extraction, and reasoning. Extensive experimental results on various models, e.g., Llama2-7B reveal that Self-Tuning consistently exhibits superior performance across all knowledge acquisition tasks and excels in preserving previous knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
