自聚焦：通过多级聚焦从无监督文本中增强合成指令的多样性和难度
Self-Foveate: Enhancing Diversity and Difficulty of Synthesized Instructions from Unsupervised Text via Multi-Level Foveation
AI总结：
Self-Foveate通过多级聚焦方法提升无监督文本合成指令的多样性和难度，采用细粒度到整体模式的提取策略，并结合重合成模块提高指令质量。
摘要：
从无监督文本中合成高质量的指令数据是训练大语言模型（LLMs）的一种有前景的范式，然而针对这一任务的自动化方法仍然在合成指令的多样性和难度方面存在显著局限。为了解决这些挑战，我们提出了Self-Foveate，一种由LLM驱动的指令合成方法。受人类视觉感知的层次结构启发，Self-Foveate引入了一种“微散射-宏”多级聚焦方法，该方法引导在三个互补的粒度层级上提取文本信息，从细粒度细节通过跨区域连接到整体模式，从而增强合成指令的多样性和难度。此外，还加入了一个重合成模块，以提高指令对源文本的忠实度及其整体质量。在多个无监督语料库和多种模型架构上的全面实验表明，Self-Foveate在各方面均优于现有方法。我们公开发布了代码，网址为https://github.com/Mubuky/Self-Foveate
英文摘要：
Synthesizing high-quality instruction data from unsupervised text is a promising paradigm for training large language models (LLMs), yet automated methods for this task still exhibit significant limitations in the diversity and difficulty of synthesized instructions. To address these challenges, we propose Self-Foveate, an LLM-driven method for instruction synthesis. Inspired by hierarchical human visual perception, Self-Foveate introduces a "Micro-Scatter-Macro" multi-level foveation methodology that guides the extraction of textual information at three complementary granularities, from fine-grained details through cross-region connections to holistic patterns, thereby enhancing both the diversity and difficulty of synthesized instructions. Furthermore, a re-synthesis module is incorporated to improve the fidelity of instructions to source text and their overall quality. Comprehensive experiments across multiple unsupervised corpora and diverse model architectures demonstrate that Self-Foveate consistently outperforms existing methods. We publicly release our code at https://github.com/Mubuky/Self-Foveate
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
