ReaderLM-v2:用于HTML转Markdown和JSON的小型语言模型
ReaderLM-v2: Small Language Model for HTML to Markdown and JSON
AI总结:
本文提出ReaderLM-v2,一个15亿参数的紧凑语言模型,通过三阶段数据合成流水线和统一训练框架,高效将HTML转为Markdown或JSON,在基准测试中超越GPT-4o等大模型15-20%,计算需求更低。
摘要:
我们提出了ReaderLM-v2,一个15亿参数的紧凑型语言模型,专为高效网页内容提取而设计。该模型可处理长达512K tokens的文档,将杂乱的HTML高精度地转换为整洁的Markdown或JSON格式,使其成为大型语言模型基础化的理想工具。该模型的有效性源于两项关键创新:(1)三阶段数据合成流水线,通过迭代起草、优化和审查网页内容提取过程,生成高质量、多样化的训练数据;(2)统一训练框架,结合了连续预训练与多目标优化。深入评估表明,ReaderLM-v2在精心策划的基准测试中,性能比GPT-4o-2024-08-06及其他更大模型高出15-20%,在处理超过100K tokens的文档时表现尤为出色,同时显著降低了计算需求。
英文摘要:
We present ReaderLM-v2, a compact 1.5 billion parameter language model designed for efficient web content extraction. Our model processes documents up to 512K tokens, transforming messy HTML into clean Markdown or JSON formats with high accuracy -- making it an ideal tool for grounding large language models. The model's effectiveness results from two key innovations: (1) a three-stage data synthesis pipeline that generates high quality, diverse training data by iteratively drafting, refining, and critiquing web content extraction; and (2) a unified training framework combining continuous pre-training with multi-objective optimization. Intensive evaluation demonstrates that ReaderLM-v2 outperforms GPT-4o-2024-08-06 and other larger models by 15-20\% on carefully curated benchmarks, particularly excelling at documents exceeding 100K tokens, while maintaining significantly lower computational requirements.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。