arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.14062cs.AIcs.CL
收藏

知何时该收手:通过多阶段飞行拒绝实现令牌高效的大语言模型合成数据生成

Know When To Fold 'Em: Token-Efficient LLM Synthetic Data Generation via Multi-Stage In-Flight Rejection

Department of Computer Science University of Houston(计算机科学系休斯顿大学);IBM Research(IBM研究院)

Anjir Ahmed Chowdhury, Syed Zawad, Feng Yan

AI总结:

本文提出MSIFR框架,通过在生成过程中早期检测低质量轨迹以减少令牌浪费,提升合成数据生成效率,实验表明其在多个模型和基准上显著降低令牌消耗并保持准确性。

摘要:

尽管大型语言模型(LLMs)在合成数据生成中被广泛用于训练后流程,现有方法通常在应用质量过滤器前生成完整输出,导致大量令牌被浪费在最终被丢弃的样本上。为解决此问题,我们提出了多阶段飞行拒绝(MSIFR),一种轻量级、无需训练的框架,能够在生成过程中中间检查点检测并终止低质量生成轨迹。MSIFR将生成过程分解为连续阶段,并应用快速基于规则的验证器来识别算术不一致、幻觉模式和格式违规,从而实现早期拒绝故障样本。我们正式将飞行拒绝定义为一个顺序决策过程,并证明任何非平凡的丢弃策略都会减少预期令牌消耗,当拒绝发生在生成管道更早的阶段时,阶段间的节省会增加。我们进一步证明条件效用估计形成一个马尔可夫链,确保早期飞行拒绝不会偏移保留样本的预期效用。在五个指令微调模型和七个推理基准上,MSIFR作为独立方法减少了11%-77%的令牌消耗,当与早期退出方法结合时,可高达78.2%,同时保持或提高评估准确性。这些结果证实了MSIFR为改进LLM合成数据生成效率提供了一种实用机制,无需额外训练或架构更改。

英文摘要:

While synthetic data generation with large language models (LLMs) is widely used in post-training pipelines, existing approaches typically generate full outputs before applying quality filters, leading to substantial token waste on samples that are ultimately discarded. To address this, we propose Multi-Stage In-Flight Rejection (MSIFR), a lightweight, training-free framework that detects and terminates low-quality generation trajectories at intermediate checkpoints before they reach full completion. MSIFR decomposes the generation process into sequential stages and applies fast rule-based validators to identify arithmetic inconsistencies, hallucination patterns, and formatting violations, enabling early rejection of faulty samples. We formalize in-flight rejection as a sequential decision process and show that any non-trivial discard policy reduces expected token consumption, with stage-wise savings increasing when rejection occurs earlier in the generation pipeline. We further demonstrate that conditional utility estimates form a martingale, ensuring that early, in-flight rejection does not bias the expected utility of retained samples. Across five instruction-tuned models and seven reasoning benchmarks, MSIFR reduces token consumption by 11%-77% as a standalone method, and up to 78.2% when combined with early-exit methods, while preserving or improving evaluation accuracy. These results confirm that MSIFR provides a practical mechanism for improving the efficiency of LLM-based synthetic data generation without additional training or architectural changes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑