语言模型中用于稀有事件估计的加权集成采样
Weighted Ensemble Sampling for Rare-Event Estimation in Language Models
AI总结:
该研究针对语言模型稀有事件概率估计效率低的问题,提出无训练的加权集成(WE)框架,利用LLM内部表示分配采样预算,经实验验证可提升估计的效率与可靠性。
摘要:
大型语言模型的罕见但严重的故障在部署规模下会变得具有实际重要性。使用直接蒙特卡洛采样估计其概率效率极低,而基于提议的方法通常需要额外学习一个提议分布。我们提出了加权集成(Weighted Ensemble, WE),这是一种用于语言模型稀有事件估计的无训练框架。WE仅从原始语言模型中采样,并通过定期将采样预算重新分配给更可能达到目标稀有事件的前缀,改进了直接蒙特卡洛方法。为了在不学习单独提议分布的情况下指导这种重新分配,我们利用大型语言模型(LLM)的内部表示作为前缀级信号,并通过实验验证了其有效性。我们对所提出的算法进行了理论分析。在涉及安全相关任务的实验中,涵盖了小规模和现代大规模LLM,结果表明WE提高了稀有事件概率估计的效率和可靠性。
英文摘要:
Rare but severe failures of large language models can become practically significant at deployment scale. Estimating their probabilities with direct Monte Carlo sampling is highly inefficient, while proposal-based methods typically require auxiliary learning of an additional proposal distribution. We introduce \textbf{Weighted Ensemble (WE)}, a training-free framework for rare-event estimation in language models. WE samples exclusively from the original language model and improves upon direct Monte Carlo by periodically reallocating the sampling budget toward prefixes that appear more likely to reach the target rare event. To guide this reallocation without learning a separate proposal distribution, we exploit internal LLM representations as prefix-level signals and empirically validate their effectiveness. We provide a theoretical analysis of the proposed algorithm. Experiments on safety-related tasks spanning both small-scale and modern large-scale LLMs demonstrate that WE improves the efficiency and reliability of rare-event probability estimation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。