arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

简单的密集语言生成:确定性、随机化与多阶算法

Dense Language Generation Made Simple: Deterministic, Randomized, and Multi-Order Algorithms

Ziyi Cai, Shuangping Li, Yiheng Shen, Kangning Wang, Peng Zhang

arXiv 2608.01320首次发表:更新:

发表机构

Rutgers University; Yale University; Meta(罗格斯大学; 耶鲁大学; Meta)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本文提出简单统一框架,证明确定性算法的最优lower density保证为1/2,随机化可提升至1-1/e,且能同时适配多种重要性顺序的最优保证。

AI 中文摘要

极限语言生成是一种理论框架,用于研究生成器如何从正例流中学习生成新的有效字符串。在该模型中,对手从可数族中选择一种未知语言,并以任意顺序枚举其元素,而生成器最终必须仅输出尚未出现在枚举中的该语言元素。因此,可靠生成通过两个最终保证形式化:相对于观测数据的有效性和新颖性。为进一步量化生成器输出的广度,Kleinberg和Wei(FOCS 2025、STOC 2026)引入了 lower density(输出覆盖度)作为衡量指标。给定表示可能输出重要性或相关性的顺序,lower density 是当 n 增长时,生成器在目标语言前 n 个元素出现在数据中之前所输出的元素占该 n 个元素的渐近下界。Kleinberg和Wei证明确定性算法的最优 lower density 保证为 1/2。本文开发了一种简单且统一的框架以获取最优 lower density 保证:首先,提出一种确定性算法,其能达到 1/2 的最优保证,且分析比现有工作简单得多;随后,通过两个扩展展示该框架的灵活性:第一,针对无知对手,随机化将最优保证提升至 1-1/e;第二,对于任意有限的顺序集合,可同时实现每个顺序对应的最优确定性和随机化保证,因此适配多种重要性或相关性概念不会损失最优保证。

英文摘要

Language generation in the limit is a theoretical framework for studying how a generator can learn to produce new valid strings from a stream of positive examples. In this model, an adversary chooses an unknown language from a countable family and enumerates its elements in an arbitrary order, while the generator must eventually output only elements of the language that have not yet appeared in the enumeration. Reliable generation is thus formalized through two eventual guarantees: validity and novelty relative to the observed data. To further quantify the breadth of the generator's outputs, Kleinberg and Wei (FOCS 2025, STOC 2026) introduced lower density as a measure of output coverage. Given an order representing the importance or relevance of possible outputs, lower density is the asymptotic lower bound, as $n$ grows, on the fraction of the first $n$ elements of the target language that the generator outputs before they appear in the data. Kleinberg and Wei showed that $1/2$ is the optimal lower-density guarantee for deterministic algorithms. We develop a simple and unified framework for obtaining optimal lower-density guarantees. We first give a deterministic algorithm that recovers the optimal guarantee of $1/2$ with a significantly simpler analysis than prior work. We then demonstrate the flexibility of our framework through two extensions. First, against an oblivious adversary, randomization raises the optimal guarantee to $1-1/e$. Second, for any finite collection of orders, the optimal deterministic and randomized guarantees can be achieved simultaneously with respect to every order, so accommodating multiple notions of importance or relevance entails no loss in the optimal guarantee.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑