序列图的实现与语言模型中的歧义性
Sequence graphs realizations and ambiguity in language models
AI总结：
本文从组合与算法角度研究语言模型中序列图的可实现性与歧义性，针对不同窗口大小、方向性和权重设置，给出了多项式时间算法、XP算法及整数规划与动态规划求解方法，并证明了多种变体的计算困难性。
摘要：
几种流行的语言模型将输入文本 $x$ 中的局部上下文表示为词袋。这种表示自然地通过序列图来编码，其顶点是 $x$ 中出现的不同单词，边表示两个单词在大小为 $w$ 的滑动窗口内（有序）共现。然而，这种压缩表示通常不是双射的：某些可能具有歧义，允许作为序列的多种实现，而另一些可能不允许任何实现。在本文中，我们从组合和算法角度研究序列图的可实现性和歧义性。我们考虑在多种设置下序列图实现的存在性和枚举：窗口大小 $w$、图方向性的存在/缺失，以及权重的存在/缺失。当 $w=2$ 时，我们为所有情况提供了多项式时间的可实现性和枚举算法，除了无向/加权设置，我们证明了其枚举的 $\#$P-困难性。对于 $w \ge 3$，我们证明了所有变体的困难性，即使将 $w$ 视为常数，但无向无权情况是显著例外，我们为这两个问题提出了 XP 算法，由于相应的 $W[1]-$ 困难结果，该算法是紧致的。最后，我们提出了一个整数规划公式来解决可实现性问题，以及一个动态规划算法来解决中等规模实例的枚举问题。这项工作留下了这两个问题是否属于 NP 的开放问题，由于存在大小在实例编码上呈指数级的最小实现，这是一个非平凡的问题。
英文摘要：
Several popular language models represent local contexts in an input text $x$ as bags of words. Such representations are naturally encoded by a sequence graph whose vertices are the distinct words occurring in $x$, with edges representing the (ordered) co-occurrence of two words within a sliding window of size $w$. However, this compressed representation is not generally bijective: some may be ambiguous, admitting several realizations as a sequence, while others may not admit any realization. In this paper, we study the realizability and ambiguity of sequence graphs from a combinatorial and algorithmic point of view. We consider the existence and enumeration of realizations of a sequence graph under multiple settings: window size $w$, presence/absence of graph orientation, and presence/absence of weights (multiplicities). When $w=2$, we provide polynomial time algorithms for realizability and enumeration in all cases except the undirected/weighted setting, where we show the $\#$P-hardness of enumeration. For $w \ge 3$, we prove the hardness of all variants, even when $w$ is considered as a constant, with the notable exception of the undirected unweighted case for which we propose XP algorithms for both problems, tight due to a corresponding $W[1]-$hardness result. We conclude with an integer program formulation to solve the realizability problem, and a dynamic programming algorithm to solve the enumeration problem in instances of moderate sizes. This work leaves open the membership to NP of both problems, a non-trivial question due to the existence of minimum realizations having size exponential on the instance encoding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
