arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.07644cs.FLcs.DSmath.PR
收藏

期望编辑距离的可计算界与蒙特卡洛估计

Computable Bounds and Monte Carlo Estimates of the Expected Edit Distance

Gianfranco Bilardi, Michele Schimd

AI总结:

针对随机字符串期望归一化编辑距离及其极限难精确计算的问题,本文给出可计算界、精确算法、高置信蒙特卡洛估计和渐近趋紧的组合下界,并改进及补充了多组数值结果。

摘要:

编辑距离是衡量字符串之间不相似程度的一种度量,广泛应用于计算生物学、语音识别和机器学习。设 $e_k(n)$ 表示从大小为 $k$ 的字母表中独立随机抽取的两个长度为 $n$ 的字符串之间的平均编辑距离。对于 $k \geq 2$,如何高效计算 $α_{k}(n)=e_k(n)/n$ 以及 $α_{k}=\lim_{n \to \infty}α_{k}(n)$(已知该极限存在)的精确值,仍是一个开放问题。本文证明了 $α_k(n)-Q(n)\leq α_k\leq α_k(n)$,其中特定的 $Q(n)=Θ(\sqrt{\log n/n})$,该结果意味着 $α_k$ 是可计算的。文中探讨了 $α_k(n)$ 的精确计算,给出了一个运行时间为 $T=\mathcal{O}(n^2k\min(3^n,k^n))$ 的算法,但这一复杂度使其实际用途有限。本文还基于 McDiarmid 不等式分析了统计估计,说明如何在良好精度、高置信水平和合理计算时间下评估 $α_k(n)$,例如 $n$ 最高可达 25 万。相应地,文中得到了 $α_k$ 的宽度约为 $10^{-2}$ 的 99.9% 置信区间。作者利用关于编辑脚本的组合论证,解析刻画了一个可高效计算的 $α_k$ 下界 $β_k^*$,满足 $\lim_{k\to\infty}β_k^*=1$。一般而言,$β_k^*\leq α_k\leq 1-1/k$;当 $k$ 大于几十时,计算 $β_k^*$ 远比生成宽度为 $1-1/k-β_k^*$ 的带置信区间的良好统计估计更快。本文发展的技术改进了此前发表的大多数数值结果,并给出了以往未曾报告的字母表大小和字符串长度的结果。

英文摘要:

The edit distance is a metric of dissimilarity between strings, widely applied in computational biology, speech recognition, and machine learning. Let $e_k(n)$ denote the average edit distance between random, independent strings of $n$ characters from an alphabet of size $k$. For $k \geq 2$, it is an open problem how to efficiently compute the exact value of $α_{k}(n) = e_k(n)/n$ as well as of $α_{k} = \lim_{n \to \infty} α_{k}(n)$, a limit known to exist. This paper shows that $α_k(n)-Q(n) \leq α_k \leq α_k(n)$, for a specific $Q(n)=Θ(\sqrt{\log n / n})$, a result which implies that $α_k$ is computable. The exact computation of $α_k(n)$ is explored, leading to an algorithm running in time $T=\mathcal{O}(n^2k\min(3^n,k^n))$, a complexity that makes it of limited practical use. An analysis of statistical estimates is proposed, based on McDiarmid's inequality, showing how $α_k(n)$ can be evaluated with good accuracy, high confidence level, and reasonable computation time, for values of $n$ say up to a quarter million. Correspondingly, 99.9\% confidence intervals of width approximately $10^{-2}$ are obtained for $α_k$. Combinatorial arguments on edit scripts are exploited to analytically characterize an efficiently computable lower bound $β_k^*$ to $α_k$, such that $ \lim_{k \to \infty} β_k^*=1$. In general, $β_k^* \leq α_k \leq 1-1/k$; for $k$ greater than a few dozens, computing $β_k^*$ is much faster than generating good statistical estimates with confidence intervals of width $1-1/k-β_k^*$. The techniques developed in the paper yield improvements on most previously published numerical values as well as results for alphabet sizes and string lengths not reported before.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑