使用加权Dowker复合体的格式方言的统计检测
Statistical detection of format dialects using the weighted Dowker complex
摘要：
本文提供了一个实验验证，概率模型的文件行为时，消费的一组预先存在的解析器。文件行为是通过读取文件时产生的一组标准化的布尔“消息”来衡量的。通过阈值的后验概率，表现出一组特定的消息的文件是从一个特定的方言，我们的模型产生了一个实用的分类算法为两种方言。我们证明，这两种方言的阈值算法可以从主要由一种方言组成的训练集引导。一种方言的文件行为的（参数）理论和（非参数）经验分布都产生良好的分类性能，并且优于基于简单计数消息的分类。我们的理论框架依赖于每种方言中消息的统计独立性。违反这一假设是可检测的，并允许格式分析师识别方言之间的“边界”。因此，格式分析师可以大大减少他们在制定方言检测的新标准时需要考虑的文件数量，因为他们只需要考虑显示模糊消息模式的文件。
英文摘要：
This paper provides an experimentally validated, probabilistic model of file behavior when consumed by a set of pre-existing parsers. File behavior is measured by way of a standardized set of Boolean "messages" produced as the files are read. By thresholding the posterior probability that a file exhibiting a particular set of messages is from a particular dialect, our model yields a practical classification algorithm for two dialects. We demonstrate that this thresholding algorithm for two dialects can be bootstrapped from a training set consisting primarily of one dialect. Both the (parametric) theoretical and the (non-parametric) empirical distributions of file behaviors for one dialect yield good classification performance, and outperform classification based on simply counting messages. Our theoretical framework relies on statistical independence of messages within each dialect. Violations of this assumption are detectable and allow a format analyst to identify "boundaries" between dialects. A format analyst can therefore greatly reduce the number of files they need to consider when crafting new criteria for dialect detection, since they need only consider the files that exhibit ambiguous message patterns.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
