作为R或Tetrad程序中的标准实现,科学家最广泛或最有效地使用的因果搜索算法具有严重的维度限制,使其不适合大数据问题,而不会牺牲准确性。然而,执行方面的改进是可能的。我们探索贪婪等价搜索的优化,允许在13分钟内搜索50,000个变量的问题,在四个处理器,16 G膝上计算机上的1000个样本的稀疏模型。我们在匹兹堡超级计算中心的超级计算机节点上用18小时完成了1000个样本对1,000,000个变量的稀疏模型的问题,该节点具有40个处理器和384 G RAM。同样的算法可以应用于离散数据,但离散得分较慢,尽管离散实现目前在我们的实验中没有扩展;我们已经设法在具有1000个样本的稀疏模型中扩展到约10,000个变量。
英文摘要:
As standardly implemented in R or the Tetrad program, causal search algorithms used most widely or effectively by scientists have severe dimensionality constraints that make them inappropriate for big data problems without sacrificing accuracy. However, implementation improvements are possible. We explore optimizations for the Greedy Equivalence Search that allow search on 50,000-variable problems in 13 minutes for sparse models with 1000 samples on a four-processor, 16G laptop computer. We finish a problem with 1000 samples on 1,000,000 variables in 18 hours for sparse models on a supercomputer node at the Pittsburgh Supercomputing Center with 40 processors and 384 G RAM. The same algorithm can be applied to discrete data, with a slower discrete score, though the discrete implementation currently does not scale as well in our experiments; we have managed to scale up to about 10,000 variables in sparse models with 1000 samples.
Department of Philosophy(哲学系);Carnegie Mellon University(卡内基梅隆大学);Department of Psychiatry & Behavioral Sciences(精神病学与行为科学系);University of Minnesota(明尼苏达大学)
Reply to: Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent set
Amazon Quantum Solutions Lab(亚马逊量子解决方案实验室);AWS Center for Quantum Computing(AWS量子计算中心)