In the process of knowledge discovery and representation in large datasets using formal concept analysis, complexity plays a major role in identifying all the formal concepts and constructing the concept lattice(digraph of the concepts). For identifying the formal concepts and constructing the digraph from the identified concepts in very large datasets, various distributed algorithms are available in the literature. However, the existing distributed algorithms are not very well suitable for concept generation because it is an iterative process. The existing algorithms are implemented using distributed frameworks like MapReduce and Open MP, these frameworks are not appropriate for iterative applications. Hence, in this paper we proposed efficient distributed algorithms for both formal concept generation and concept lattice digraph construction in large formal contexts using Apache Spark. Various performance metrics are considered for the evaluation of the proposed work, the results of the evaluation proves that the proposed algorithms are efficient for concept generation and lattice graph construction in comparison with the existing algorithms.
Scalable Exact Parent Sets Identification in Bayesian Networks Learning with Apache Spark
Department of Computer Science and Engineering University at Buffalo(布法罗大学计算机科学与工程系);Department of Computer Science and Engineering Department of Biomedical Informatics University at Buffalo(布法罗大学生物医学信息学系计算机科学与工程系(注:此处按原文机构名合并,实际通常以主要系部为准,原文同时列出两个系部,均需提取));University at Buffalo(布法罗大学(注:经检查,原HTML中作者的Affiliation部分明确包含的机构单位为“Department of Computer Science and Engineering University at Buffalo”和“Department of Computer Science and Engineering Department of Biomedical Informatics University at Buffalo”,但根据提取规则,机构粒度到大学/学院/公司/研究院级别即可,此处“University at Buffalo”为大学,“Department of Computer Science and Engineering”为系部,“Department of Biomedical Informatics”为系部。按要求提取到大学/学院/公司/研究院级别,因此主要机构为“University at Buffalo”,但原文中作者的Affiliation完整内容包含多个系部和大学,需严格按原文提取作者所属机构的英文名称,中文翻译需自然准确。正确提取应为:{)
LLM-based Automated Theorem Proving Hinges on Scalable Synthetic Data Generation
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术全国重点实验室);School of Computing and Mathematical Sciences, Birkbeck, University of London(伦敦大学伯贝克学院计算与数学科学学院)
Lean Meets Theoretical Computer Science: Scalable Synthesis of Theorem Proving Challenges in Formal-Informal Pairs
D-CHAB, ETH Zurich, Zurich, Switzerland.;D-INFK, ETH Zurich, Zurich, Switzerland.;ETH AI Center, Zurich, Switzerland.;University of Pennsylvania, PA, USA.;Independent Researcher.