Dynamic networks, especially those representing social networks, undergo constant evolution of their community structure over time. Nodes can migrate between different communities, communities can split into multiple new communities, communities can merge together, etc. In order to represent dynamic networks with evolving communities it is essential to use a dynamic model rather than a static one. Here we use a dynamic stochastic block model where the underlying block model is different at different times. In order to represent the structural changes expressed by this dynamic model the network will be split into discrete time segments and a clustering algorithm will assign block memberships for each segment. In this paper we show that using an ensemble of clustering assignments accommodates for the variance in scalable clustering algorithms and produces superior results in terms of pairwise-precision and pairwise-recall. We also demonstrate that the dynamic clustering produced by the ensemble can be visualized as a flowchart which encapsulates the community evolution succinctly.
Though much work has been done on ensemble clustering in data mining, the application of ensemble methods to community detection in networks is in its infancy. In this paper, we propose two ensemble methods: ENDISCO and MEDOC. ENDISCO performs disjoint community detection. In contrast, MEDOC performs disjoint, overlapping, and fuzzy community detection and represents the first ever ensemble method for fuzzy and overlapping community detection. We run extensive experiments with both algorithms against both synthetic and several real-world datasets for which community structures are known. We sh…
University of Houston（休斯顿大学）；Texas Center for Superconductivity, University of Houston（休斯顿大学德克萨斯超导中心）
We introduce an ensemble learning scheme for community detection in complex networks. The scheme uses a Machine Learning algorithmic paradigm we call Extremal Ensemble Learning. It uses iterative extremal updating of an ensemble of network partitions, which can be found by a conventional base algorithm, to find a node partition that maximizes modularity. At each iteration, core groups of nodes that are in the same community in every ensemble partition are identified and used to form a reduced network. Partitions of the reduced network are then found and used to update the ensemble. The smaller…
Social media has revolutionized human communication and styles of interaction. Due to its easiness and effective medium, people share and exchange information, carry out discussion on various events, and express their opinions. For effective policy making and understanding the response of a community on different events, we need to monitor and analyze the social media. In social media, there are some users who are more influential, for example, a famous politician may have more influence than a common person. These influential users belong to specific communities. The main object of this resea…
Recognizing number of communities and detecting community structures of complex network are discussed in this paper. As a visual and feasible algorithm, block model has been successfully applied to detect community structures in complex network. In order to measure the quality of the block model, we first define an objective function WQ value. For obtaining block model B of a network, GSA algorithm is applied to optimize WQ with the help of random keys. After executing processes AO (Adding Ones) and RO (Removing Ones) on block model B, the number of communities of a network can be recognized d…
While there has been a plethora of approaches for detecting disjoint communities from real-world complex networks, some methods for detecting overlapping community structures have also been recently proposed. In this work, we argue that, instead of developing separate approaches for detecting overlapping communities, a promising alternative is to infer the overlapping communities from multiple disjoint community structures. We propose an ensemble-based approach, called EnCoD, that leverages the solutions produced by various disjoint community detection algorithms to discover the overlapping co…
We propose a novel method to test the existence of community structure of undirected real-valued edge-weighted graph. The method is based on Wigner semicircular law on the asymptotic behavior of the random distribution for eigenvalues of a real symmetric matrix. We provide a theoretical foundation for this method and report on its performance in synthetic and real data, suggesting that our method outperforms other state-of-the-art methods.
Over the last couple of decades, Social Networks have connected people on the web from across the globe and have become a crucial part of our daily life. These networks have also rapidly grown as platforms for propagating products, ideas, and opinions to target a wider audience. This calls for the need to find influential nodes in a network for a variety of reasons, including the curb of misinformation being spread across the networks, advertising products efficiently, finding prominent protein structures in biological networks, etc. In this paper, we propose Modified Community Diversity (MCD)…
Given a set ${\cal AL}$ of community detection algorithms and a graph $G$ as inputs, we propose two ensemble methods $\mathtt{EnDisCO}$ and $\mathtt{MeDOC}$ that (respectively) identify disjoint and overlapping communities in $G$. $\mathtt{EnDisCO}$ transforms a graph into a latent feature space by leveraging multiple base solutions and discovers disjoint community structure. $\mathtt{MeDOC}$ groups similar base communities into a meta-community and detects both disjoint and overlapping community structures. Experiments are conducted at different scales on both synthetically generated networks…
Peer-to-peer (P2P) botnets have become one of the major threats in network security for serving as the infrastructure that responsible for various of cyber-crimes. Though a few existing work claimed to detect traditional botnets effectively, the problem of detecting P2P botnets involves more challenges. In this paper, we present PeerHunter, a community behavior analysis based method, which is capable of detecting botnets that communicate via a P2P structure. PeerHunter starts from a P2P hosts detection component. Then, it uses mutual contacts as the main feature to cluster bots into communitie…
Revealing the community structure exhibited by real networks is a fundamental phase towards a comprehensive understanding of complex systems beyond the local organization of their components. Community detection techniques help on providing insights into understanding the local organization of the components of networks. In this study we identify and investigate the overlapping community structure of an interesting and unique case of study: the Estonian network of payments. In order to perform the study, we use the Clique Percolation Method and explore statistical distribution functions of the…
Community detection is a key data analysis problem across different fields. During the past decades, numerous algorithms have been proposed to address this issue. However, most work on community detection does not address the issue of statistical significance. Although some research efforts have been made towards mining statistically significant communities, deriving an analytical solution of p-value for one community under the configuration model is still a challenging mission that remains unsolved. The configuration model is a widely used random graph model in community detection, in which t…
Core-periphery structure and community structure are two typical meso-scale structures in complex networks. Though the community detection has been extensively investigated from different perspectives, the definition and the detection of core-periphery structure have not received much attention. Furthermore, the detection problems of the core-periphery and community structure were separately investigated. In this paper, we develop a unified framework to simultaneously detect core-periphery structure and community structure in complex networks. Moreover, there are several extra advantages of ou…
Detecting community structure in social networks is a fundamental problem empowering us to identify groups of actors with similar interests. There have been extensive works focusing on finding communities in static networks, however, in reality, due to dynamic nature of social networks, they are evolving continuously. Ignoring the dynamic aspect of social networks, neither allows us to capture evolutionary behavior of the network nor to predict the future status of individuals. Aside from being dynamic, another significant characteristic of real-world social networks is the presence of leaders…
Multilayer networks capture pairwise relationships between the components of complex systems across multiple modes or scales of interactions. An important meso-scale feature of these networks is measured though their community structure, which defines groups of strongly connected nodes that exist within and across network layers. Because interlayer edges can describe relationships between different modalities, scales, or time points, it is essential to understand how communities change and evolve across layers. A popular method for detecting communities in multilayer networks consists of maxim…
One key challenge in Social Network Analysis is to design an efficient and accurate community detection procedure as a means to discover intrinsic structures and extract relevant information. In this paper, we introduce a novel strategy called (COIN), which exploits COncept INterestingness measures to detect communities based on the concept lattice construction of the network. Thus, unlike off-the-shelf community detection algorithms, COIN leverages relevant conceptual characteristics inherited from Formal Concept Analysis to discover substantial local structures. On the first stage of COIN, w…
Tutte Institute for Mathematics and Computing（塔特数学与计算研究所）
We recently proposed a new ensemble clustering algorithm for graphs (ECG) based on the concept of consensus clustering. We validated our approach by replicating a study comparing graph clustering algorithms over benchmark graphs, showing that ECG outperforms the leading algorithms. In this paper, we extend our comparison by considering a wider range of parameters for the benchmark, generating graphs with different properties. We provide new experimental results showing that the ECG algorithm alleviates the well-known resolution limit issue, and that it leads to better stability of the partitio…
How to detect a small community in a large network is an interesting problem, including clique detection as a special case, where a naive degree-based $χ^2$-test was shown to be powerful in the presence of an Erdős-Renyi background. Using Sinkhorn's theorem, we show that the signal captured by the $χ^2$-test may be a modeling artifact, and it may disappear once we replace the Erdős-Renyi model by a broader network model. We show that the recent SgnQ test is more appropriate for such a setting. The test is optimal in detecting communities with sizes comparable to the whole network, but has neve…
Peer-to-peer (P2P) botnets have become one of the major threats in network security for serving as the fundamental infrastructure for various cyber-crimes. More challenges are involved in the problem of detecting P2P botnets, despite a few work claimed to detect centralized botnets effectively. We propose Enhanced PeerHunter, a network-flow level community behavior analysis based system, to detect P2P botnets. Our system starts from a P2P network flow detection component. Then, it uses "mutual contacts" to cluster bots into communities. Finally, it uses network-flow level community behavior an…
Community detection on social media has attracted considerable attention for many years. However, existing methods do not reveal the relations between communities. Communities can form alliances or engage in antagonisms due to various factors, e.g., shared or conflicting goals and values. Uncovering such relations can provide better insights to understand communities and the structure of social media. According to social science findings, the attitudes that members from different communities express towards each other are largely shaped by their community membership. Hence, we hypothesize that…
Network communities represent mesoscopic structure for understanding the organization of real-world networks, where nodes often belong to multiple communities and form overlapping community structure in the network. Due to non-triviality in finding the exact boundary of such overlapping communities, this problem has become challenging, and therefore huge effort has been devoted to detect overlapping communities from the network. In this paper, we present PVOC (Permanence based Vertex-replication algorithm for Overlapping Community detection), a two-stage framework to detect overlapping communi…
We consider the algorithmic problem of community detection in networks. Given an undirected friendship graph $G=\left(V,E\right)$, a subset $S\subseteq V$ is an $\left(α,β\right)$-community if: * Every member of the community is friends with an $α$-fraction of the community; * Every non-member is friends with at most a $β$-fraction of the community. Arora et al [AGSS12] gave a quasi-polynomial time algorithm for enumerating all the $\left(α,β\right)$-communities for any constants $α>β$. Here, we prove that, assuming the Exponential Time Hypothesis (ETH), quasi-polynomial time is in fact necess…
Link streams model interactions over time in a wide range of fields. Under this model, the challenge is to mine efficiently both temporal and topological structures. Community detection and change point detection are one of the most powerful tools to analyze such evolving interactions. In this paper, we build on both to detect stable community structures by identifying change points within meaningful communities. Unlike existing dynamic community detection algorithms, the proposed method is able to discover stable communities efficiently at multiple temporal scales. We test the effectiveness o…
An important aspect of preventing fake news dissemination is to proactively detect the likelihood of its spreading. Research in the domain of fake news spreader detection has not been explored much from a network analysis perspective. In this paper, we propose a graph neural network based approach to identify nodes that are likely to become spreaders of false information. Using the community health assessment model and interpersonal trust we propose an inductive representation learning framework to predict nodes of densely-connected community structures that are most likely to spread fake news…
Huawei Noah's Ark Lab（华为诺亚方舟实验室）；University of South Florida（南佛罗里达大学）；University of Michigan（密歇根大学）；GIPSA-lab, University Grenoble-Alpes（格勒诺布尔-阿尔卑斯大学GIPSA实验室）；MIT-IBM Watson AI Lab, IBM Research（IBM研究院MIT-IBM沃森人工智能实验室）
We propose a method for simultaneously detecting shared and unshared communities in heterogeneous multilayer weighted and undirected networks. The multilayer network is assumed to follow a generative probabilistic model that takes into account the similarities and dissimilarities between the communities. We make use of a variational Bayes approach for jointly inferring the shared and unshared hidden communities from multilayer network observations. We show that our approach outperforms state-of-the-art algorithms in detecting disparate (shared and private) communities on synthetic data as well…
Community detection (CD) algorithms are applied to Hi-C data to discover new communities of loci in the 3D conformation of human and mouse DNA. We find that CD has some distinct advantages over pre-existing methods: (1) it is capable of finding a variable number of communities, (2) it can detect communities of DNA loci either adjacent or distant in the 1D sequence, and (3) it allows us to obtain a principled value of k, the number of communities present. Forcing k = 2, our method recovers earlier findings of Lieberman-Aiden, et al. (2009), but letting k be a parameter, our method obtains as op…
We present a new online algorithm for detecting overlapping communities. The main ingredients are a modification of an online k-means algorithm and a new approach to modelling overlap in communities. An evaluation on large benchmark graphs shows that the quality of discovered communities compares favorably to several methods in the recent literature, while the running time is significantly improved.
Nowadays, identification and detection community structures in complex networks is an important factor in extracting useful information from networks. Label propagation algorithm with near linear-time complexity is one of the most popular methods for detecting community structures, yet its uncertainty and randomness is a defective factor. Merging LPA with other community detection metrics would improve its accuracy and reduce instability of LPA. Considering this point, in this paper we tried to use edge betweenness centrality to improve LPA performance. On the other hand, calculating edge betw…
The University of New South Wales（新南威尔士大学）；Shanghai Jiao Tong University（上海交通大学）；Antai College of Economics & Management, Shanghai Jiao Tong University（上海交通大学安泰经济与管理学院）；Zhejiang Gongshang University（浙江工商大学）
The study of time-varying (dynamic) networks (graphs) is of fundamental importance for computer network analytics. Several methods have been proposed to detect the effect of significant structural changes in a time series of graphs. The main contribution of this work is a detailed analysis of a dynamic community graph model. This model is formed by adding new vertices, and randomly attaching them to the existing nodes. It is a dynamic extension of the well-known stochastic blockmodel. The goal of the work is to detect the time at which the graph dynamics switches from a normal evolution -- whe…
The clustering ensemble paradigm has emerged as an effective tool for community detection in multilayer networks, which allows for producing consensus solutions that are designed to be more robust to the algorithmic selection and configuration bias. However, one limitation is related to the dependency on a co-association threshold that controls the degree of consensus in the community structure solution. The goal of this work is to overcome this limitation with a new framework of ensemble-based multilayer community detection, which features parameter-free identification of consensus communitie…
Community structure is one of the most prominent features of complex networks. Community structure detection is of great importance to provide insights into the network structure and functionalities. Most proposals focus on static networks. However, finding communities in a dynamic network is even more challenging, especially when communities overlap with each other. In this article , we present an online algorithm, called OLCPM, based on clique percolation and label propagation methods. OLCPM can detect overlapping communities and works on temporal networks with a fine granularity. By locally…
Comprehensive, Efficient Large-Scale Community Detection via Structural Entropy Game
Kunming University of Science and Technology（昆明理工大学）；Yunnan Key Laboratory of Artificial Intelligence（云南省人工智能重点实验室）；Beihang University（北京航空航天大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
National Research University Higher School of Economics（国立研究大学高等经济学院）；University of Thessaloniki（塞萨洛尼基大学）；Intel Corp（英特尔公司）
In this paper, we present a new method for detecting overlapping communities in networks with a predefined number of clusters called LPAM (Link Partitioning Around Medoids). The overlapping communities in the graph are obtained by detecting the disjoint communities in the associated line graph employing link partitioning and partitioning around medoids which are done through the use of a distance function defined on the set of nodes. We consider both the commute distance and amplified commute distance as distance functions. The performance of the LPAM method is evaluated with computational exp…
Maintaining the integrity of long-term data collection is an essential scientific practice. As a field evolves, so too will that field's measurement instruments and data storage systems, as they are invented, improved upon, and made obsolete. For data streams generated by opaque sociotechnical systems which may have episodic and unknown internal rule changes, detecting and accounting for shifts in historical datasets requires vigilance and creative analysis. Here, we show that around 10\% of day-scale word usage frequency time series for Twitter collected in real time for a set of roughly 10,0…
Complex networks usually expose community structure with groups of nodes sharing many links with the other nodes in the same group and relatively few with the nodes of the rest. This feature captures valuable information about the organization and even the evolution of the network. Over the last decade, a great number of algorithms for community detection have been proposed to deal with the increasingly complex networks. However, the problem of doing this in a private manner is rarely considered. In this paper, we solve this problem under differential privacy, a prominent privacy concept for r…
Tutte Institute for Mathematics and Computing（塔特数学与计算研究所）
We propose an ensemble clustering algorithm for graphs (ECG), which is based on the Louvain algorithm and the concept of consensus clustering. We validate our approach by replicating a recently published study comparing graph clustering algorithms over artificial networks, showing that ECG outperforms the leading algorithms from that study. We also illustrate how the ensemble obtained with ECG can be used to quantify the presence of community structure in the graph.
Sichuan University（四川大学）；Cyber Science Research Institute, Sichuan University（四川大学网络空间安全研究院）
The pervasiveness of offensive language on the social network has caused adverse effects on society, such as abusive behavior online. It is urgent to detect offensive language and curb its spread. Existing research shows that methods with community structure features effectively improve the performance of offensive language detection. However, the existing models deal with community structure independently, which seriously affects the effectiveness of detection models. In this paper, we propose an end-to-end method based on community structure and text features for offensive language detection…
University of California, Irvine（加州大学欧文分校）；University of Southern California（南加州大学）
Community detection in graphs has been extensively studied both in theory and in applications. However, detecting communities in hypergraphs is more challenging. In this paper, we propose a tensor decomposition approach for guaranteed learning of communities in a special class of hypergraphs modeling social tagging systems or folksonomies. A folksonomy is a tripartite 3-uniform hypergraph consisting of (user, tag, resource) hyperedges. We posit a probabilistic mixed membership community model, and prove that the tensor method consistently learns the communities under efficient sample complexit…
Detection of non-overlapping and overlapping communities are essentially the same problem. However, current algorithms focus either on finding overlapping or non-overlapping communities. We present a generalized framework that can identify both non-overlapping and overlapping communities, without any prior input about the network or its community distribution. To do so, we introduce a vertex-based metric, GenPerm, that quantifies by how much a vertex belongs to each of its constituent communities. Our community detection algorithm is based on maximizing the GenPerm over all the vertices in the…
