Potts model is a powerful tool to uncover community structure in complex networks. Here, we propose a new framework to reveal the optimal number of communities and stability of network structure by quantitatively analyzing the dynamics of Potts model. Specifically we model the community structure detection Potts procedure by a Markov process, which has a clear mathematical explanation. Then we show that the local uniform behavior of spin values across multiple timescales in the representation of the Markov variables could naturally reveal the network's hierarchical community structure. In addition, critical topological information regarding to multivariate spin configuration could also be inferred from the spectral signatures of the Markov process. Finally an algorithm is developed to determine fuzzy communities based on the optimal number of communities and the stability across multiple timescales. The effectiveness and efficiency of our algorithm are theoretically analyzed as well as experimentally validated.
We show that the recently introduced label propagation method for detecting communities in complex networks is equivalent to find the local minima of a simple Potts model. Applying to empirical data, the number of such local minima was found to be very high, much larger than the number of nodes in the graph. The aggregation method for combining information from more local minima shows a tendency to fragment the communities into very small pieces.
We report on an exceptionally accurate spin-glass-type Potts model for community detection. With a simple algorithm, we find that our approach is at least as accurate as the best currently available algorithms and robust to the effects of noise. It is also competitive with the best currently available algorithms in terms of speed and size of solvable systems. We find that the computational demand often exhibits superlinear scaling L^1.3 where L is the number of edges in the system, and we have applied the algorithm to synthetic systems as large as 40x10^6 nodes and over 1x10^9 edges. A previou…
The analysis of stability of community structure is an important problem for scientists from many fields. Here, we propose a new framework to reveal hidden properties of community structure by quantitatively analyzing the dynamics of Potts model. Specifically we model the Potts procedure of community structure detection by a Markov process, which has a clear mathematical explanation. Critical topological information regarding to multivariate spin configuration could also be inferred from the spectral significance of the Markov process. We test our framework on some example networks and find it…
We examine a global disorder transition when identifying community structure in an arbitrary complex network. Earlier, we illustrated [Phil. Mag. 92, 406 (2012)] that "community detection" (CD) generally exhibits disordered (or unsolvable) and ordered (solvable) phases of both high and low computational complexity along with corresponding transitions from regular to chaotic dynamics in derived systems. Using an exact generalized dimensional reduction inequality, multivariate Tutte polynomials, and other considerations, we illustrate how increasing the number of communities q emulates increasin…
Phase transitions in spin glass type systems and, more recently, in related computational problems have gained broad interest in disparate arenas. In the current work, we focus on the "community detection" problem when cast in terms of a general Potts spin glass type problem. As such, our results apply to rather broad Potts spin glass type systems. Community detection describes the general problem of partitioning a complex system involving many elements into optimally decoupled "communities" of such elements. We report on phase transitions between solvable and unsolvable regimes. Solvable regi…
Based on brief review of approaches for community identification and measurement for sensitivity characterization, the accuracy and precision of several approaches for detecting communities in weighted networks are investigated. In weighted networks, the community structure should take both links and link weights into account and the partition of networks should be evaluated by weighted modularity $Q^w$. The results reveal that link weight has important effects on communities especially in dense networks. Potts model and Weighted Extremal Optimization (WEO) algorithm work well on weighted netw…
Community detection of network flows conventionally assumes one-step dynamics on the links. For sparse networks and interest in large-scale structures, longer timescales may be more appropriate. Oppositely, for large networks and interest in small-scale structures, shorter timescales may be better. However, current methods for analyzing networks at different timescales require expensive and often infeasible network reconstructions. To overcome this problem, we introduce a method that takes advantage of the inner-workings of the map equation and evades the reconstruction step. This makes it pos…
To better understand the flows of ideas or information through social and biological systems, researchers develop maps that reveal important patterns in network flows. In practice, network flow models have implied memoryless first-order Markov chains, but recently researchers have introduced higher-order Markov chain models with memory to capture patterns in multi-step pathways. Higher-order models are particularly important for effectively revealing actual, overlapping community structure, but higher-order Markov chain models suffer from the curse of dimensionality: their vast parameter space…
Multiscale community detection can be viewed from a dynamical perspective within the Markov Stability framework, which uses the diffusion of a Markov process on the graph to uncover intrinsic network substructures across all scales. Here we reformulate multiscale community detection as a max-sum length vector partitioning problem with respect to the set of time-dependent node vectors expressed in terms of eigenvectors of the transition matrix. This formulation provides a geometric interpretation of Markov Stability in terms of a time-dependent spectral embedding, where the Markov time acts as …
In recent years, there has been a surge of interest in community detection algorithms for complex networks. A variety of computational heuristics, some with a long history, have been proposed for the identification of communities or, alternatively, of good graph partitions. In most cases, the algorithms maximize a particular objective function, thereby finding the `right' split into communities. Although a thorough comparison of algorithms is still lacking, there has been an effort to design benchmarks, i.e., random graph models with known community structure against which algorithms can be ev…
We propose a generative model to detect globally optimal community structures in networks by utilizing random walks. Sophisticated parameter optimization algorithms are developed based on the Markov chain Monte Carlo methods to overcome limitations of the EM algorithm, which has been used in previous works but is sometimes trapped in local optima depending on initial conditions. We apply the algorithms to synthetic and real-world networks to examine their performance in terms of precision and robustness of detected communities. It is found that the Gibbs samplers outperform the previous approa…
Community detection in networks refers to the process of seeking strongly internally connected groups of nodes which are weakly externally connected. In this work, we introduce and study a community definition based on internal edge density. Beginning with the simple concept that edge density equals number of edges divided by maximal number of edges, we apply this definition to a variety of node and community arrangements to show that our definition yields sensible results. Our community definition is equivalent to that of the Absolute Potts Model community detection method (Phys. Rev. E 81, 0…
Random walks on networks is the standard tool for modelling spreading processes in social and biological systems. This first-order Markov approach is used in conventional community detection, ranking, and spreading analysis although it ignores a potentially important feature of the dynamics: where flow moves to may depend on where it comes from. Here we analyse pathways from different systems, and while we only observe marginal consequences for disease spreading, we show that ignoring the effects of second-order Markov dynamics has important consequences for community detection, ranking, and i…
Detecting communities in complex networks accurately is a prime challenge, preceding further analyses of network characteristics and dynamics. Until now, community detection took into account only positively valued links, while many actual networks also feature negative links. We extend an existing Potts model to incorporate negative links as well, resulting in a method similar to the clustering of signed graphs, as dealt with in social balance theory, but more general. To illustrate our method, we applied it to a network of international alliances and disputes. Using data from 1993--2001, it …
Detection of overlapping communities in complex networks has motivated recent research in the relevant fields. Aiming this problem, we propose a Markov dynamics based algorithm, called UEOC, which means, 'unfold and extract overlapping communities'. In UEOC, when identifying each natural community that overlaps, a Markov random walk method combined with a constraint strategy, which is based on the corresponding annealed network (degree conserving random network), is performed to unfold the community. Then, a cutoff criterion with the aid of a local community function, called conductance, which…
We use a Potts model community detection algorithm to accurately and quantitatively evaluate the hierarchical or multiresolution structure of a graph. Our multiresolution algorithm calculates correlations among multiple copies ("replicas") of the same graph over a range of resolutions. Significant multiresolution structures are identified by strongly correlated replicas. The average normalized mutual information, the variation of information, and other measures in principle give a quantitative estimate of the "best" resolutions and indicate the relative strength of the structures in the graph.…
Detecting community structure in real-world networks is a challenging problem. Recently, it has been shown that the resolution of methods based on optimizing a modularity measure or a corresponding energy is limited; communities with sizes below some threshold remain unresolved. One possibility to go around this problem is to vary the threshold by using a tuning parameter, and investigate the community structure at variable resolutions. Here, we analyze the resolution limit and multiresolution behavior for two different methods: a q-state Potts method proposed by Reichard and Bornholdt, and a …
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The specific implementation of the quality function and the resulting optimal community structure thus become dependent both on the type of Markov process and on the specific Markov times considered. For instance, if we use a natural Markov chain dynamics and discount its stationary distribution -- that is, we take as reference process the dynamics at infinite time -- …
The detection of community structure in networks is intimately related to finding a concise description of the network in terms of its modules. This notion has been recently exploited by the Map equation formalism (M. Rosvall and C.T. Bergstrom, PNAS, 105(4), pp.1118--1123, 2008) through an information-theoretic description of the process of coding inter- and intra-community transitions of a random walker in the network at stationarity. However, a thorough study of the relationship between the full Markov dynamics and the coding mechanism is still lacking. We show here that the original Map co…
Detecting communities from complex networks has recently triggered great interest. Aiming at this problem, a new ant colony optimization strategy building on the Markov random walks theory, which is named as MACO, is proposed in this paper. The framework of ant colony optimization is taken as the basic framework in this algorithm. In each iteration, a Markov random walk model is employed as heuristic rule; all of the ants local solutions are aggregated to a global one through an idea of clustering ensemble, which then will be used to update a pheromone matrix. The strategy relies on the progre…
Complex systems made of interacting elements are commonly abstracted as networks, in which nodes are associated with dynamic state variables, whose evolution is driven by interactions mediated by the edges. Markov processes have been the prevailing paradigm to model such a network-based dynamics, for instance in the form of random walks or other types of diffusions. Despite the success of this modelling perspective for numerous applications, it represents an over-simplification of several real-world systems. Importantly, simple Markov models lack memory in their dynamics, an assumption often n…
Central European University（中欧大学）；ISI Foundation（ISI基金会）；University of Bath（巴斯大学）
We present a Markov chain Monte Carlo scheme based on merges and splits of groups that is capable of efficiently sampling from the posterior distribution of network partitions, defined according to the stochastic block model (SBM). We demonstrate how schemes based on the move of single nodes between groups systematically fail at correctly sampling from the posterior distribution even on small networks, and how our merge-split approach behaves significantly better, and improves the mixing time of the Markov chain by several orders of magnitude in typical cases. We also show how the scheme can b…
Most methods proposed to uncover communities in complex networks rely on combinatorial graph properties. Usually an edge-counting quality function, such as modularity, is optimized over all partitions of the graph compared against a null random graph model. Here we introduce a systematic dynamical framework to design and analyze a wide variety of quality functions for community detection. The quality of a partition is measured by its Markov Stability, a time-parametrized function defined in terms of the statistical properties of a Markov process taking place on the graph. The Markov process pr…
Identifying communities (or clusters), namely groups of nodes with comparatively strong internal connectivity, is a fundamental task for deeply understanding the structure and function of a network. Yet, there is a lack of formal criteria for defining communities and for testing their significance. We propose a sharp definition which is based on a significance threshold. By means of a lumped Markov chain model of a random walker, a quality measure called "persistence probability" is associated to a cluster. Then the cluster is defined as an "$α$-community" if such a probability is not smaller …
Community structure is one of the most important features of real networks and reveals the internal organization of the nodes. Many algorithms have been proposed but the crucial issue of testing, i.e. the question of how good an algorithm is, with respect to others, is still open. Standard tests include the analysis of simple artificial graphs with a built-in community structure, that the algorithm has to recover. However, the special graphs adopted in actual tests have a structure that does not reflect the real properties of nodes and communities found in real networks. Here we introduce a ne…
We introduce a novel algorithm of community detection that maintains dynamically a community structure of a large network that evolves with time. The algorithm maximizes the modularity index thanks to the construction of a randomized hierarchical clustering based on a Monte Carlo Markov Chain (MCMC) method. Interestingly, it could be seen as a dynamization of Louvain algorithm (see Blondel et Al, 2008) where the aggregation step is replaced by the hierarchical instrumental probability.
To connect structure, dynamics and function in systems with multibody interactions, network scientists model random walks on hypergraphs and identify communities that confine the walks for a long time. The two flow-based community-detection methods Markov stability and the map equation identify such communities based on different principles and search algorithms. But how similar are the resulting communities? We explain both methods' machinery applied to hypergraphs and compare them on synthetic and real-world hypergraphs using various hyperedge-size biased random walks and time scales. We fin…
Grouping objects into clusters based on similarities or weights between them is one of the most important problems in science and engineering. In this work, by extending message passing algorithms and spectral algorithms proposed for unweighted community detection problem, we develop a non-parametric method based on statistical physics, by mapping the problem to Potts model at the critical temperature of spin glass transition and applying belief propagation to solve the marginals corresponding to the Boltzmann distribution. Our algorithm is robust to over-fitting and gives a principled way to …
The investigation of community structure in networks is a task of great importance in many disciplines, namely physics, sociology, biology and computer science where systems are often represented as graphs. One of the challenges is to find local communities from a local viewpoint in a graph without global information in order to reproduce the subjective hierarchical vision for each vertex. In this paper we present the improvement of an information dynamics algorithm in which the label propagation of nodes is based on the Markovian flow of information in the network under cognitive-inspired con…
Community detection remains an important problem in data mining, owing to the lack of scalable algorithms that exploit all aspects of available data - namely the directionality of flow of information and the dynamics thereof. Most existing methods use measures of connectedness in the graphical structure. In this paper, we present a fast, scalable algorithm to detect communities in directed, weighted graph representations of social networks by simulating flow of information through them. By design, our algorithm naturally handles undirected or unweighted networks as well. Our algorithm runs in …
Community detection in complex networks is a topic of considerable recent interest within the scientific community. For dealing with the problem that genetic algorithm are hardly applied to community detection, we propose a genetic algorithm with ensemble learning (GAEL) for detecting community structure in complex networks. GAEL replaces its traditional crossover operator with a multi-individual crossover operator based on ensemble learning. Therefore, GAEL can avoid the problems that are brought by traditional crossover operator which is only able to mix string blocks of different individual…
Modularity was introduced as a measure of goodness for the community structure induced by a partition of the set of vertices in a graph. Then, it also became an objective function used to find good partitions, with high success. Nevertheless, some works have shown a scaling limit and certain instabilities when finding communities with this criterion. Modularity has been studied proposing several formalisms, as hamiltonians in a Potts model or laplacians in spectral partitioning. In this paper we present a new probabilistic formalism to analyze modularity, and from it we derive an algorithm bas…
The stochastic block model is able to generate different network partitions, ranging from traditional assortative communities to disassortative structures. Since the degree-corrected stochastic block model does not specify which mixing pattern is desired, the inference algorithms, which discover the most likely partition of the networks nodes, are likely to get trapped in the local optima of the log-likelihood. Here we introduce a new model constraining nodes' internal degrees ratios in the objective function to stabilize the inference of block models from the observed network data. Given the …
We propose a one parameter family of random walk processes on hypergraphs, where a parameter biases the dynamics of the walker towards hyperedges of low or high cardinality. We show that for each value of the parameter the resulting process defines its own hypergraph projection on a weighted network. We then explore the differences between them by considering the community structure associated to each random walk process. To do so, we generalise the Markov stability framework to hypergraphs and test it on artificial and real-world hypergraphs.
Time-varying community structures widely exist in various real-world networks. However, the spreading dynamics on this kind of network has not been fully studied. To this end, we systematically study the effects of time-varying community structures on social contagions. We first propose a non-Markovian social contagion model on time-varying community networks based on the activity driven network model, in which an individual adopts a behavior if and only if the accumulated behavioral information it has ever received reaches a threshold. Then, we develop a mean-field theory to describe the prop…
In many applications, it is common practice to obtain a network from interaction counts by thresholding each pairwise count at a prescribed value. Our analysis calls attention to the dependence of certain methods, notably Newman--Girvan modularity, on the choice of threshold. Essentially, the threshold either separates the network into clusters automatically, making the algorithm's job trivial, or erases all structure in the data, rendering clustering impossible. By fitting the original interaction counts as given, we show that minor modifications to classical statistical methods outperform th…
Many real systems can be represented as networks whose analysis can be very informative regarding the original system's organisation. In the past decade community detection received a lot of attention and is now an active field of research. Recently stability was introduced as a new measure for partition quality. This work investigates stability as an optimisation criterion that exploits a Markov process view of networks to enable multi-scale community detection. Several heuristics and variations of an algorithm optimising stability are presented as well as an application to overlapping commun…
From traffic flows on road networks to electrical signals in brain networks, many real-world networks contain modular structures of different sizes and densities. In the networks where modular structures emerge due to coupling between nodes with similar dynamical functions, we can identify them using flow-based community detection methods. However, these methods implicitly assume that communities are dense or clique-like which can shatter sparse communities due to a field-of-view limit inherent in one-step dynamics. Taking multiple steps with shorter or longer Markov time enables us to effecti…
Communities in directed networks have often been characterized as regions with a high density of links, or as sets of nodes with certain patterns of connection. Our approach for community detection combines the optimization of a quality function and a spectral clustering of a deformation of the combinatorial Laplacian, the so-called magnetic Laplacian. The eigenfunctions of the magnetic Laplacian, that we call magnetic eigenmaps, incorporate structural information. Hence, using the magnetic eigenmaps, dense communities including directed cycles can be revealed as well as "role" communities in …
In evolving complex systems such as air traffic and social organizations, collective effects emerge from their many components' dynamic interactions. While the dynamic interactions can be represented by temporal networks with nodes and links that change over time, they remain highly complex. It is therefore often necessary to use methods that extract the temporal networks' large-scale dynamic community structure. However, such methods are subject to overfitting or suffer from effects of arbitrary, a priori imposed timescales, which should instead be extracted from data. Here we simultaneously …
Monash University Malaysia（马来西亚蒙纳士大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Xiamen University Malaysia（厦门大学马来西亚分校）
We present a unified statistical framework for characterizing community structure of brain functional networks that captures variation across individuals and evolution over time. Existing methods for community detection focus only on single-subject analysis of dynamic networks; while recent extensions to multiple-subjects analysis are limited to static networks. To overcome these limitations, we propose a multi-subject, Markov-switching stochastic block model (MSS-SBM) to identify state-related changes in brain community organization over a group of individuals. We first formulate a multilayer…
In the light of several major epidemic events that emerged in the past two decades, and emphasized by the COVID-19 pandemics, the non-Markovian spreading models occurring on complex networks gained significant attention from the scientific community. Following this interest, in this article, we explore the relations that exist between the non-Markovian SEIS (Susceptible--Exposed--Infectious--Susceptible) and the classical Markov SIS, as basic re-occurring virus spreading models in complex networks. We investigate the similarities and seek for equivalences both for the discrete-time and the con…
Random walks on simple graphs in connection with electrical resistor networks lead to the definition of Markov chains with transition probability matrix in terms of electrical conductances. We extend this definition to an effective transition matrix $P_{ij}$ to account for the probability of going from vertex $i$ to any vertex $j$ of the original connected graph $G$. Also, we present an algorithm based on the definition of this effective transition matrix among vertices in the network to extract a topological feature related to the manner graph $G$ has been organized. This topological feature …
In network science, a group of nodes connected with each other at higher probability than with those outside the group is referred to as a community. From the perspective that individual communities are associated with functional modules constituting complex systems described by networks, discovering communities is primarily important for understanding overall functions of these systems. Much effort has been devoted to developing methods to detect communities in networks since the early days of network science. Nevertheless, the method to reveal key characteristics of communities in real-world…
