We report on an exceptionally accurate spin-glass-type Potts model for community detection. With a simple algorithm, we find that our approach is at least as accurate as the best currently available algorithms and robust to the effects of noise. It is also competitive with the best currently available algorithms in terms of speed and size of solvable systems. We find that the computational demand often exhibits superlinear scaling L^1.3 where L is the number of edges in the system, and we have applied the algorithm to synthetic systems as large as 40x10^6 nodes and over 1x10^9 edges. A previous stumbling block encountered by popular community detection methods is the so-called "resolution limit." Being a "local" measure of community structure, our Potts model is free from this resolution-limit effect, and it further remains a local measure on weighted and directed graphs. We also address the mitigation of resolution-limit effects for two other popular Potts models.
We examine a global disorder transition when identifying community structure in an arbitrary complex network. Earlier, we illustrated [Phil. Mag. 92, 406 (2012)] that "community detection" (CD) generally exhibits disordered (or unsolvable) and ordered (solvable) phases of both high and low computational complexity along with corresponding transitions from regular to chaotic dynamics in derived systems. Using an exact generalized dimensional reduction inequality, multivariate Tutte polynomials, and other considerations, we illustrate how increasing the number of communities q emulates increasin…
We show that the recently introduced label propagation method for detecting communities in complex networks is equivalent to find the local minima of a simple Potts model. Applying to empirical data, the number of such local minima was found to be very high, much larger than the number of nodes in the graph. The aggregation method for combining information from more local minima shows a tendency to fragment the communities into very small pieces.
Potts model is a powerful tool to uncover community structure in complex networks. Here, we propose a new framework to reveal the optimal number of communities and stability of network structure by quantitatively analyzing the dynamics of Potts model. Specifically we model the community structure detection Potts procedure by a Markov process, which has a clear mathematical explanation. Then we show that the local uniform behavior of spin values across multiple timescales in the representation of the Markov variables could naturally reveal the network's hierarchical community structure. In addi…
Phase transitions in spin glass type systems and, more recently, in related computational problems have gained broad interest in disparate arenas. In the current work, we focus on the "community detection" problem when cast in terms of a general Potts spin glass type problem. As such, our results apply to rather broad Potts spin glass type systems. Community detection describes the general problem of partitioning a complex system involving many elements into optimally decoupled "communities" of such elements. We report on phase transitions between solvable and unsolvable regimes. Solvable regi…
Based on brief review of approaches for community identification and measurement for sensitivity characterization, the accuracy and precision of several approaches for detecting communities in weighted networks are investigated. In weighted networks, the community structure should take both links and link weights into account and the partition of networks should be evaluated by weighted modularity $Q^w$. The results reveal that link weight has important effects on communities especially in dense networks. Potts model and Weighted Extremal Optimization (WEO) algorithm work well on weighted netw…
The analysis of stability of community structure is an important problem for scientists from many fields. Here, we propose a new framework to reveal hidden properties of community structure by quantitatively analyzing the dynamics of Potts model. Specifically we model the Potts procedure of community structure detection by a Markov process, which has a clear mathematical explanation. Critical topological information regarding to multivariate spin configuration could also be inferred from the spectral significance of the Markov process. We test our framework on some example networks and find it…
We use a Potts model community detection algorithm to accurately and quantitatively evaluate the hierarchical or multiresolution structure of a graph. Our multiresolution algorithm calculates correlations among multiple copies ("replicas") of the same graph over a range of resolutions. Significant multiresolution structures are identified by strongly correlated replicas. The average normalized mutual information, the variation of information, and other measures in principle give a quantitative estimate of the "best" resolutions and indicate the relative strength of the structures in the graph.…
Detecting communities in complex networks accurately is a prime challenge, preceding further analyses of network characteristics and dynamics. Until now, community detection took into account only positively valued links, while many actual networks also feature negative links. We extend an existing Potts model to incorporate negative links as well, resulting in a method similar to the clustering of signed graphs, as dealt with in social balance theory, but more general. To illustrate our method, we applied it to a network of international alliances and disputes. Using data from 1993--2001, it …
Detecting community structure in real-world networks is a challenging problem. Recently, it has been shown that the resolution of methods based on optimizing a modularity measure or a corresponding energy is limited; communities with sizes below some threshold remain unresolved. One possibility to go around this problem is to vary the threshold by using a tuning parameter, and investigate the community structure at variable resolutions. Here, we analyze the resolution limit and multiresolution behavior for two different methods: a q-state Potts method proposed by Reichard and Bornholdt, and a …
Multiscale community detection can be viewed from a dynamical perspective within the Markov Stability framework, which uses the diffusion of a Markov process on the graph to uncover intrinsic network substructures across all scales. Here we reformulate multiscale community detection as a max-sum length vector partitioning problem with respect to the set of time-dependent node vectors expressed in terms of eigenvectors of the transition matrix. This formulation provides a geometric interpretation of Markov Stability in terms of a time-dependent spectral embedding, where the Markov time acts as …
Community structure is one of the most important features of real networks and reveals the internal organization of the nodes. Many algorithms have been proposed but the crucial issue of testing, i.e. the question of how good an algorithm is, with respect to others, is still open. Standard tests include the analysis of simple artificial graphs with a built-in community structure, that the algorithm has to recover. However, the special graphs adopted in actual tests have a structure that does not reflect the real properties of nodes and communities found in real networks. Here we introduce a ne…
Various attempts have been made in recent years to solve the Resolution Limit (RL) problem in community detection by considering variants of the modularity metric in the detection algorithms. These metrics purportedly largely mitigate the RL problem and are preferable to modularity in many realistic scenarios. However, they are not generally suitable for analyzing weighted networks or for detecting hierarchical community structure. Resolution limit problems can be complicated, though, and in particular it can be unclear when it should be considered as problem. In this paper, we introduce a met…
Community detection in networks refers to the process of seeking strongly internally connected groups of nodes which are weakly externally connected. In this work, we introduce and study a community definition based on internal edge density. Beginning with the simple concept that edge density equals number of edges divided by maximal number of edges, we apply this definition to a variety of node and community arrangements to show that our definition yields sensible results. Our community definition is equivalent to that of the Absolute Potts Model community detection method (Phys. Rev. E 81, 0…
Detecting communities in large networks has drawn much attention over the years. While modularity remains one of the more popular methods of community detection, the so-called resolution limit remains a significant drawback. To overcome this issue, it was recently suggested that instead of comparing the network to a random null model, as is done in modularity, it should be compared to a constant factor. However, it is unclear what is meant exactly by "resolution-limit-free", that is, not suffering from the resolution limit. Furthermore, the question remains what other methods could be classifi…
Modularity, since its introduction, has remained one of the most widely used metrics to assess the quality of community structure in a complex network. However the resolution limit problem associated with modularity limits its applicability to networks with community sizes smaller than a certain scale. In the past various attempts have been made to solve this problem. More recently a new metric, modularity density, was introduced for the quality of community structure in networks in order to solve some of the known problems with modularity, particularly the resolution limit problem. Modularity…
Recently, a type of multi-resolution methods in community detection was introduced, which can adjust the resolution of modularity by modifying the modularity function with tunable resolution parameters, such as those proposed by Arenas, Fernandez and Gomez and by Reichardt and Bornholdt. In this paper, we show that these methods still have the intrinsic limitation-large communities may have been split before small communities become visible-because it is at the cost of the community stability that the enhancement of the modularity resolution is obtained. The theoretical results indicated that …
Community detection is of considerable importance for analyzing the structure and function of complex networks. Many real-world networks may possess community structures at multiple scales, and recently, various multi-resolution methods were proposed to identify the community structures at different scales. In this paper, we present a type of multi-resolution methods by using the generalized self-loop rescaling strategy. The self-loop rescaling strategy provides one uniform ansatz for the design of multi-resolution community detection methods. Many quality functions for community detection can…
A community detection algorithm is considered to have a resolution limit if the scale of the smallest modules that can be resolved depends on the size of the analyzed subnetwork. The resolution limit is known to prevent some community detection algorithms from accurately identifying the modular structure of a network. In fact, any global objective function for measuring the quality of a two-level assignment of nodes into modules must have some sort of resolution limit or an external resolution parameter. However, it is yet unknown how the resolution limit affects the so-called map equation, wh…
An algorithm for the detection of overlapping natural communities in networks was proposed by Lancichinetti, Fortunato, and Kertesz (LFK) last year. The LFK algorithm constructs natural communities of (in principle) all nodes of a graph by maximising the local fitness of communities. The resulting modules can overlap. The generation of communities can easily be repeated for many values of resolution; thus allowing different views on the network at different resolutions. We implemented the main idea of the LFK algorithm---to generate natural communities of each node of a network---in a differen…
The maximization of generalized modularity performs well on networks in which the members of all communities are statistically indistinguishable from each other. However, there is no theory bounding the maximization performance in more realistic networks where edges are heterogeneously distributed within and between communities. Using the random graph properties, we establish asymptotic theoretical bounds on the resolution parameter for which the generalized modularity maximization performs well. From this new perspective on random graph model, we find the resolution limit of modularity maximi…
Community detection is the process of assigning nodes and links in significant communities (e.g. clusters, function modules) and its development has led to a better understanding of complex networks. When applied to sizable networks, we argue that most detection algorithms correctly identify prominent communities, but fail to do so across multiple scales. As a result, a significant fraction of the network is left uncharted. We show that this problem stems from larger or denser communities overshadowing smaller or sparser ones, and that this effect accounts for most of the undetected communitie…
Community detection algorithms attempt to find the best clusters of nodes in an arbitrary complex network. Multi-scale ("multiresolution") community detection extends the problem to identify the best network scale(s) for these clusters. The latter task is generally accomplished by analyzing community stability simultaneously for all clusters in the network. In the current work, we extend this general approach to define local multiresolution methods, which enable the extraction of well-defined local communities even if the global community structure is vaguely defined in an average sense. Towar…
It is well-known that community detection methods based on modularity optimization often fails to discover small communities. Several objective functions used for community detection therefore involve a resolution parameter that allows the detection of communities at different scales. We provide an explicit upper bound on the community size of communities resulting from the optimization of several of these functions. We also show with a simple example that the use of the resolution parameter may artificially force the complete disaggregation of large and densely connected communities.
Most methods proposed to uncover communities in complex networks rely on combinatorial graph properties. Usually an edge-counting quality function, such as modularity, is optimized over all partitions of the graph compared against a null random graph model. Here we introduce a systematic dynamical framework to design and analyze a wide variety of quality functions for community detection. The quality of a partition is measured by its Markov Stability, a time-parametrized function defined in terms of the statistical properties of a Markov process taking place on the graph. The Markov process pr…
We investigate the possibility of global optimization-based overlapping community detection, using link community framework. We first show that partition density, the original quality function used in link community detection method, is not suitable as a quality function for global optimization because it prefers breaking communities into triangles except in highly limited conditions. We analytically derive those conditions and confirm it with computational results on direct optimization of various synthetic and real-world networks. To overcome this limitation, we propose alternative approache…
Modularity maximization is the most popular technique for the detection of community structure in graphs. The resolution limit of the method is supposedly solvable with the introduction of modified versions of the measure, with tunable resolution parameters. We show that multiresolution modularity suffers from two opposite coexisting problems: the tendency to merge small subgraphs, which dominates when the resolution is low; the tendency to split large subgraphs, which dominates when the resolution is high. In benchmark networks with heterogeneous distributions of cluster sizes, the simultaneo…
The concept of community detection has long been used as a key device for handling the mesoscale structures in networks. Suitably conducted community detection reveals various embedded informative substructures of network topology. However, regarding the practical usage of community detection, it has always been a tricky problem to assign a reasonable community resolution for networks of interest. Because of the absence of the unanimously accepted criterion, most of the previous studies utilized rather ad hoc heuristics to decide the community resolution. In this work, we harness the concept o…
Communities of vertices within a giant network such as the World-Wide Web are likely to be vastly smaller than the network itself. However, Fortunato and Barthélemy have proved that modularity maximization algorithms for community detection may fail to resolve communities with fewer than $\sqrt{L/2}$ edges, where $L$ is the number of edges in the entire network. This resolution limit leads modularity maximization algorithms to have notoriously poor accuracy on many real networks. Fortunato and Barthélemy's argument can be extended to networks with weighted edges as well, and we derive this cor…
Aiming at improving the efficiency and accuracy of community detection in complex networks, we proposed a new algorithm, which is based on the idea that communities could be detected from subnetworks by comparing the internal and external cohesion of each subnetwork. In our method, similar nodes are firstly gathered into meta-communities, which are then decided to be retained or merged through a multilevel label propagation process, until all of them meet our community criterion. Our algorithm requires neither any priori information of communities nor optimization of any objective function. Ex…
Grouping objects into clusters based on similarities or weights between them is one of the most important problems in science and engineering. In this work, by extending message passing algorithms and spectral algorithms proposed for unweighted community detection problem, we develop a non-parametric method based on statistical physics, by mapping the problem to Potts model at the critical temperature of spin glass transition and applying belief propagation to solve the marginals corresponding to the Boltzmann distribution. Our algorithm is robust to over-fitting and gives a principled way to …
Multilayer networks capture pairwise relationships between the components of complex systems across multiple modes or scales of interactions. An important meso-scale feature of these networks is measured though their community structure, which defines groups of strongly connected nodes that exist within and across network layers. Because interlayer edges can describe relationships between different modalities, scales, or time points, it is essential to understand how communities change and evolve across layers. A popular method for detecting communities in multilayer networks consists of maxim…
Applied network science often involves preprocessing network data before applying a network-analysis method, and there is typically a theoretical disconnect between these steps. For example, it is common to aggregate time-varying network data into windows prior to analysis, and the tradeoffs of this preprocessing are not well understood. Focusing on the problem of detecting small communities in multilayer networks, we study the effects of layer aggregation by developing random-matrix theory for modularity matrices associated with layer-aggregated networks with $N$ nodes and $L$ layers, which a…
The problem of community detection is relevant in many disciplines of science and modularity optimization is the widely accepted method for this purpose. It has recently been shown that this approach presents a resolution limit by which it is not possible to detect communities with sizes smaller than a threshold which depends on the network size. Moreover, it might happen that the communities resulting from such an approach do not satisfy the usual qualitative definition of commune, i.e., nodes in a commune are more connected among themselves than to nodes outside the commune. In this article …
Community structure is one of the most important features of complex networks. Modularity-based methods for community detection typically rely on heuristic algorithms to optimize a specific community quality function. Such methods are limited by two major defects: (1) the resolution limit problem, which prohibits communities of heterogeneous sizes being simultaneously detected, and (2) divergent outputs of the heuristic algorithm, which make it difficult to differentiate relevant and irrelevant results. In this paper, we propose an improved method for community detection based on a scalable co…
Detecting community structure is fundamental to clarify the link between structure and function in complex networks and is used for practical applications in many disciplines. A successful method relies on the optimization of a quantity called modularity [Newman and Girvan, Phys. Rev. E 69, 026113 (2004)], which is a quality index of a partition of a network into communities. We find that modularity optimization may fail to identify modules smaller than a scale which depends on the total number L of links of the network and on the degree of interconnectedness of the modules, even in cases wher…
We propose a new local, deterministic and parameter-free algorithm that detects fuzzy and crisp overlapping communities in a weighted network and simultaneously reveals their hierarchy. Using a local fitness function, the algorithm greedily expands natural communities of seeds until the whole graph is covered. The hierarchy of communities is obtained analytically by calculating resolution levels at which communities grow rather than numerically by testing different resolution levels. This analytic procedure is not only more exact than its numerical alternatives such as LFM and GCE but also muc…
Optimizing statistical measures for community structure is one of the most popular strategies for community detection, but many of them lack the flexibility of resolution and thus are incompatible with multi-scale communities of networks. Here, we further studied a statistical measure of interest for community detection, asymptotic surprise, an asymptotic approximation of surprise. We discussed the critical behaviors of asymptotic surprise in phase transition of community partition theoretically. Then, according to the theoretical analysis, a multi-resolution method based on asymptotic surpris…
Mining community structures from the complex network is an important problem across a variety of fields. Many existing community detection methods detect communities through optimizing a community evaluation function. However, most of these functions even have high values on random graphs and may fail to detect small communities in the large-scale network (the so-called resolution limit problem). In this paper, we introduce two novel node-centric community evaluation functions by connecting correlation analysis with community detection. We will further show that the correlation analysis can pr…
Graph theory provides a powerful framework to investigate brain functional connectivity networks and their modular organization. However, most graph-based methods suffer from a fundamental resolution limit that may have affected previous studies and prevented detection of modules, or communities, that are smaller than a specific scale. Surprise, a resolution-limit-free function rooted in discrete probability theory, has been recently introduced and applied to brain networks, revealing a wide size-distribution of functional modules, in contrast with many previous reports. However, the use of Su…
We demonstrate an exact equivalence between two widely used methods of community detection in networks, the method of modularity maximization in its generalized form which incorporates a resolution parameter controlling the size of the communities discovered, and the method of maximum likelihood applied to the special case of the stochastic block model known as the planted partition model, in which all communities in a network are assumed to have statistically similar properties. Among other things, this equivalence provides a mathematically principled derivation of the modularity function, cl…
Many complex networks show signs of modular structure, uncovered by community detection. Although many methods succeed in revealing various partitions, it remains difficult to detect at what scale some partition is significant. This problem shows foremost in multi-resolution methods. We here introduce an efficient method for scanning for resolutions in one such method. Additionally, we introduce the notion of "significance" of a partition, based on subgraph probabilities. Significance is independent of the exact method used, so could also be applied in other methods, and can be interpreted as …
To understand the structure and organization of a large-scale social, biological or technological network, it can be helpful to describe and extract local communities or modules of the network. In this article, we develop a neurodynamic framework to describe the local communities which correspond to the stable states of a neuro-system built based on the network. The quantitative criteria to describe the neurodynamic system can cover a large range of objective functions. The resolution limit of these functions enable us to propose a generic criterion to explore multi-resolution local communitie…
Modularity based community detection encompasses a number of widely used, efficient heuristics for identification of structure in networks. Recently, a belief propagation approach to modularity optimization provided a useful guide for identifying non-trivial structure in single-layer networks in a way that other optimization heuristics have not. In this paper, we extend modularity belief propagation to multilayer networks. As part of this development, we also directly incorporate a resolution parameter. We show that adjusting the resolution parameter affects the convergence properties of the a…
We study the community structure of networks representing voting on resolutions in the United Nations General Assembly. We construct networks from the voting records of the separate annual sessions between 1946 and 2008 in three different ways: (1) by considering voting similarities as weighted unipartite networks; (2) by considering voting similarities as weighted, signed unipartite networks; and (3) by examining signed bipartite networks in which countries are connected to resolutions. For each formulation, we detect communities by optimizing network modularity using an appropriate null mode…
The recent boom of large-scale Online Social Networks (OSNs) both enables and necessitates the use of parallelisable and scalable computational techniques for their analysis. We examine the problem of real-time community detection and a recently proposed linear time - O(m) on a network with m edges - label propagation or "epidemic" community detection algorithm. We identify characteristics and drawbacks of the algorithm and extend it by incorporating different heuristics to facilitate reliable and multifunctional real-time community detection. With limited computational resources, we employ th…
Community detection is a fundamental and important problem in network science, as community structures often reveal both topological and functional relationships between different components of the complex system. In this paper, we first propose a gradient descent framework of modularity optimization called vector-label propagation algorithm (VLPA), where a node is associated with a vector of continuous community labels instead of one label. Retaining weak structural information in vector-label, VLPA outperforms some well-known community detection methods, and particularly improves the perform…
In this paper, we first discuss the definition of modularity (Q) used as a metric for community quality and then we review the modularity maximization approaches which were used for community detection in the last decade. Then, we discuss two opposite yet coexisting problems of modularity optimization: in some cases, it tends to favor small communities over large ones while in others, large communities over small ones (so called the resolution limit problem). Next, we overview several community quality metrics proposed to solve the resolution limit problem and discuss Modularity Density (Qds) …
Community structure represents the local organization of complex networks and the single most important feature to extract functional relationships between nodes. In the last years, the problem of community detection has been reformulated in terms of the optimization of a function, the Newman-Girvan modularity, that is supposed to express the quality of the partitions of a network into communities. Starting from a recent critical survey on modularity optimization, pointing out the existence of a resolution limit that poses severe limits to its applicability, we discuss the general issue of the…
Heuristic and exact modularity optimization with size-constrained communities
Kellogg School of Management, Northwestern University（西北大学凯洛格管理学院）；Department of Mechanical and Industrial Engineering, University of Toronto（多伦多大学机械与工业工程系）；Centre for Science and Technology Studies (CWTS), Leiden University（莱顿大学科学技术研究中心）；Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington（印第安纳大学布鲁明顿分校卢迪信息、计算与工程学院复杂网络与系统研究中心）
University of Colorado（科罗拉多大学）；BioFrontiers Institute（生物前沿研究所）
In bipartite networks, community structures are restricted to being disassortative, in that nodes of one type are grouped according to common patterns of connection with nodes of the other type. This makes the stochastic block model (SBM), a highly flexible generative model for networks with block structure, an intuitive choice for bipartite community detection. However, typical formulations of the SBM do not make use of the special structure of bipartite networks. Here we introduce a Bayesian nonparametric formulation of the SBM and a corresponding algorithm to efficiently find communities in…
Identifying community structure in networks is an issue of particular interest in network science. The modularity introduced by Newman and Girvan [Phys. Rev. E 69, 026113 (2004)] is the most popular quality function for community detection in networks. In this study, we identify a problem in the concept of modularity and suggest a solution to overcome this problem. Specifically, we obtain a new quality function for community detection. We refer to the function as Z-modularity because it measures the Z-score of a given division with respect to the fraction of the number of edges within communit…
The investigation of community structure in networks is a task of great importance in many disciplines, namely physics, sociology, biology and computer science where systems are often represented as graphs. One of the challenges is to find local communities from a local viewpoint in a graph without global information in order to reproduce the subjective hierarchical vision for each vertex. In this paper we present the improvement of an information dynamics algorithm in which the label propagation of nodes is based on the Markovian flow of information in the network under cognitive-inspired con…
Community detection is a significant and challenging task in network research. Nowadays, plenty of attention has been focused on local methods of community detection. Among them, community detection with a greedy algorithm typically starts from the identification of local essential nodes called central nodes of the network; communities expand later from these central nodes by optimizing a modularity function. In this paper, we propose a new central node indicator and a new modularity function. Our central node indicator, which we call local centrality indicator (LCI), is as efficient as the we…
Modularity-based algorithms used for community detection have been increasing in recent years. Modularity and its application have been generating controversy since some authors argue it is not a metric without disadvantages. It has been shown that algorithms that use modularity to detect communities suffer a resolution limit and, therefore, it is unable to identify small communities in some situations. In this work, we try to apply a density optimization of communities found by the label propagation algorithm and study what happens regarding modularity of optimized results. We introduce a met…
Community structure is an important structural property that extensively exists in various complex networks. In the past decade, much attention has been paid to the design of community-detection methods, but analyzing the behaviors of the methods is also of interest in the theoretical research and real applications. Here, we focus on an important measure for community structure, significance [Sci. Rep. 3 (2013) 2930]. Specifically, we study the effect of various network parameters on this measure in detail, analyze the critical behaviors of it in partition transition, and analytically give the…
Community structure is an important property of complex networks. An automatic discovery of such structure is a fundamental task in many disciplines, including sociology, biology, engineering, and computer science. Recently, several community discovery algorithms have been proposed based on the optimization of a quantity called modularity (Q). However, the problem of modularity optimization is NP-hard, and the existing approaches often suffer from prohibitively long running time or poor quality. Furthermore, it has been recently pointed out that algorithms based on optimizing Q will have a res…
Link streams model interactions over time in a wide range of fields. Under this model, the challenge is to mine efficiently both temporal and topological structures. Community detection and change point detection are one of the most powerful tools to analyze such evolving interactions. In this paper, we build on both to detect stable community structures by identifying change points within meaningful communities. Unlike existing dynamic community detection algorithms, the proposed method is able to discover stable communities efficiently at multiple temporal scales. We test the effectiveness o…
