Various attempts have been made in recent years to solve the Resolution Limit (RL) problem in community detection by considering variants of the modularity metric in the detection algorithms. These metrics purportedly largely mitigate the RL problem and are preferable to modularity in many realistic scenarios. However, they are not generally suitable for analyzing weighted networks or for detecting hierarchical community structure. Resolution limit problems can be complicated, though, and in particular it can be unclear when it should be considered as problem. In this paper, we introduce a metric that we call generalized modularity density $Q_g$ that eliminates the RL problem at any desired resolution and is easily extendable to study weighted and hierarchical networks. We also propose a benchmark test to quantify the resolution limit problem, examine various modularity-like metrics to show that the new metric $Q_g$ performs best, and show that $Q_g$ can identify modular structure in real-world and artificial networks that is otherwise hidden.
Modularity, since its introduction, has remained one of the most widely used metrics to assess the quality of community structure in a complex network. However the resolution limit problem associated with modularity limits its applicability to networks with community sizes smaller than a certain scale. In the past various attempts have been made to solve this problem. More recently a new metric, modularity density, was introduced for the quality of community structure in networks in order to solve some of the known problems with modularity, particularly the resolution limit problem. Modularity…
Modularity is widely used to effectively measure the strength of the disjoint community structure found by community detection algorithms. Although several overlapping extensions of modularity were proposed to measure the quality of overlapping community structure, there is lack of systematic comparison of different extensions. To fill this gap, we overview overlapping extensions of modularity to select the best. In addition, we extend the Modularity Density metric to enable its usage for overlapping communities. The experimental results on four real networks using overlapping extensions of mo…
Modularity is widely used to effectively measure the strength of the community structure found by community detection algorithms. However, modularity maximization suffers from two opposite yet coexisting problems: in some cases, it tends to favor small communities over large ones while in others, large communities over small ones. The latter tendency is known in the literature as the resolution limit problem. To address them, we propose to modify modularity by subtracting from it the fraction of edges connecting nodes of different communities and by including community density into modularity.…
Identifying community structure in networks is an issue of particular interest in network science. The modularity introduced by Newman and Girvan [Phys. Rev. E 69, 026113 (2004)] is the most popular quality function for community detection in networks. In this study, we identify a problem in the concept of modularity and suggest a solution to overcome this problem. Specifically, we obtain a new quality function for community detection. We refer to the function as Z-modularity because it measures the Z-score of a given division with respect to the fraction of the number of edges within communit…
Modularity maximization has been one of the most widely used approaches in the last decade for discovering community structure in networks of practical interest in biology, computing, social science, statistical mechanics, and more. Modularity is a quality function that measures the difference between the number of edges found within clusters minus the number of edges one would statistically expect to find based on random chance. We present a natural generalization of modularity based on the difference between the actual and expected number of walks within clusters, which we call walk-modulari…
In this paper, we first discuss the definition of modularity (Q) used as a metric for community quality and then we review the modularity maximization approaches which were used for community detection in the last decade. Then, we discuss two opposite yet coexisting problems of modularity optimization: in some cases, it tends to favor small communities over large ones while in others, large communities over small ones (so called the resolution limit problem). Next, we overview several community quality metrics proposed to solve the resolution limit problem and discuss Modularity Density (Qds) …
We show here that the problem of maximizing a family of quantitative functions, encompassing both the modularity (Q-measure) and modularity density (D-measure), for community detection can be uniformly understood as a combinatoric optimization involving the trace of a matrix called modularity Laplacian. Instead of using traditional spectral relaxation, we apply additional nonnegative constraint into this graph clustering problem and design efficient algorithms to optimize the new objective. With the explicit nonnegative constraint, our solutions are very close to the ideal community indicator …
Heterogeneous networks are networks consisting of different types of nodes and multiple types of edges linking such nodes. While community detection has been extensively developed as a useful technique for analyzing networks that contain only one type of nodes, very few community detection techniques have been developed for heterogeneous networks. In this paper, we propose a modularity based community detection framework for heterogeneous networks. Unlike existing methods, the proposed approach has the flexibility to treat the number of communities as an unknown quantity. We describe a Louvain…
Community detection is a significant and challenging task in network research. Nowadays, plenty of attention has been focused on local methods of community detection. Among them, community detection with a greedy algorithm typically starts from the identification of local essential nodes called central nodes of the network; communities expand later from these central nodes by optimizing a modularity function. In this paper, we propose a new central node indicator and a new modularity function. Our central node indicator, which we call local centrality indicator (LCI), is as efficient as the we…
Many real-world complex networks exhibit a community structure, in which the modules correspond to actual functional units. Identifying these communities is a key challenge for scientists. A common approach is to search for the network partition that maximizes a quality function. Here, we present a detailed analysis of a recently proposed function, namely modularity density. We show that it does not incur in the drawbacks suffered by traditional modularity, and that it can identify networks without ground-truth community structure, deriving its analytical dependence on link density in generic …
Modularity is widely used to effectively measure the strength of the disjoint community structure found by community detection algorithms. Several overlapping extensions of modularity were proposed to measure the quality of overlapping community structure. However, all these extensions differ just in the way they define the belonging coefficient and belonging function. Yet, there is lack of systematic comparison of different extensions. To fill this gap, we overview overlapping extensions of modularity and generalize them with a uniform definition enabling application of different belonging co…
Modularity maximization using greedy algorithms continues to be a popular approach toward community detection in graphs, even after various better forming algorithms have been proposed. Apart from its clear mechanism and ease of implementation, this approach is persistently popular because, presumably, its risk of algorithmic failure is not well understood. This Rapid Communication provides insight into this issue by estimating the algorithmic performance limit of modularity maximization. This is achieved by counting the number of metastable states under a local update rule. Our results offer …
We demonstrate an exact equivalence between two widely used methods of community detection in networks, the method of modularity maximization in its generalized form which incorporates a resolution parameter controlling the size of the communities discovered, and the method of maximum likelihood applied to the special case of the stochastic block model known as the planted partition model, in which all communities in a network are assumed to have statistically similar properties. Among other things, this equivalence provides a mathematically principled derivation of the modularity function, cl…
Modularity maximization is the most popular technique for the detection of community structure in graphs. The resolution limit of the method is supposedly solvable with the introduction of modified versions of the measure, with tunable resolution parameters. We show that multiresolution modularity suffers from two opposite coexisting problems: the tendency to merge small subgraphs, which dominates when the resolution is low; the tendency to split large subgraphs, which dominates when the resolution is high. In benchmark networks with heterogeneous distributions of cluster sizes, the simultaneo…
We introduce a new algorithm for modularity-based community detection in large networks. The algorithm, which we refer to as a smart local moving algorithm, takes advantage of a well-known local moving heuristic that is also used by other algorithms. Compared with these other algorithms, our proposed algorithm uses the local moving heuristic in a more sophisticated way. Based on an analysis of a diverse set of networks, we show that our smart local moving algorithm identifies community structures with higher modularity values than other algorithms for large-scale modularity optimization, among…
The modularity of a network quantifies the extent, relative to a null model network, to which vertices cluster into community groups. We define a null model appropriate for bipartite networks, and use it to define a bipartite modularity. The bipartite modularity is presented in terms of a modularity matrix B; some key properties of the eigenspectrum of B are identified and used to describe an algorithm for identifying modules in bipartite networks. The algorithm is based on the idea that the modules in the two parts of the network are dependent, with each part mutually being used to induce the…
Community structure is an important property that captures inhomogeneities common in large networks, and modularity is one of the most widely used metrics for such community structure. In this paper, we introduce a principled methodology, the Spectral Graph Forge, for generating random graphs that preserves community structure from a real network of interest, in terms of modularity. Our approach leverages the fact that the spectral structure of matrix representations of a graph encodes global information about community structure. The Spectral Graph Forge uses a low-rank approximation of the m…
Many networks of interest in the sciences, including a variety of social and biological networks, are found to divide naturally into communities or modules. The problem of detecting and characterizing this community structure has attracted considerable recent attention. One of the most sensitive detection methods is optimization of the quality function known as "modularity" over the possible divisions of a network, but direct application of this method using, for instance, simulated annealing is computationally costly. Here we show that the modularity can be reformulated in terms of the eigenv…
The maximization of generalized modularity performs well on networks in which the members of all communities are statistically indistinguishable from each other. However, there is no theory bounding the maximization performance in more realistic networks where edges are heterogeneously distributed within and between communities. Using the random graph properties, we establish asymptotic theoretical bounds on the resolution parameter for which the generalized modularity maximization performs well. From this new perspective on random graph model, we find the resolution limit of modularity maximi…
Some temporal networks, most notably citation networks, are naturally represented as directed acyclic graphs (DAGs). To detect communities in DAGs, we propose a modularity for DAGs by defining an appropriate null model (i.e., randomized network) respecting the order of nodes. We implement a spectral method to approximately maximize the proposed modularity measure and test the method on citation networks and other DAGs. We find that the attained values of the modularity for DAGs are similar for partitions that we obtain by maximizing the proposed modularity (designed for DAGs), the modularity f…
We study networks that display community structure -- groups of nodes within which connections are unusually dense. Using methods from random matrix theory, we calculate the spectra of such networks in the limit of large size, and hence demonstrate the presence of a phase transition in matrix methods for community detection, such as the popular modularity maximization method. The transition separates a regime in which such methods successfully detect the community structure from one in which the structure is present but is not detected. By comparing these results with recent analyses of maximu…
Community structures detection is one of the fundamental problems in complex network analysis towards understanding the topology structures of the network and the functions of it. Nonnegative matrix factorization (NMF) is a widely used method for community detection, and modularity Q and modularity density D are criteria to evaluate the quality of community structures. In this paper, we establish the connections between Q, D and NMF for the first time. Q maximization can be approximately reformulated under the framework of NMF with Frobenius norm, especially when $n$ is large, and D maximizati…
Modularity based community detection encompasses a number of widely used, efficient heuristics for identification of structure in networks. Recently, a belief propagation approach to modularity optimization provided a useful guide for identifying non-trivial structure in single-layer networks in a way that other optimization heuristics have not. In this paper, we extend modularity belief propagation to multilayer networks. As part of this development, we also directly incorporate a resolution parameter. We show that adjusting the resolution parameter affects the convergence properties of the a…
Maximizing the modularity of a network is a successful tool to identify an important community of nodes. However, this combinatorial optimization problem is known to be NP-complete. Inspired by recent nonlinear modularity eigenvector approaches, we introduce the modularity total variation $TV_Q$ and show that its box-constrained global maximum coincides with the maximum of the original discrete modularity function. Thus we describe a new nonlinear optimization approach to solve the equivalent problem leading to a community detection strategy based on $TV_Q$. The proposed approach relies on the…
Modularity-based algorithms used for community detection have been increasing in recent years. Modularity and its application have been generating controversy since some authors argue it is not a metric without disadvantages. It has been shown that algorithms that use modularity to detect communities suffer a resolution limit and, therefore, it is unable to identify small communities in some situations. In this work, we try to apply a density optimization of communities found by the label propagation algorithm and study what happens regarding modularity of optimized results. We introduce a met…
In this paper a simple but efficient real-time detecting algorithm is proposed for tracking community structure of dynamic networks. Community structure is intuitively characterized as divisions of network nodes into subgroups, within which nodes are densely connected while between which they are sparsely connected. To evaluate the quality of community structure of a network, a metric called modularity is proposed and many algorithms are developed on optimizing it. However, most of the modularity based algorithms deal with static networks and cannot be performed frequently, due to their high c…
Many networks including social networks, computer networks, and biological networks are found to divide naturally into communities of densely connected individuals. Finding community structure is one of fundamental problems in network science. Since Newman's suggestion of using \emph{modularity} as a measure to qualify the goodness of community structures, many efficient methods to maximize modularity have been proposed but without a guarantee of optimality. In this paper, we propose two polynomial-time algorithms to the modularity maximization problem with theoretical performance guarantees. …
This paper investigates community detection by modularity maximisation on bipartite networks. In particular we are interested in how the operation of projection, using one node set of the bipartite network to infer connections between nodes in the other set, interacts with community detection. We first define a notion of modularity appropriate for a projected bipartite network and outline an algorithm for maximising it in order to partition the network. Using both real and synthetic networks we compare the communities found by five different algorithms, where each algorithm maximises a differe…
We investigate the impact of community structure on information diffusion with the linear threshold model. Our results demonstrate that modular structure may have counter-intuitive effects on information diffusion when social reinforcement is present. We show that strong communities can facilitate global diffusion by enhancing local, intra-community spreading. Using both analytic approaches and numerical simulations, we demonstrate the existence of an optimal network modularity, where global diffusion require the minimal number of early adopters.
Detecting communities in a network, based only on the adjacency matrix, is a problem of interest to several scientific disciplines. Recently, Zhang and Moore have introduced an algorithm in [P. Zhang and C. Moore, Proceedings of the National Academy of Sciences 111, 18144 (2014)], called mod-bp, that avoids overfitting the data by optimizing a weighted average of modularity (a popular goodness-of-fit measure in community detection) and entropy (i.e. number of configurations with a given modularity). The adjustment of the relative weight, the "temperature" of the model, is crucial for getting a…
Multiscale community detection can be viewed from a dynamical perspective within the Markov Stability framework, which uses the diffusion of a Markov process on the graph to uncover intrinsic network substructures across all scales. Here we reformulate multiscale community detection as a max-sum length vector partitioning problem with respect to the set of time-dependent node vectors expressed in terms of eigenvectors of the transition matrix. This formulation provides a geometric interpretation of Markov Stability in terms of a time-dependent spectral embedding, where the Markov time acts as …
We introduce the Convex Hull of Admissible Modularity Partitions (CHAMP) algorithm to prune and prioritize different network community structures identified across multiple runs of possibly various computational heuristics. Given a set of partitions, CHAMP identifies the domain of modularity optimization for each partition ---i.e., the parameter-space domain where it has the largest modularity relative to the input set---discarding partitions with empty domains to obtain the subset of partitions that are "admissible" candidate community structures that remain potentially optimal over indicated…
Community detection is one of the pivotal tools for discovering the structure of complex networks. Majority of community detection methods rely on optimization of certain quality functions characterizing the proposed community structure. Perhaps, the most commonly used of those quality functions is modularity. Many heuristics are claimed to be efficient in modularity maximization, which is usually justified in relative terms through comparison of their outcomes with those provided by other known algorithms. However as all the approaches are heuristics, while the complete brute-force is not fea…
Spectral analysis has been successfully applied at the detection of community structure of networks, respectively being based on the adjacency matrix, the standard Laplacian matrix, the normalized Laplacian matrix, the modularity matrix, the correlation matrix and several other variants of these matrices. However, the comparison between these spectral methods is less reported. More importantly, it is still unclear which matrix is more appropriate for the detection of community structure. This paper answers the question through evaluating the effectiveness of these five matrices against the ben…
Given a graph of interactions, a module (also called a community or cluster) is a subset of nodes whose fitness is a function of the statistical significance of the pairwise interactions of nodes in the module. The topic of this paper is a model-based community finding approach, commonly referred to as modularity clustering, that was originally proposed by Newman and has subsequently been extremely popular in practice. Various heuristic methods are currently employed for finding the optimal solution. However, the exact computational complexity of this approach is still largely unknown. To this…
Current community detection algorithms operate by optimizing a statistic called modularity, which analyzes the distribution of positively weighted edges in a network. Modularity does not account for negatively weighted edges. This paper introduces a dual assortative modularity measure (DAMM) that incorporates both positively and negatively weighted edges. We describe the the DAMM statistic and illustrate its utility in a community detection algorithm. We evaluate the efficacy of the algorithm on both computer generated and real-world networks, showing that DAMM broadens the domain of networks …
Heuristic and exact modularity optimization with size-constrained communities
Kellogg School of Management, Northwestern University（西北大学凯洛格管理学院）；Department of Mechanical and Industrial Engineering, University of Toronto（多伦多大学机械与工业工程系）；Centre for Science and Technology Studies (CWTS), Leiden University（莱顿大学科学技术研究中心）；Center for Complex Networks and Systems Research, Luddy School of Informatics, Computing, and Engineering, Indiana University Bloomington（印第安纳大学布鲁明顿分校卢迪信息、计算与工程学院复杂网络与系统研究中心）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；University of Oxford（牛津大学）
Because networks can be used to represent many complex systems, they have attracted considerable attention in physics, computer science, sociology, and many other disciplines. One of the most important areas of network science is the algorithmic detection of cohesive groups (i.e., "communities") of nodes. In this paper, we algorithmically detect communities in social networks and image data by optimizing multislice modularity. A key advantage of modularity optimization is that it does not require prior knowledge of the number or sizes of communities, and it is capable of finding network partit…
Recently, a type of multi-resolution methods in community detection was introduced, which can adjust the resolution of modularity by modifying the modularity function with tunable resolution parameters, such as those proposed by Arenas, Fernandez and Gomez and by Reichardt and Bornholdt. In this paper, we show that these methods still have the intrinsic limitation-large communities may have been split before small communities become visible-because it is at the cost of the community stability that the enhancement of the modularity resolution is obtained. The theoretical results indicated that …
One of the most widely used methods for community detection in networks is the maximization of the quality function known as modularity. Of the many maximization techniques that have been used in this context, some of the most conceptually attractive are the spectral methods, which are based on the eigenvectors of the modularity matrix. Spectral algorithms have, however, been limited by and large to the division of networks into only two or three communities, with divisions into more than three being achieved by repeated two-way division. Here we present a spectral algorithm that can directly …
Modularity for multilayer networks, also called multislice modularity, is parametric to a resolution factor and an inter-layer coupling factor. The former is useful to express layer-specific relevance and the latter quantifies the strength of node linkage across the layers of a network. However, such parameters can be set arbitrarily, thus discarding any structure information at graph or community level. Other issues are related to the inability of properly modeling order relations over the layers, which is required for dynamic networks. In this paper we propose a new definition of modularity …
Many social networks and complex systems are found to be naturally divided into clusters of densely connected nodes, known as community structure (CS). Finding CS is one of fundamental yet challenging topics in network science. One of the most popular classes of methods for this problem is to maximize Newman's modularity. However, there is a little understood on how well we can approximate the maximum modularity as well as the implications of finding community structure with provable guarantees. In this paper, we settle definitely the approximability of modularity clustering, proving that appr…
The modularity is a quality function in community detection, which was introduced by Newman and Girvan (2004). Community detection in graphs is now often conducted through modularity maximization: given an undirected graph $G=(V,E)$, we are asked to find a partition $\mathcal{C}$ of $V$ that maximizes the modularity. Although numerous algorithms have been developed to date, most of them have no theoretical approximation guarantee. Recently, to overcome this issue, the design of modularity maximization algorithms with provable approximation guarantees has attracted significant attention in the …
We consider three distinct and well studied problems concerning network structure: community detection by modularity maximization, community detection by statistical inference, and normalized-cut graph partitioning. Each of these problems can be tackled using spectral algorithms that make use of the eigenvectors of matrix representations of the network. We show that with certain choices of the free parameters appearing in these spectral algorithms the algorithms for all three problems are, in fact, identical, and hence that, at least within the spectral approximations used here, there is no di…
We examine a community structure in random graphs of size $n$ and link probability $p/n$ determined with the Newman greedy optimization of modularity. Calculations show that for $p<1$ communities are nearly identical with clusters. For $p=1$ the average sizes of a community $s_{av}$ and of the giant community $s_g$ show a power-law increase $s_{av}\sim n^{α'}$ and $s_g\sim n^α$. From numerical results we estimate $α'\approx 0.26(1)$, $α\approx 0.50(1)$, and using the probability distribution of sizes of communities we suggest that $α'=α/2$ should hold. For $p>1$ the community structure remains…
Complex systems are usually illustrated by networks which captures the topology of the interactions between the entities. To better understand the roles played by the entities in the system one needs to uncover the underlying community structure of the system. In recent years, systems with interactions that have various types or can change over time between the entities have attracted an increasing research attention. However, algorithms aiming to solve the key problem - community detection - in multilayer networks are still limited. In this work, we first introduce the multilayer network mode…
In this paper we investigate how the modularity of model and real-world social networks affect their robustness and the efficacy of node attack (removal) strategies based on node degree (ID) and node betweenness (IB). We build Barabasi-Albert model networks with different modularity by a new ad hoc algorithm that rewire links forming networks with community structure. We traced the network robustness using the largest connected component (LCC). We find that higher level of modularity decreases the model network robustness under both attack strategies, i.e. model network with higher community s…
We propose a novel method to find the community structure in complex networks based on an extremal optimization of the value of modularity. The method outperforms the optimal modularity found by the existing algorithms in the literature. We present the results of the algorithm for computer simulated and real networks and compare them with other approaches. The efficiency and accuracy of the method make it feasible to be used for the accurate identification of community structure in large complex networks.
The generalized version of modularity for multilayer networks, a.k.a. multislice modularity, is characterized by two model parameters, namely resolution factor and inter-layer coupling factor. The former corresponds to a notion of layer-specific relevance, whereas the inter-layer coupling factor represents the strength of node connections across the network layers. Despite the potential of this approach, the setting of both parameters can be arbitrarily selected, without considering specific characteristics from the topology of the multilayer network as well as from an available community stru…
Multi-layer networks are networks on a set of entities (nodes) with multiple types of relations (edges) among them where each type of relation/interaction is represented as a network layer. As with single layer networks, community detection is an important task in multi-layer networks. A large group of popular community detection methods in networks are based on optimizing a quality function known as the modularity score, which is a measure of presence of modules or communities in networks. Hence a first step in community detection is defining a suitable modularity score that is appropriate fo…
Agglomerative clustering is a well established strategy for identifying communities in networks. Communities are successively merged into larger communities, coarsening a network of actors into a more manageable network of communities. The order in which merges should occur is not in general clear, necessitating heuristics for selecting pairs of communities to merge. We describe a hierarchical clustering algorithm based on a local optimality property. For each edge in the network, we associate the modularity change for merging the communities it links. For each community vertex, we call the pr…
A "quantitative function" for community detection called modularity density has been proposed by Li, Zhang, Wang, Zhang, and Chen in $[$Phys. Rev. E 77, 036109 (2008)$]$. We study the modularity density maximization problem and we discuss some features of the optimal solution. More precisely, we show that in the optimal solution there can be communities having negative modularity density, and we propose a modification of the original formulation to overcome this issue. Moreover, we show that a clique can be divided into two or more parts when maximizing the modularity density. We also compare …
Modularity is a widely used measure for evaluating community structure in networks. The definition of modularity involves a comparison of within-community edges in the observed network and that number in an equivalent randomized network. This equivalent randomized network is called the null model, which serves as a reference. To make the comparison significant, the null model should characterize some features of the observed network. However, the null model in the original definition of modularity is unrealistically mixed, in the sense that any node can be linked to any other node without pref…
Community detection in networks is the process of identifying unusually well-connected sub-networks and is a central component of many applied network analyses. The paradigm of modularity optimization stipulates a partition of the network's vertices which maximizes the difference between the fraction of edges within groups (communities) and the expected fraction if edges were randomly distributed. The modularity objective function incorporates the network's topology exclusively and has been extensively studied whereas the integration of constraints or external information on community composit…
