Communities are fundamental entities for the characterization of the structure of real networks. The standard approach to the identification of communities in networks is based on the optimization of a quality function known as "modularity". Although modularity has been at the center of an intense research activity and many methods for its maximization have been proposed, not much it is yet known about the necessary conditions that communities need to satisfy in order to be detectable with modularity maximization methods. Here, we develop a simple theory to establish these conditions, and we successfully apply it to various classes of network models. Our main result is that heterogeneity in the degree distribution helps modularity to correctly recover the community structure of a network and that, in the realistic case of scale-free networks with degree exponent $γ< 2.5$, modularity is always able to detect the presence of communities.
We have analyzed the detectability limits of network communities in the framework of the popular Girvan and Newman benchmark. By carefully taking into account the inevitable stochastic fluctuations that affect the construction of each and every instance of the benchmark, we come to the conclusions that the native, putative partition of the network is completely lost even before the in-degree/out-degree ratio becomes equal to the one of a structure-less Erdös-Rényi network. We develop a simple iterative scheme, analytically well described by an infinite branching-process, to provide an estimate…
Detectability of hierarchical communities in networks
Department of Data Analytics and Digitalisation, School of Business and Economics, Maastricht University（数据分析与数字化学院，经济学院，马斯特里赫特大学）；Department of Computer Science, RWTH Aachen University（计算机科学系，亚琛工业大学）
We investigate the widely encountered problem of detecting communities in multiplex networks, such as social networks, with an unknown arbitrary heterogeneous structure. To improve detectability, we propose a generative model that leverages the multiplicity of a single community in multiple layers, with no prior assumption on the relation of communities among different layers. Our model relies on a novel idea of incorporating a large set of generic localized community label constraints across the layers, in conjunction with the celebrated Stochastic Block Model (SBM) in each layer. Accordingly…
We investigate signed networks with community structure with respect to their spectrum and their evolution under a dynamical model of structural balance, a prominent theory of signed social networks. The spectrum of the adjacency matrix generated by a stochastic block model with two equal size communities shows detectability transitions in which the community structure becomes manifest when its signal eigenvalue appears outside the main spectral band. The spectrum also exhibits "sociality" transitions involving the homogeneous structure representing the average tie value. We derive expressions…
We prove the existence of an asymptotic phase transition threshold on community detectability for the spectral modularity method [M. E. J. Newman, Phys. Rev. E 74, 036104 (2006) and Proc. National Academy of Sciences. 103, 8577 (2006)] under a stochastic block model. The phase transition on community detectability occurs as the inter-community edge connection probability $p$ grows. This phase transition separates a sub-critical regime of small $p$, where modularity-based community detection successfully identifies the communities, from a super-critical regime of large $p$ where successful comm…
As a fundamental structure in real-world networks, in addition to graph topology, communities can also be reflected by abundant node attributes. In attributed community detection, probabilistic generative models (PGMs) have become the mainstream method due to their principled characterization and competitive performances. Here, we propose a novel PGM without imposing any distributional assumptions on attributes, which is superior to the existing PGMs that require attributes to be categorical or Gaussian distributed. Based on the block model of graph structure, our model incorporates the attrib…
Community detection is an important tool for exploring and classifying the properties of large complex networks and should be of great help for spatial networks. Indeed, in addition to their location, nodes in spatial networks can have attributes such as the language for individuals, or any other socio-economical feature that we would like to identify in communities. We discuss in this paper a crucial aspect which was not considered in previous studies which is the possible existence of correlations between space and attributes. Introducing a simple toy model in which both space and node attri…
University College London（伦敦大学学院）；University of Bologna（博洛尼亚大学）；Scuola Normale Superiore（比萨高等师范学校）
We study the inference of a model of dynamic networks in which both communities and links keep memory of previous network states. By considering maximum likelihood inference from single snapshot observations of the network, we show that link persistence makes the inference of communities harder, decreasing the detectability threshold, while community persistence tends to make it easier. We analytically show that communities inferred from single network snapshot can share a maximum overlap with the underlying communities of a specific previous instant in time. This leads to time-lagged inferenc…
In this paper, we study the sensitivity of the spectral clustering based community detection algorithm subject to a Erdos-Renyi type random noise model. We prove phase transitions in community detectability as a function of the external edge connection probability and the noisy edge presence probability under a general network model where two arbitrarily connected communities are interconnected by random external edges. Specifically, the community detection performance transitions from almost perfect detectability to low detectability as the inter-community edge connection probability exceeds …
There has been a surge of interest in community detection in homogeneous single-relational networks which contain only one type of nodes and edges. However, many real-world systems are naturally described as heterogeneous multi-relational networks which contain multiple types of nodes and edges. In this paper, we propose a new method for detecting communities in such networks. Our method is based on optimizing the composite modularity, which is a new modularity proposed for evaluating partitions of a heterogeneous multi-relational network into communities. Our method is parameter-free, scalabl…
Statistical significance of network clustering has been an unresolved problem since it was observed that community detection algorithms produce false positives even in random graphs. After a phase transition between undetectable and detectable cluster structures was discovered, the connection between spectra of adjacency matrices and detectability limits were shown, and both were calculated for a wide range of networks with arbitrary degree distributions and community structure. In practice the full eigenspectrum is not known, and whether a given network has any communities within detectabilit…
Modularity is a popular measure of community structure. However, maximizing the modularity can lead to many competing partitions, with almost the same modularity, that are poorly correlated with each other. It can also produce illusory "communities" in random graphs where none exist. We address this problem by using the modularity as a Hamiltonian at finite temperature, and using an efficient Belief Propagation algorithm to obtain the consensus of many partitions with high modularity, rather than looking for a single partition that maximizes it. We show analytically and numerically that the pr…
We introduce an analytical statistical method to characterize the communities detected in heterogeneous complex systems. By posing a suitable null hypothesis, our method makes use of the hypergeometric distribution to assess the probability that a given property is over-expressed in the elements of a community with respect to all the elements of the investigated set. We apply our method to two specific complex networks, namely a network of world movies and a network of physics preprints. The characterization of the elements and of the communities is done in terms of languages and countries for…
Identifying communities in networks is a fundamental and challenging problem of practical importance in many fields of science. Current methods either ignore the heterogeneous distribution of nodal degrees or assume prior knowledge of the number of communities. Here we propose an efficient hypothesis test for community detection based on quantifying dissimilarities between graphs. Given a random graph, the null hypothesis is that it is of degree-corrected Erdös-Rényi type. We compare the dissimilarity between them by a measure incorporating the vertex distance distribution, the clustering coef…
Rotation dynamics of eigenvectors of modular network adjacency matrices under random perturbations are presented. In the presence of $q$ communities, the number of eigenvectors corresponding to the $q$ largest eigenvalues form a "community" eigenspace and rotate together, but separately from that of the "bulk" eigenspace spanned by all the other eigenvectors. Using this property, the number of modules or clusters in a network can be estimated in an algorithm-independent way. A general argument and derivation for the theoretical detectability limit for sparse modular networks with $q$ communiti…
Applied network science often involves preprocessing network data before applying a network-analysis method, and there is typically a theoretical disconnect between these steps. For example, it is common to aggregate time-varying network data into windows prior to analysis, and the tradeoffs of this preprocessing are not well understood. Focusing on the problem of detecting small communities in multilayer networks, we study the effects of layer aggregation by developing random-matrix theory for modularity matrices associated with layer-aggregated networks with $N$ nodes and $L$ layers, which a…
The problem of community detection is relevant in many scientific disciplines, from social science to statistical physics. Given the impact of community detection in many areas, such as psychology and social sciences, we have addressed the issue of modifying existing well performing algorithms by incorporating elements of the domain application fields, i.e. domain-inspired. We have focused on a psychology and social network - inspired approach which may be useful for further strengthening the link between social network studies and mathematics of community detection. Here we introduce a commun…
Community structure is one of the most important features of complex networks. Modularity-based methods for community detection typically rely on heuristic algorithms to optimize a specific community quality function. Such methods are limited by two major defects: (1) the resolution limit problem, which prohibits communities of heterogeneous sizes being simultaneously detected, and (2) divergent outputs of the heuristic algorithm, which make it difficult to differentiate relevant and irrelevant results. In this paper, we propose an improved method for community detection based on a scalable co…
Recently, a phase transition has been discovered in the network community detection problem below which no algorithm can tell which nodes belong to which communities with success any better than a random guess. This result has, however, so far been limited to the case where the communities have the same size or the same average degree. Here we consider the case where the sizes or average degrees are different. This asymmetry allows us to assign nodes to communities with better-than- random success by examining their local neighborhoods. Using the cavity method, we show that this removes the de…
Community detection algorithms are fundamental tools to understand organizational principles in social networks. With the increasing power of social media platforms, when detecting communities there are two possi- ble sources of information one can use: the structure of social network and node attributes. However structure of social networks and node attributes are often interpreted separately in the research of community detection. When these two sources are interpreted simultaneously, one common as- sumption shared by previous studies is that nodes attributes are correlated with communities.…
Modularity based community detection encompasses a number of widely used, efficient heuristics for identification of structure in networks. Recently, a belief propagation approach to modularity optimization provided a useful guide for identifying non-trivial structure in single-layer networks in a way that other optimization heuristics have not. In this paper, we extend modularity belief propagation to multilayer networks. As part of this development, we also directly incorporate a resolution parameter. We show that adjusting the resolution parameter affects the convergence properties of the a…
Many empirical networks have community structure, in which nodes are densely interconnected within each community (i.e., a group of nodes) and sparsely across different communities. Like other local and meso-scale structure of networks, communities are generally heterogeneous in various aspects such as the size, density of edges, connectivity to other communities and significance. In the present study, we propose a method to statistically test the significance of individual communities in a given network. Compared to the previous methods, the present algorithm is unique in that it accepts diff…
Much of the community detection literature studies structural communities, communities defined solely by the connectivity patterns of the network. Often, networks contain additional metadata which can inform community detection such as the grade and gender of students in a high school social network. In this work, we introduce a tuning parameter to the content map equation that allows users of the Infomap community detection algorithm to control the metadata's relative importance for identifying network structure. On synthetic networks, we show that our algorithm can overcome the structural de…
Recent years have seen increased interest in the online presence of extreme right groups. Although originally composed of dedicated websites, the online extreme right milieu now spans multiple networks, including popular social media platforms such as Twitter, Facebook and YouTube. Ideally therefore, any contemporary analysis of online extreme right activity requires the consideration of multiple data sources, rather than being restricted to a single platform. We investigate the potential for Twitter to act as a gateway to communities within the wider online network of the extreme right, given…
The $\mathbb{S}^1$ model has been a central geometric model in the development of the field of network geometry. It has been mainly studied in its homogeneous regime, in which angular coordinates are independently and uniformly scattered on the circle. We now investigate if the model can generate networks with targeted topological features and soft communities, that is, heterogeneous angular distributions. Under these circumstances, hidden degrees must depend on angular coordinates and we propose a method to estimate them. We conclude that the model can be topologically invariant with respect …
In this paper, we propose some new notions to characterize and analyze the communities. The new notions are general characters of the communities or local structures of networks. At first, we introduce the notions of internal dominating set and external dominating set of a community. We show that most communities in real networks have a small internal dominating set and a small external dominating set, and that the internal dominating set of a community keeps much of the information of the community. Secondly, based on the notions of the internal dominating set and the external dominating set,…
The study of community structure has been a hot topic of research over the last years. But, while successfully applied in several areas, the concept lacks of a general and precise notion. Facts like the hierarchical structure and heterogeneity of complex networks make it difficult to unify the idea of community and its evaluation. The global functional known as modularity is probably the most used technique in this area. Nevertheless, its limits have been deeply studied. Local techniques as the ones by Lancichinetti et al. and Palla et al. arose as an answer to the resolution limit and degener…
Over the last decade, random hyperbolic graphs have proved successful in providing geometric explanations for many key properties of real-world networks, including strong clustering, high navigability, and heterogeneous degree distributions. These properties are ubiquitous in systems as varied as the internet, transportation, brain or epidemic networks, which are thus unified under the hyperbolic network interpretation on a surface of constant negative curvature. Although a few studies have shown that hyperbolic models can generate community structures, another salient feature observed in real…
Heterogeneous networks are networks consisting of different types of nodes and multiple types of edges linking such nodes. While community detection has been extensively developed as a useful technique for analyzing networks that contain only one type of nodes, very few community detection techniques have been developed for heterogeneous networks. In this paper, we propose a modularity based community detection framework for heterogeneous networks. Unlike existing methods, the proposed approach has the flexibility to treat the number of communities as an unknown quantity. We describe a Louvain…
The maximization of generalized modularity performs well on networks in which the members of all communities are statistically indistinguishable from each other. However, there is no theory bounding the maximization performance in more realistic networks where edges are heterogeneously distributed within and between communities. Using the random graph properties, we establish asymptotic theoretical bounds on the resolution parameter for which the generalized modularity maximization performs well. From this new perspective on random graph model, we find the resolution limit of modularity maximi…
In the past decade, cities have experienced rapid growth, expansion, and changes in their community structure. Many aspects of critical urban infrastructure are closely coupled with the human communities that they serve. Urban communities are composed of a multiplex of overlapping factors which can be distinguished into cultural, religious, social-economic, political, and geographical layers. In this paper, we review how increasingly available heterogeneous mobile big data sets can be leveraged to detect the community interaction structure using natural language processing and machine learning…
In this paper we study a simple cascading process in a structured heterogeneous population, namely, a network composed of two loosely coupled communities. We demonstrate that under certain conditions the cascading dynamics in such a network has a two--tiered structure that characterizes activity spreading at different rates in the communities. We study the dynamics of the model using both simulations and an analytical approach based on annealed approximation, and obtain good agreement between the two. Our results suggest that network modularity might have implications in various applications, …
Identifying topologically well-defined communities that are also homogeneous w.r.t. attributes carried by the nodes that compose them is a challenging social network analysis task. We address such a problem by introducing Eva, a bottom-up low complexity algorithm designed to identify network hidden mesoscale topologies by optimizing structural and attribute-homophilic clustering criteria. We evaluate the proposed approach on heterogeneous real-world labeled network datasets, such as co-citation, linguistic, and social networks, and compare it with state-of-art community discovery competitors. …
The hidden metric space behind complex network topologies is a fervid topic in current network science and the hyperbolic space is one of the most studied, because it seems associated to the structural organization of many real complex systems. The Popularity-Similarity-Optimization (PSO) model simulates how random geometric graphs grow in the hyperbolic space, reproducing strong clustering and scale-free degree distribution, however it misses to reproduce an important feature of real complex networks, which is the community organization. The Geometrical-Preferential-Attachment (GPA) model was…
This paper introduces a new concept of least community that is as homogeneous as a random graph, and develops a new community detection algorithm from the perspective of homogeneity or heterogeneity. Based on this concept, we adopt head/tail breaks - a newly developed classification scheme for data with a heavy-tailed distribution - and rely on edge betweenness given its heavy-tailed distribution to iteratively partition a network into many heterogeneous and homogeneous communities. Surprisingly, the derived communities for any self-organized and/or self-evolved large networks demonstrate very…
Social interactions vary in time and appear to be driven by intrinsic mechanisms, which in turn shape the emerging structure of the social network. Large-scale empirical observations of social interaction structure have become possible only recently, and modelling their dynamics is an actual challenge. Here we propose a temporal network model which builds on the framework of activity-driven time-varying networks with memory. The model also integrates key mechanisms that drive the formation of social ties - social reinforcement, focal closure and cyclic closure, which have been shown to give ri…
Community detection and hierarchy extraction are usually thought of as separate inference tasks on networks. Considering only one of the two when studying real-world data can be an oversimplification. In this work, we present a generative model based on an interplay between community and hierarchical structures. It assumes that each node has a preference in the interaction mechanism and nodes with the same preference are more likely to interact, while heterogeneous interactions are still allowed. The sparsity of the network is exploited for implementing a more efficient algorithm. We demonstra…
Multilayer networks or MLNs (also called multiplexes or network of networks) are being used extensively for modeling and analysis of data sets with multiple entity and feature types as well as their relationships. As the concept of communities and hubs are used for these analysis, a structure-preserving definition for them on MLNs (that retains the original MLN structure and node/edge labels and types) and its efficient detection are critical. There is no structure-preserving definition of a community for a MLN as most of the current analyses aggregate a MLN to a single graph. Although there i…
We examine world migration as a social-spatial network of countries connected via movements of people. We assess how multilateral migratory relationships at global, regional, and local scales coexist ("glocalization"), divide ("polarization"), or form an interconnected global system ("globalization"). To do this, we decompose the world migration network (WMN) into communities---sets of countries with denser than expected migration connections---and characterize their pattern of local (i.e., intracommunity) and global (i.e., intercommunity) connectivity. We distinguish community signatures---"c…
Inter-firm organizations, which play a driving role in the economy of a country, can be represented in the form of a customer-supplier network. Such a network exhibits a heavy-tailed degree distribution, disassortative mixing and a prominent community structure. We analyze a large-scale data set of customer-supplier relationships containing data from one million Japanese firms. Using a directed network framework, we show that the production network exhibits the characteristics listed above. We conduct detailed investigations to characterize the communities in the network. The topology within s…
The design of an efficient curing policy, able to stem an epidemic process at an affordable cost, has to account for the structure of the population contact network supporting the contagious process. Thus, we tackle the problem of allocating recovery resources among the population, at the lowest cost possible to prevent the epidemic from persisting indefinitely in the network. Specifically, we analyze a susceptible-infected-susceptible epidemic process spreading over a weighted graph, by means of a first-order mean-field approximation. First, we describe the influence of the contact network on…
Multilayer networks are a useful way to capture and model multiple, binary or weighted relationships among a fixed group of objects. While community detection has proven to be a useful exploratory technique for the analysis of single-layer networks, the development of community detection methods for multilayer networks is still in its infancy. We propose and investigate a procedure, called Multilayer Extraction, that identifies densely connected vertex-layer sets in multilayer networks. Multilayer Extraction makes use of a significance based score that quantifies the connectivity of an observe…
The structure of real-world multilayer infrastructure systems usually exhibits anisotropy due to constraints of the embedding space. For example, geographical features like mountains, rivers and shores influence the architecture of critical infrastructure networks. Moreover, such spatial networks are often non-homogeneous but rather have a modular structure with dense connections within communities and sparse connections between neighboring communities. When the networks of the different layers are interdependent, local failures and attacks may propagate throughout the system. Here we study th…
A classic measure of ecological stability describes the tendency of a community to return to equilibrium after small perturbation. While many advances show how the network structure of these communities severely constrains such tendencies, few if any of these advances address one of the most fundamental properties of network structure: heterogeneity among nodes with different numbers of links. Here we systematically explore this property of "degree heterogeneity" and find that its effects on stability systematically vary with different types of interspecific interactions. Degree heterogeneity …
