DOCSLIB.ORG

Exploratory Analysis of Networks

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Exploratory Analysis of Networks Exploratory Analysis of Networks James D. Wilson Data Institute Conference 2017 James D. Wilson (USF) Exploratory Analysis of Networks 1 / 35 Summarizing Networks Goals 1 Provide one number summaries of network 2 Develop hypotheses about observed data 3 Motivate predictive models 4 Augment standard multivariate analyses James D. Wilson (USF) Exploratory Analysis of Networks 2 / 35 At first glance: Network Summary Statistics Measures of Connectivity Degree: number of edges incident to each node (popularity) For directed networks, we consider In- and Out-Degree Geodesic distance: shortest path between two nodes Diameter: longest geodesic distance Clustering coefficient: fraction of triangles to total triples Reciprocity: fraction of reciprocated ties (directed graphs only) James D. Wilson (USF) Exploratory Analysis of Networks 3 / 35 Examples in Social Networks James D. Wilson (USF) Exploratory Analysis of Networks 4 / 35 At first glance: Network Summary Statistics Measures of Nodal Influence / Centrality Centrality: how ”central” is a node in the observed network? Degree: popularity of a node Eigenvector: self- neighbor- importance Betweenness: extent+ to which a node lies on the shortest path between two nodes Closeness: mean distance from node to all other vertices Authorities: vertices with useful information on a topic Hubs: vertices that point to authorities James D. Wilson (USF) Exploratory Analysis of Networks 5 / 35 Examples Internet Search PageRank Algorithm Popular web-search scoring and search algorithm (Google) Score based on eigenvector centrality HITS Algorithm (Kleinberg, 1999) Hyperlink-induced topic search Uses hub and authority scores as basis for measuring importance of webpages James D. Wilson (USF) Exploratory Analysis of Networks 6 / 35 Example: Recommendation Systems Figure: eye2data.blogspot.com Want individual (node) with most influence James D. Wilson (USF) Exploratory Analysis of Networks 7 / 35 Example: Epidemics Where did this outbreak begin? James D. Wilson (USF) Exploratory Analysis of Networks 8 / 35 Community Structure Informally: Communities in a network are subgraphs C1;:::; Ck n such that ⊆ [ ] Edge density within sets Ci is large Edge density between sets Ci is small James D. Wilson (USF) Exploratory Analysis of Networks 9 / 35 Community Structure In the adjacency matrix, re-ordering the rows and columns according to community labels / modules will result in densely connected “blocks” along the diagonal Networks with community structure are said to be assortative James D. Wilson (USF) Exploratory Analysis of Networks 10 / 35 Aims of Community Detection Aim: Capture relevant structure of a complex system Example 1: Facebook friendship networks User friendships Geographic location of user → Example 2: Human Connectome Clustering of regions signify “functional regions” James D. Wilson (USF) Exploratory Analysis of Networks 11 / 35 Community Detection Approaches In general, community detection (when well-defined mathematically) is NP-hard. Thus, identifying communities requires approximate algorithms That being the case, there is no shortage of computational algorithms to identify communities We will describe several key approaches - i.e., ways to define community structure. For each of these approaches there are many algorithms available (which we won’t detail here) Several review articles and one 100 + review on algorithms James D. Wilson (USF) Exploratory Analysis of Networks 12 / 35 Key Community Detection Approaches Min-cut Identify cut of vertices that "cuts" the fewest edges Modularity Partition that deviates most from organization in random graph Spectral Focus on spectral properties of graph Laplacian Stochastic Block Model Model-based approach. Relies on maximum likelihood estimation Extraction Local, significance based algorithms James D. Wilson (USF) Exploratory Analysis of Networks 13 / 35 The Min-k-Cut Approach Goal (Min-cut Max flow problem): Find the partition of vertices Π C1 ::: Ck whose communities have the minimum number of edges between them (Goldberg and Tarjan, 1988) = ∪ ∪ The cut of two communities C1; C2 n is: 1 ⊂ [ ] cut C1; C2 Ai;j I i C1; j C2 2 i;j ( ) = Q ( ∈ ∈ ) We seek the partition - k−1 k Min-k-Cut G argminΠ cut C`; Cm `=1 m=`+1 ( ) = Q Q ( ) James D. Wilson (USF) Exploratory Analysis of Networks 14 / 35 Min-k-Cut Question: What happens if we search for the Min-2-cut when there are nodes that are only connected to one other node? James D. Wilson (USF) Exploratory Analysis of Networks 15 / 35 Normalized Cut Problem with Min-Cut: Tends to find many singleton communities! To address this, one can normalize the cut between two communities by their size (Ratio-Cut) or by their volume (Norm-cut) Normalized-Cut (Shi and Malik, 2000): Define the volume of a collection B n as: vol B di . Then, i∈B ⊂ [ ] ( ) = Q k−1 k cut B`; Bm cut B`; Bm Min-Norm-k-Cut G argminΠ `=1 m=`+1 vol B` vol Bm ( ) ( ) ( ) = Q Q + ( ) ( ) James D. Wilson (USF) Exploratory Analysis of Networks 16 / 35 Normalized Cut Addresses the issue of singleton communities but ... Issue: When k 2, finding the solution to the Norm-Cut is NP-hard. > Fortunately, an approximate solution can be found! James D. Wilson (USF) Exploratory Analysis of Networks 17 / 35 Connected Components A connected component of an undirected graph is a collection of vertices C V such that There is⊆ a path between u and v for all u; v C There is no path between u and v for u C and∈ v V C ∈ ∈ Note: Partitioning a network into its connected components is an “extreme” example of community detection. So we want to identify communities that are “like” disjoint connected components James D. Wilson (USF) Exploratory Analysis of Networks 18 / 35 Spectral Clustering and The Graph Laplacian n×n Define D diag d1;:::; dn R where Graph Laplacian= ( L: ) ∈ L D A = − Normalized graph laplacian Lnorm: −1 −1 Lnorm D L I D A = = − James D. Wilson (USF) Exploratory Analysis of Networks 19 / 35 Key Property of the Graph Laplacian Theorem 1. Let G be an undirected graph with non-negative weights and let Lnorm be its normalized graph laplacian. Let k the multiplicity of the eigenvalue 0 of Lnorm. Then, (1) k is= the number of connected components C1;:::; Ck in G (2) The eigenspace of 0 is spanned by the indicator vectors 1Ci Key Point: If G clustered into k disjoint connected components, then we can perfectly identify the k clusters using the k smallest eigenvectors James D. Wilson (USF) Exploratory Analysis of Networks 20 / 35 Spectral Clustering Algorithm n×n Input: Adjacency matrix A R+ , number of communities k 1 Calculate normalized graph∈ laplacian Lnorm 2 Compute X the n k matrix of the k smallest eigenvectors of Lnorm = × 3 Cluster the rows of X using k-means Output: Clusters C1;:::; Ck James D. Wilson (USF) Exploratory Analysis of Networks 21 / 35 Properties of Spectral Clustering Requires a prespecified number of clusters k Works perfectly in an ideal scenario Requires the use of another clustering method (k-means) The solution to a relaxed version of the normalized-cut problem Reference (seriously, read this): Ulrike Von Luxburg "A tutorial on spectral clustering" (2006) James D. Wilson (USF) Exploratory Analysis of Networks 22 / 35 Stochastic Block Model (SBM) Model-based approach to community detection G V n ; E with binary adjacency matrix A Assumes= ( = [ that] G) has k blocks generated as follows: 1 Community labels c c1;:::; cn generated at random: = iid( ) c1;:::; cn multinomial 1; π π1; : : : ; πk ∼ ( = { }) 2 Conditional on c, A u; v are independent Bernoulli rvs with ( ) E A u; v c Pcu ;cv Reference: Holland, et al. "Stochastic[ ( )S block] = models: first steps" (1983) James D. Wilson (USF) Exploratory Analysis of Networks 23 / 35 Stochastic Block Model (SBM) Observe G Go, calculate likelihood Θ Go; k with Θ P; c Finding c becomes= an estimation problem:L( S ) = { } Θ arg max Θ Go; k Θ ̂ Requires approximate algorithms= L like( MCMCS ) or variational EM Issue: Approximate algorithms can be slow! James D. Wilson (USF) Exploratory Analysis of Networks 24 / 35 Modularity Aim: find the partition of G whose communities contain the highest density of edges relative to the expected density of edges Remarks: Requires a notion of what a random network looks like The choice of a null network model affects resulting communities This is the most widely adopted approach to community detection! Reference: Mark E Newman "Modularity and community structure in networks" (2004) James D. Wilson (USF) Exploratory Analysis of Networks 25 / 35 Modularity Graph G V n ; E , adjacency matrix A Au;v Modularity= ( ):= Measures[ ] ) the “significance"= of[ partition] c: Q 1 dud v c A u; v I cu cv 2 E u;v 2 E ( ) Q( ) = Q ( ) − { = } Measures the averageS S departure of observedS S edge density from expected edge density James D. Wilson (USF) Exploratory Analysis of Networks 26 / 35 Modularity Maximization Aim: Find the labels c∗ 1;:::; k n that maximizes modularity: ∈ { } c∗ arg max c NP hard optimization problem= {Q} Many approximate algorithms developed Reference: Santo Fortunato, "Community detection in graphs" (2009). [100+ page review paper] James D. Wilson (USF) Exploratory Analysis of Networks 27 / 35 Community Extraction Basic Idea: Identify communities Ci V one at a time via iterative search Remove/avoid C1;:::; Ci⊆when searching for Ci+1 Virtues: Possible to accommodate overlap Automatic selection of number of communities Parallelizable! Can easily scale to large networks. James D. Wilson (USF) Exploratory Analysis of Networks 28 / 35 Community Extraction Methods Methods: OSLOM: Lancichinetti, et al. "Finding statistically significant communities in networks" (2011) – resampling based method Extraction: Zhao, et al. "Community extraction for social networks" (2011) – score-based residualizing ESSC: Wilson, et al. "A testing based extraction algorithm for identifying significant communities in networks" (2014) – hypothesis testing based extraction James D. Wilson (USF) Exploratory Analysis of Networks 29 / 35 Significance based Community Extraction James D.
Load more

Recommended publications
  • Arxiv:1705.10225V8 [Stat.ML] 6 Feb 2020
    Bayesian stochastic blockmodelinga Tiago P. Peixotoy Department of Mathematical Sciences and Centre for Networks and Collective Behaviour, University of Bath, United Kingdom and ISI Foundation, Turin, Italy This chapter provides a self-contained introduction to the use of Bayesian inference to ex- tract large-scale modular structures from network data, based on the stochastic blockmodel (SBM), as well as its degree-corrected and overlapping generalizations. We focus on non- parametric formulations that allow their inference in a manner that prevents overfitting, and enables model selection. We discuss aspects of the choice of priors, in particular how to avoid underfitting via increased Bayesian hierarchies, and we contrast the task of sampling network partitions from the posterior distribution with finding the single point estimate that maximizes it, while describing efficient algorithms to perform either one. We also show how inferring the SBM can be used to predict missing and spurious links, and shed light on the fundamental limitations of the detectability of modular structures in networks. arXiv:1705.10225v8 [stat.ML] 6 Feb 2020 aTo appear in “Advances in Network Clustering and Blockmodeling,” edited by P. Doreian, V. Batagelj, A. Ferligoj, (Wiley, New York, 2019 [forthcoming]). y [email protected] 2 CONTENTS I. Introduction 3 II. Structure versus randomness in networks 3 III. The stochastic blockmodel (SBM) 5 IV. Bayesian inference: the posterior probability of partitions 7 V. Microcanonical models and the minimum description length principle (MDL) 11 VI. The “resolution limit” underfitting problem, and the nested SBM 13 VII. Model variations 17 A. Model selection 18 B. Degree correction 18 C.
    [Show full text]
  • On Community Structure in Complex Networks: Challenges and Opportunities
    On community structure in complex networks: challenges and opportunities Hocine Cherifi · Gergely Palla · Boleslaw K. Szymanski · Xiaoyan Lu Received: November 5, 2019 Abstract Community structure is one of the most relevant features encoun- tered in numerous real-world applications of networked systems. Despite the tremendous effort of a large interdisciplinary community of scientists working on this subject over the past few decades to characterize, model, and analyze communities, more investigations are needed in order to better understand the impact of community structure and its dynamics on networked systems. Here, we first focus on generative models of communities in complex networks and their role in developing strong foundation for community detection algorithms. We discuss modularity and the use of modularity maximization as the basis for community detection. Then, we follow with an overview of the Stochastic Block Model and its different variants as well as inference of community structures from such models. Next, we focus on time evolving networks, where existing nodes and links can disappear, and in parallel new nodes and links may be introduced. The extraction of communities under such circumstances poses an interesting and non-trivial problem that has gained considerable interest over Hocine Cherifi LIB EA 7534 University of Burgundy, Esplanade Erasme, Dijon, France E-mail: hocine.cherifi@u-bourgogne.fr Gergely Palla MTA-ELTE Statistical and Biological Physics Research Group P´azm´any P. stny. 1/A, Budapest, H-1117, Hungary E-mail: [email protected] Boleslaw K. Szymanski Department of Computer Science & Network Science and Technology Center Rensselaer Polytechnic Institute 110 8th Street, Troy, NY 12180, USA E-mail: [email protected] Xiaoyan Lu Department of Computer Science & Network Science and Technology Center Rensselaer Polytechnic Institute 110 8th Street, Troy, NY 12180, USA E-mail: [email protected] arXiv:1908.04901v3 [physics.soc-ph] 6 Nov 2019 2 Cherifi, Palla, Szymanski, Lu the last decade.
    [Show full text]
  • Community Detection with Dependent Connectivity∗
    Submitted to the Annals of Statistics arXiv: arXiv:0000.0000 COMMUNITY DETECTION WITH DEPENDENT CONNECTIVITY∗ BY YUBAI YUANy AND ANNIE QUy University of Illinois at Urbana-Champaigny In network analysis, within-community members are more likely to be connected than between-community members, which is reflected in that the edges within a community are intercorrelated. However, existing probabilistic models for community detection such as the stochastic block model (SBM) are not designed to capture the dependence among edges. In this paper, we propose a new community detection approach to incorporate intra-community dependence of connectivities through the Bahadur representation. The pro- posed method does not require specifying the likelihood function, which could be intractable for correlated binary connectivities. In addition, the pro- posed method allows for heterogeneity among edges between different com- munities. In theory, we show that incorporating correlation information can achieve a faster convergence rate compared to the independent SBM, and the proposed algorithm has a lower estimation bias and accelerated convergence compared to the variational EM. Our simulation studies show that the pro- posed algorithm outperforms the popular variational EM algorithm assuming conditional independence among edges. We also demonstrate the application of the proposed method to agricultural product trading networks from differ- ent countries. 1. Introduction. Network data has arisen as one of the most common forms of information collection. This is due to the fact that the scope of study not only focuses on subjects alone, but also on the relationships among subjects. Networks consist of two components: (1) nodes or ver- tices corresponding to basic units of a system, and (2) edges representing connections between nodes.
    [Show full text]
  • Developing a Credit Scoring Model Using Social Network Analysis
    Developing a Credit Scoring Model Using Social Network Analysis Ahmad Abd Rabuh UP821483 This thesis is submitted in fulfilment of the requirements for the award of the degree of Doctor of Philosophy in the School of Business and Law at the University of Portsmouth November 2020 1 Author’s Declaration Whilst registered as a candidate for the above degree, I have not been registered for any other research award. The results and conclusions embodied in this thesis are the work of the named candidate and have not been submitted for any other academic award. Signature: Name: Ahmad Abd Rabuh Date: the 30th of April 2020 Word Count: 63,809 2 Acknowledgements As I witness this global pandemic, my greatest gratitude goes to my mother, Hanan, for her moral support and encouragement through the good, the bad and the ugly. During the period of my PhD revision, I received mentoring and help from my future lifetime partner, Kawthar. Also, I have been blessed with the help of wonderful people from Scholars at Risk, Rose Anderson and Sarina Rosenthal, who assisted me in the PhD application and scholarship processes. Additionally, I cannot be thankful enough to the Associate Dean of Research, Andy Thorpe, who supported me in every possible way during my time at the University of Portsmouth. Finally, many thanks to my supervisors, Mark Xu and Renatas Kizys for their continuous guidance. 3 Developing a Credit Scoring Model Using Social Network Analysis Table of Contents Abstract ......................................................................................................................................... 10 CHAPTER 1: INTRODUCTION ................................................................................................. 12 1.1. Research Questions ........................................................................................................ 16 1.2. Aims and Objectives .....................................................................................................
    [Show full text]
  • On Spectral Embedding Performance and Elucidating Network Structure in Stochastic Block Model Graphs
    On spectral embedding performance and elucidating network structure in stochastic block model graphs Joshua Cape and Minh Tang and Carey E. Priebe Department of Applied Mathematics and Statistics Johns Hopkins University August 16, 2018 Abstract Statistical inference on graphs often proceeds via spectral methods involving low- dimensional embeddings of matrix-valued graph representations, such as the graph Laplacian or adjacency matrix. In this paper, we analyze the asymptotic information- theoretic relative performance of Laplacian spectral embedding and adjacency spectral embedding for block assignment recovery in stochastic block model graphs by way of Chernoff information. We investigate the relationship between spectral embedding per- formance and underlying network structure (e.g. homogeneity, affinity, core-periphery, (un)balancedness) via a comprehensive treatment of the two-block stochastic block model and the class of K-block models exhibiting homogeneous balanced affinity struc- ture. Our findings support the claim that, for a particular notion of sparsity, loosely speaking, “Laplacian spectral embedding favors relatively sparse graphs, whereas adja- cency spectral embedding favors not-too-sparse graphs.” We also provide evidence in support of the claim that “adjacency spectral embedding favors core-periphery network structure.” 2010 Mathematics Subject Classification: Primary 62H30; Secondary 62B10. arXiv:1808.04855v1 [math.ST] 14 Aug 2018 Keywords: Statistical network analysis; random graphs; stochastic block model; Laplacian spectral embed- ding; adjacency spectral embedding; Chernoff information; vertex clustering. 1 1 Preface The stochastic block model (SBM) (Holland et al., 1983) is a simple yet ubiquitous network model capable of capturing community structure that has been widely studied via spectral methods in the mathematics, statistics, physics, and engineering communities.
    [Show full text]
  • Link Prediction in Large-Scale Complex Networks (Application to Bibliographical Networks)
    N◦ attribué par la bibliothèque | _ | _ | _ | _ | _ | _ | _ | _ | _ | _ | Université Paris Nord Doctoral Thesis Link Prediction in Large-scale Complex Networks (Application to Bibliographical Networks) Author: Manisha Pujari A thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science in the research team Apprentissage Artificiel et Applications LIPN CNRS UMR-7030 Jury: Reviewer: Céline Robardet Professor INSA Lyon Bénédicte le Grand Professor Université Paris 1 Panthéon Sorbonne Examiner: Aldo Gangemi Professor SPC, Université Paris 13 Christophe Prieur Associate Professor, HDR Université Paris Diderot Director: Céline Rouveirol Professor SPC, Université Paris 13 Supervisor: Rushed Kanawati Associate Professor SPC, Université Paris 13 “ The more I learn, the more I realize how much I don’t know. ” Albert Einstein Abstract Link Prediction in Large-scale Complex Networks (Application to Bibliographical Networks) In this work, we are interested to tackle the problem of link prediction in complex net- works. In particular, we explore topological dyadic approaches for link prediction. Dif- ferent topological proximity measures have been studied in the scientific literature for finding the probability of appearance of new links in a complex network. Supervised learning methods have also been used to combine the predictions made or information provided by different topological measures. They create predictive models using various topological measures. The problem of supervised learning for link prediction is a difficult problem especially due to the presence of heavy class imbalance. In this thesis, we search different alternative approaches to improve the performance of different dyadic approaches for link prediction.
    [Show full text]
  • Dynamic Stochastic Block Models, Clustering and Segmentations in Dynamic Graphs Marco Corneli
    Dynamic stochastic block models, clustering and segmentations in dynamic graphs Marco Corneli To cite this version: Marco Corneli. Dynamic stochastic block models, clustering and segmentations in dynamic graphs. Mathematics [math]. Université Paris 1 - Panthéon Sorbonne, 2017. English. tel-01644866 HAL Id: tel-01644866 https://hal.archives-ouvertes.fr/tel-01644866 Submitted on 22 Nov 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Université Paris 1 Pahntéon-Sorbonne Laboratoire SAMM Thèse présentée en première version en vu d’obtenir le grade de Docteur, spécialité "Mathématiques Appliquées" par Marco Corneli Dynamic stochastic block models, clustering and segmentation in dynamic graphs Thèse soutenue le 17 Novembre 2017 devant le jury composé de : M. Christophe Biernacki Université Lille 1 Sciences et Technologies (Rapporteur) M. Thomas Brendan Murphy University College Dublin (Rapporteur) M. Allou-Badara Samé IFSTTAR (Examinateur) Mme. Clémence Magnien CNRS (Examinatrice) Mme. Sophie Lèbre Université Paul Valéry Montpellier 3 (Examinatrice) M. Fabrice Rossi Université Paris 1 Panthéon-Sorbonne (Directeur) M. Pierre Latouche Université Paris 1 Panthéon-Sorbonne (Co-encadrant) Alla mia famiglia... Remerciements e remercie Fabrice Rossi et Pierre Latouche pour m’avoir fait confiance J et avoir encadré ce travail de thèse.
    [Show full text]
  • Efficient Inference in Stochastic Block Models with Vertex Labels
    1 Efficient inference in stochastic block models with vertex labels Clara Stegehuis and Laurent Massoulie´ Abstract—We study the stochastic block model with two connectivity matrix and d the average degree in the network. communities where vertices contain side information in the form Underneath the Kesten-Stigum threshold, no algorithm is able of a vertex label. These vertex labels may have arbitrary label to infer the community memberships better than a random distributions, depending on the community memberships. We analyze a linearized version of the popular belief propagation guess, even though a community structure may be present [14]. algorithm. We show that this algorithm achieves the highest In this setting, it is even impossible to discriminate between accuracy possible whenever a certain function of the network a graph generated by the stochastic block model and an parameters has a unique fixed point. Whenever this function has Erdos˝ Renyi´ random graph with the same average degree, multiple fixed points, the belief propagation algorithm may not even though a community structure is present [13]. Above the perform optimally. We show that increasing the information in the vertex labels may reduce the number of fixed points and Kesten-Stigum threshold, the communities can be efficiently hence lead to optimality of belief propagation. reconstructed [11], [15]. A popular algorithm for community detection is belief propagation [5] (BP). This algorithm starts with initial beliefs I. INTRODUCTION on the community memberships, and iterates until these beliefs converge to a fixed point. Above the Kesten-Stigum threshold, Many real-world networks contain community structures: a fixed point that is correlated with the true community mem- groups of densely connected nodes.
    [Show full text]
  • A Review of Stochastic Block Models and Extensions for Graph Clustering
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Newcastle University E-Prints Lee and Wilkinson Applied Network Science (2019) 4:122 Applied Network Science https://doi.org/10.1007/s41109-019-0232-2 REVIEW Open Access A review of stochastic block models and extensions for graph clustering Clement Lee1* and Darren J. Wilkinson2,3 *Correspondence: [email protected] Abstract 1Department of Mathematics and There have been rapid developments in model-based clustering of graphs, also known Statistics, Lancaster University, Lancaster, United Kingdom as block modelling, over the last ten years or so. We review different approaches and Full list of author information is extensions proposed for different aspects in this area, such as the type of the graph, the available at the end of the article clustering approach, the inference approach, and whether the number of groups is selected or estimated. We also review models that combine block modelling with topic modelling and/or longitudinal modelling, regarding how these models deal with multiple types of data. How different approaches cope with various issues will be summarised and compared, to facilitate the demand of practitioners for a concise overview of the current status of these areas of literature. Keywords: Model-based clustering, Stochastic block models, Mixed membership models, Topic modelling, Longitudinal modelling, Statistical inference Introduction Stochastic block models (SBMs) are an increasingly popular class of models in statistical analysis of graphs or networks. They can be used to discover or understand the (latent) structure of a network, as well as for clustering purposes.
    [Show full text]
  • Interpretable Stochastic Block Influence Model: Measuring Social Influence Among Homophilous Communities
    Interpretable Stochastic Block Influence Model: measuring social influence among homophilous communities YAN LENG, MIT Media Lab, McCombs School of Business at UT-Austin TARA SOWRIRAJAN, Harvard University, MIT Media Lab ALEX PENTLAND, MIT Media Lab Decision-making on networks can be explained by both homophily and social influence. While homophily drives the formation of communities with similar characteristics, social influence occurs both within and between communities. Social influence can be reasoned through role theory, which indicates that the influences among individuals depend on their roles and the behavior of interest. To operationalize these social science theories, we empirically identify the homophilous communities and use the community structures to capture the “roles”, which affect the particular decision-making processes. We propose a generative model named Stochastic Block Influence Model and jointly analyze both the network formation and the behavioral influence within and between different empirically-identified communities. To evaluate the performance and demonstrate the interpretability of our method, we study the adoption decisions of microfinance in an Indian village. We show that although individuals tend to form links within communities, there are strong positive and negative social influences between communities, supporting the weak tie theory. Moreover, we find that communities with shared characteristics are associated with positive influence. In contrast, the communities with a lack of overlap are associated with negative influence. Our framework facilitates the quantification of the influences underlying decision communities and is thus a useful tool for driving information diffusion, viral marketing, and technology adoptions. Key Words: Social influence; Homophily; Stochastic Block Model; Community structure; Generative model 1 INTRODUCTION We are living in an increasingly connected society [4, 26, 28, 37].
    [Show full text]
  • Barabasi-Albert Scale-Free Networks • Repeat Measurements Multiple Times and Plot Histograms of Assortativity
    Random Networks © Mario Freese Hiroki Sayama [email protected] Small-World Phenomenon “Bacon Number” • BN = 1 for those who co-starred with Kevin Bacon in a film • BN = 2 for those who co-starred with actors/actresses with BN=1 … • Mostly BN <= 3 !! • The largest finite BN = 8 !! 3 “Erdös Number” • EN = 1 for those who co-authored a paper with a Hungarian mathematician Paul Erdös (1913-1996) • EN = 2 for those who co-authored a paper with authors with EN=1 … • Mostly EN <= 7 !! • The largest finite EN = 13 !! FYI - Hiroki’s EN=4 (by Bing/Microsoft Academic Search) 4 3.5 degrees of separation in FB • https://research.facebook.com/blog/three-and-a- half-degrees-of-separation/ 5 “Small-world” phenomenon • Most real-world networks are remarkably “small” – Despite a huge number of nodes involved – Even if connections are relatively sparse • Why? 6 Random Networks Classical explanation: Erdös-Rényi random network model • A network made of N nodes • Each node pair is connected randomly and independently with probability p • A small characteristic path length is realized because of randomness – Number of nodes reachable from a single node within k steps increases exponentially with k 8 Exercise • Create and plot a few ER random networks using NetworkX • Measure their properties – Network density – Characteristic path length – Clustering coefficient – Degree distribution – etc. 9 Limitation of ER networks • ER random networks have very few loops or local clusters if connection probability is small • Real-world networks are often clustered with
    [Show full text]
  • Dynamic Stochastic Blockmodel Regression for Social Networks: Application to International Militarized Conflicts∗
    Dynamic Stochastic Blockmodel Regression for Social Networks: Application to International Militarized Conflicts∗ Santiago Olivellay Tyler Prattz Kosuke Imaix First Draft: July 12, 2018 This Draft: December 23, 2019 Abstract A primary goal of social science research is to understand how latent group mem- berships predict the dynamic process of social network evolution. In the modeling of international conflicts scholars hypothesize that membership in geopolitical coalitions shapes the decision to engage in militarized conflict. Such theories explain the ways in which nodal and dyadic characteristics affect the evolution of relational ties over time via their effects on group memberships. To aid the empirical testing of these arguments, we develop a dynamic model of social networks by combining a hidden Markov model with a mixed-membership stochastic blockmodel that identifies latent groups underlying the network structure. Unlike existing models, we incorporate co- variates that predict node membership in latent groups as well as the direct formation of edges between dyads. While prior substantive research often assumes the deci- sion to engage in militarized conflict is independent across states and static over time, we demonstrate that conflict patterns are driven by states’ evolving membership in geopolitical blocs. Changes in monadic covariates like democracy shift states between coalitions, generating heterogeneous effects on conflict over time and across states. Keywords: clustering, hidden Markov model, latent variable modeling, mixed-membership stochastic blockmodel, network modeling ∗The methods described in this paper can be implemented via the open-source statistical software, Net- Mix, available at https://CRAN.R-project.org/package=NetMix. yAssistant Professor of Political Science, UNC-Chapel Hill.
    [Show full text]