Zachary Karate Club

Tutorial: Social Network Models and Data

http://www.stanford.edu/~jugander/ec-tutorial/

Johan Ugander, Microsoft Research (with Jure Leskovec, Stanford) ! ACM EC Tutorial June 16, 2015 Social networks: mapping structure

n=33

First “sociogram”: 8th grade students studying in proximity

• J Moreno (1934) “Who shall survive?: A new approach to the problem of human interrelations.”

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Social networks: mapping structure

Moreno’s “chance sociogram”: a random graph null model

• J Moreno (1934) “Who shall survive?: A new approach to the problem of human interrelations.” The digital microscope

n>1,400,000,000

“The emergence of ‘cyberspace’ and the World Wide Web is like the discovery of a new continent.” - Jim Gray, 1998 Turing Award address Processes on social networks

1940 election: Hybrid seed corn Tetracycline two-step theory of opinion leaders

Lazarsfeld et al. ’55 Ryan-Gross ‘43 Coleman-Katz-Menzel ‘57 Watts-Dodds ‘07

• B Ryan, N Gross (1943) “The diffusion of hybrid seed corn in two Iowa communities”, Rural sociology. • P Lazarsfeld; B Berelson, H Gaudet (1948) “The People's Choice. How the Voter Makes up His Mind in a Presidential Campaign”. • E Katz, P Lazarsfeld (1955) “Personal Influence, The part played by people in the flow of mass communications”. • E Katz (1957) “The Two-Step Flow of Communication: An Up-To-Date Report on an Hypothesis”. Political Opinion Quarterly. • J Coleman, E Katz, H Menzel (1957) “The diffusion of an innovation among physicians”, Sociometry. • D Watts, P Dodds (2007) “Influentials, Networks, and Public Opinion Formation” Journal of Consumer Research. Processes on social networks

Hybrid seed corn (Ryan-Gross): 5 stages: awareness, interest, evaluation, trial, adoption .678

4 0

:5 OPerc.en+ hsaring - • ac.ceptinq _ " [-20 "2 LI Vi 0,; t- I- r1 0 f- I-

:5 f- f- o r rio. .. .. 11 I r r I I':'? .... · H 26 27 28 H "305• or 56 -4C 41 "'" -=YI::AR 2. Percentage of operators first hearing and percentageSurvey of operatorsof n=259 accepting farmers hylJlld ,eed in specified years. • B Ryan, N Gross (1943) “The diffusion of hybrid seed corn in two Iowa communities”, Rural sociology. • P Lazarsfeld; B Berelson,mental H Gaudetstage.) (1948) This “The spread People's of Choice. knowledge How the was Voter highly Makes concentrated up His Mind in a Presidential Campaign”. • E Katz, P Lazarsfeldin (1955) the three“Personal years Influence, 1929, The1930 part and played 1931. by people During in the thisflow ofbrief mass period communications”. • E Katz (1957) “Theabout Two-Step 60 Flowpercent of Communication: of the operators An Up-To-Date first learned Report of on the an Hypothesis”.seed. Political Opinion Quarterly. • J Coleman, E Katz, H MenzelVery' roughly (1957) “The there diffusion is a oflag an innovationof about among5 years physicians between'”, Sociometry. the curve • D Watts, P Dodds (2007)of information “Influentials, andNetworks, the curve and Public of acceptance.Opinion Formation However,” Journal ofone Consumer could Research. scarcely say that the time patterns of the two aspects of diffusion were closely similar. The spread of information followed a pattern even less similar to a normal frequency distrihution than the curve of adoption.36 The preliminary stages of diffusion covered a longer time span in terms of adoption than in knowledge. The modal frequency in knowledge came 7 years after the first operator heard of the seed; the modal frequency in adoption occurred 10 years after the trait was first accepted. Where.as the adoption curve is definitely "bell-shaped," the spread of knowledge curve is asym- metrical and even more highly concentrated around the mode. The adoption curve itself shows a long period of slow growth followed by a great wave of acceptance, which in turn is followed by a rela- tively short period in which the remaining stragglers accepted the new seed. It is clear that the acceptance of the seed for use was delayed some time after initial contact. The. lag between first information and first adoption was 5.5 years for all operators. This lag, however, varied markedly for those who adopted the seed early and those who adopted it late. (See table 2.) Thus the mean number of years before acceptance, after initial information, was 1.6 for those adopting prior to 1934,. For those adopting in 1934 through 1936, the lag was increased to 4.4 years; for those adopting in 1937 to 1939, there was a delay of 6.4 years; and for the most resistant the delay amounted to 9.2 years. 36Neither curve is in fact a normal frequency. See Ryan and Gross, op. cit. Digital experimental microscope

Massive experiments to test theories of social processes on large-scale networks.

Experiment on n=61,000,000 Facebook users

Not just FB: Telenor service experiment (n=46,000), LinkedIn, others.

• Bond et al. (2012) “A 61-Million-Person Experiment in Social Influence and Political Mobilization”, Nature. • J Bjelland et al. (2015) “Investigating Social Influence Through Large-Scale Field Experimentation”, NetMob. Digital experimental microscope

Massive experiments to test theories of social processes on large-scale networks. RESEARCH LETTER

abInformational message 2.1 Social Social Today is Election Day What’s this? • close message message Find your polling place on the U.S. 01553761 1.8 versus versus Politics Page and click the "I Voted" People on Facebook Voted button to tell your friends you voted. informational control VOTE l Voted 1.5 message

1.2

Social message 0.9

Today is Election Day What’s this? • close 0.6 01553761

Find your polling place on the U.S. on own behaviour (%)

Politics Page and click the "I Voted" People on Facebook Voted Direct effect of treatment button to tell your friends you voted. VOTE 0.3 l Voted

0 Self- Search for Validated Validated Jaime Settle, Jason Jones, and 18 other friends have voted. reported polling voting voting voting place

FigureExperiment 1 | The experiment on n=61,000,000 and direct effects. a, b, ExamplesFacebook of the informationalusers message and social message Facebook treatments (a) and their direct effect on voting behaviour (b). Vertical lines indicate s.e.m. (they are too small to be seen for the first two bars). who received the social message were 0.39% (s.e.m., 0.17%; t-test, Figure 2 shows that the observed per-friend treatment effects increase P 5 0.02) more likely to vote than users who received no message at astie-strength increases.All ofthe observed treatment effectsfall outside Not just FB: Telenorall. Similarly, the service difference in voting experiment between those who received(n=46,000), the the null distribution LinkedIn, for expressed others. vote (Fig. 2b), suggesting that they are social message and those who received the informational message was significantly different from chance outcomes. For validated vote 0.39% (s.e.m., 0.17%; t-test, P 5 0.02), suggesting that seeing faces of (Fig. 2c), the observed treatment effect is near zero for weak ties, but • Bond et al. (2012) “A 61-Million-Personfriends significantly contributedExperiment to thein Social overall effectInfluence of the messageand Political on it Mobilization spikes upwards”, and Nature. falls outside the null distribution for the top two • J Bjelland et al. (2015) “Investigatingreal-world voting. Social In fact,Influence turnout Through among those Large-Scale who received Field the Experimentationdeciles. This suggests”, NetMob. that strong ties are important for the spread of informational message was identical to turnout among those in the real-world voting behaviour. Finally, the treatment effect for polling control group (treatment effect 0.00%, s.e.m., 0.28%; P 5 0.98), which place search gradually increases (Fig. 2d), with several of the effects raises doubts about the effectiveness of information-only appeals to falling outside the 95% confidence interval of the null distribution. vote in this context. To simplify the analysis and reporting of results, we arbitrarily These results show that online political mobilization can have a define ‘close friends’ as people who were in the eightieth percentile direct effect on political self-expression, information seeking and or higher (decile 9) of frequency of interaction among all friendships in real-world voting behaviour, and that messages including cues from the sample (see the Supplementary Information). ‘Friends’ are all other an individual’s social network are more effective than information- Facebook friends who had less interaction. A total of 60,491,898 (98%) only appeals. But what about indirect effects that spread from person users in our sample had at least 1 close friend, with the average user to person in the social network? Users in our sample had on average having about 10 close friends (compared with an average of 139 friends 149 Facebook friends, with whom they share social information, who were not close). although many of these relationships constitute ‘weak ties’. Past The results suggest that users were about 0.011% (95% confidence research indicates that close friends have a stronger behavioural effect interval (CI) of null distribution 20.009% to 0.010%) more likely to on each other than do acquaintances or strangers9,11,13,21. We therefore engage in an act of political self-expression by clicking on the I Voted expected mobilization to spread more effectively online through button than they would have been had their friend seen no message. ‘strong ties’. Similarly, for each close friend who received the social message, an To distinguish users who are likely to have close relationships, we individual was on average 0.099% (null 95% CI –0.042% to 0.048%) used the degree to which Facebook friends interacted with each other on more likely to express voting. the site (see Supplementary Information for more detail). Higher levels We also found an effect in the validated vote sample. For each close of interaction indicate that friends are more likely to be physically friend who received the social message, a user was 0.224% (null 95% CI proximate and suggest a higher level of commitment to the friendship, –0.181% to 0.174%) more likely to vote than they would have been had more positive affect between the friends, and a desire for the friendship their close friend received no message. Similarly, for information- to be socially recognized29.Wecountedthenumberofinteractions seeking behaviour we found that for each close friend who received between each pair of friends and categorized them by decile, ranking the social message, a user was 0.012% (null 95% CI –0.012% to 0.012%) them from the lowest to highest percentage of interactions. A validation more likely to click the link to find their polling place than they would study (see Supplementary Information) shows that friends in the highest have been had their close friends received no message. In both cases decile are those most likely to be close friends in real life (Fig. 2a). there was no evidence that other friends had an effect (see We then used these categories to estimate the effect of the mobil- Supplementary Information). Thus, ordinary Facebook friends may ization message on a user’s friends. Random assignment means that affect online expressive behaviour, but they do not seem to affect any relationship between the message a user receives and a friend’s private or real-world political behaviours. In contrast, close friends behaviour is not due to shared attributes, as these attributes are not seem to have influenced all three. correlated with the treatment (see Supplementary Information). To The magnitude of these contagion effects are small per friend, but it measure a per-friend treatment effect, we compared behaviour in the is important to remember that they result from a single message, and in friends connected to a user who received the social message to beha- many cases it was not possible to change the target’s behaviour. For viour in the friends connected to a user in the control group. To example, users may have already voted by absentee ballot before account for dependencies in the network, we simulate the null distri- Election Day, or they may have logged in to Facebook too late to vote bution using a network permutation method (see the Supplementary or to influence other users’ voting behaviour. In other words, all effects Information). Monte Carlo simulations suggest that this method measured here are intent-to-treat effects rather than treatment-on- minimizes the risk of false positives and recovers true causal effects treated effects, which would be greater if we had better information without bias (see Supplementary Information). about who was eligible to receive the treatment.

.678

4 0

:5 OPerc.en+ hsaring - • ac.ceptinq _ • " 1940s-50s: Early theories, early data [-20 "2 LI Vi 0,; t- I- r1 0 f- I-

:5 f- f- o r rio. .. .. 11 I r r I I':'? .... · H 26 27 28 -= H "305• or 56 -4C 41 ! "'" YI::AR 2. Percentage of operators first hearing and percentage of operators accepting hylJlld ,eed in specified years.

mental stage.) This spread of knowledge was highly concentrated in the three years 1929, 1930 and 1931. During this brief period about 60 percent of the operators first learned of the seed. • 1960s-90s: Theory refinement/testing Very' roughly there is a lag of about 5 years between' the curve of information and the curve of acceptance. However, one could scarcely say that the time patterns of the two aspects of diffusion were closely similar. The spread of information followed a pattern even less similar to a normal frequency distrihution than the curve of adoption.36 The preliminary stages of diffusion covered a longer time span in terms of adoption than in knowledge. The modal ! frequency in knowledge came 7 years after the first operator heard of the seed; the modal frequency in adoption occurred 10 years after the trait was first accepted. Where.as the adoption curve is definitely "bell-shaped," the spread of knowledge curve is asym- metrical and even more highly concentrated around the mode. The adoption curve itself shows a long period of slow growth followed • 2000s: Large-scale data by a great wave of acceptance, which in turn is followed by a rela- tively short period in which the remaining stragglers accepted the new seed. It is clear that the acceptance of the seed for use was delayed some time after initial contact. The. lag between first information and first adoption was 5.5 years for all operators. This lag, however, varied markedly for those who adopted the seed early and ! those who adopted it late. (See table 2.) Thus the mean number of years before acceptance, after initial information, was 1.6 for those adopting prior to 1934,. For those adopting in 1934 through 1936, the lag was increased to 4.4 years; for those adopting in 1937 to 1939, there was a delay of 6.4 years; and for the most resistant • the delay amounted to 9.2 years. 2010s: Large-scale experiments 36Neither curve is in fact a normal frequency. See Ryan and Gross, op. cit.

Designing/analyzing experiments to develop/test network theories = Big opportunity Outline of this tutorial

• Part I: From Theory to Data (9:00-10:00)

• What network?

• From karate to communities

• Influence, instrumented

• Short Break (very short)

• Part II: Experimentation and Causal Inference (10:05-11:00)

• Homophily vs. contagion

• Influence experiments

• Network experiments Part I: from Theory to Data Testing a theory with data ! • Homans (1950): Small groups of people create a social structure that contains many clique subgroups and a ranking system. ! • Davis & Leinhardt (1967): Operational statement using subgraph frequencies: 7 triads less frequent than random model predicts.

• G Homans (1950) “The Human Group.” • J Davis, S Leinhardt (1971) “The structure of positive interpersonal relations in small groups,” Sociological Theories in Progress. fsmlrrn adhnernhrznal ntefigure). the the in is horizontally run as individuals hence The visible, between (and rank run clearly similar edges of grade. is undirected most grade to that with according fact rank scale colored of a and on increase rank vertically inferred arranged of nodes ex- with be networks, together. would school the than single a rank in higher students all a school were have in the pected students may temporarily status) hence are highest students and When therefore grade, 8th (and school. junior-high the oldest high feeder in to the are up in move they students grade senior before most schools, the is grade Unlike a gives have grades. not 9th do di two and significant these 8th statistically grades, the consecutive of for pairs results other the in seen be schools. averaged all grade, in school of individuals function a all as over rank Rescaled 5: FIG. are and points data compa- the are of errors sizes shown. the Measurement than not out-degr smaller degrees. or (b) of with in-degree, sum rable (a) the a for (c) over networks averaged and degree, all versus in rank rescaled individuals of Plots 4: FIG. Average rank ial,i i.6w hwa culeapeo n of one of example actual an show we 6 Fig. in Finally, Ave ra ge ra n k 0.5 0 1 0.2 0.4 0.6 0.8 1.0 p> 0 a b (c) (b) (a) 5 0 In-degree . 5.Ti a eettefc htte8th the that fact the reflect may This 95). 7 10 15 8 024 ff rnei vrg ak(a rank average in erence 9 Out-degree Grade

Extending a theory6 with more data 10 ! 8 0

11 • Ball-Newman (2013): Maximum likelihood5 inference of status according Degree to Homans’ theory. 10 12 15 t -test 20 • ee, Examined 84 high school networksll for correlates of Homans status. novdsc ssxo ethnicity. or sex individuals the as of indepen- such characteristics involved essentially other be of On average to on status. seem dent sug- rankings to popularity, hand, correspond overall other may and the rankings age the as that such gesting status measures social traditional of so with rankings significantly the correlate that find derived unreciprocated we and and account, into reciprocated friendships maximum- complete both a from taking ranks developed such networks, inferring have for We method likelihood one. with con- friendship higher-ranked friendships claiming a individual unreciprocated low lower-ranked a from most of ranked that sist be in- such can one high students only to that by find claimed We friendships dividual. and participat- both individuals by ing dis- claimed the friendships on between particularly high tinction focusing American schools, in junior-high students and between friendships of works orlto ewe akadgaei h network. rath the in model), grade an clear the and the rank of (i.e., Note between ranking. calculation correlation distribution maximum-likelihood Carlo the merely posterior Monte than the the over within average average an as e undirected and di vertical grade, the colored to on according rank colored (rescaled) vertices with network axis, sample A 6: FIG.

nti ae,w aeaaye ag e fnet- of set large a analyzed have we paper, this In Rank 0.25 0.75 0.5 1.0 0 ﬀ rnl rmdrce de.Rn scalculated is Rank edges. directed from erently Conclusions Shape Form Undirected Edge Type Directed Grade 12 11 10 dges • G Homans (1950) “The Human Group9 8 .”7 er • J Davis, S Leinhardt (1971) “The structure of positive interpersonal relations in small groups,” Sociological6 Theories in Progress. • B Ball, MEJ Newman (2013) “Friendship networks and social status.” Network Science. What network? ! “Name generators” in sociology show huge difference between social networks generated by questions: ! “Who do you know?” “Who are your three closest friends?" “With whom do you discuss important matters?”

• K Campbell, B Lee (1991) “Name generators in surveys of personal networks.” Social Networks. • M Resnick et al. (1997) “Protecting adolescents from harm: findings from the national longitudinal study on adolescent health.” JAMA. • C Apicella, F Marlowe, J Fowler, N Christakis (2012) “Social networks and cooperation in hunter-gatherers,” Nature. • B Ball, MEJ Newman (2013) “Friendship networks and social status.” Network Science. What network? ! “Name generators” in sociology show huge difference between social networks generated by questions: ! “Who do you know?” “Who are your three closest friends?" “With whom do you discuss important matters?” ! Ball-Newman used AddHealth, which has issues that requires care. ! “Up to 10 people with whom they are friends, with a maximum of five being male and 5 being female”

RESEARCH SUPPLEMENTARY INFORMATION

“With whom they would like to live in the next camp” ! “To whom they would give an actual gift of honey”

Supplementary Figure S2: Example of one poster set for one sex (women). These posters were used to elicit social ties.

54 | WWW.NATURE.COM/NATURE Online Social Networks Online Social Networks

Acquaintances, often international. Business and personal. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online.

Mostly professional connections, some friends. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online.

Mostly professional connections, some friends.

Virtual acquaintances, often interest-driven. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online.

Mostly professional connections, some friends.

Virtual acquaintances, often interest-driven.

Photography-interested real-world/virtual friends. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online.

Mostly professional connections, some friends.

Virtual acquaintances, often interest-driven.

Photography-interested real-world/virtual friends.

People you talk to on the phone, including customer service. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online.

Mostly professional connections, some friends.

Virtual acquaintances, often interest-driven.

Photography-interested real-world/virtual friends.

People you talk to on the phone, including customer service.

Close friends who exercise. Online Social Networks

Acquaintances, often international. Business and personal.

2004: classmates, 2015: most people you know who are online.

Mostly professional connections, some friends.

Virtual acquaintances, often interest-driven.

Photography-interested real-world/virtual friends.

People you talk to on the phone, including customer service.

Close friends who exercise.

Sometimes personal, sometimes professional, sometimes both. Online Social Networks

Some differences: • Design aspects • Personal vs. professional • Strong vs. weak (Onnela et al. 2007) • Virtual/real-world acquaintances (Jacobs et al. 2015) • Single interest vs. diverse interest networks • Co-tag friends vs. news feed friends vs. chat friends • Phone calls vs. texts • …

• JP Onnela et al. (2007) “Structure and tie strengths in mobile communication networks,” PNAS. • AZ Jacobs, SF Way, J Ugander, A Clauset (2015) “Assembling thefacebook: Using Heterogeneity to Understand Online Social Network Assembly,” WebSci. Networks: a matter of scale

• 34-node network of a recreational karate club [Zachary 1978] • 4436-node network of email exchange over 3-months at corporate research lab [Adamic-Adar 2003] • 43,553-node network of email exchange over 2 years at a large university [Kossinets-Watts 2006] • 4.4-million-node network of declared friendships on a blogging community [Liben-Nowell et al. 2005, Backstrom et al. 2006] • 240-million-node network of all IM communication over a month on Microsoft Instant Messenger [Leskovec-Horvitz 2008] • 721-million-node network of Facebook friendships between all active users in May 2011 [Ugander et al. 2011] • 1.44-billion-node network of Facebook monthly active users in March 2015. Zachary Karate Club

• Wayne Zachary, sociologist interested in group dynamics.

• Studied a karate club for 3 years (’70-’72)

• Club formed factions around instructor (1) and Club President (34).

• Zachary was interested in if faction structure could be predicted.

• Method?

! Zachary Karate Club

• Wayne Zachary, sociologist interested in group dynamics.

• Studied a karate club for 3 years (’70-’72)

• Club formed factions around instructor (1) and Club President (34).

• Zachary was interested in if faction structure could be predicted.

• Method?

• Network Flow! Applied Ford-Fulkerson, found group split was predicted by min-cut. From Karate to Communities

• Zachary “objective function” for community detection: does algorithm predict how a group fissions when led by two rival leaders? From Karate to Communities

• Zachary “objective function” for community detection: does algorithm predict how a group fissions when led by two rival leaders? • Other objectives • Modularity maximization: • Has “resolution limit” • Conductance (normalized min-cut): • Produces balanced partitions; spectral guarantees

• MEJ Newman, M Girvan (2004) "Finding and evaluating community structure in networks,” Physical Rev E. • S Fortunato, M Barthelemy (2007) "Resolution limit in community detection," PNAS. • J Shi, J Malik (2000) “Normalized cuts and image segmentation,” IEEE Trans Pattern Analysis and Machine Intelligence. • E Mossel, J Neeman, A Sly (2012) “Stochastic block models and reconstruction” From Karate to Communities

• Zachary “objective function” for community detection: does algorithm predict how a group fissions when led by two rival leaders? • Other objectives

n n • Modularity maximization: a b

• Has “resolution limit” paa pab • Conductance (normalized min-cut): • Produces balanced partitions; spectral guarantees • Ability to recover Stochastic Block Model: pab pbb • Stylized model in absence of ground truth data

• Zachary “objective function” for community detection: does algorithm predict how a group fissions when led by two rival leaders? • Other objectives

n n • Modularity maximization: a b

• Prob. of adoption depends on the number of friends who have adopted (Bass 1969, Granovetter 1978) • What is the shape? Diminishing returns? Critical mass? ! ! ! !

! Prob."of"adop1on" Prob."of"adop1on" k"="number"of"friends"adop1ng" k"="number"of"friends"adop1ng"

• F Bass (1969) "A new product growth for model consumer durables". Management Science. • M Granovetter (1978) "Threshold models of collective action," American Journal Sociology. • D Watts, P Dodds (2007) “Influentials, Networks, and Public Opinion Formation” Journal of Consumer Research. Inﬂuence, instrumented

Backstrom et al. 2006: Probability of Leskovec et al. 2006: Probability of The Dynamics of Viral Marketing 19 joining LiveJournal group buying a DVD

0.06 0.08

0.05 0.06 0.04

0.03 0.04

0.02

Probability of Buying Probability of Buying 0.02

Prob."of"joining" 0.01

0 0 2 4 6 8 10 10 20 30 40 50 60 k"(number"of"friends"in"the"community)"Incoming Recommendations Incoming Recommendations (a) Books (b) DVD 0.2 0.2

0.15 0.15 • L Backstrom, D Huttenlocher, J Kleinberg, X Lan (2006) "Group formation in large social networks: membership, growth, and evolution," KDD. 0.1 0.1 • J Leskovec, LA Adamic, BA Huberman (2006) "The dynamics of viral marketing," EC. • D Centola, V Eguiluz, M Macy (2007) "Cascade dynamics of complex propagation," Physica A. • D Centola, M Macy (2007) "Complex contagions and the weakness of long ties" American Journal Sociology.

Probability of Buying 0.05 Probability of Buying 0.05

0 0 1 2 3 4 5 6 7 8 2 4 6 8 10 12 14 16 Incoming Recommendations Incoming Recommendations (c) Music (d) Video

Figure 8: Probability of buying a book (DVD) given a number of incoming recommendations.

incoming recommendations on a particular book. The maximum was 30 incoming recommendations. For these reasons we cut-off the plot when the number of observations becomes too small and the error bars too large. We calculate the purchase probabilities and the standard errors of the estimates which we use to plot the error bars in the following way. We regard each point as a binomial random variable. Given the number of observations n, let m be the number of successes, and k (k=n-m) the number of failures. In our case, m is the number of people that first purchased a product after receiving r recommendations on it, and k is the number of people that received the total of r recommendations on a product (till the end of the dataset) but did purchase it, then the estimated probability of purchasing isp ˆ = m/n and the standard error spˆ of estimatep ˆ is spˆ = !p(1 p)/n. Figure 8(a) shows that, overall, book recommendations are rarely followed.− Even more surprisingly, as more and more recommendations are received, their success decreases. We observe a peak in probability of buying at 2 incoming recommendations and then a slow drop. This implies that if a person doesn’t buy abookafterthefirst recommendation, but receives another, they are more likely to be persuaded by the second recommendation. But thereafter, they are less likelytorespondtoadditional Influence, instrumented

Backstrom et al. 2006: Probability of Leskovec et al. 2006: Probability of The Dynamics of Viral Marketing 19 joining LiveJournal group buying a DVD

0.06 0.08

0.05 0.06 0.04

0.03 0.04

0.02

Probability of Buying Probability of Buying 0.02

Prob."of"joining" 0.01

0 0 2 4 6 8 10 10 20 30 40 50 60 k"(number"of"friends"in"the"community)"Incoming Recommendations Incoming Recommendations (a) Books (b) DVD 0.2 0.2 Complex contagion? 0.15 0.15 • L Backstrom, D Huttenlocher, J Kleinberg, X Lan (2006) "Group formation in large social networks: membership, growth, and evolution," KDD. 0.1 0.1 • J Leskovec, LA Adamic, BA Huberman (2006) "The dynamics of viral marketing," EC. • D Centola, V Eguiluz, M Macy (2007) "Cascade dynamics of complex propagation," Physica A. • D Centola, M Macy (2007) "Complex contagions and the weakness of long ties" American Journal Sociology.

Probability of Buying 0.05 Probability of Buying 0.05

0 0 1 2 3 4 5 6 7 8 2 4 6 8 10 12 14 16 Incoming Recommendations Incoming Recommendations (c) Music (d) Video

Figure 8: Probability of buying a book (DVD) given a number of incoming recommendations.

▪ Adoption as a simple function of ‘contact neighborhood’ size:

susceptible susceptible susceptible

≤ ≤

• J Ugander, L Backstrom, C Marlow, J Kleinberg (2012) “Structural diversity in social contagion,” PNAS. Influence and graph structure

▪ Adoption as a simple function of ‘contact neighborhood’ size:

susceptible susceptible susceptible

≤ ≤

susceptible susceptible

co-worker sibling vs.

co-worker co-worker

• J Ugander, L Backstrom, C Marlow, J Kleinberg (2012) “Structural diversity in social contagion,” PNAS. Structural diversity

Conversion rate on invitations to Facebook as a function of graph, “f(G)”?

size 2: size 3: size 4:

2.5 2.5 − 2.5 − − 2.0 − 2.0 − 2.0 − − − − − − 1.5 1.5 1.5 − − − − − − − − − − − − 1.0 1.0 1.0 − − − − − 0.5 0.5 0.5 − Relative conversion rate conversion Relative Relative conversion rate conversion Relative Relative conversion rate conversion Relative 0 0 0

Contact Contact Contact neighborhood: neighborhood: neighborhood:

• J Ugander, L Backstrom, C Marlow, J Kleinberg (2012) “Structural diversity in social contagion,” PNAS. Part II: Experimentation and Causal Inference

http://www.stanford.edu/~jugander/ec-tutorial/ Is obesityThe Spread contagious? of Obesity in a Large Social Network Over 32 Years Figure 3. Effect of Social and Geographic Distance from Obese Alters on the Probability of an Ego’s Obesity in 100 the Social Network of the Framingham Heart Study. Examination 1 Examination 2 Panel A shows the mean effect of an ego’s social prox- 80 Examination 3 imity to an obese alter; this effect is derived by compar- Examination 4 ing the conditional probability of obesity in the observed Examination 5 network with the probability of obesity in identical net- 60 Examination 6 works (with topology preserved) in which the same Examination 7 number of obese persons is randomly distributed. The social distance between the alter and the ego is repre- 40 sented by degrees of separation (1 denotes one degree of separation from the alter, 2 denotes two degrees of separation from the alter, and so forth). The examina- 20 tion took place at seven time points. Panel B shows the mean effect of an ego’s geographic proximity to an 0

obese alter. We ranked all geographic distances (derived from geocoding) between the homes of directly connected egos and alters (i.e., just those pairs at one degree 12345 6 of separation) and created six groups of equal size. This figure shows the average distances for the six mileage groups: 1 denotes 0 miles (i.e., closest to the " alter’s home), 2 denotes 0.26 mile, 3 denotes 1.5 miles, 100 “comparing4 denotesthe conditional3.4 miles, 5 denotes 9.3probability miles, and 6 denotes of obesity in the observed network with the 471 miles (i.e., farthest from the alter’s home). There is probability noof trend obesity in geographic in distance. identical I bars for networksboth panels (with 80topology preserved) in which the show 95% confidence intervals based on 1000 simula- same numbertions. Toof convert obese miles topersons kilometers, multiply is randomly by 1.6. distributed” 60

poraneous obesity (changing from 0 to 1), using 40 • N Christakis, J Fowler1000 randomly (2007) "The drawn Spread sets of of estimates Obesity in from a Large the Social Network over 32 Years," New England J of Medicine. coefficient-covariance matrix and assuming mean • C Shalizi, A Thomas (2011) "Homophily and contagion are generically confounded20 in observational social network studies," 29 Sociological Methodsvalues & for Research. all other variables. All tests were two-tailed. The sensitivity of the results was as-

sessed with multiple additional analyses (see the Supplementary Appendix). 12345 6 $ + - Results

RETAKE 1st Figure 1 depicts the largest connected subcom- ICM AUTHOR: Christakis work with simulated networks with the same 2nd ponent of the social network in the year 2000. REG F FIGURE: 3 of 4 network topology and the same overall preva- 3rd This network is sufficiently dense to obscure CASE Revised much of the underlying structure, although re- lence of obesityEMail as the observed network,Line 4-Cbut with * ARTIST: ts H/T H/T the incidenceEnon of obesity randomly distributed 22p3 gions of the network with clusters of obese or Combo among the nodes (in what we call “random body- nonobese persons can be seen. Figure 2 illus- )(% & ' (! mass–index networks”).#-+ . + If clustering is occur- trates the spread of obesity between adjoining +, - nodes in a part of the network over time. A video ring, then the probability that an alter will be (available with the full text of this article at www. obese, givenJOB: that35704 an ego is known to be obese,ISSUE: 07-26-07 nejm.org) depicts the evolution of the largest should be higher in the observed network than component of the network and shows the prog- in the random body-mass–index networks. What ress of the obesity epidemic over the 32-year study we call the “reach” of the clusters is the point, in period. terms of an alter’s degree of separation from any Figure 3A characterizes clusters within the given ego, at which the probability of an alter’s entire network more formally. To quantify these obesity is no longer related to whether the ego clusters, we compared the whole observed net- is obese. In all of the examinations (from 1971 through 2003), the risk of obesity among alters

n engl j med 357;4 www.nejm.org july 26, 2007 375 Knock-out experiments

Z(i) X(i) X(j) Z(j)

Symbol Meaning Y(i,t-1) A(i,j) Y(j,t-1) i, j Individuals Z Observed Traits X Latent Traits Y(i,t) Y(j,t) Y Observed Outcomes

Figure 1: Causal graph allowing for latent variables (X) to inﬂuence both manifest network ties Aij and manifest Figure 2: Notational guide to behaviors (Y ). terms used in this investigation.

or indeed X, to vary over time, as is readily verified by drawing the appropriate graphs. Finally, adding a third individual to the graph would not help: even if they were, say, assumed to be linked to i but not j or vice versa, Y (t) X i i ! Aij and Yj(t 1) Xj Aij would remain confounding paths. How then might we get! identifiability? It may be that very stringent parametric assumptions would suce, though we have not been able to come up with any which would be suce7 Otherwise, we must keep X from being la- • N Christakis, J Fowler tent,(2007) or, "The more Spread precisely, of Obesity either in the a Large components Social Network of X that over influence32 Years,"Y Newmust England be J of Medicine. • C Shalizi, A Thomas (2011)made "Homophily observable (Figureand contagion3a), or are those generically parts of confoundedX which influence in observational the social social tie network studies," Sociological Methods formation& Research.A (Figure 3b). In either case the confounding arcs go away, and the • 8 E Bakshy et al. (2012)direct “The Role e↵ect of ofSocialYj( tNetworks1) on inYi (Informationt) becomes Diffusion identifiable.,” WWW.It is noteworthy that the most successful attempts at explicit modeling that handle both homophily and influence, as found in the work of Leenders (1995); Steglich et al. (2004) involves, all at once, strong parametric (exponential-family) assumptions, plus the assumption that observable covariates carry all of the dependence from X to Y and A; the latter is also implicitly assumed by the matching methods of Aral et al. (2009). Whether we face the unidentifiable situation of Figure 1, or the identifiable case of Figure 3, currently depends upon subject-matter knowledge rather than statistical techniques. It may be possible to adapt algorithms, such as those in

7In particular, making all of the relations between continuous variables in Figure 1 linear, with independent noise for each variable, is not enough — the confounding path continues to prevent identifiability even in a linear model. 8Elwert and Christakis (2008)isanotherinterestingapproach.Ine↵ect, they introduce athirdnode,callitk,wheretheycanassumethatYi is not influenced by Yk,butthe homophily is the same. Estimating the apparent influence of Yk on Yi then shows the extent of confounding to due purely to homophily; if Yi is more dependent than this on Yj ,theexcess is presumably due to actual causal influence.

7 Knock-out experiments

Z(i) X(i) X(j) Z(j)

Symbol Meaning Y(i,t-1) A(i,j) Y(j,t-1) i, j Individuals Z Observed Traits x X Latent Traits Y(i,t) Y(j,t) Y Observed Outcomes

Figure 1: Causal graph allowing for latent variables (X) to inﬂuence both manifest network ties Aij and manifest Figure 2: Notational guide to behaviors (Y ). terms used in this investigation.

7 Knock-out experiments

Z(i) X(i) X(j) Z(j)

Symbol Meaning Y(i,t-1) A(i,j) Y(j,t-1) i, j Individuals Z Observed Traits x X Latent Traits Y(i,t) Y(j,t) Y Observed Outcomes

Figure 1:“Feed Causal Condition” graph allowing for “No Feed Condition” latent variables (X) to inﬂuence both manifest network ties Aij and manifest Figure 2: Notational guide to behaviors (Y ). terms used in this investigation.

or indeed X, to vary over time, as is readily verified by drawing the appropriate graphs. Finally, adding a third individual to the graph would not help: even if they were, say, assumed to be linked to i but not j or vice versa, Y (t) X i i ! Aij and Yj(t 1) Xj Aij would remain confounding paths. How then might we get! identifiability? It may be that very stringent parametric assumptions would suce, though we have not been able to come up with any which would be suce7 Otherwise, we must keep X from being la- • N Christakis, J Fowler tent,(2007) or, "The more Spread precisely, of Obesity either in the a Large components Social Network of X that over influence32 Years,"Y Newmust England be J of Medicine. • C Shalizi, A Thomas (2011)made "Homophily observable (Figureand contagion3a), or are those generically parts of confoundedX which influence in observational the social social tie network studies," Sociological Methods formation& Research.A (Figure(a)3b). In either case the confounding(b) arcs go away, and the • E Bakshy et al. (2012) “TheFigure Role 2: of An Social example ofNetworks the Facebook in News Information Feed interface for Diffusion a hypothetical,” subject WWW.8 who has a link (high- directlighted e↵ect in red) of assignedYj(t to the1) (a) onfeed orY (b)i(tno) feed becomescondition. identifiable. It is noteworthy that the most successful attempts at explicit modeling that handle both homophily

and influence,user before she has as the opportunity found toin share the that work content oftheLeendersno feed condition,(1995 a shared); URLSteglich is on average deliveredet al. (2004) herself. Additional unobserved correlations may arise due to to over 99% of its potential targets. involves,external all influence at via once, e-mail, instant strong messaging, parametric and other (exponential-family)All activity relating to subject-URL pairs assumptions, assigned to ei- plus social networking sites. These causal relationships are illus- ther experimental condition is logged, including feed expo- the assumptiontrated in Figure 1. From that the observablefigure, one can see that covariates all sures, censored carry exposures,all of and the clicks to dependence the URL (from the from X unobservable correlations can be identified by blocking the feed or other sources, like messaging). Directed shares, such to Y causaland relationshipA; the between latter the Facebook is also feed and implicitly sharing. as a assumed link that is included by in the a private matching Facebook message methods or of Our experiment therefore randomizes subjects with respect explicitly posted on a friend’s wall, are not a↵ected by the Aral toet whether al. they(2009 receive). social signals about friends’ sharing assignment procedure. If a subject-URL pair is assigned to behavior of certain Web pages via the Facebook feed. an experimental condition, and the subject clicks on con- Whether we face the unidentifiabletent situation containing that of URL Figure in any interface1, or other the than identifiable the feed, that subject-URL pair is removed from the experiment. case3.1 of Figure Assignment3, Procedure currently depends uponOur subject-matter experiment, which took place knowledge over the span of seven rather than Subject-URL pairs are randomly assigned at the time of weeks, includes 253,238,367 subjects, 75,888,466 URLs, and statisticaldisplay to eithertechniques. the no feed or the Itfeed maycondition. be Stories possible1,168,633,941 to adapt unique subject-URL algorithms, pairs. such as those in that contain links to a URL assigned to the no feed condi- 7 tion for the subject are never displayed in the subject’s feed. 3.2 Ensuring Data Quality InThose particular, assigned to the makingfeed condition all are of not the removed relations from betweenThreats to continuous data quality include variables using content in that Figure was 1 linear, the feed, and appear in the subject’s feed as normal (Fig- or may have been previously seen by subjects on Facebook with independenture 2). Pairs are deterministically noise for assigned each to variable, a condition at is notpriorenough to the experiment, — the content confounding that subjects may path have seen continues to preventthe time identifiability of display, so any subsequent even in share a linearof the same model. URL through interfaces on Facebook other than feed, spam, and 8 by any of a subject’s friends is also assigned to the same con- malicious content. We address these issues in a number of Elwertdition. To and improve Christakis the statistical power(2008 of our)isanotherinterestingapproach.Ine results, twice ways. First, we only consider content that↵ wasect, shared they by introduce athirdnode,callitas many pairs were assignedk,wheretheycanassumethat to the no feed condition. Be- the subjects’ friendsYi onlyis not after the influenced start of the experiment. by Yk,butthe cause removal from the feed occurs on a subject-URL basis, This enables our experiment to accurately capture the first homophilyand we include is the only same. a small fraction Estimating of subject-URL the pairs apparent in time a influence subject is exposed of Y tok aon link inYi thethen feed, and shows ensures the extent of confounding to due purely to homophily; if Yi is more dependent than this on Yj ,theexcess is presumably due to actual causal influence.

7 Knock-out experiments

Z(i) X(i) X(j) Z(j)

Symbol Meaning Y(i,t-1) A(i,j) Y(j,t-1) i, j Individuals Z Observed Traits x X Latent Traits Y(i,t) Y(j,t) Y Observed Outcomes

Feed condition:

or indeed X, to vary over time, as is readily verified by drawing the appropriate 7.37x graphs. Finally, adding a third individual to the graph would not help: evenmore if likely to share. they were, say, assumed to be linked to i but not j or vice versa, Y (t) X i i ! Aij and Yj(t 1) Xj Aij would remain confounding paths. How then might we get! identifiability? It may be that very stringent parametric assumptions would suce, though we have not been able to come up with any which would be suce7 Otherwise, we must keep X from being la- • N Christakis, J Fowler tent,(2007) or, "The more Spread precisely, of Obesity either in the a Large components Social Network of X that over influence32 Years,"Y Newmust England be J of Medicine. • C Shalizi, A Thomas (2011)made "Homophily observable (Figureand contagion3a), or are those generically parts of confoundedX which influence in observational the social social tie network studies," Sociological Methods formation& Research.A (Figure(a)3b). In either case the confounding(b) arcs go away, and the • E Bakshy et al. (2012) “TheFigure Role 2: of An Social example ofNetworks the Facebook in News Information Feed interface for Diffusion a hypothetical,” subject WWW.8 who has a link (high- directlighted e↵ect in red) of assignedYj(t to the1) (a) onfeed orY (b)i(tno) feed becomescondition. identifiable. It is noteworthy that the most successful attempts at explicit modeling that handle both homophily

7 Influence maximization

General model (Kempe et al. 2003):

• When u tries to influence v: success based on set of nodes S that already tried & failed

• Success functions pv(u,S): S" v# • Independent cascades: pv(u,S) = puv

• Threshold: if |S|=k: pv(u,S)=1 else 0 u • Diminishing returns: pv(u,S) ≥ pv(u,T) if S subset of T Most inﬂuenal set of size k: the set S of k nodes producing largest expected cascade size f(S) if acvated. ! NP-hard, diminishing returns funcon yields (1-1/e)-factor approximaon.

• D Kempe, J Kleinberg, E Tardos (2003) "Maximizing the spread of influence through a social network," KDD. • J Leskovec et al. (2007) "Cost-effective outbreak detection in networks," KDD. REPORTS

ing the correlation of influence and susceptibility 2) The “influentials” and “susceptibles” highly susceptible users (Fig. 4, panel IV). The across all individuals and the assortativity of hypotheses are orthogonal claims. Both influ- clustering of influentials suggests the existence influence and susceptibility across all individ- ential individuals and noninfluential individu- of a multiplier effect of infecting a highly influ- uals and their peers in the network. We calcu- als have approximately the same distribution ential individual. However, such individuals also lated individual influence and susceptibility scores of susceptibility to influence among their peers; tend to have peers with only average susceptibil- as the product of the estimated hazard ratios of hence, being influential is not simply a conse- ity, making predictions about which effect would individuals’ attributes for a broader sample of quence of having susceptible peers (Fig. 4, dominate difficult without more evidence. Addi- 12 million users with 85 million relationships. panel II). Both influence and susceptibility play tional empirical and simulation studies should The analysis combines the estimated impact of aroleinthepeer-to-peerdiffusionoftheproduct. therefore examine the effects of the assortativity each demographic attribute on influence and sus- Combining studies of influence with studies of of influence and susceptibility on the diffusion ceptibility to calculate individuals’ overall influ- susceptibility will therefore likely improve our of behaviors, products, and diseases. ence and susceptibility scores. For example, a understanding of the diffusion of behavioral Analyzing the heat maps in Fig. 4 is not 35-year-old single female has an influence score contagions. sufficient to identify optimal intervention targets, equal to exp(binfl, >31 + binfl, single + binfl, female). 3) There are more people with high influ- because more information is needed about the The following inferences can be drawn from our ence scores than high susceptibility scores (Fig. network structure around candidate targets in results: 4, panel I), which suggests that, in our context, each region. For example, an individual with 1) Highly influential individuals tend not to targeting should focus on the attributes of cur- high influence and high peer susceptibility in the be susceptible, highly susceptible individuals tend rent adopters (e.g., giving individuals incentives upper right quadrant of in Fig. 4, panel II, may Influence andnot to be influential, susceptibility and almost no one is both to influence their peers) rather than attributes of seem like a good target, but may be of low de- highly influential and highly susceptible to in- their peers (e.g., giving individuals with suscep- gree or may be isolated. The network diagrams fluence (Fig. 4, panel I). This implies that influ- tible peers incentives to adopt). to the left of the heat maps show the assorta- ential individuals are less likely to adopt the 4) Influentials cluster in the network. As tivity of influence and susceptibility in ego product as a consequence of natural influence shown in Fig. 4, panel III, influential individuals networks from different regions combined with •! (Aral-Walker 2012): Randomizedprocesses (i.e., in the absence experiment of targeting); hence, connected using to other influential Facebook peers are approx- “app”information onfor their networkmovie structures, such as targeting influentials with low propensities to imately twice as influential as baseline users. In network degree and the distribution of influence ! recommendations. spontaneously adopt would be a potentially vi- contrast, we find a tendency for less suscepti- and susceptibility across peers in the network. ! able promotion strategy. ble users to cluster together and no clusters of Analyzing networks in different regions of the • Influence and susceptibility found to be decoupled (here, for movies)

! Identifying Influential and Susceptible Members of Social Networks

• Manydisplays the notificationresults inbox, where on a Facebook traits user may view that and click on notificationscorrelate delivered to her inbox. The notification inbox is private and only visible to users logged into their Facebook accounts.

withIt is not visible influence/susceptibility. to peers visiting other users’ profile pages.

Figures S2 (left) and S3 (right)

Fig. 4. Scores for 12 million Facebook users (collected from users who in- IV, ego susceptibility (y axis) and peer susceptibility (x axis). The heat maps stalled one of several other Facebook applications developed by the company) do not provide information on network structure, which can be important for The procedure to randomize the delivery targets of automated notificationswith 85 is million illustrated relationships in Figure S4. areAs calculated by means of hazard rate estimates informing targeting decisions. The diagrams to the left of the heat maps application users engaged in actions on the application during the courserelative of normal to the use, baselinefor example hazard when in the influence and susceptibility model show the assortativity of influence and susceptibility in ego networks drawn • described in the text. The resulting heat maps are shown at the right. Panel I from the regions of the heat maps labeled A, B, C, and D. Nodes in the S Aral,they D rated Walker a movie or (2012)friended a celebrity "Identifying, packets of notifications Influential displaysinforming thetheir percentageand friends ofSusceptible their of use people of (ego) Members with predicted influence of Social (y axis) Networks," and networks areScience. sized in proportion to theirpredictedinfluence(largernodes • D Centolathe application (2010) were automatically "The spreadgenerated in response of behavior to those actionspredicted andin deliveredan susceptibility online to their (randomlyx axis).social Panels network II to IV display experiment," the percentage of ego- Science.are more influential) and are shaded and placed relative to their predicted peer relationships: panel II, ego influence (y axis) and peer susceptibility (x susceptibility (redder nodes and nodes closer to ego are more susceptible; targeted Facebook peers. Each packet contained a fixed number axis); of notifications, panel III, each ego o influencef which was (y axis) and peer influence (x axis); and panel grayer nodes and nodes farther from ego are less susceptible).

randomly targeted to a specific peer of the application user. This process was repeated for each action the

user took on the application.4 The number of notifications 340that a particular peer of an application user 20 JULY 2012 VOL 337 SCIENCE www.sciencemag.org

received at any given time was a function of a random Poisson process that depended only on the

4 Facebook enforces a maximum limit of the number of notifications that an application can send on behalf of its users, as a spam prevention measure.

network registered Ͼ14 billion page views and sent 3.9 billion Do we need experiments? messages over 89.3 million distinct relationships. For details about the service, the data, and descriptive statistics see the Data section • (Aral et al. 2009): Yahoo! Go service, 2007, n=27.4 million. of the SI. Evidence of Assortative Mixing and Temporal Clustering We observe strong evidence of bothnetwork assortative registered mixing and tem-Ͼ14 billion page views and sent 3.9 billion poral clustering in Go adoption. Atmessages the end of the over 5-month 89.3 period, million distinct relationships. For details about adopters have a 5-fold higher percentage of adopters in their local networks (t Ϫ stat ϭ 100.12, p Ͻ 0.001;thek.s. service,Ϫ stat ϭ 0.06, thep Ͻ data,0.001) and descriptive statistics see the Data section and receive a 5-fold higher percentageof the of messagesSI. from adopters than nonadopters (t Ϫ stat ϭ 88.30, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.17, p Ͻ 0.001). Both the number and percentage of one’s local network who have adopted are highly predictiveEvidence of one’s propensity of Assortative to adopt Mixing and Temporal Clustering (Logistic: ␤(#) ϭ 0.153, p Ͻ 0.001; We␤(%) ϭ observe1.268, p Ͻ 0.001), strong and evidence to of both assortative mixing and tem- adopt earlier (Hazard Rate: ␤(#) ϭ 0.10, p Ͻ 0.001; ␤(%) ϭ 0.003, p Ͻ 0.001). The likelihood of adoptionporal increases clustering dramatically in Gowith adoption. At the end of the 5-month period, the number of adopter friends (Fig. 2C), and correspondingly, adopters are more likely to have moreadopters adopter have friends a (Fig. 5-fold 2B), higher percentage of adopters in their local Fig. 1. Diffusion of Yahoo!Is this Go social over time. influence? (A–C and D–F)Twosubgraphsofthe mirroring prior evidence on productnetworks adoption in (t networksϪ stat (29).ϭ 100.12, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.06, p Ͻ 0.001) Yahoo! IM network colored by adoption states on July 4 (the Go launch date), Adoption decisions among friends also cluster in time. We • P Rosenbaum, AugustD Rubin (1983) 10, and "The October central 29,role 2007.of the propensity For animations score in of observational the diffusion studies of Yahoo! for causal Go effects,"randomly Biometrika. reassigned all Go adoptionand times receive (while maintaining a 5-fold the higher percentage of messages from adopters • D Rubin (2006)over “Matched time sampling see Movies for causal S1 and effects” S2. • S Aral, L Muchnik, A Sundararajan (2009) "Distinguishing influence-based contagion from homophily-driven diffusionadoption in dynamic frequency distribution overthan time) nonadopters and compared observed (t Ϫ stat ϭ 88.30, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.17, networks," PNAS. dyadic differences in adoption times among friends to differences ! among friends with randomly reassignedp Ͻ 0.001). adoption Both times, a the proce- number and percentage of one’s local network A iii launched in July 2007 (Yahoo! Go) (Fig. 2 ), and ( )precise dure known as the ‘‘shuffle test’’ ofwho social influence have adopted (25). Compared are highly predictive of one’s propensity to adopt attribute and dynamic behavioral data on users’ demographics, with these randomly reassigned adoption times, friends␤ areϭ between p Ͻ ␤ ϭ p Ͻ geographic location, mobile device type and usage, and per-day 100% and 500% more likely to adopt(Logistic: within 2 days(#) of each0.153, other, 0.001; (%) 1.268, 0.001), and to page views of different types of content (e.g., sports, weather, news, after which the temporal interdependenceadopt earlier of adoption (Hazard among Rate: ␤(#) ϭ 0.10, p Ͻ 0.001; ␤(%) ϭ 0.003, finance, and photo sharing) from desktop, mobile, and Go plat- friends disappears (Fig. 1D). p Ͻ forms. Much of these data, such as mobile device usage and page Evidence of assortative mixing and0.001). temporal clustering The likelihood may of adoption increases dramatically with views of different types of content, provide fine-grained proxies for suggest peer influence in Go adoption,the but number is by no means of adopter conclu- friends (Fig. 2C), and correspondingly, individuals’ tastes and preferences. The complete set of covariates sive. Demographic, behavioral, andadopters preference are similarities more could likely to have more adopter friends (Fig. 2B), includes 40 time-varying and 6 time-invariant individual and net- simultaneously drive friendship and adoption, creating assortative Fig.work 1. characteristics.Diffusion Taken of Yahoo! together, Go the over sampled time. users (A–C of theand IMD–F)Twosubgraphsofthemixing. Such homophily could alsomirroring explain the temporal prior clustering evidence on product adoption in networks (29). Yahoo! IM network colored by adoption states on July 4 (the Go launch date), Adoption decisions among friends also cluster in time. We August 10, and October 29, 2007. For animations of the diffusion of Yahoo! Go randomly reassigned all Go adoption times (while maintaining the over time see Movies S1 and S2. adoption frequency distribution over time) and compared observed dyadic differences in adoption times among friends to differences among friends with randomly reassigned adoption times, a proce- launched in July 2007 (Yahoo! Go) (Fig. 2A), and (iii)precise dure known as the ‘‘shuffleSOCIAL SCIENCES test’’ of social influence (25). Compared attribute and dynamic behavioral data on users’ demographics, with these randomly reassigned adoption times, friends are between geographic location, mobile device type and usage, and per-day 100% and 500% more likely to adopt within 2 days of each other, page views of different types of content (e.g., sports, weather, news, after which the temporal interdependence of adoption among finance, and photo sharing) from desktop, mobile, and Go plat- friends disappears (Fig. 1D). forms. Much of these data, such as mobile device usage and page Evidence of assortative mixing and temporal clustering may views of different types of content, provide fine-grained proxies for suggest peer influence in Go adoption, but is by no means conclu- individuals’ tastes and preferences. The complete set of covariates sive. Demographic, behavioral, and preference similarities could includes 40 time-varying and 6 time-invariant individual and net- simultaneously drive friendship and adoption, creating assortative work characteristics. Taken together, the sampled users of the IM mixing. Such homophily could also explain the temporal clustering

Fig. 2. Assortative mixing and temporal clustering. (A)ThenumberofGoadoptersperdayfromJuly1toOctober29,2007.(B)Thefractionofadoptersand nonadopters with a given number of adopter friends. (C)Theratioofthelikelihoodofadoptiongivenn adopter friends Pa(n)andthelikelihoodofadoptiongiven 0adopterfriendsPa(0) where the number of adopter friends is assessed at the time of adoption. (D)Frequencyofobserveddyadicdifferencesinadoptiontimesbetween friends compared with differences in adoption times between friends with randomly reassigned adoption times. ⌬t ϭ ti Ϫ tj,whereti represents the time of i’s adoption.

Aral et al. PNAS ͉ December 22, 2009 ͉ vol. 106 ͉ no. 51 ͉ 21545 SOCIAL SCIENCES

Aral et al. PNAS ͉ December 22, 2009 ͉ vol. 106 ͉ no. 51 ͉ 21545 Do we need experiments?

• (Aral et al. 2009): Yahoo! Go service, 2007, n=27.4 million.

Is this social influence? ~50%

• P Rosenbaum, D Rubin (1983) "The central role of the propensity score in observational studies for causal effects," Biometrika. • D Rubin (2006) “Matched sampling for causal effects” • S Aral, L Muchnik, A Sundararajan (2009) "Distinguishing influence-based contagion from homophily-driven diffusion in dynamic networks," PNAS. !

Fig. 4. Influence and homophily effects in Go adoption. (A and B)Alltreatedadopters(filledcircles)andthenumberoftreatedadoptersthatcanbeexplainedby homophily (open circles) per day (A)andcumulativelyovertime(B). (C–E)Treatmenteffectsarethendisplayedwhentheaveragestrengthofego’stiestoadopter friends (measured by the volume of IM message traffic) is greater than and less than the median under random and propensity score matching (C); the clustering coefficient in the network around ego is greater than and less than the median (D); and ego’s page views of news content are greater than and less than the median (E).

more similar to one another and more dissimilar vis-a`-vis the rest that account for homophily, and limitations in observability are of the population. Influence is also overestimated to a greater likely to make our estimates of the homophily effect even more degree in large clusters of adopters because in these clusters the conservative. Second, a distinct but related body of literature homophily effect is more pronounced. Large clusters of adopters examines selection and influence processes in the co-evolution of tend to be more similar to one another, creating greater risk of behaviors and network structure in cases where tie formation is overestimation of influence in the very cliques that seem to be the likely to be a function of the behavior in question [see Snijders et most susceptible to contagious spread. We also find that different al. (34)]. In our context (and in many important contexts) link subsets of the population, characterized by distributions of formation is not likely to be driven by the behavior in question—Go individual and relational characteristics such as the strength of adoption is unlikely to drive friendship. However, extending these ties and local clustering, display various susceptibilities to po- methods to account for selection processes could prove useful in tential influence. cases where selection effects are more prevalent. Third, Yahoo! Go Our work is not without limitations. First, although we measure 2.0 does not exhibit direct network externalities and its adoption is individuals’ dynamic characteristics, preferences, and behaviors in not likely to be driven by the desire to communicate with one’s great detail, the data are not necessarily comprehensive. Although friends by using the application. We suspect that peer influence the matching process accounts for homophily on all observed effects differ for products with direct network externalities and characteristics and those unobserved or latent characteristics that therefore encourage the application of these methods to influence are correlated with what we observe, unobserved and uncorrelated estimation in the adoption of such products. latent homophily and unobserved confounding factors or contex- Understanding the dynamic mechanisms that govern contagion tual effects (such as correlated exposure to advertising among processes in networks is critical in numerous scientific disciplines friends or information from common unobserved friends) may also and for the development of effective social policy, public health contribute to assortative mixing and temporal clustering. The actions, and marketing strategies. A key challenge in identifying the methods therefore establish upper bounds of influence estimates existence and strength of true contagions is to distinguish peer

21548 ͉ www.pnas.org͞cgi͞doi͞10.1073͞pnas.0908800106 Aral et al. network registered Ͼ14 billion page views and sent 3.9 billion messages over 89.3 million distinct relationships. For details about (Propensity score) matching the service, the data, and descriptive statistics see the Data section of the SI. Group B Evidence of Assortative Mixing and Temporal Clustering We observe strong evidence of bothnetwork assortative registered mixing and tem-Ͼ14 billion page views and sent 3.9 billion poral clustering in Go adoption. Atmessages the end of the over 5-month 89.3 period, million distinct relationships. For details about Group A adopters have a 5-fold higher percentage of adopters in their local networks (t Ϫ stat ϭ 100.12, p Ͻ 0.001;thek.s. service,Ϫ stat ϭ 0.06, thep Ͻ data,0.001) and descriptive statistics see the Data section and receive a 5-fold higher percentageof the of messagesSI. from adopters than nonadopters (t Ϫ stat ϭ 88.30, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.17, p Ͻ 0.001). Both the number and percentage of one’s local network who have adopted are highly predictiveEvidence of one’s propensity of Assortative to adopt Mixing and Temporal Clustering (Logistic: ␤(#) ϭ 0.153, p Ͻ 0.001; We␤(%) ϭ observe1.268, p Ͻ 0.001), strong and evidenceto of both assortative mixing and tem- adopt earlier (Hazard Rate: ␤(#) ϭ 0.10, p Ͻ 0.001; ␤(%) ϭ 0.003, p Ͻ 0.001). The likelihood of adoptionporal increases clustering dramatically in Gowith adoption. At the end of the 5-month period, the number of adopter friends (Fig. 2C), and correspondingly, adopters are more likely to have moreadopters adopter have friends a (Fig. 5-fold 2B), higher percentage of adopters in their local Fig. 1. Diffusion of Yahoo! Go over time. (A–C and D–F)Twosubgraphsofthe mirroring prior evidence on productnetworks adoption in (t networksϪ stat (29).ϭ 100.12, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.06, p Ͻ 0.001) Yahoo! IM network colored by adoption states on July 4 (the Go launch date), Adoption decisions among friends also cluster in time. We August 10, and October 29, 2007. For animations of the diffusion of Yahoo! Go randomly reassigned all Go adoptionand times receive (while maintaining a 5-fold the higher percentage of messages from adopters over time see Movies S1 and S2. adoption frequency distribution overthan time) nonadopters and compared observed (t Ϫ stat ϭ 88.30, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.17, dyadic differences in adoption times among friends to differences among friends with randomly reassignedp Ͻ 0.001). adoption Both times, a the proce- number and percentage of one’s local network A iii launched in July 2007 (Yahoo! Go) (Fig. 2 ), and ( )precise dure known as the ‘‘shuffle test’’ ofwho social influence have adopted (25). Compared are highly predictive of one’s propensity to adopt attribute and dynamic behavioral data on users’ demographics, with these randomly reassigned adoption times, friends␤ areϭ between p Ͻ ␤ ϭ p Ͻ geographic location, mobile device type and usage, and per-day 100% and 500% more likely to adopt(Logistic: within 2 days(#) of each0.153, other, 0.001; (%) 1.268, 0.001), and to page views of different types of content (e.g., sports, weather, news, after which the temporal interdependenceadopt earlier of adoption (Hazard among Rate: ␤(#) ϭ 0.10, p Ͻ 0.001; ␤(%) ϭ 0.003, finance, and photo sharing) from desktop, mobile, and Go plat- friends disappears (Fig. 1D). p Ͻ forms. Much of these data, such as mobile device usage and page Evidence of assortative mixing and0.001). temporal clustering The likelihood may of adoption increases dramatically with views of different types of content, provide fine-grained proxies for suggest peer influence in Go adoption,the but number is by no means of adopterconclu- friends (Fig. 2C), and correspondingly, individuals’ tastes and preferences. The complete set of covariates sive. Demographic, behavioral, andadopters preference are similarities more could likely to have more adopter friends (Fig. 2B), includes 40 time-varying and 6 time-invariant individual and net- simultaneously drive friendship and adoption, creating assortative Fig.work 1. characteristics.Diffusion Taken of Yahoo! together, Go the over sampled time. users (A–C of theand IMD–F)Twosubgraphsofthemixing. Such homophily could alsomirroring explain the temporal prior clustering evidence on product adoption in networks (29). • P Rosenbaum, D Rubin (1983) "The central role of the propensityYahoo! score IM in network observational colored studies by adoption for causal states effects," on JulyBiometrika. 4 (the Go launch date), Adoption decisions among friends also cluster in time. We • D Rubin (2006) “Matched sampling for causal effects” August 10, and October 29, 2007. For animations of the diffusion of Yahoo! Go randomly reassigned all Go adoption times (while maintaining the • S Aral, L Muchnik, A Sundararajan (2009) "Distinguishing influence-basedover time see contagionMovies S1 from and homophily-driven S2. diffusion in dynamic adoption frequency distribution over time) and compared observed networks," PNAS. ! dyadic differences in adoption times among friends to differences among friends with randomly reassigned adoption times, a proce- launched in July 2007 (Yahoo! Go) (Fig. 2A), and (iii)precise dure known as the ‘‘shuffleSOCIAL SCIENCES test’’ of social influence (25). Compared attribute and dynamic behavioral data on users’ demographics, with these randomly reassigned adoption times, friends are between geographic location, mobile device type and usage, and per-day 100% and 500% more likely to adopt within 2 days of each other, page views of different types of content (e.g., sports, weather, news, after which the temporal interdependence of adoption among finance, and photo sharing) from desktop, mobile, and Go plat- friends disappears (Fig. 1D). forms. Much of these data, such as mobile device usage and page Evidence of assortative mixing and temporal clustering may views of different types of content, provide fine-grained proxies for suggest peer influence in Go adoption, but is by no means conclu- individuals’ tastes and preferences. The complete set of covariates sive. Demographic, behavioral, and preference similarities could includes 40 time-varying and 6 time-invariant individual and net- simultaneously drive friendship and adoption, creating assortative work characteristics. Taken together, the sampled users of the IM mixing. Such homophily could also explain the temporal clustering

Aral et al. PNAS ͉ December 22, 2009 ͉ vol. 106 ͉ no. 51 ͉ 21545 SOCIAL SCIENCES

Aral et al. PNAS ͉ December 22, 2009 ͉ vol. 106 ͉ no. 51 ͉ 21545 network registered Ͼ14 billion page views and sent 3.9 billion messages over 89.3 million distinct relationships. For details about (Propensity score) matching the service, the data, and descriptive statistics see the Data section of the SI. Group B Evidence of Assortative Mixing and Temporal Clustering We observe strong evidence of bothnetwork assortative registered mixing and tem-Ͼ14 billion page views and sent 3.9 billion poral clustering in Go adoption. Atmessages the end of the over 5-month 89.3 period, million distinct relationships. For details about Group A adopters have a 5-fold higher percentage of adopters in their local networks (t Ϫ stat ϭ 100.12, p Ͻ 0.001;thek.s. service,Ϫ stat ϭ 0.06, thep Ͻ data,0.001) and descriptive statistics see the Data section and receive a 5-fold higher percentageof the of messagesSI. from adopters than nonadopters (t Ϫ stat ϭ 88.30, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.17, p Ͻ 0.001). Both the number and percentage of one’s local network who have adopted are highly predictiveEvidence of one’s propensity of Assortative to adopt Mixing and Temporal Clustering (Logistic: ␤(#) ϭ 0.153, p Ͻ 0.001; We␤(%) ϭ observe1.268, p Ͻ 0.001), strong and evidenceto of both assortative mixing and tem- adopt earlier (Hazard Rate: ␤(#) ϭ 0.10, p Ͻ 0.001; ␤(%) ϭ 0.003, p Ͻ 0.001). The likelihood of adoptionporal increases clustering dramatically in Gowith adoption. At the end of the 5-month period, the number of adopter friends (Fig. 2C), and correspondingly, adopters are more likely to have moreadopters adopter have friends a (Fig. 5-fold 2B), higher percentage of adopters in their local Fig. 1. Diffusion of Yahoo! Go over time. (A–C and D–F)Twosubgraphsofthe mirroring prior evidence on productnetworks adoption in (t networksϪ stat (29).ϭ 100.12, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.06, p Ͻ 0.001) Yahoo! IM network colored by adoption states on July 4 (the Go launch date), Adoption decisions among friendsand also receive cluster a in 5-fold time. We higher percentage of messages from adopters MatchAugust Group 10, and October A (friends 29, 2007. For animations of early of the diffusion adopters) of Yahoo! Go torandomly reassigned all Go adoption times (while maintaining the over time see Movies S1 and S2. adoption frequency distribution overthan time) nonadopters and compared observed (t Ϫ stat ϭ 88.30, p Ͻ 0.001; k.s. Ϫ stat ϭ 0.17, Group B (rest of graph) so that the two dyadic differences in adoption times among friends to differences among friends with randomly reassignedp Ͻ 0.001). adoption Both times, a the proce- number and percentage of one’s local network A iii groupslaunched have in July 2007 the (Yahoo! same Go) (Fig.demographics. 2 ), and ( )precise dure known as the ‘‘shuffle test’’ ofwho social influence have adopted (25). Compared are highly predictive of one’s propensity to adopt attribute and dynamic behavioral data on users’ demographics, with these randomly reassigned adoption times, friends␤ areϭ between p Ͻ ␤ ϭ p Ͻ geographic location, mobile device! type and usage, and per-day 100% and 500% more likely to adopt(Logistic: within 2 days(#) of each0.153, other, 0.001; (%) 1.268, 0.001), and to Roughly: If grouppage views B has, of different e.g., types oftoo content few (e.g., sports,young weather, people, news, after which the temporal interdependenceadopt earlier of adoption (Hazard among Rate: ␤(#) ϭ 0.10, p Ͻ 0.001; ␤(%) ϭ 0.003, finance, and photo sharing) from desktop, mobile, and Go plat- friends disappears (Fig. 1D). p Ͻ 0.001). The likelihood of adoption increases dramatically with thenforms. “double Much of these count data, such them”; as mobile deviceplus usage 46 and other page Evidence of assortative mixing and temporal clustering may views of different types of content, provide fine-grained proxies for suggest peer influence in Go adoption,the but number is by no means of adopterconclu- friends (Fig. 2C), and correspondingly, individuals’ tastes and preferences.traits. The complete set of covariates sive. Demographic, behavioral, andadopters preference are similarities more could likely to have more adopter friends (Fig. 2B), includes 40 time-varying and 6 time-invariant individual and net- simultaneously drive friendship and adoption, creating assortative Fig.work 1. characteristics.Diffusion Taken of Yahoo! together, Go the over sampled time. users (A–C of theand IMD–F)Twosubgraphsofthemixing. Such homophily could alsomirroring explain the temporal prior clustering evidence on product adoption in networks (29). • P Rosenbaum, D Rubin (1983) "The central role of the propensityYahoo! score IM in network observational colored studies by adoption for causal states effects," on JulyBiometrika. 4 (the Go launch date), Adoption decisions among friends also cluster in time. We • D Rubin (2006) “Matched sampling for causal effects” August 10, and October 29, 2007. For animations of the diffusion of Yahoo! Go randomly reassigned all Go adoption times (while maintaining the • S Aral, L Muchnik, A Sundararajan (2009) "Distinguishing influence-basedover time see contagionMovies S1 from and homophily-driven S2. diffusion in dynamic adoption frequency distribution over time) and compared observed networks," PNAS. ! dyadic differences in adoption times among friends to differences among friends with randomly reassigned adoption times, a proce- launched in July 2007 (Yahoo! Go) (Fig. 2A), and (iii)precise dure known as the ‘‘shuffleSOCIAL SCIENCES test’’ of social influence (25). Compared attribute and dynamic behavioral data on users’ demographics, with these randomly reassigned adoption times, friends are between geographic location, mobile device type and usage, and per-day 100% and 500% more likely to adopt within 2 days of each other, page views of different types of content (e.g., sports, weather, news, after which the temporal interdependence of adoption among finance, and photo sharing) from desktop, mobile, and Go plat- friends disappears (Fig. 1D). forms. Much of these data, such as mobile device usage and page Evidence of assortative mixing and temporal clustering may views of different types of content, provide fine-grained proxies for suggest peer influence in Go adoption, but is by no means conclu- individuals’ tastes and preferences. The complete set of covariates sive. Demographic, behavioral, and preference similarities could includes 40 time-varying and 6 time-invariant individual and net- simultaneously drive friendship and adoption, creating assortative work characteristics. Taken together, the sampled users of the IM mixing. Such homophily could also explain the temporal clustering

Aral et al. PNAS ͉ December 22, 2009 ͉ vol. 106 ͉ no. 51 ͉ 21545 SOCIAL SCIENCES

Aral et al. PNAS ͉ December 22, 2009 ͉ vol. 106 ͉ no. 51 ͉ 21545 Propensity score matching

• (Aral et al. 2009): Yahoo! Go service, 2007.

~50% of users were adopting due to latent traits

21548 ͉ www.pnas.org͞cgi͞doi͞10.1073͞pnas.0908800106 Aral et al. A/B testing under network effects A/B testing under network effects A/B testing under network effects Causal inference & network effects

Universe A Universe B

Fundamental problem: want to compare (average treatment effect, ATE), but can’t observe network in both states at once.

• J Ugander, B Karrer, L Backstrom, J Kleinberg (2013) "Graph Cluster Randomization: Network Exposure to Multiple Universes," KDD. • D Eckles, B Karrer, J Ugander (2014) "Design and analysis of experiments in networks: Reducing bias from interference," arXiv. • S Athey, D Eckles, G Imbens (2015) "Exact P-values for Network Interference," arXiv. Direct vs. indirect effects Universe A Direct effect

Indirect effect Universe B

• P Aronow, C Samii (2013) "Estimating average causal effects under interference between units," arXiv. • C Manski (2013) "Identification of treatment response with social interactions," The Econometrics Journal. Experiments with interference

Chat/communication services Social product design

Market Mechanisms (ads, labor, etc) Content ranking models Design & Analysis Design & Analysis

Surrounded

Surrounded Design & Analysis

Surrounded

Remember me?

Surrounded Design & Analysis

Control Treatment Average Treatment Effect Treatment: Response: Average Treatment Effect: zi =1

Yi(zi = 1) = 60 min on site/day 1 n z =0 ⌧ = (Y (z = 1) Y (z = 0)). i n i i i i i=1 Yi(zi = 0)= 45 min on site/day X Average Treatment Effect Treatment: Response: Average Treatment Effect: zi =1

Yi(zi = 1) = 60 min on site/day 1 n z =0 ⌧ = (Y (z = 1) Y (z = 0)). i n i i i i i=1 Yi(zi = 0)= 45 min on site/day X

▪ A/B testing: ▪ IID randomization, inverse probability weighting ! 1 n 1[Z = 1] 1[Z = 0] ! ⌧ˆ(Z)= Y (Z = 1) i Y (Z = 0) i n i i Pr(Z = 1) i i Pr(Z = 0) ! i=1 i i X ✓ ◆ Average Treatment Effect Treatment: Response: Average Treatment Effect: zi =1

Yi(zi = 1) = 60 min on site/day 1 n z =0 ⌧ = (Y (z = 1) Y (z = 0)). i n i i i i i=1 Yi(zi = 0)= 45 min on site/day X

▪ A/B testing: ▪ IID randomization, inverse probability weighting ! n 1 1[Zi = 1] 1[Zi = 0] ~ ! ⌧ˆ(Z)= Y (Z = 1) Y (Z = 0) Z = 1 n i i Pr(Z = 1) i i Pr(Z = 0) ! i=1 i i X ✓ ◆ ▪ Introduce sets: 1 0 , 1 n , Y (Z 0) Y (Z = ~0) i ✓{ } i 2 i ⇡ i 0 n 1 ~ ! i 0, 1 , Yi(Z i ) Yi(Z = 1) ✓{ } 2 ⇡ Z = ~0 ▪ ATE with interference (Aronow & Samii 2012, Manski 2013): 1 n Y (Z)1[Z 1] Y (Z)1[Z 0] ⌧ˆ(Z)= i 2 i i 2 i n Pr(Z 1) Pr(Z 0) i=1 i i X ✓ 2 2 ◆ Analysis: “network exposure” ▪ Two treatment conditions: treatment/control. ▪ When are people network exposed to their treatment condition? ▪ Neighborhood exposure to treatment/control: ▪ Full neighborhood exposure: you and all neighbors ▪ Fractional neighborhood exposure: you and ≥q% neighbors ▪ Many more notions are plausible Design & Analysis

Control Treatment Design: how to assign? Design: how to assign? Design: graph cluster randomization Assign vertices according to graph clusters: ▪ Form clusters; assign each cluster to treatment with probability p ▪ Assign all vertices to their cluster’s assignment Design: graph cluster randomization Assign vertices according to graph clusters: ▪ Form clusters; assign each cluster to treatment with probability p ▪ Assign all vertices to their cluster’s assignment

▪ Probability of full neighborhood exposure: p (# clusters connected to i) ▪ Probability of fractional neighborhood exposure: dynamic program Algorithms for clustering ▪ Facebook: 1B+ vertices, 100B+ edges (countries still 100M+ vertices) ▪ Require highly scalable methods:

▪ Label Propagation, Louvain method: [Zhu & Ghahramani 2002, Blondel et al. 2008]

▪ Balanced Label Propagation: [Ugander & Backstrom 2013]

▪ Streaming/Restreaming graph partitioning: [Stanton & Kliot 2012, Tsourakakis et al. 2012, Nishimura & Ugander 2013]

▪ ✏ -net clustering: Variance bounds on graphs with restricted growth [Ugander et al. 2013, Karger & Ruhl 2002, Gupta, Krauthgamer & Lee 2003] Network Experimentation

Initialization Design Outcome Generation Analysis

Treatment Response Treatment weight Control Control weight

Figure 1: Model of the network experimentation process, consisting of (i) initialization, ▪ Initialization:which generates the An graph empirical and vertex graph characteristics, or graph (ii)model design , which determines the ▪ Design:randomization Graph scheme, cluster(iii) outcomerandomization generation , which observes or simulates behavior, and (iv) analysis, which constructs an estimator. We examine the bias and variance ▪ Outcomeof treatment generation: effect estimators Observe under different behavior design (or and observe analysis model) methods for varied initialization and outcome generation processes. ▪ Analysis: Discerning effective treatment

Clusteredtreatment todesign all vertices. & well-founded Let Z be a vector analysis giving each can vertex’s reduce treatment bias assignment,and variance. so that Yi(Z = z) is the potential outcome of interest for vertex i when Z is set to z. Then the quantity of interest is a contrast between two such treatment vectors, 1 ⌧(z ,z )= E[Y (Z = z ) Y (Z = z )], (1) 1 0 N i 1 i 0 i X where N is the number of units and z1 and z0 are two treatment assignments. Note that each vertex’s potential outcome is a function of the global treatment assignment vector Z. Additional assumptions will thus be required for ⌧ to be identiﬁable. This is closely connected to what Holland (1988) calls the fundamental problem of causal inference — that one can only observe a unit’s response under a single treatment. The difference is that here we can only observe all vertices’ responses under a single global treatment. To identify ⌧, it is sufﬁcient to assume SUTVA and that treatment assignment is ignorable. In the absence of assuming SUTVA, other assumptions will be necessary.

2.1 Initialization Initialization is everything that occurs prior to the experiment. This includes network formation and the processes that produce vertex characteristics and prior behaviors. In some cases, we may regard this process as random, and so wish to understand design and analysis decisions averaged over instances of this process; for example, we may wish to average over a distribution of networks that corresponds to particular network formation model. In other cases, we may regard the outcome of this process as ﬁxed; for example, we may be working with a particular network and vertices with particular characteristics, which we wish to condition on in planning our design and analysis. When initialization is complete, we have a particular network G =(V,E) with adjacency matrix A. In the simulations we present here, G is either a particular real network we observe or it is generated as a small world network (Watts and Strogatz, 1998). The small world network model has three parameters: the network size N,

4 REPORTS

ing the correlation of influence and susceptibility 2) The “influentials” and “susceptibles” highly susceptible users (Fig. 4, panel IV). The across all individuals and the assortativity of hypotheses are orthogonal claims. Both influ- clustering of influentials suggests the existence influence and susceptibility across all individ- ential individuals and noninfluential individu- of a multiplier effect of infecting a highly influ- uals and their peers in the network. We calcu- als have approximately the same distribution ential individual. However, such individuals also lated individual influence and susceptibility scores of susceptibility to influence among their peers; tend to have peers with only average susceptibil- as the product of the estimated hazard ratios of hence, being influential is not simply a conse- ity, making predictions about which effect would individuals’ attributes for a broader sample of quence of having susceptible peers (Fig. 4, dominate difficult without more evidence. Addi- 12 million users with 85 million relationships. panel II). Both influence and susceptibility play tional empirical and simulation studies should The analysis combines the estimated impact of aroleinthepeer-to-peerdiffusionoftheproduct. therefore examine the effects of the assortativity each demographic attribute on influence and sus- Combining studies of influence with studies of of influence and susceptibility on the diffusion ceptibility to calculate individuals’ overall influ- susceptibility will therefore likely improve our of behaviors, products, and diseases. ence and susceptibility scores. For example, a understanding of the diffusion of behavioral Analyzing the heat maps in Fig. 4 is not 35-year-old single female has an influence score contagions. sufficient to identify optimal intervention targets, equal to exp(binfl, >31 + binfl, single + binfl, female). 3) There are more people with high influ- because more information is needed about the The following inferences can be drawn from our ence scores than high susceptibility scores (Fig. network structure around candidate targets in results: 4, panel I), which suggests that, in our context, each region. For example, an individual with 1) Highly influential individuals tend not to targeting should focus on the attributes of cur- high influence and high peer susceptibility in the be susceptible, highly susceptible individuals tend rent adopters (e.g., giving individuals incentives upper right quadrant of in Fig. 4, panel II, may not to be influential, and almost no one is both to influence their peers) rather than attributes of seem like a good target, but may be of low de- highly influential and highly susceptible to in- their peers (e.g., giving individuals with suscep- gree or may be isolated. The network diagrams fluence (Fig. 4, panel I). This implies that influ- tible peers incentives to adopt). to the left of the heat maps show the assorta- ential individuals are less likely to adopt the 4) Influentials cluster in the network. As tivity of influence and susceptibility in ego product as a consequence of natural influence shown in Fig. 4, panel III, influential individuals networks from different regions combined with Summary processes (i.e., in the absence of targeting); hence, connected to other influential peers are approx- information on their network structures, such as targeting influentials with low propensities to imately twice as influential as baseline users. In network degree and the distribution of influence spontaneously adopt would be a potentially vi- contrast, we find a tendency for less suscepti- and susceptibility across peers in the network. able promotion strategy. ble users to cluster together and no clusters of Analyzing networks in different regions of the

▪ Experiments to test social network theories continue to require innovative large-scale computation.

▪ Accelerating trend from small data to big data to Fig. 4. Scores for 12 million Facebook users (collected from users who in- IV, ego susceptibility (y axis) and peer susceptibility (x axis). The heat maps stalled one of several other Facebook applications developed by the company) do not provide information on network structure, which can be important for with 85 million relationships are calculated by means of hazard rate estimates informing targeting decisions. The diagrams to the left of the heat maps relative to the baseline hazard in the influence and susceptibility model show the assortativity of influence and susceptibility in ego networks drawn described in the text. The resulting heat maps are shown at the right. Panel I from the regions of the heat maps labeled A, B, C, and D. Nodes in the big experiments. displays the percentage of people (ego) with predicted influence (y axis) and networks are sized in proportion to theirpredictedinfluence(largernodes predicted susceptibility (x axis). Panels II to IV display the percentage of ego- are more influential) and are shaded and placed relative to their predicted peer relationships: panel II, ego influence (y axis) and peer susceptibility (x susceptibility (redder nodes and nodes closer to ego are more susceptible; axis); panel III, ego influence (y axis) and peer influence (x axis); and panel grayer nodes and nodes farther from ego are less susceptible).

340 20 JULY 2012 VOL 337 SCIENCE www.sciencemag.org ▪ Challenge: influence maximization with realism.

▪ Challenge: improve theory for network data from disparate contexts (generalizability).

▪ Challenge: Propensity scores and null model statistics at scale and in difficult experiments.

▪ Challenge: applications of graph clustering/ community detection in experimental design