Hidden Community Detection in Online Forums

Daniel Salz Nicholas Benavides Jonathan Li Stanford University Stanford University Stanford University [email protected] [email protected] [email protected]

Abstract cial networks, communities take on a very re- latable meaning, as communities of users are This project investigated both dominant and densely connected and may share common in- hidden community structure on a network of in- terests, backgrounds, or experiences. Online fo- teractions between subreddits on Reddit.com. We rums, such as Reddit, are generally divided into identified moderate community structure using communities based on the topic of the forum (ie the Louvain algorithm, and we also detected some r/nba for NBA fans or r/politics for those inter- hidden community structure with the HICODE ested in political news and discussions). The algorithm. We also explored a method to de- users in a specific forum can be thought of as tect future communities that involved predicting members of a community, but we can also think new edges and their weights, adding the predicted about groups of these forums as communities in edges to the existing network, and re-running the their own right. Louvain and HICODE algorithms to determine Within the field of community detection, there changes in the community structure. While the has been some interesting research conducted on edge weight prediction was a difficult task, we the topic of hidden community detection, which found that a linear regression model using 256- attempts to identify weaker community structure dimension node embeddings yielded the best re- that often goes undetected by the popular commu- sults. When we added predicted edges to the net- nity detection algorithms. Although these hidden work, we found slightly weaker community struc- communities are more difficult to identify, they ture, no evidence of hidden community structure can reveal unique insights into the functioning of as determined HICODE, and fewer communities a network. HICODE, one such algorithm for hid- overall, suggesting that lines between communi- den community detection, removes edges from ties may become more blurred as the network ma- the graph to try and uncover more subtle commu- tures and experiences more interactions between nities that are dominated by the density of edges forums. in the original graph. This paper aims to analyze both the dominant and hidden community structure of subreddits on 1. Introduction Reddit, leveraging a combination of established and new techniques to understand how the algo- Community detection is an important task in rithms perform in different conditions. We also the field of network analysis, as it can reveal in- introduce a new way to think about hidden com- formation about the structure of a network and munity detection that adds edges to the graph how the flow of information can occur. In so-

1 rather than removing them. By predicting new community as well as crowd-sourced sentiment edges and running existing community detection of the posts to identify negative mobilizations. algorithms on the resulting graph, we can detect The analysis relies on two variants of the PageR- hidden communities that are not present in the ank algorithm, which are used to measure the current graph but may appear in the network in flow of information from the perspectives of at- the future. With this information, moderators of tackers and defenders in the conflict, respec- these online forums will have greater insight into tively. Finally, the paper presents an LSTM model the interactions between forums as well as how that leverages data from both the post text and they can expect those interactions to change in the the user-community interaction network to pre- future. dict the likelihood of a post leading to a conflict between communities, achieving a significantly higher AUC than the baseline model. 2. Related Work In A Unified Framework for Community De- In their paper Hidden Community Detection in tection and Network Representation Learning, Social Networks, He et. al. [1] highlight the Tu et. al. [3] propose a unified framework utility and importance of detecting hidden com- for community detection and network representa- munity structures in graphs, sparse structures that tion learning (NRL) called community-enhanced can be tangled with the structure of the dom- network representation learning (CNRL). The inant community structure and are thus harder method considers each vertex grouped into mul- to detect. He et al note that the existing liter- tiple overlapping communities. Different from ature and community detection algorithms often typical NRL where vertex embeddings are learnt are able to find dominant community structures from local context vertices, CNRL uses both lo- but have difficulty revealing hidden community cal and global community information. They im- structures. He et al introduce HICODE, an al- plemented two different strategies with CNRL, gorithm for detecting hidden communities within a statistical based and an embedding based one, graphs that builds upon existing community de- of which both performed similarly. The ini- tection algorithms. HICODE works by reveal- tial step of the method consists of perform- ing hidden communities in layers, first isolating ing a Community-enhanced DeepWalk over the dominant communities and then weakening the network to generate “word” sequences as well connections within those communities and rean- as community assignments and updated ver- alyzing the resulting graph for new community tex representations. By doing this representa- structure. This process is repeated to reveal mul- tional learning process, they create representa- tiple layers of hidden communities, and has been tions for vertices, communities, and community found to produce state-of-the-art results in reveal- distributions of the vertices. From their experi- ing both strong and weak community structure. ments both the embedding and statistical based Kumar et. al.’s Community Interaction and community enhanced deep walk/node2vec im- Conflict on the Web [2] examines the nature plementations consistently and significantly out- of conflict between communities on Reddit.com. performed current methods such as DeepWalk, Without ground truth labels indicating when an LINE, node2vec, SDNE, MNMF, and ComE. intercommunity conflict occurs, the authors rely on hyperlinks between communities and a null model of user activity that detects cases where a hyperlink mobilizes a substantial number of users in one community to comment in another 3. Dataset graph, which revealed that most nodes have de- gree ≤ 10, while a small fraction of nodes have 3.1. Community Detection Dataset degree ≥ 100. We utilized the SNAP Reddit Hyperlink Net- work Dataset, which is a directed, signed, tem- poral, and attributed graph. Nodes in the graph are subreddits (online forums within Reddit), and edges between subreddits signify a post from one subreddit hyperlinking to the other subreddit, with the edge weight corresponding to either a -1 (negative sentiment) or 1 (neutral or positive sen- timent). Each edge (post) is annotated with the timestamp, the sentiment of the source commu- nity post towards the target community post, and the text property vector of the source post contain- ing 86 features related to post content. The graph contains 55,863 nodes (subreddits) and 858,490 Figure 1. Degree Distribution Plot edges (posts). This dataset was generated as part We also ran PageRank on our graph to un- of the Kumar et. al. paper [2]. derstand what subreddits were central in the net- The dataset distinguishes between posts that work of hyperlinks. From these results, it ap- contain the hyperlink in the title of the post and pears that the most influential subreddits tend to posts that contain the hyperlink in the body of be larger subreddits about more general topics the post. From this, we generated a single undi- such as ’askreddit’, ’bestof’, ’funny’, and ’pics’. rected, weighted graph for the community detec- tion task. To do so, we first took the union of 3.2. Edge Weight Prediction Dataset nodes and edges where the hyperlink was in the From our graph, we generated a dataset that title and where the hyperlink was in the body to could be used for edge weight prediction. Given generate a larger graph. From there, removed the a set of node embeddings, we created a positive directionality of the edges, and collapsed multi- example for each edge in the graph by concate- ple edges between nodes into a single edge with nating the node embeddings associated with the weight equal to the number of hyperlinked posts edge, resulting in 309,667 positive examples. The between the subreddits. We chose to convert the assigned label is the weight of the edge. With- graph to be undirected because most community out a ground truth for negative examples, we detection algorithms require an undirected graph, leveraged a heuristic that pairs of nodes that are and we chose to include edge weights because 5 or more hops apart are unlikely to be related it enabled us to capture richer information about and have hyperlinks between them in the future. the interactions between subreddits. Our resulting To generate negative examples, we randomly se- graph contains 67,180 nodes and 309,667 edges. lected 300,000 node pairs whose nodes had dis- Conducting a basic exploration of the dataset, tance ≥ 5 between them, concatenated the node we found that the graph has 712 connected embeddings for each pair of nodes, and assigned components, with a giant component containing the example a label of 0 since no edge exists be- 65,648 nodes and the other 711 connected com- tween the nodes. We also generated an infer- ponents containing 9 or fewer nodes. We also ence dataset of 300,000 examples by randomly examined the degree distribution of nodes in the selecting pairs of nodes that are not connected in the graph and concatenating their embeddings to ations of refinement, form examples, which we use to predict future PT t=1 Qt edges. Rt = T · Q0 T = 5 4. Methods In our implementation we use . For both algorithms, our primary evaluation 4.1. Baseline Community Detection metric for the quality of the detected communi- ties was modularity, which is denoted by Q and For our baseline community detection algo- described by the following equation: rithm, we chose the Louvain algorithm for its   strong performance and ability to scale well to 1 X kikj Q = Aij − δ (ci, cj) large graphs [4] and leveraged an implementa- 2m 2m i,j tion from the NetworkX Python library [5]. The In the equation, 2m is the sum of the adjacency Louvain algorithm greedily optimizes modular- matrix A, A is the weight of the edge between ity by iteratively changing node community as- ij node i and node j, k represents the degree of signments and collapsing communities into meta- i node i, and δ(c , c ) is an indicator function that nodes. i j evaluates to 1 if nodes i and j are in the same 4.2. HICODE: Hidden Community Detection. community and 0 otherwise. We also implemented the HICODE algorithm 4.3. Edge Weight Prediction [1] for hidden community detection since there The other main task in this project was pre- was not a python implementation readily avail- dicting new edges in the graph along with their able. The HICODE algorithm is composed of corresponding weights. To accomplish this, we two stages, identification and refinement. In the decomposed the task into two parts - generating identification stage, HICODE leverages a base node embeddings and training predictive models. community detection algorithm (Louvain, CNM, Givran-Newman, etc.) to generate layers, or par- 4.3.1 Node Embeddings titions of nodes into communities. It first applies the base detection algorithm, in our case Lou- In order to train regression models on our graph vain, then weakens the structure of the detected to predict new edges and their corresponding communities by reducing the weights on intra- weights, we needed to generate embeddings that community edges. Louvain is then re-run to de- we could use as a dataset for prediction. For this tect the next layer and the process repeats itself project, we experimented with 2 types of node for nL number of layers. embeddings - recursive feature aggregation as de- In the refinement stage, HICODE improves scribed by the RolX algorithm [6] and node2vec the layers detected by weakening the community embeddings [7]. structure of the communities in other hidden lay- For our baseline embedding method, we chose ers. This is done by reducing the edge weights recursive feature aggregation described by the in the same fashion as in the identification stage. RolX algorithm because it was relatively simple This refinement process is repeated for a select (did not need to be trained) but still had been number of iterations, and a metric Rt, the aver- shown to perform well. For each node, we com- age improvement ratio of the modularity metric is puted 6 features - the number of outgoing edges, calculated. HICODE will choose nL by taking nL the sum of outgoing edge weights, the number that maximizes Rt. of edges in the egonet, the sum of edge weights Rt is given in the following equation for T iter- in the egonet, the number of edges crossing the egonet, and the sum of edge weights crossing the 4.3.2 Regression Models egonet. We also added three types of aggretation - For our baseline edge weight prediction model, mean aggregation, sum aggregation, and max ag- we chose linear regression for its simplicity, train- gregation. We found that 2 iterations of the recur- ing speed, and interpretability of the coefficients. sive feature aggregation was an optimal tradeoff We also trained several deep regression models. between speed and performance, resulting in 96- For the deep learning models we elected to use dimensional node embeddings. a similar strategy as for node2vec and take advan- For a more advanced embedding method, we tage of the large amount of compute available to chose node2vec given its strong empirical per- train many models. All models were trained us- formance and ability to optimize training for our ing the tensorflow-keras framework. First a node specific application. The node2vec algorithm embedding setting was randomly chosen from learns node embeddings by executing a number a directory that had the previously mentioned of random walks to collect neighborhood char- node2vec and rolx embeddings. After choosing acteristics. Node2Vec differs from typical ran- the embeddings a Neural Network is built, either dom walks by incorprating both a BFS and DFS in the form of a dense neural network (DNN) strategy which can be individually tweaked by hy- or recurrent neural network (RNN), from sam- perparameters to orchestrate more local or global pling parameters from pre-defined discrete lists. behavior. Once the random walks have been For both types of models the number of layers, collected, stochastic gradient descent is applied hidden layer sizes, learning rate, and seed were to maximize the log-probability of observing a randomly chosen. The maximum possible hid- neighbor of a node given its embedding [7]. den layer size was 512 and the maximum possi- Our implementation of node2vec used the [put ble number of layers was 8 and 3 for the DNN or github ref] node2vec library primarily because all RNNs respectively. DNN architectures were fully other robust implementations we found used large connected layers with relu activations and RNN amounts of memory for each training session and architectures were LSTM cells with tanh activa- made it infeasible to train more than a couple tions. The DNN architecture additionally utilized in parallel even on large machines. For each dropout with a rate of either 0, .25, or .5. node2vec model the training procedure randomly For all of our edge weight prediction models, sampled hyperparameters from defined discrete we round predictions to the nearest integer value lists for: number of walks per node, length of with a floor of 0 and use the root mean square er- each walk, initial seed, embedding dimension ror (RMSE) and mean absolute error (MAE) as size, learning rate, BFS and DFS parameters. Ini- our evaluation metrics. Both of these metrics are tially creating the node embeddings and applying common evaluation metrics for regression models regression was one process (if an embedding set- and measure the average deviation between the ting that had already existed was sampled again actual and predicted values, taking on a value in it was simply reloaded), however the CPU and the interval [0, ∞). memory requirements from node2vec and GPU requirements from the deep learning models did 5. Experiments not interact well on the same machine. It was significantly faster and cheaper to train node2vec 5.1. Community Detection on Base Graph models in parallel on a machine with many cores For our first experiment, we ran our two and large main memory, while DL models were community detection algorithms, Louvain and trained on a several GPU core machine. HICODE, on the base graph. HICODE identified 2 layers of communities, and the results of the al- gorithms are shown below. strong enough to be detected by Louvain unless intra-community edge weights are reduced. It Algorithm Modularity Communities also appears that nodes in the community of cities detected by Louvain are dispersed among other Louvain 0.4765 784 communities in the first layer of HICODE. HICODE Layer 1 0.4782 779 Investigating the communities detected by the HICODE Layer 2 0.4732 780 second layer of HICODE, the results seem Figure 2. Baseline Community Detection to differ only slightly from the first HICODE Both algorithms identify moderately strong layer. We still observe strong communities community structure and identify a very similar around Health/Fitness, Bitcoin, Soccer, Sports, number of communities. The difference in mod- and Movies/TV Shows. One interesting note ularity scores between the partitions is not signif- about the partition in this layer is that some of icant. Furthermore, manually examining the de- the motorsports subreddits moved from the Sports tected communities provides deeper insight into community to the Bitcoin community, suggesting the differences between the results of the parti- that there may be some overlap between propo- tions. nents of Bitcoin and fans of motorsports. Analyzing the partition generated by Louvain, Based on all of these results, both quantitative we find that the subreddits are generally clus- and qualitative, we conclude that our graph does tered by similar topics. For example, there is a have some hidden community structure, as evi- community of gaming subreddits, containing fo- denced by the similar modularity scores for hid- rums such as rddtgaming, xboxone, battlefield4, den community layers detected by HICODE com- smashbros, and grandtheftautov. Other strong pared to the Louvain baseline as well as exam- communities of subreddits include but are not ples of more subtle groupings within the detected limited to Health/Fitness, Soccer, Bitcoin, Drugs, communities. Cities, Travel/Foreign Countries, and Mother- hood/Babies. Overall, there are approximately 50 large communities, with the remaining communi- ties nearly all having fewer than 5 subreddits in them. Examining the communities detected in the 5.2. Edge Weight Prediction Model Performance first layer of HICODE, we find similar strong For each embedding method, we trained sev- communities such as Health/Fitness (containing eral regression models to predict edge weights be- subreddits such as judo, musclebuilding, yoga, tween pairs of nodes that are not adjacent to one myfitnesspal, p90x, keto), Soccer, Drugs, and another in the base graph. The tables below sum- TV Shows/Movies. The Soccer community is marize the performance of the baseline model and particularly interesting, as it also contains sev- best deep model for each embedding. All metrics eral Spanish-speaking subreddits, such as cien- were computed on the test set. For each model, cia (meaning science in Spanish) and espanol, we report the RMSE and MAE values, which help that were not grouped with soccer subreddits in us understand the differences in the kinds of er- the Louvain partition. This example highlights ror that the models make, but ultimately our pri- a hidden community of subreddits frequented mary evaluation metric is RMSE because edges by Spanish-speakers, who browse Spanish sub- with high weights have more impact on the com- reddits and are also likely to be fans of soc- munity detection results, so we we want to place cer, whose hyperlinks between subreddits are not a higher weight for errors on these edges. Model RMSE MAE window of 5. None of the best models resulted Linear Regression 0.967 0.356 from the RNN architecture, likely due to the em- DNN 1.123 0.5765 beddings not having a clear sequential meaning. For all of the node2vec embeddings, the linear Figure 3. Models w/ RolX Embedding regression models outperformed the deep mod- els on both RMSE and MAE, suggesting that the The best deep model with RolX embeddings linear regression models make fewer large errors was a DNN with 4 dense layers of dimensions of and smaller errors overall. The difference in per- 256, 256, 128, and 32, respectively. Examining formance between linear regression and the deep the results of the models trained with RolX em- models was stark for the 64-dimension and 256- beddings, we see that the linear regression model dimension embeddings and was much smaller for strictly outperforms the dense neural network on the 512-dimension embeddings. As the embed- both RMSE and MAE, which tells us that the lin- dings grew larger, we did see a drop in RMSE, so ear regression model makes fewer large errors due it’s clear that larger node2vec embeddings were to the lower RMSE as well as smaller errors over- beneficial to the DNNs. However, that trend was all given a lower MAE. not the case for the linear regression models, with the 256-dimension embeddings performing the Model Dim RMSE MAE best in terms of RMSE among all of the mod- Linear Regression 64 0.931 0.382 els we trained. This result suggests that the 512- Linear Regression 256 0.916 0.366 dimension embedding may have had too many co- Linear Regression 512 0.946 0.386 efficients for the linear regression to learn given DNN 64 2.393 0.659 the size of our dataset. DNN 256 2.274 0.708 Comparing results between the models trained DNN 512 1.306 0.452 with RolX embeddings and the models trained Figure 4. Models w/ Node2Vec Embedding with node2vec embeddings, it’s inconclusive which embedding method is best. The best overall The best deep model with 64-dimension model was linear regression with 256-dimension Node2Vec embeddings was a DNN with 2 dense node2vec embeddings, and the best deep model of layers of dimensions 256 and 32, respec- was a DNN with RolX embeddings. Furthermore, tively. The optimal 64-dimension embeddings all of the linear regression models with node2vec had a learning rate of 0.05, 250 epochs, 20 walks embeddings outperformed the linear regression per node with a walk length of 20, and a window model with RolX embeddings, but the DNN with of 5. The best deep model with 256-dimension RolX embeddings outperformed all of the deep Node2Vec embeddings was a DNN with 3 dense models trained on node2vec embeddings. of layers of dimensions 64, 64, and 32, respec- From all of these results, we can draw sev- tively. The optimal 256-dimension embeddings eral conclusions. First, it’s clear that predicting had a learning rate of 0.01, 250 epochs, 20 walks edge weights, at least on this particular graph, is per node with a walk length of 40, and a win- a challenging problem given the moderate perfor- dow of 10. The best deep model with 512- mance of all the models. Second, it seems that dimension Node2Vec embeddings was a DNN larger node2vec embeddings are beneficial for the with 2 dense of layers of dimensions 512 and 32, DNNs, although the simpler RolX embeddings respectively. The optimal 64-dimension embed- yielded better results. Finally, we conclude that dings had a learning rate of 0.05, 250 epochs, 100 our best approach to this task was a simple lin- walks per node with a walk length of 10, and a ear regression, specifically with 256-dimension node2vec embeddings. communities interact with one another through hyperlinks between their forums. 5.3. Future Community Detection on Modified Comparing the differences in the communities Graphs detected in the two graphs, we see that the re- After analyzing the results of our edge weight sults vary significantly. For the graph that added prediction models, we selected the best linear re- edges predicted by the linear regression model, gression model (256-dim node2vec embeddings) the new edges reduced the graph to a single con- and the best deep model (DNN with RolX embed- nected component, which led to both algorithms dings) to predict new edges in the graph model. detecting fewer than 40 communities, compared For each model, we took all non-zero predic- to nearly 800 in the original graph. In this tions on the inference dataset and added the pre- set of communities, we still saw a large gam- dicted edges to the graph with their correspond- ing community, but the gaming community also ing predicted weights. Then, we ran Louvain and seemed to combine many technology subreddits HICODE on the resulting graphs to understand such as cloudcomputing, unicode, and internetex- how we might expect the dominant and hidden plorer. It also appears that the Health/Fitness and community structure to change in the future. The Movies/TV Shows communities were merged to- table below summarizes the results of running the gether, and the Religion and Drugs communities community detection algorithms on the modified were combined as well. With this graph, we still graphs. observed strong communities around the topics of Soccer, Bitcoin, and Sports, to name a few. Algorithm Modularity Communities Analyzing the communities detected in the graph that added edges predicted by the dense Louvain 0.4507 37 neural network, we found that the resulting graph HICODE Layer 1 0.4525 36 had 286 connected components, which is still a Figure 5. Community Detection on Graph significant reduction from the 712 in the origi- with Linear Regression Predictions nal graph, which explains a drop in the number of communities detected to approximately 200. The Algorithm Modularity Communities algorithms detected 39 large communities, with the rest of the communities containing fewer than Louvain 0.4392 200 5 nodes. In this partitioning of the graph, we HICODE Layer 1 0.4357 198 see all of the strong communities identified pre- Figure 6. Community Detection on Graph viously, such as Gaming, Health/Fitness, Soccer, with DNN Predictions Bitcoin, and Religion. However, compared to the In this case, the optimal number of HICODE results from the first modified graph, we do not layers was 1, meaning that the algorithm did not see these any of these large communities merged identify any hidden community structure. With together, which is the major difference between one layer of partitions for HICODE, the algorithm the two sets of partitions. just runs the base algorithm in the identification From these results, we predict that a future stage and does not execute the refinement stage. version of the network will have slightly weaker Compared to the community detection results on community structure, will not exhibit hidden the original graph, there is slightly weaker com- community structure, and that we will see a re- munity structure in the modified graphs, mean- duction in the number of distinct communities in ing that we would expect slightly weaker commu- the network. nity structure in the network in the future as more 6. Conclusion and the number of communities will likely drop as the network matures and observes increased in- Through our analysis, we identified moderate teractions between forums. community structure in the Reddit Hyperlink Net- work, given modularity scores around 0.5. We also detected two layers of hidden community 7. Future Work structure with the HICODE algorithm, which we For future work, a more comprehensive analy- determined by similar modularity scores to run- sis of the multiple community partitionings pro- ning Louvain on the original network as well as duced by HICODE could produce more insight manual analysis of the identified communities. on additional hidden structures within the dataset. From there, we tested our method for identify- This could be achieved by a more automated ap- ing future communities by adding predicted edges proach to identify when nodes change communi- to the existing network. We trained a variety of ties for different partititions of the graph. One linear regression, DNN, and RNN models using challenge of the edge weight prediction task was both RolX and Node2Vec embeddings and con- a lack of ground truth for negative examples. ducted an extensive hyperparameter search on the Thus, obtaining true negative examples for pairs DNN and RNN models. Ultimately, the base- of nodes that will not be in the connected in the line linear regression models performed the best, graph or a deeper investigation of heuristics to use with 256-dimension Node2Vec embeddings being for generating negative examples may be bene- the mest embedding method for linear regression. ficial. Another area that could benefit from fur- We did notice an improvement in DNN perfor- ther investigation is methods to represent edges mance for increasingly large Node2Vec embed- for prediction. For this project, we used the con- dings, but these models were outperformed by a catenation of node embeddings, but there may be DNN with RolX embeddings. The best linear re- better ways to represent the edges that could lead gression models used Node2Vec embeddings, but to improved model performance. the best DNN model utilized RolX embeddings, so it doesn’t seem that a particular embedding 8. Contributions type is preferred in all cases. When we added the edges predicted by the best Daniel Salz was primarily responsible for train- models to the network and re-ran our commu- ing the node2vec embeddings and deep learning nity detection algorithms, we observed slightly models for edge weight prediction, initial graph weaker community structure and fewer identi- generation, and detecting communities with the fied communities. These results were caused by Louvain algorithm. Nicholas Benavides was pri- the new edges increasing the size of the graph’s marily responsible for creating the prediction and giant component, and the graph became fully inference datasets, generating RolX embeddings, connected after adding the edges predicted by implementing an edge weight prediction baseline, the linear regression model with 256-dimension edge weight prediction model evaluation, and an- Node2Vec embeddings. We also did not ob- alyzing the detected communities. Johnny Li serve hidden community structure on the mod- was primarily responsible for implementing the ified graphs as detected by the HICODE al- HICODE algorithm and detecting communities gorithm. Generally, the detected communities with the HICODE algorithm. All three group on these graphs led to some communities being members were equally responsible for the liter- merged together, suggesting that distinctions be- ature review. tween communities may become less clear-cut References chanics: Theory + Experiment. Volume October 2008. https://arxiv.org/ [1] Kun He, Yingru Li, Sucheta Soundarajan, pdf/0803.0476 John E. Hopcroft. 2017. Hidden Community Detection in Social Networks. Information [5] Aric A. Hagberg, Daniel A. Schult and Sciences Volume 245. https://arxiv. Pieter J. Swart. 2008. Exploring network org/abs/1702.07462 structure, dynamics, and function using Net- workX. Proceedings of the 7th Python in [2] Srijan Kumar, William L Hamilton, Jure Science Conference. Leskovec, Dan Jurafsky. 2018. Community Interaction and Conflict on the Web. Pro- [6] Keith Henderson, Brian Gallagher, Tina ceedings of the 2018 World Wide Web Con- Eliassi-Rad, Hanghang Tong, Sugato Basu, ference on World Wide Web. https:// Leman Akoglu, Danai Koutra, Christos arxiv.org/pdf/1803.03697 Faloutsos, and Lei Li. 2012. RolX: Struc- tural role extraction mining in large graphs. [3] Cunchao Tu, Xiangkai Zeng, Hao Wang, Proceedings of the ACM SIGKDD Interna- Zhengyan Zhang, Zhiyuan Liu, Maosong tional Conference on Knowledge Discovery Sun, Bo Zhang, Leyu Lin. 2019. A Unified and Data Mining. https://web.eecs. Framework for Community Detection and umich.edu/˜dkoutra/papers/12- Network Representation Learning. IEEE kdd-recursiverole.pdf Transactions on Knowledge and Data En- gineering. Volume 31. https://arxiv. [7] Aditya Grover and Jure Leskovec. org/abs/1611.06645 2016. node2vec: Scalable Feature Learning for Networks. Proceedings [4] Vincent D. Blondel, Jean-Loup Guillaume, of the 22nd ACM SIGKDD Inter- Renaud Lambiotte, and Etienne Lefebvre. national Conference on Knowledge 2008. Fast unfolding of communities in Discovery and Data Mining. https: large networks. Journal of Statistical Me- //arxiv.org/abs/1607.00653