ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Lesson 10 Visualising and Analysing Network Data
Mentor: Dr. Kam Tin Seong Associate Professor of Information Systems (Practice) School of Information Systems, Singapore Management University
Content • Introduction to Graph Visual Analytics • Graph Visualisation in Actions • Basic Principles of Graph – Network data sets – Graph data format • Network Visualisation and Analysis – Network visualisation and analysis process model – Graph layouts and visual attributes – Network metrics • Network visualisation and analysis tools – NodeXL – Gephi
2
10-1 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Network in real World • Physical – Transportation (i.e. road, port, rail, etc) – Utility (electricity, water, gas, network cable, etc) – Natural (river, etc) • Abstract – Social media (i.e. e-mail, Facebook, Twitter, Wikipedia, etc) – Organisation (i.e. NGO, politics, customer-company, staff-to-staff, criminal, terrorist, disease, etc)
3
Classical Graph Theory • The Seven Bridges of Königsberg is a historically notable problem in mathematics. Its negative resolution by Leonhard Euler in 1735 laid the foundations of graph theory and prefigured the idea of topology.
Source: http://en.wikipedia.org/wiki/Seven_Bridges_of_K%C3%B6nigsberg
4
10-2 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Classical Graph Visualisation and Analysis • Using sociogram, also know as network graph
Early 20th-century social graph used in studying workflows among employees
Source: Valdis Krebs (2010) “Your Choice Reveal Who You Are: Mining and Visualizing Social Patterns” in Beautiful Visualization.
5
Graph Visualisation in action 1 • Graph visualisation is used to understand online social network - Sociogram
6
10-3 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Visualisation in action 2 • Graph visualisation is used to show how the news are all connected by degrees of separation
Link: http://whichlight.github.io/reddit-network- vis/?discussion=http://reddit.com/r/gaming/comments/3d2ewz/ninten do_president_satoru_iwata_passes_away/
7
Graph Visualisation in action 3 • Using graph visualisation to understand business networks
8
10-4 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Visualisation in action 4 • Graph visualisation is used to reveal voting patterns among United States senators
9
Graph Visualisation in action 5 • The social graph above shows the email flows amongst a large project team.
10
10-5 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Visualisation in action 6 • (a) Similarity graph of log entries and (b) Similarity graph of network scans
(b) (a)
Source: Graph Drawing for Security Visualization
11
Graph Visualisation in action 7 • Networks of the below universities are expanded in a breadth first manner up to the depth of 2, (showing university, alumni and companies they are associated with through employment, investment or other activities) • Size of the node reflects degree of the node (scaled logarithmically).
Source: http://www.innovation-ecosystems.org/wp-content/uploads/2010/12/2011.educon.pdf
12
10-6 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Visualisation in actin 8 • SNA in Construction Industry
Source: Pryke, S.D.”Analysing construction project coalitions: exploring the application of social network analysis”, Construction Management and Economics, (2004), 22. pp. 787-797.
Graph Visualisation in actin 9 • Community structure in protein-protein interaction networks.
Source: http://arxiv.org/pdf/0906.0612.pdf
14
10-7 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Visualisation in action 9 • Spatial Network Analysis for Multi-modal Urban Transport Systems
Source: http://www.snamuts.com/melbourne.html 15
Graph Visualisation in action 10 • Global Maritime Transport
http://onlinelibrary.wiley.com/doi/10.1111/j.1471-0374.2011.00355.x/pdf
16
10-8 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
To learn more, go to Visual Complexity
Source: http://www.visualcomplexity.com/vc/index.cfm?domain=Transportation%20Networks
17
Source of network data I • Unstructured data (for example, Tweets)
18
10-9 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Source of network data II • Node and link dataset (for example, phone call)
19
Source of network data III • Transaction records (for example, EZLink card)
20
10-10 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Source of network data
• Company directory such as OneSource and opencorporates (https://opencorporates.com/)
21
An undirected link table • Focus on interaction but not the direction. – For example C1 -> C6 and C6 -> C1 are considered as one interaction. • The table is symmetric.
22
10-11 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
A directed link table • Focus on both interaction and direction. – For example C1 -> C6 and C6 -> C1 are considered as two different interactions. • The table is asymmetric.
23
Graph Data File Formats • GraphML, an XML-based file format for graphs. • GXL, graph exchange format based on XML. • Trivial Graph Format, simple text based format. • GML is another widely used graph exchange format. • DGML, Directed Graph Markup Language from Microsoft. • XGMML, an XML-based graph markup language closely related to GML. • Dot Language, a format for describing graphs and their presentation, for the Graphviz set of tools.
24
10-12 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph database -
Reference: http://neo4j.com/developer/get-started/
Graph representation • On the left is a normal graph, in the centre is a graph in which each edge is given a numerical value, and to the right is a directed graph.
26
10-13 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Complete Graph • A complete graph is a simple undirected graph in which every pair of distinct vertices is connected by a unique edge.
27
A sample graph • A complete graph is a simple undirected graph in which every pair of distinct vertices (also known as nodes) are connected by an unique edge.
edge vertex
28
10-14 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Basic Visual Attributes
29
A directed graph • Have a clear origin and destination. • Also known as asymmetric edges. • Suitable for representing network with non reciprocal relationships such as Twitter.
30
10-15 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
A weighted graph • A weighted edge includes values associated with each edge that indicate the strength or frequency of tie. – For example, numbers of calls between two staffs.
31
A weighted graph
• Edges with different thickness are used to represent the monthly calls by staffs.
32
10-16 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
An ego-centric graph • Network consisting of an individual and their immediate peers (Heer & Boyd, 2005)
33
A Multimodel Graph
• Social network connecting different types of vertices. – For example, a network may connect peers to discussion forums and blog posts they have commented on.
SeriousEats visualization emphasizing the most important people (black circles), forums (orange squares), and blogs (blue diamonds).
34
10-17 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Bipartite Graph • A graph whose vertices can be divided into two disjoint sets U and V such that every edge connects a vertex in U to one in V; that is, U and V are independent sets. • Equivalently, a bipartite graph is a graph that does not contain any odd- length cycles.
35
Affiliation Networks - Bipartite Graph
• Nodes are partitions into two subsets and all Two-mode view of the Southern lines are between pairs of nodes belonging to Women social event dataset different subsets • As there are g actors and h events, there are g + h nodes • The lines on the graph represent the relation “is affiliated with” from the perspective of the actor and the relation “has as a member” from the perspective of the event. • No two actors are adjacent and no two events are adjacent. If pairs of actors are reachable, it is only via paths containing one or more events. Similarly, if pairs of events are reachable, it is only via paths containing one or more actors. Valdis Krebs (2010) “Your Choice Reveal Who You Are: Mining and Visualizing Social Patterns” in Beautiful Visualization.
36
10-18 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Measures of Power and Influence - Network Metrics • A collection of statistical measures to report: – the connectivity of a node within a network, – the complexity of a network, – the clusters or sub- groups within a network.
37
Network Metrics: Degree • Degree, the number of direct connections a node has. • Degree is often interpreted in terms of the immediate risk of node for catching whatever is flowing through the network (such as a virus, or some information). • If the network is directed (meaning that ties have direction), then we usually define two separate measures of degree centrality, namely indegree and outdegree. • Indegree is a count of the number of ties directed to the node, and outdegree is the number of ties that the node directs to others. • For positive relations such as friendship or advice, we normally interpret indegree as a form of popularity, and outdegree as gregariousness.
38
10-19 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Network Metrics: Degree
39
Network Metrics: Betweenness centrality • Betweenness is a centrality measure of a vertex within a graph (there is also edge betweenness, which is not discussed here). • Vertices that occur on many shortest paths between other vertices have higher betweenness than those that do not.
40
10-20 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Network Metrics: Betweenness centrality
41
Network Metrics: Closeness Centrality • In graph theory closeness is a centrality measure of a vertex within a graph. Vertices that are 'shallow' to other vertices (that is, those that tend to have short geodesic distances to other vertices with in the graph) have higher closeness. • Closeness is preferred in network analysis to mean shortest-path length, as it gives higher values to more central vertices, and so is usually positively associated with other measures such as degree.
42
10-21 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Network Metrics: Closeness Centrality
43
Network Metrics: Eigenvector Centrality • A measure of the importance of a node in a network. • It assigns relative scores to all nodes in the network based on the principle that connections to high-scoring nodes contribute more to the score of the node in question than equal connections to low-scoring nodes.
44
10-22 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Network Metrics: Eigenvector Centrality
45
Network Metrics: Clustering Coefficient • A measure on how connected a vertex’s neighbours are to one another. More specifically, it is the number of edges connecting a vertex’s neighbours divided by the total number of possible edges between the vertex’s neighbour.
46
10-23 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Network Metrics: Clustering Coefficient
47
Summary of Network Metrics
48
10-24 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Layouts
49
Force-Directed Layout • http://bl.ocks.org/mbostock/4062045
Source: https://en.wikipedia.org/wiki/Force-directed_graph_drawing
50
10-25 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Node-Only Layout
Link: http://www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html
51
Time-Oriented Layout
Linked: http://neuralengr.com/asifr/journals/
52
10-26 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Radial Hierarchical Layout
Link: http://mbostock.github.io/d3/talk/20111018/cluster.html
53
Tree Hierarchical Layout
Link: http://mbostock.github.io/d3/talk/20111018/tree.html
54
10-27 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Geographic Layout • Air transportation network for Brazil.
Source: http://www.researchgate.net/publication/260109852_Using_Network_Sciences_to_Evaluate_the_Brazilian_Airline_Network
55
Additional Visual Attributes
56
10-28 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Motif Simplification – Basic concept
57
Motif Simplification Methods
58
10-29 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
NodeXL Implementation of Motif Simplification
Source: http://hcil2.cs.umd.edu/trs/2012-29/2012-29.pdf
59
Chord Diagrams
Link: http://projects.delimited.io/experiments/chord-transitions/demos/trade.html
60
10-30 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Sankey Diagrams
Link: http://bost.ocks.org/mike/sankey/
61
BiPartite Layout
Link: http://bl.ocks.org/NPashaP/9796212
62
10-31 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Adjacency Matrix
63
Hive Plot
Link: http://www.hiveplot.net/
64
10-32 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Hive Plot using d3.js
Link: http://www.konstantinkashin.com/blog/2013/12/22/network-visualization-with-d3js/
65
GeoVisual Analytics Application IV
Source: http://xliberation.com/googlecharts/d3flights.html
66
10-33 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
GeoVisual Analytics Application V
Reference: http://www.kiln.it/projects/opencorporates and https://opencorporates.com/viz/financial/index.html#bp/sg
67
Network Visualisation and Analysis Process Model (Hansen, D. L. et. al. 2009)
68
10-34 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Swiss Knife for Graph Visualisation and Analysis I
• NodeXL, an open-source template for Microsoft® Excel® 2007 and 2010 that makes it easy to explore network graphs.
Homepage: http://nodexl.codeplex.com/
69
Swiss Knife for Graph Visualisation and Analysis II
Reference: https://gephi.org/
70
10-35 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Swiss Knife for Graph Visualisation and Analysis III • Cytoscape
Homepage: http://www.cytoscape.org/
71
Web enabled Graph Visualisation Libraries • Graph – Sigma.js – Cytoscape.js (http://js.cytoscape.org/)
• Generic – Vis.js (http://visjs.org/index.html) – JavaScript InfoVis Toolkit (http://philogb.github.io/jit/demos.html) – D3.js (http://d3js.org/)
Link: http://stackoverflow.com/questions/7034/graph-visualization-library-in-javascript
72
10-36 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Sigma.js • (http://sigmajs.org/)
73
Interactive Network Visualisation in D3.js I • d3.js force directed graph examples
Source: http://www.d3noob.org/2013/03/d3js-force-directed-graph-examples.html
74
10-37 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Interactive Network Visualisation in D3.js II
Source: http://flowingdata.com/2012/08/02/how-to-make-an-interactive-network-visualization/ 75
Interactive Network Visualisation in D3.js III • This application is created using d3.js
For more information: http://groups.google.com/group/d3-js/browse_thread/thread/d1b6ec5c59aa576f
76
10-38 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
d3.js force directed graph examples IV
Source: http://dhs.stanford.edu/dh/networks/
77
d3.js force directed graph examples V
Source: http://christophergandrud.github.io/networkD3/
78
10-39 ISSS608 Visual Analytics and Applications 11/18/2015 Lesson 10: Visualising and Analysing Network Data
Graph Visualisation in action 8 • Visualising physical network such as transport
79
Reference • Brath, R. and Jonker, D. (2015) Graph Analysis and Visualization. John Wiley & Sons, Indianapolis, IN. • Hansen, D., Shneiderman, B. and Smith, M.A. (2011) Analyzing Social Media Networks with NodeXL. Elservier Inc. USA. • Cherven, K. (2015) Mastering Gephi Network Visualization. PACKT Publishing, Birmingham, UK.
80
10-40