Webometric Network Analysis Webometric Network Analysis Mapping Cooperation and Geopolitical Connections between Local Government Administration on the Web Kim Holmberg ÅBO 2009 ÅBO AKADEMIS FÖRLAG – ÅBO AKADEMI UNIVERSITY PRESS CIP Cataloguing in Publication Holmberg , Kim Webometric network analysis : mapping cooperation and geopolitical connections between local government administration on the web / Kim Holmberg. – Åbo : Åbo Akademi University Press, 2009. Diss.: Åbo Akademi University. ISBN 978-951-765-510-1 ISBN 978-951-765-510-1 ISBN 978-951-765-511-8 (digital) Painosalama Oy Åbo 2009 Acknowledgements This research started as a mixed collection of very diffuse ideas that gradually gained focus and evolved into the research that now has been completed and written down. Without the advice and encouragement of certain persons this work would not have been possible, or at least it would have looked very different. To these persons I direct my deep gratitude. First of all I want to thank my supervisors, professors Gunilla Widén-Wulff and Mike Thelwall, for their advice and support throughout this process. I have been very fortunate to have two such knowledgeable and generous supervisors. Gunilla has given me the freedom to pursue my own path and make my own discoveries, but she has always been there to help me find my way back to the path when I have strayed too far from it. It has been a great privilege to have Mike as my second supervisor. Mike’s vast knowledge and experience of webometrics, great technical skills, and unparalleled helpfulness have been extremely valuable throughout this research. I also want to thank professors Mariam Ginman and Sara von Ungern-Sternberg for convincing me to pursue a doctoral degree, a decision that I have never regretted. I have been very fortunate to be able to spend some time with some extraordinary persons and pick their brains. I want to express my deepest gratitude to the Nordic School of Library and Information Science, NORSLIS, for granting me funding to travel and visit some people that have had a great impact on my research and that today I am really happy to be able to call my friends. At an early stage of this research I visited professor Olle Persson in Umeå, Sweden. I am deeply grateful to Olle for sharing his time and knowledge with me. I have also travelled to Copenhagen, Denmark, to discuss my research with Doctor Lennart Björneborn. Lennart’s innovative ideas, support and encouragement have been more valuable to me than I can express in words. During my visits to University of Wolverhampton, UK, to meet with Mike, I have met members of his research group and special thanks goes to one of the members; my colleague and good friend Doctor David Stuart. At the very end of my research David did me a huge favour by reading my manuscript and trying to find holes in my thinking and by pointing out the biggest errors in my grammar and spelling. I am very happy that professors Liwen Vaughan and Peter Ingwersen agreed to be my pre-reviewers. I am in more debt to them than they may know. I attended the ISSI 2005 conference in Stockholm, Sweden, as a “tourist” and it was professor Vaughan’s presentation about co-inlinking that got me interested in webometrics. At the very beginning of my PhD studies, I shared i an office with professor Ingwersen for a week. The talks we had were very valuable and encouraging for a “new” researcher who was not quite sure what he wanted to do. Now they have both played an important part at the very end of my research, giving me such valuable and constructive feedback on my manuscript, and in a way, closing the circle. I want to thank all my colleagues at the Department of Information Studies at Åbo Akademi University for all the inspiring discussions we have had, both on- and off-topic; it has been a great privilege and an enjoyable time to work with you. I also want to thank all the senior researchers, colleagues and new friends that I have met at the conferences, doctoral forums and courses that I have had the great fortune to attend during this research; thank you all for commenting on my research and for asking all the tough questions. This work would not have been possible without the financial support of some generous organizations. My most humble gratitude goes to Åbo Akademi University Foundation, Academy of Finland, Alfred Kordelin Foundation, Waldemar von Frenckell’s Foundation, and TOP Foundation for the financial support I have received during this research. I am in great debt to my parents who have always supported me and encouraged me to study for as long as I want, and now I can say that I studied all the way. I want to thank my sister Nea and all my friends for providing such pleasant and well needed breaks from reading and writing every now and then. And last, but definitively not least, I want to thank my beloved wife Eva, for her love, companionship, and understanding for those countless hours that I have spent in front of the computer screen or submerged in some books or articles. Thank you for keeping my feet on the ground and for reminding me that there is more to life than webometrics. ii Contents 1. INTRODUCTION ................................................................................................. 1 1.1. LINK ANALYSIS .................................................................................................... 3 1.2. REGION OF FINLAND PROPER ................................................................................ 5 1.3. MOTIVATION AND OBJECTIVE ................................................................................ 7 1.3.1. Research questions ............................................................................... 8 1.4. STRUCTURE OF THE DISSERTATION ........................................................................ 11 2. NETWORK ANALYSIS ....................................................................................... 13 2.1. REAL WORLD NETWORKS .................................................................................... 13 2.1.1. Terminology ........................................................................................ 15 2.2. THE NATURE OF THE WEB ................................................................................... 20 2.2.1. The size of the Web ............................................................................. 21 2.2.2. The dynamic nature of the Web .......................................................... 23 2.2.3. Small worlds on the Web .................................................................... 24 2.2.4. Power laws on the Web ...................................................................... 27 2.2.5. Preferential attachment ...................................................................... 28 2.3. METHODS IN NETWORK ANALYSIS ....................................................................... 30 2.3.1. Centrality measures ............................................................................ 31 2.3.1.1. Freeman Degree Centrality ........................................................................ 32 2.3.1.2. Closeness Centrality ................................................................................... 33 2.3.1.3. Betweenness Centrality ............................................................................. 34 2.3.2. Reciprocity ........................................................................................... 35 2.3.3. Clustering ............................................................................................ 36 2.3.4. Cliques, n-cliques, k-plexes and k-cores .............................................. 39 2.3.5. Matrix similarity measures ................................................................. 40 2.3.5.1. Simple match ............................................................................................. 41 2.3.5.2. Jaccard coefficient ..................................................................................... 42 2.3.5.3. Matching of existing links .......................................................................... 43 2.3.5.4. Precision/Recall ......................................................................................... 43 2.3.5.5. Quadratic Assignment Protocol ................................................................. 44 2.3.6. From valued to binary ......................................................................... 45 2.3.7. Collapsing networks ............................................................................ 47 2.3.8. Visualizing networks ........................................................................... 48 2.4. CONCLUSIONS .................................................................................................. 51 3. WEBOMETRICS ................................................................................................ 52 3.1. DEVELOPMENT OF WEBOMETRICS ........................................................................ 53 iii 3.2. LINK DATA COLLECTION ...................................................................................... 56 3.2.1. Web crawlers ...................................................................................... 57 3.2.1.1. Crawling ethics ........................................................................................... 58 3.2.2. Search engines .................................................................................... 59 3.2.2.1. Overlapping coverage of Web search engines ..........................................