Quantitative Measurement of Parliamentary Accountability Using Text As Data: the Canadian House of Commons, 1945-2015 by Tanya W
Total Page：16
File Type：pdf, Size：1020Kb
Quantitative measurement of parliamentary accountability using text as data: the Canadian House of Commons, 1945-2015 by Tanya Whyte A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Graduate Department of Political Science University of Toronto c Copyright 2019 by Tanya Whyte Abstract Quantitative measurement of parliamentary accountability using text as data: the Canadian House of Commons, 1945-2015 Tanya Whyte Doctor of Philosophy Graduate Department of Political Science University of Toronto 2019 How accountable is Canada’s Westminster-style parliamentary system? Are minority parliaments more accountable than majorities, as contemporary critics assert? This dissertation develops a quanti- tative measurement approach to investigate parliamentary accountability using the text of speeches in Hansard, the historical record of proceedings in the Canadian House of Commons, from 1945-2015. The analysis makes a theoretical and methodological contribution to the comparative literature on legislative debate, as well as an empirical contribution to the Canadian literature on Parliament. I propose a trade-off model in which parties balance communication about goals of office-seeking (accountability) or policy-seeking (ideology) in their speeches. Assuming a constant context of speech, I argue that lexical similarity between government and opposition speeches is a valid measure of parlia- mentary accountability, while semantic similarity is an appropriate measure of ideological polarization. I develop a computational approach for measuring lexical and semantic similarity using word vectors and the doc2vec algorithm for word embeddings. To validate my measurement approach, I perform a qualitative case study of the 38th and 39th Parliaments, two successive minority governments with alternating governing parties. I find that similarity scores are positively related with the substantive quality of opposition questions and government responses. In the quantitative analysis phase, I study Question Periods from 1975-2010 and daily debates from 1945-2015 using the lexical similarity measurement. I find that minority parliaments are more account- able than majority governments since the 30th Parliament, but find no significant relationship between government seat percentage and parliamentary accountability. I show that Parliament becomes more accountable as a government’s popularity decreases. However, the data more strongly support a non- linear model. A structural break analysis yields one significant break at 33%: below this critical value, polling popularity and parliamentary accountability are positively related, and above, are negatively re- lated. Finally, I confirm that the correlation between measures of lexical and semantic similarity varies in strength and direction across parliamentary sessions, suggesting two distinct generative processes are indeed at work. ii Acknowledgements Thanks to my supervisors, Grace Skogstad and Chris Cochrane, and the third member of my committee, Arthur Spirling; the internal and external members of my exam committee, Ludovic Rheault and Justin Grimmer; academic colleagues including but not limited to Kaspar Beelen, Steven Bernstein, Hanil Chang, Adrienne Davidson, Rod Haddow, Maxime Héeroux-Legault, Graeme Hirst, Vincent Hopkins, Peter Loewen, John McAndrews, Denver McNeney, Nona Naderi, Federico Nanni, Ed Schatz, Lior Sheffer, Graham White, and Linda White; department graduate administrators Carolynn Branton and Louis Tentsos; mentors and teachers during prior studies at the University of Alberta and Old Scona Academic High School; the Government of Canada, including SSHRC for its financial support, and the staffs of the Library of Parliament and Canadiana at Library and Archives Canada; and my colleagues at Receptiviti, especially my boss, Sharan Karanth. Finally, my thanks and love to my parents, Annie and Tony Whyte, my sister, Andrea Whyte, and my husband, Kevin Chan. iii Contents 1 Introduction 1 1.1 Canadian Literature on Parliament . .4 1.2 The Parliamentary Decline Thesis . 10 2 Literature Review: Responsible Government and Parliamentary Debate 14 2.1 Responsible Government and Accountability . 14 2.1.1 Individual Ministerial Responsibility . 16 2.1.2 Collective Ministerial Responsibility . 16 2.1.3 Role of the Opposition . 18 2.2 Comparative Theories of Parliamentary Debate . 21 2.2.1 Proksch and Slapin (2014) . 24 2.2.2 Bäck and Debus (2016) . 27 3 Research Design 30 3.1 Textual Debate Models: Peterson and Spirling (2018) . 30 3.2 Accountability, Ideology, and the Lexical Gap . 32 3.2.1 Theoretical Framework . 34 3.2.2 Empirical Model and Measurement . 36 3.3 Validation and Prediction . 38 3.3.1 Part 1: Qualitative Assessment of Opposition-Minister Exchanges in the 38th and 39th Parliaments . 38 3.3.2 Part 2: Quantitative Study of Daily Debates (1945-2015) and Oral Question Period (1975-2010) . 39 4 Methodology 45 4.1 Content Analysis . 45 4.2 Dictionary Methods . 47 4.3 Lexicographic and Scaling Methods . 49 4.4 Supervised and Unsupervised Machine Learning . 50 4.5 Latent Semantic Analysis . 52 4.5.1 Probabilistic Linguistics . 53 4.5.2 Limitations of LSA . 54 4.6 Measuring Lexical Similarity: Cosine Similarity and the Term-Document Matrix . 55 iv 4.7 Measuring Semantic Similarity: Probabilistic Vector Representations of Words and Para- graphs . 57 4.8 Study Dataset . 61 5 Qualitative Validation: Opposition-Minister Exchanges in the 38th and 39th Parlia- ments 62 5.1 38th Parliament: Liberal Minority . 63 5.1.1 Historical Overview . 63 5.1.2 Topics of Debate . 68 5.1.3 Natural Resources . 69 5.1.4 Health . 72 5.1.5 Citizenship and Immigration . 76 5.1.6 David Dingwall . 81 5.1.7 Sponsorship Program . 82 5.2 39th Parliament: Conservative Minority . 85 5.2.1 Historical Overview . 85 5.2.2 Topics of Debate . 89 5.2.3 Child Care . 93 5.2.4 The Environment . 96 5.2.5 Ethics . 99 5.2.6 Airbus . 100 5.3 Summary: Comparison of 38th and 39th Parliaments . 103 6 Quantitative Analysis: Question Period, 1975-2010 and Daily Debate, 1945-2015 105 7 Conclusion 127 7.1 Ideas for Further Research . 130 Appendices 134 A Qualitative Results: Additional Analysis 135 A.1 38th Parliament quantitative tests . 135 A.2 39th Parliament quantitative tests . 137 A.3 Model: lexical similarity vs. word count differential . 140 B Quantitative Results: Additional Analysis 141 B.1 Polling data breakpoints analysis . 141 B.2 Replication of polling analysis with all opposition speeches . 145 B.3 Comparison of alternative fits (loess, lm, spline, polynomial) for polling data model . 145 B.4 Parliament-level daily debate model using 1975- data subset . 149 B.5 Daily debate model including random effects using 1975- data subset . 150 B.6 Model of lexical and semantic similarity, Question Period data . 150 B.7 Interaction of government poll popularity and previous popularity . 152 B.8 Effect size calculations . 152 B.9 Simulated distribution of similarity scores . 153 v C Technical Information 156 C.1 Preprocessing . 156 C.2 Computation and Analysis . 156 D Model of Parliamentary Speech Content 158 E Lipad Digitization of Canadian Hansard Debates 162 vi List of Tables 5.1 Topics of Opposition-Minister Exchanges in Question Period, 38th Parliament . 69 5.2 Natural Resources (38th Parliament) High Similarity Speech Pair Examples . 70 5.3 Natural Resources (38th Parliament) Low Similarity Speech Pair Examples . 71 5.4 Health (38th Parliament) High Similarity Speech Pair Examples . 73 5.5 Health (38th Parliament) Low Similarity Speech Pair Examples . 74 5.6 Citizenship and Immigration (38th Parliament) High Similarity Speech Pair Examples . 77 5.7 Citizenship and Immigration (38th Parliament) Low Similarity Speech Pair Examples . 78 5.8 David Dingwall (38th Parliament) High Similarity Speech Pair Examples . 79 5.9 David Dingwall (38th Parliament) Low Similarity Speech Pair Examples . 80 5.10 Sponsorship Program (38th Parliament) High Similarity Speech Pair Examples . 83 5.11 Sponsorship Program (38th Parliament) Low Similarity Speech Pair Examples . 84 5.12 Topics of Opposition-Minister Exchanges in Question Period, 39th Parliament . 90 5.13 Child Care (39th Parliament) High Similarity Speech Pair Examples . 91 5.14 Child Care (39th Parliament) Low Similarity Speech Pair Examples . 92 5.15 The Environment (39th Parliament) High Similarity Speech Pair Examples . 94 5.16 The Environment (39th Parliament) Low Similarity Speech Pair Examples . 95 5.17 Ethics (39th Parliament) High Similarity Speech Pair Examples . 97 5.18 Ethics (39th Parliament) Low Similarity Speech Pair Examples . 98 5.19 Airbus (39th Parliament) High Similarity Speech Pair Examples . 101 5.20 Airbus (39th Parliament) Low Similarity Speech Pair Examples . 102 6.1 Models 1, 2, and 3 investigate the effect of majority status, governing party, government poll popularity, and seat percentage controlled by the government on lexical similarity scores. All models use the Question Period dataset. Mode1 1 studies individual Question Period observations and includes random effects for quarter, session, and parliament, while Models 2 and 3 are calculated on mean quarterly similarity scores and include random effects for session and parliament. 108 6.2 Models 4 and 5 investigate the effect of majority status, governing party, and seat percent- age controlled by the government on lexical similarity scores. Both models are calculated on individual daily observations from the daily debate dataset and include random effects for quarter, session, and parliament.