Jaakko Rinta-Filppula IS STATIC TYPE CHECKING WORTH IT? On the pros and cons of adding a static type checker to an existing codebase Master of Science Thesis Faculty of Information Technology and Communication Sciences Examiners: Prof. Kari Systä D.Sc. Matti Rintala April 2021 i ABSTRACT Jaakko Rinta-Filppula: Is static type checking worth it? Master of Science Thesis Tampere University Master’s Programme in Information Technology April 2021 Even though type systems are a well-researched topic in theoretical computer science, there are relatively few studies on the differences between static and dynamic type checking for software development. There are controlled studies that focus on specific aspects of the development such as speed and code maintainability but due to their nature, these studies consider rather small codebases. This thesis augments the results of the previous research by gathering real-world data on the experiences of using static type checkers designed for dynamically typed languages. The goal of the thesis is to find out if any benefits are observed by using these static type checking tools and whether those benefits justify any possible drawbacks. To answer these ques- tions, an online survey was conducted to gather the experiences of a total of 138 other developers. In addition to the online survey, the author performed an experiment where he adopted Sorbet, a static type checker for Ruby, in a large codebase (approximately 80,000 lines of code) that multiple developers are working on simultaneously. In the survey, almost everyone who had used these tools told that the tools are at least some- what beneficial. A majority of them said that using a static type checker is very beneficial. Usinga static type checker was found to improve all of the different areas of development that were asked about in the survey: code reliability, development speed, API usability, maintainability, working with unfamiliar parts of the code, and confidence in refactoring and writing new code. Overall the responses were very positive with only a small amount of negative answers. In the exploratory experiment performed by the author the results were also positive. Even though there were problems that prevented the author from using the full potential of the type checker, undiscovered problems in the codebase were surfaced. The process of setting up and working with the tools was pleasant. All in all, it seems that using these kinds of static type checkers in dynamically typed languages can provide significant benefits and improve the software development process. Keywords: static typing, dynamic typing, gradual typing, type systems, type checkers, software development The originality of this thesis has been checked using the Turnitin OriginalityCheck service. ii TIIVISTELMÄ Jaakko Rinta-Filppula: Kannattaako staattinen tyyppitarkistus? Diplomityö Tampereen yliopisto Tietotekniikan DI-ohjelma Huhtikuu 2021 Vaikka tyyppijärjestelmiä ja niiden teoriaa on tutkittu paljon tietojenkäsittelytieteessä, on staat- tisen ja dynaamisen tyyppitarkistuksen vaikutuksista ohjelmistokehitykseen verrattaen vähän tut- kimuksia. Kehityksen eri osa-alueisiin kohdistuvia vertailukokeita on tehty, mutta tämän tutkimus- tavan luonteen vuoksi ne koskettavat melko pieniä koodikantoja. Tämä diplomityö laajentaa aikai- sempien tutkimusten tuloksia keräämällä kehittäjien kokemuksia dynaamisesti tyypitetyille ohjel- mointikielille suunniteltujen staattisten tyyppitarkistimien käytöstä. Työn tavoitteena on selvittää, onko näiden tyyppitarkistimien käytöstä havaittavaa hyötyä, se- kä arvioida hyötyjen suuruutta mahdollisiin haittoihin verrattuna. Näihin kysymyksiin pyritään vas- taamaan kyselylomakkeella, jolla selvitettiin ohjelmistokehittäjien kokemuksia tyyppitarkistustyö- kaluista. Kyselyyn vastasi 138 osallistujaa. Sähköisen kyselyn lisäksi kirjoittaja suoritti kokeelli- sen tutkimuksen, jossa hän otti käyttöön Sorbet-nimisen työkalun isossa, noin 80 000 rivin Ruby- projektissa, jonka parissa työskentelee useita kehittäjiä samanaikasesti. Lähes kaikki kyselyyn vastanneista kertoivat tyyppitarkistustyökalujen olevan ainakin jokseen- kin hyödyllisiä. Suurin osa heistä luokitteli työkalut erittäin hyödyllisiksi. Staattisen tyyppitarkisti- men koettiin parantavan kaikkia kyselyssä käsiteltyjä kehityksen osa-alueita: koodin luotettavuus, kehitysnopeus, rajapintojen käytettävyys, ylläpidettävyys, entuudestaan tuntemattomien osien pa- rissa työskentely sekä varmuus koodia refaktoroidessa ja uutta koodia kirjoitettaessa. Kokonai- suudessa kyselyn vastaukset olivat erittäin positiivisia ja huonoja kokemuksia oli hyvin vähän. Kirjoittajan oman tutkimuksen tulokset olivat myös positiivisia. Huolimatta ilmenneistä ongel- mista, jotka estivät tyyppitarkistuksen täysimääräisen käytön, koodikannasta ilmeni aikaisemmin löytymättömiä ongelmia. Lisäksi työkalun käyttöönotto ja sen käyttäminen oli miellyttävää. Kaiken kaikkiaan tällaisten dynaamisesti tyypitetyille ohjelmointikielille kehitettyjen staattisten tyyppitar- kistimien käyttö voi tuoda merkkitäviä hyötyjä ja parantaa ohjelmistokehitysprosessia. Avainsanat: staattinen tyypitys, dynaaminen tyypitys, asteittainen tyypitys, tyyppijärjestelmät, tyyp- pitarkistimet, ohjelmistokehitys Tämän julkaisun alkuperäisyys on tarkastettu Turnitin OriginalityCheck -ohjelmalla. iii PREFACE Student syndrome refers to planned procrastination, when, for example, a student will only start to apply themselves to an assignment at the last possi- ble moment before its deadline. (Wikipedia, Student syndrome) This thesis is a testament to the above definition of the student syndrome. Some, includ- ing myself, may have doubted whether it will ever be finished at all. Nevertheless, with the right amount of deadline-induced stress, anything seems to be possible. I would like to thank Flockler and especially my boss, Toni Hopponen, for providing me with the ability to work on this thesis. Thank you for allowing me to allocate company time for the thesis even though it is only loosely related to work. I want to thank the examiners Prof. Kari Systä and D.Sc. Matti Rintala for providing guidance throughout the thesis process. A special thanks to my friends Mika Kuitunen and Sakari Kapanen for giving their opinions on the online survey questions, your help was much appreciated. Last but absolutely not least, I want to thank you, Kaisa, for supporting me through this sometimes overwhelming and desperate process. Your continued encouragement, help and proofreading have been vital to the success of this thesis. Thank you for pushing me to write when I didn’t feel like writing at all, and for bringing so much joy to everyday life to counterbalance the thesis work. Tampere, 26th April 2021 Jaakko Rinta-Filppula iv CONTENTS 1. Introduction............................1 2. Type systems...........................3 2.1 Static checking.........................4 2.2 Dynamic checking.......................5 2.3 Type conversions and type safety.................6 3. Static type checkers for dynamically typed languages...........9 3.1 Different design approaches...................9 3.2 Ruby............................. 12 3.3 JavaScript.......................... 13 3.4 Python............................ 13 4. Studies on static vs. dynamic type systems............... 15 4.1 Controlled studies....................... 15 4.2 Studies on existing codebases.................. 19 4.3 Other studies......................... 21 4.4 Summary of existing studies................... 21 5. Methodology........................... 23 5.1 Exploratory testing....................... 23 5.2 Online survey......................... 24 6. Results and discussion....................... 28 6.1 Exploratory testing results.................... 28 6.2 Online survey results...................... 32 6.3 Summary of results and discussion................ 38 7. Conclusion............................ 40 References.............................. 42 Appendix A: Static type checker survey................... 45 v LIST OF FIGURES 5.1 Flowchart of the survey . 25 6.1 Years of programming experience . 32 6.2 What programming languages do you know? . 33 6.3 Number of people working on the codebase . 34 6.4 Type checking coverage . 34 6.5 Experiences of using static type checkers . 36 6.6 Expectations (E) of static type checking effects compared to experiences (X) of using them . 37 vi LIST OF TABLES 2.1 Examples of languages with different type system characteristics. Adapted from [15]. .7 6.1 File strictness level statistics at the start and end of the exploratory testing 29 6.2 Call-site-level typedness statistics . 30 6.3 Codebase size statistics (lines of code) . 34 vii LIST OF PROGRAMS AND ALGORITHMS 2.1 An example C++ program with a type error . .4 2.2 Resulting compiler error when trying to compile program 2.1 . .4 2.3 An online store payment example in Python . .5 2.4 Resulting runtime type error when trying to execute program 2.3 . .6 2.5 Problematic line from program 2.3 converted to JavaScript . .6 3.1 An example of type annotations with custom syntax (TypeScript) . 10 3.2 An example of type annotations as comments in JavaScript using Google Closure Compiler . 11 3.3 An example of type annotations using existing language syntax in Ruby with Sorbet . 11 3.4 An example of type annotations using a separate file (Ruby and RBS) . 12 6.1 Using safe navigation operator to prevent a crash in Ruby . 31 viii LIST OF TERMS AND ABBREVIATIONS call-site-level typedness a Sorbet metric that measures the number