Federal University of Pernambuco Center of Informatics Bachelor’s Program in Computer Engineering A TypeScript program generator based on Alloy Gabriela Araujo Britto Recife 2020 Federal University of Pernambuco Center of Informatics Gabriela Araujo Britto A TypeScript program generator based on Alloy A B.Sc. Dissertation presented to the Bachelor’s Program in Computer Engineering of the Center of Informatics of Federal University of Pernambuco in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Engineering. Advisor: Leopoldo Motta Teixeira Recife 2020 Abstract Refactoring is the process of modifying code to improve its internal structure, without altering its external behavior. To aid the programmer in the process of applying common refactorings to code, many IDEs provide refactoring implementations that automate the refactoring process. However, in doing so, the refactoring developers must concern themselves with guaranteeing that their refactoring does not change program behavior. Formally verifying that a refactoring preserves behavior is a complex and costly task. Therefore, in practice, developers use tests as an alternative. Testing a refactoring implementation requires programs as test inputs. Manually writing such programs can be tedious and difficult, as there may be many language features to consider. This work proposes a technique for generating TypeScript programs that can be used as input to test refactorings. We implement this technique in a tool called TSDolly. TSDolly uses an Alloy specification of TypeScript language and the Alloy Analyzer to generate instances for this specification, which are then transformed into TypeScript programs. The majority of the generated programs (at least 97.45%) compiled without errors. We applied five supported refactorings to our generated programs. All of the generated programs could be refactored. Those results show that TSDolly generates programs that can be used to test refactorings, and, more specifically, that can be used to test behavior preservation of refactorings. We were able to find a bug where a refactoring provided by the TypeScript compiler project introduced a compilation error when applied to some previously compilable programs. This suggests that TSDolly can be useful for finding bugs in TypeScript refactoring implementations. Keywords: refactoring, program generation iii Resumo Refatoração é o processo de modificar código para melhorar sua estrutura interna, sem mod- ificar seu comportamento externo. Para auxiliar o programador no processo de aplicação de refatorações comuns, muitas IDEs fornecem implementações de refatorações que automatizam esse processo. Porém, ao fazer isso, os desenvolvedores de refatorações precisam se preocu- par com garantir que sua refatoração não altere o comportamento de programas. Verificar formalmente que uma refatoração preserva comportamento é uma tarefa custosa e complexa. Portanto, na prática, desenvolvedores utilizam testes como alternativa. Testar a implementação de uma refatoração requer programas para serem utilizados como entrada dos testes. Escr- ever tais programas manualmente pode ser tedioso e difícil, visto que podem existir muitas características de linguagem de programação a serem consideradas. Este trabalho propõe uma técnica para gerar programas TypeScript que podem ser usados como entrada para testar refa- torações. Implementamos essa técnica em uma ferramenta chamada TSDolly. TSDolly usa uma especificação Alloy da linguagem TypeScript, e usa o Alloy Analyzer para gerar instâncias para essa especificação. Essas instâncias são então transformadas em programas TypeScript. A maioria dos programas gerados (pelo menos 97.45%) foi compilada sem erros. Aplicamos cinco refatorações suportadas em nossos programas gerados. Todos os programas gerados pud- eram ser refatorados. Esses resultados mostram que TSDolly gera programas que podem ser usados para testar refatorações, e, mais especificamente, que podem ser usados para testar se refatorações preservam o comportamento de programas. Encontramos um bug em que uma refatoração fornecida pelo projeto do compilador de TypeScript introduz um erro de compi- lação em programas que eram previamente compiláveis. Isso sugere que TSDolly pode ser útil para encontrar bugs em implementações de refatorações de TypeScript. Palavras-chave: refatoração, geração de programas iv List of Figures 3.1 Technique used by TSDolly to generate programs and apply refactorings to those programs.8 3.2 Signatures and relations of the TypeScript specification written in Alloy. 11 3.3 An instance generated by Alloy that represents a TypeScript program. 22 v List of Tables 4.1 Metrics collected during TSDolly experiments. 32 4.2 Execution time of TSDolly experiments. 35 vi List of Listings 2.2.1 The same code written in TypeScript and JavaScript.6 3.2.1 Grammar of the subset of TypeScript we have specified using Alloy. 12 3.2.2 Fact that expresses what is a valid variable access. 13 3.3.1 Example of the “Convert parameters to destructured object” refactoring. 14 3.3.2 Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Convert parameters to destructured object” refactoring. 14 3.3.3 Example of the “Convert to template string” refactoring. 15 3.3.4 Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Convert to template string” refactoring. 15 3.3.5 Fact that guarantees a concatenation expression is in fact a string concatenation. 16 3.3.6 Example of the “Generate ‘get’ and ‘set’ accessors” refactoring. 17 3.3.7 Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Generate ‘get’ and ‘set’ accessors” refactoring. 17 3.3.8 Example of the “Extract symbol” refactoring extracting a function. 18 3.3.9 Example of the “Extract symbol” refactoring extracting a literal value. 19 3.3.10Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Extract Symbol” refactoring. 20 3.3.11Exampleof the “Move to a new file” refactoring. 21 3.3.12Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Move to a new file” refactoring. 21 3.7.1 Interfaces that describe the result object produced by TSDolly. 28 4.3.1 Example of the error introduced by the “Generate ‘get’ and ‘set’ accessors” refactoring. 34 vii List of Acronyms API Application Programming Interface. AST Abstract Syntax Tree. IDE Integrated Development Environment. JSON JavaScript Object Notation. viii Contents 1 Introduction1 1.1 Goals2 1.2 Structure2 2 Background3 2.1 Alloy3 2.1.1 Alloy language3 2.1.2 Alloy Analyzer5 2.2 TypeScript6 2.2.1 TypeScript refactorings6 3 TSDolly8 3.1 TSDolly overview8 3.2 TypeScript specification in Alloy9 3.3 Refactorings 13 3.3.1 Convert parameters to destructured object 13 3.3.2 Convert to template string 14 3.3.3 Generate ‘get’ and ‘set’ accessors 16 3.3.4 Extract Symbol 16 3.3.5 Move to a new file 20 3.4 Instance generation 20 3.5 Building TypeScript programs 23 3.6 Applying refactorings 25 3.7 TSDolly output 26 4 Evaluation 29 4.1 Metrics 29 4.2 Experiments 30 4.2.1 Variable parameters 30 4.2.2 Fixed parameters 30 4.3 Results 31 5 Conclusion 36 5.1 Related work 36 5.2 Future work 38 ix CHAPTER 1 Introduction Refactoring is the process of modifying code with the goal of improving its internal structure without altering its external behavior [1]. Refactoring can improve code readability and orga- nization, making it easier to maintain. For instance, renaming a variable can make the meaning of that variable more clear. To aid the programmer in the process of applying common refactorings to code, many IDEs provide refactoring implementations that automate the refactoring process. However, in doing so, refactoring developers must be careful that their refactoring does not change the program behavior. Otherwise, applying the refactoring could introduce bugs in the refactored code. For instance, when implementing a refactoring for renaming variables, developers need to consider if there already is a variable with the new name, and they need to make sure the refactoring updates all occurrences of the renamed variable. To formally verify that a refactoring for a given programming language preserves behavior, one would need to have a formal specification of the language. Moreover, one would also need to have a specification of the refactoring, along with a proof that it preserves program behavior. However, that task is usually too complex and costly to be used in practice. In addition to that, in some cases, determining when a refactoring preserves the behavior of a program is an undecidable problem [2]. Therefore, in practice, developers use tests as an alternative to formally verifying refactoring implementations [3,4]. Although test suites do not guarantee the absence of bugs, they can reveal their presence. This way, they can increase the developers’ confidence in their refactoring implementation. To test their refactoring implementations, developers can write manual tests. This requires writing programs to be used as test inputs. This can be a tedious and difficult task: the developer must manually write a lot of programs, and decide what features of the language must be present in those programs. With that in mind, Soares et al. [3] proposed a technique for testing refactoring imple- mentations for the Java programming language. Their technique consists of generating Java programs, applying refactorings to those programs and then checking if the programs have the same behavior before and after refactoring. The program generation is implemented by a tool called JDolly. JDolly is based on two related technologies: Alloy [5], a formal specification language, and the Alloy Analyzer [5], a model finder tool that analyzes Alloy specifications and can check properties and generate structures that satisfy such specifications. The Alloy language is used to specify a subset of the Java language, and the Alloy Analyzer is then used to generate structures that satisfy the Java specification. The structures generated are converted into actual Java programs, which JDolly outputs.