A Typescript Program Generator Based on Alloy

Federal University of Pernambuco Center of Informatics

Bachelor’s Program in Computer Engineering

A TypeScript program generator based on Alloy

Gabriela Araujo Britto

Recife 2020 Federal University of Pernambuco Center of Informatics

Gabriela Araujo Britto

A TypeScript program generator based on Alloy

A B.Sc. Dissertation presented to the Bachelor’s Program in Computer Engineering of the Center of Informatics of Federal University of Pernambuco in partial fulﬁllment of the requirements for the degree of Bachelor of Science in Computer Engineering.

Advisor: Leopoldo Motta Teixeira

Recife 2020 Abstract

Refactoring is the process of modifying code to improve its internal structure, without altering its external behavior. To aid the programmer in the process of applying common refactorings to code, many IDEs provide refactoring implementations that automate the refactoring process. However, in doing so, the refactoring developers must concern themselves with guaranteeing that their refactoring does not change program behavior. Formally verifying that a refactoring preserves behavior is a complex and costly task. Therefore, in practice, developers use tests as an alternative. Testing a refactoring implementation requires programs as test inputs. Manually writing such programs can be tedious and difficult, as there may be many language features to consider. This work proposes a technique for generating TypeScript programs that can be used as input to test refactorings. We implement this technique in a tool called TSDolly. TSDolly uses an Alloy specification of TypeScript language and the Alloy Analyzer to generate instances for this specification, which are then transformed into TypeScript programs. The majority of the generated programs (at least 97.45%) compiled without errors. We applied five supported refactorings to our generated programs. All of the generated programs could be refactored. Those results show that TSDolly generates programs that can be used to test refactorings, and, more specifically, that can be used to test behavior preservation of refactorings. We were able to find a bug where a refactoring provided by the TypeScript compiler project introduced a compilation error when applied to some previously compilable programs. This suggests that TSDolly can be useful for finding bugs in TypeScript refactoring implementations.

Keywords: refactoring, program generation

iii Resumo

Refatoração é o processo de modificar código para melhorar sua estrutura interna, sem modificar seu comportamento externo. Para auxiliar o programador no processo de aplicação de refatorações comuns, muitas IDEs fornecem implementações de refatorações que automatizam esse processo. Porém, ao fazer isso, os desenvolvedores de refatorações precisam se preocu- par com garantir que sua refatoração não altere o comportamento de programas. Verificar formalmente que uma refatoração preserva comportamento é uma tarefa custosa e complexa. Portanto, na prática, desenvolvedores utilizam testes como alternativa. Testar a implementação de uma refatoração requer programas para serem utilizados como entrada dos testes. Escr- ever tais programas manualmente pode ser tedioso e difícil, visto que podem existir muitas características de linguagem de programação a serem consideradas. Este trabalho propõe uma técnica para gerar programas TypeScript que podem ser usados como entrada para testar refa- torações. Implementamos essa técnica em uma ferramenta chamada TSDolly. TSDolly usa uma especificação Alloy da linguagem TypeScript, e usa o Alloy Analyzer para gerar instâncias para essa especificação. Essas instâncias são então transformadas em programas TypeScript. A maioria dos programas gerados (pelo menos 97.45%) foi compilada sem erros. Aplicamos cinco refatorações suportadas em nossos programas gerados. Todos os programas gerados pud- eram ser refatorados. Esses resultados mostram que TSDolly gera programas que podem ser usados para testar refatorações, e, mais especificamente, que podem ser usados para testar se refatorações preservam o comportamento de programas. Encontramos um bug em que uma refatoração fornecida pelo projeto do compilador de TypeScript introduz um erro de compi- lação em programas que eram previamente compiláveis. Isso sugere que TSDolly pode ser útil para encontrar bugs em implementações de refatorações de TypeScript.

Palavras-chave: refatoração, geração de programas

iv List of Figures

3.1 Technique used by TSDolly to generate programs and apply refactorings to those programs.8 3.2 Signatures and relations of the TypeScript speciﬁcation written in Alloy. 11 3.3 An instance generated by Alloy that represents a TypeScript program. 22

v List of Tables

4.1 Metrics collected during TSDolly experiments. 32 4.2 Execution time of TSDolly experiments. 35

vi List of Listings

2.2.1 The same code written in TypeScript and JavaScript.6 3.2.1 Grammar of the subset of TypeScript we have specified using Alloy. 12 3.2.2 Fact that expresses what is a valid variable access. 13 3.3.1 Example of the “Convert parameters to destructured object” refactoring. 14 3.3.2 Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Convert parameters to destructured object” refactoring. 14 3.3.3 Example of the “Convert to template string” refactoring. 15 3.3.4 Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Convert to template string” refactoring. 15 3.3.5 Fact that guarantees a concatenation expression is in fact a string concatenation. 16 3.3.6 Example of the “Generate ‘get’ and ‘set’ accessors” refactoring. 17 3.3.7 Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Generate ‘get’ and ‘set’ accessors” refactoring. 17 3.3.8 Example of the “Extract symbol” refactoring extracting a function. 18 3.3.9 Example of the “Extract symbol” refactoring extracting a literal value. 19 3.3.10Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Extract Symbol” refactoring. 20 3.3.11Exampleof the “Move to a new file” refactoring. 21 3.3.12Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Move to a new file” refactoring. 21 3.7.1 Interfaces that describe the result object produced by TSDolly. 28 4.3.1 Example of the error introduced by the “Generate ‘get’ and ‘set’ accessors” refactoring. 34

vii List of Acronyms

API Application Programming Interface.

AST Abstract Syntax Tree.

IDE Integrated Development Environment.

JSON JavaScript Object Notation.

viii Contents

1 Introduction1 1.1 Goals2 1.2 Structure2 2 Background3 2.1 Alloy3 2.1.1 Alloy language3 2.1.2 Alloy Analyzer5 2.2 TypeScript6 2.2.1 TypeScript refactorings6 3 TSDolly8 3.1 TSDolly overview8 3.2 TypeScript speciﬁcation in Alloy9 3.3 Refactorings 13 3.3.1 Convert parameters to destructured object 13 3.3.2 Convert to template string 14 3.3.3 Generate ‘get’ and ‘set’ accessors 16 3.3.4 Extract Symbol 16 3.3.5 Move to a new ﬁle 20 3.4 Instance generation 20 3.5 Building TypeScript programs 23 3.6 Applying refactorings 25 3.7 TSDolly output 26 4 Evaluation 29 4.1 Metrics 29 4.2 Experiments 30 4.2.1 Variable parameters 30 4.2.2 Fixed parameters 30 4.3 Results 31 5 Conclusion 36 5.1 Related work 36 5.2 Future work 38

ix CHAPTER 1 Introduction

Refactoring is the process of modifying code with the goal of improving its internal structure without altering its external behavior [1]. Refactoring can improve code readability and orga- nization, making it easier to maintain. For instance, renaming a variable can make the meaning of that variable more clear. To aid the programmer in the process of applying common refactorings to code, many IDEs provide refactoring implementations that automate the refactoring process. However, in doing so, refactoring developers must be careful that their refactoring does not change the program behavior. Otherwise, applying the refactoring could introduce bugs in the refactored code. For instance, when implementing a refactoring for renaming variables, developers need to consider if there already is a variable with the new name, and they need to make sure the refactoring updates all occurrences of the renamed variable. To formally verify that a refactoring for a given programming language preserves behavior, one would need to have a formal specification of the language. Moreover, one would also need to have a specification of the refactoring, along with a proof that it preserves program behavior. However, that task is usually too complex and costly to be used in practice. In addition to that, in some cases, determining when a refactoring preserves the behavior of a program is an undecidable problem [2]. Therefore, in practice, developers use tests as an alternative to formally verifying refactoring implementations [3,4]. Although test suites do not guarantee the absence of bugs, they can reveal their presence. This way, they can increase the developers’ confidence in their refactoring implementation. To test their refactoring implementations, developers can write manual tests. This requires writing programs to be used as test inputs. This can be a tedious and difficult task: the developer must manually write a lot of programs, and decide what features of the language must be present in those programs. With that in mind, Soares et al. [3] proposed a technique for testing refactoring implementations for the Java programming language. Their technique consists of generating Java programs, applying refactorings to those programs and then checking if the programs have the same behavior before and after refactoring. The program generation is implemented by a tool called JDolly. JDolly is based on two related technologies: Alloy [5], a formal specification language, and the Alloy Analyzer [5], a model finder tool that analyzes Alloy specifications and can check properties and generate structures that satisfy such specifications. The Alloy language is used to specify a subset of the Java language, and the Alloy Analyzer is then used to generate structures that satisfy the Java specification. The structures generated are converted into actual Java programs, which JDolly outputs. The programs generated by JDolly are then used as input by a tool called SafeRefactor [6], which generates random tests for the JDolly pro-

1 1.1 GOALS 2 grams received as input, applies refactorings to those programs, and checks if the result of the randomly generated tests is the same for the input programs and their refactored counterparts.

1.1 Goals

This project proposes an adaptation of the technique used by the JDolly tool for the Type- Script language. Our goal is to present a technique for generating TypeScript programs that can be used as input for testing TypeScript refactorings. We implement this technique in a tool called TSDolly. In TSDolly, we use Alloy to write a specification of a subset of the Type- Script language. We use the Alloy Analyzer to generate instances for this specification. Then we transform those instances into TypeScript programs. Our goal is not to generate arbitrary TypeScript programs, but rather to generate programs to which TypeScript refactorings can be applied. For this purpose, our Alloy specification models the preconditions that programs must satisfy in order to serve as input for TypeScript refactorings. To evaluate TSDolly on our goal of generating programs to test refactorings, we used TS- Dolly to generate programs for five TypeScript refactorings. We also applied the refactorings to the generated programs. We inspected a random sample of the generated programs and found that more than 97% of them were compilable, and that all of them were successfully refactored. We also found a bug in a refactoring provided by the TypeScript compiler project where the refactoring introduces a compilation error in previously compilable programs.

1.2 Structure

This work is structured as follows: Chapter 2 describes the fundamental concepts related to Alloy and TypeScript needed to understand the remaining chapters. Chapter 3 describes the TSDolly tool implementation. Chapter 4 describes the experiments we ran to evaluate TSDolly and the results we obtained. Finally, Chapter 5 presents our conclusions, related works, and directions for future work. CHAPTER 2 Background

In this chapter we introduce basic concepts about Alloy and TypeScript.

2.1 Alloy

Alloy is an open source language and analyzer for software modeling [7]. The Alloy language is a formal speciﬁcation language that describes models, and those models can then be auto- matically analyzed by the Alloy Analyzer tool.

2.1.1 Alloy language The Alloy speciﬁcation language is declarative and based on the concept of sets of atoms, and relations between those sets. Atoms are primitive entities of a model, and a relation is a set of atom tuples [5]. When writing an Alloy speciﬁcation, we introduce a set of atoms by declaring a signature. For example, the following signature declaration declares a new set of atoms called A.

1 sig A{}

A signature can extend one other signature by using the extends keyword, as follows. If a signature A extends a signature B, that means the set introduced by A is a subset of the set introduced by B.

1 sig B{}

2 sig A extends B{

3 }

A signature can also declare fields. For instance, considering the following snippet, signature A declares a field called f. A field is a relation between the set being declared and other sets. In the example, f defines a relation whose domain is the set A, and whose range is set B.

1 sig A{

2 f: one B

3 }

4 sig B{}

3 2.1 ALLOY 4

When specifying fields, we can assign multiplicity to sets. In the previous example, we use the one multiplicity keyword, placed before B, to specify that each atom from the set A relates to exactly one atom from the set B. Other multiplicity keywords are lone, which stands for at most one atom, some, which stands for at least one atom, and set, which stands for any number of atoms. If there is no multiplicity keyword in a field declaration, the default is one. We can also assign a multiplicity to a signature. For instance, in the example that follows, the signature declaration introduces a set called A that has exactly one atom. For signature declarations, if no multiplicity is specified, the default multiplicity is set, which means the signature’s set can have any number of atoms.

1 one sig A{}

Sets and relations are the building blocks of Alloy, but we also need a way to express properties in our Alloy speciﬁcations. A fact is a way to express a property that our model should always satisfy. Consider the following example.

1 sig A{

2 f: lone B

3 }

4 sig B{}

5 fact some_A_relates_to_one_B{

6 some a: A| some a.f

7 }

Here, we have two sets, A and B, and a ﬁeld f that relates one atom of A to at most one atom of B. We also have the fact some_A_relates_to_one_B that states that there exists some atom a of set A, such that f relates a to some atom (of set B). More precisely, the fact states that there is some a of set A such that there is some atom in the set deﬁned by the expression a.f, where a.f is the set of atoms to which f maps a. The expression a.f above uses the dot operator (.). The dot operator means a composition of relations in the usual mathematical sense. Alloy allows us to also use the dot operator even on things that are not explicitly relations, by interpreting them as if they were. For instance, in the example expression a.f, a is an atom, but for the purposes of composing it with the f relation, Alloy interprets a as a singleton unary relation, that is a relation containing only the atom a. In the same way, we can compose a set and a relation by interpreting the set as a unary relation. When we write a property as a fact, we want Alloy to assume the property is always true. We might want to have a different purpose for a property. For instance, we might want to check whether a particular property is true, instead of assuming it is always true. In this case, we would write an assertion. Syntactically, the difference is that we use the assert keyword instead of fact. We can also have a property that we want to reuse in different contexts, and for that, we write a predicate using the pred keyword. 2.1 ALLOY 5

To learn the meaning of other language features that appear throughout this work, refer to the Alloy Language Reference1[5, Appendix B]. An important characteristic of the Alloy language is that its core logic is a ﬁrst-order logic. That means we cannot abstract over relations. We cannot, for instance, write a property that universally quantiﬁes over relations. We also cannot have a higher-order relation, i.e. a relation containing other relations. The effect of this choice is that while the language loses expressive power, it becomes more suitable for automated analysis [5], which for the Alloy language is done by the Alloy Analyzer tool.

2.1.2 Alloy Analyzer The Alloy Analyzer is a tool for analyzing specifications written in the Alloy language. More specifically, the Alloy Analyzer is a model finder: given a specification, the analyzer tries to find an instance – a binding of the variables to values – that satisfies the specification [7]. The Alloy Analyzer allows us to analyze a specification through commands. A command is an instruction written in our specification for the Alloy Analyzer. For instance, we can write a check command for an assertion in our specification, and the Alloy Analyzer attempts to find a counterexample. That is, an instance for which the property stated by the assertion does not hold. If the Alloy Analyzer finds a counterexample, we know that such property does not hold. Similarly, we can write a predicate in our specification, and then write a run command for that predicate, and the Alloy Analyzer attempts to find instances that satisfy that predicate. If the analyzer finds an instance, then we know that such property is satisfiable. However, the analysis performed by the Alloy Analyzer is not complete. It performs exhaustive model finding on a finite search space, that is, it tries to find instances up to a certain size. When we write a command, we specify a scope, which is a bound placed on the number of atoms of each signature an instance can have, and so the Alloy Analyzer only produces instances satisfying that scope. This limitation ensures that model finding is decidable and allows Alloy to analyze infinite models. But this also means that if the analyzer cannot find a counterexample to an assertion, it does not imply the assertion is always true, and if it cannot find an instance that satisfy a predicate, it does not imply the predicate is always false. The premise for performing checks in a finite scope is the hypothesis that a small scope is often enough to find the flaws in a specification - the small scope hypothesis [8]. For Alloy, this hypothesis means that although the Alloy Analyzer cannot prove or refute properties of our specification, it can still show us flaws through exhaustive search in smaller search spaces. The decision to perform finite scope analysis and to use a first-order logic is what allows the Alloy Analyzer to be able to analyse an Alloy specification in a fully automated fashion. Because of those constraints, the analyzer is able to translate an Alloy specification into a boolean formula and use a SAT solver to solve this formula, and then translate the solutions back into an instance of the specification’s model.

1The Alloy Language Reference is available at https://alloytools.org/download/ alloy-language-reference.pdf. 2.2 TYPESCRIPT 6

2.2 TypeScript

TypeScript is an open-source language which builds on JavaScript by adding static type deﬁni- tions [9]. JavaScript is a dynamically-typed, high-level programming language. It is known as a scripting language that runs in web browsers, providing interactive features to web pages. We can view TypeScript as an optionally typed [10] superset of JavaScript. This means that TypeScript recognizes JavaScript code, extending it with optional type annotations and the possibility of statically type-checking that code, while maintaining JavaScript’s semantics. Listing 2.2.1 shows a code snippet written in JavaScript and its TypeScript counterpart.

1 function updateName(person, newName) {

2 person.name= newName;

3 }

1 interface Person {

2 name: string;

3 age: number;

4 }

6 function updateName(person: Person, newName: string): void {

7 person.name= newName;

8 }

Listing 2.2.1: The same code written in TypeScript and JavaScript. At the top is the code written in JavaScript. At the bottom is the same code written in TypeScript, which has type annotations and an interface type declaration.

TypeScript does not have an official formal specification, but it is instead defined by its open-source compiler implementation, the TypeScript compiler. The compiler is contained in the TypeScript compiler project [11], which in addition to the compiler provides other functionalities for the TypeScript language, such as refactorings. The TypeScript compiler reads and type-checks TypeScript code and optionally emits JavaScript code. In addition to type-checking and emitting, the TypeScript compiler project provides a language service API that implements features for use by code editors and IDEs, such as "find all references", completion suggestions and, more relevant to this work, automated refactorings.

2.2.1 TypeScript refactorings The TypeScript language service API provides several automated refactorings. Each refactoring has a unique name and can have multiple related actions. Each action is a specific 2.2 TYPESCRIPT 7 transformation of a given refactoring. For example, the “Extract Symbol” refactoring provides an action called “Extract to constant in global scope”, which can be applied to a literal value (e.g. a string literal) to extract the value to a new constant variable defined in the global scope. This refactoring also has another action called “Extract to constant in enclosing scope”, which behaves similarly but extracts a literal value to its enclosing scope. The refactoring-related part of the TypeScript language service API is based on how users interact with a code editor. Interaction starts when the user puts the editor’s cursor in a given position of the source code of a file that is open in the editor, or selects a range of lines in the source code. The editor must then display information relevant to that position or code range, which in our refactoring scenario is a list of descriptions of refactoring actions that can be applied to that code location. The TypeScript refactoring API has a function that implements this behavior: given a file and a position or range of the file, the getApplicableRefact c ors function returns a list of refactoring actions applicable to that location. Given a list of refactoring actions, we can imagine the user selects one to be applied. The editor must then perform the necessary changes to the code, which could be changes to the code of the current file or changes to other files. The TypeScript refactoring API provides this feature through the getEditsForRefactor function. Once the user selects a refactoring action, the editor calls this function, passing as argument the selected refactoring action. The function then returns edits that the editor applies to the code, obtaining the refactored code, which completes the user interaction. CHAPTER 3 TSDolly

In this chapter, we present our technique. First, we provide a general overview of TSDolly, which is then detailed in the following sections.

3.1 TSDolly overview

Our technique for generating TypeScript programs is implemented in TSDolly. TSDolly is a tool that generates TypeScript programs that can be used to test automated refactoring implementations. It does so by using Alloy, a speciﬁcation language and model analyzer, to model a subset of TypeScript and to generate all TypeScript programs, up to a certain size, that satisfy this model. TSDolly then applies TypeScript refactorings to those generated programs to validate that they can indeed be used for testing refactorings. The main steps of our technique are shown in Figure 3.1. First, TSDolly uses the Alloy API to generate Alloy instances that satisfy a TypeScript speciﬁcation written in Alloy (step 1). Then, it transforms those instances into TypeScript programs using the TypeScript compiler project API (step 2). Lastly, TSDolly uses the TypeScript language service API to apply refactorings to the programs generated in the previous step (step 3). As a result, TSDolly produces generated TypeScript programs and their refactored counterparts.

Figure 3.1 Technique used by TSDolly to generate programs and apply refactorings to those programs.

TSDolly receives some parameters as input for running. First, we must provide the Al-

8 3.2 TYPESCRIPT SPECIFICATION IN ALLOY 9

loy specification file that TSDolly should use as a metamodel for the TypeScript language. This specification contains TypeScript syntactic rules, and some TypeScript semantic rules. In addition to those, the specification also describes what program structure is expected by each TypeScript refactoring supported. We must also provide the name of the refactoring that we want to target for testing. The programs generated by TSDolly will have a structure that matches the structure expected by this refactoring. TSDolly generates programs that follow the specification described in this file. Lastly, we can optionally provide a positive number as the skip argument. If we provide the skip argument, TSDolly will sample the generated Alloy instances, such that only a subset of them are transformed into TypeScript programs and analyzed in TSDolly’s pipeline. For a skip of n, for every n Alloy instances TSDolly only takes one (1) into account and ignores the remaining n − 1. TSDolly’s implementation can be found in the following repository: https://github.com/gabritto/tsdolly/tree/ ccb5a8423bb3afd3802e405d4d059afcc369abb5.

3.2 TypeScript speciﬁcation in Alloy

The programs that TSDolly generates are modeled by a specification, written in Alloy, of a subset of the TypeScript language1. In this section we describe this specification and its structure. Our Alloy specification of the TypeScript language follows some underlying principles. First, we want the Alloy instances to represent programs that will be able to serve as input for TypeScript refactorings, so we prioritized TypeScript features that are relevant for refactorings. Second, we want the Alloy instances to represent programs that do not have compilation errors. This is because we want the generated programs to be useful for testing if refactorings preserve program behavior. Behavior preservation is typically defined for programs that compile successfully, as a program with a compilation error may contain undefined behavior. Even though the TypeScript compiler is capable of emitting JavaScript code for a TypeScript program that has errors [12], we have chosen to follow the approach of focusing on generating compilable programs and adopting the strictest set of type checking rules the TypeScript compiler can enforce. Our specification is based on modeling instances with a tree structure, since ultimately we want to convert those instances into ASTs. In our specification, each Alloy signature corresponds to a node type from the TypeScript AST. For instance, there is a signature called Program that represents a TypeScript program. This signature has multiplicity one, so it is always present, it is unique, and it works as the root of our AST. There is also a Funct c ionDeclaration signature that corresponds to a function declaration node in TypeScript, and this signature is related to the Program signature through the relation declarations, similar to how a function declaration node appears as a child of the program node in an AST. Figure 3.2 contains a graph of the signatures and relations present in our specification. For

1The full Alloy speciﬁcation we used can be found in https://github.com/gabritto/tsdolly/ blob/ccb5a8423bb3afd3802e405d4d059afcc369abb5/typescript.als. 3.2 TYPESCRIPT SPECIFICATION IN ALLOY 10

reference, Listing 3.2.1 contains a grammar of the subset of TypeScript that our specification models. There are, however, some adaptations and simplifications made in our specification with regards to the node types used by the TypeScript compiler. Although the TypeScript compiler has a single node type for identifiers, in our specification we chose to have a separate signature for each kind of identifier: there is a FunctionIdentifier, a ClassIdentifier etc. This choice makes it easier to write facts using particular kinds of identifiers, and avoids producing programs with name clashes that would cause compilation errors. Another adaptation was that we chose to model class inheritance differently in our Alloy specification. In the TypeScript AST, a class declaration node has an extends property that points to an identifier referring to the class it inherits from. In our Alloy specification, we modelled class inheritance as a relation, called extend, that relates one class declaration to at most one other class declaration, instead of relating a class declaration to an identifier. This decision simplified expressing some well-formedness predicates about class inheritance contained in our specification. In addition to signatures, we also have facts that eliminate some classes of ill-formed programs from our instance space. Consider the VariableAccess signature, which refers to a variable access statement. It has a variable field whose range is the union of FieldI c dentifier and ParameterIdentifier. This means that a variable access can refer to a field or a parameter. However, we do not want a variable access to refer to a non-existent variable, so we have the ValidVariableAccess fact, shown in Listing 3.2.2, expressing the conditions under which a variable access is valid. The fact has two universal clauses, which cover the cases where a variable is accessed in the body of a function declaration and in the body of a method declaration, respectively. The first clause states that if a variable access appears in the body of a function declaration, then it must refer to a parameter identifier from that function’s parameters. In other words, v.variable, which is a ParameterIdentifier, must belong to the set of that function’s parameter names (f.parameters.name). The second clause states that if a variable access is in the body of a method declaration, it must refer to either a parameter identifier (similar to the previous clause), or it must refer to a field accessible within that method. A field is accessible within a method’s body if it belongs to that method’s class (condition expressed by v.variable in m.~methods.fields.name), or if it belongs to a superclass of that method’s class (condition expressed by v.variable in c m.~methods.êxtend.fields.name) and is not private (condition expressed by no v.variable.~name.visibility). We need some other facts to optimize the generated Alloy instances. We do not want to have a node that is not connected to the graph that represents our program, which is rooted in a node belonging to the Program signature. That lone node would not appear in the resulting TypeScript code. Therefore, we have a fact guaranteeing that each element of a signature has a parent element to which it is related, with some exceptions (like the root node, which does not have a parent). 3.2 TYPESCRIPT SPECIFICATION IN ALLOY 11 iue3.2 Figure intrsadrltoso h yecitseicto rte nAlloy. in written specification TypeScript the of relations and Signatures 3.2 TYPESCRIPT SPECIFICATION IN ALLOY 12

hprogrami ::= hdeclaration-listi hdeclaration-listi ::= hdeclarationi hdeclaration-listi | ε hdeclarationi ::= hfunction-decli | hclass-decli hfunction-decli ::= function identifier ( hparametersi ){ hblocki } hparametersi ::= hparameter-listi | ε hparameter-listi ::= hparameter-decli | hparameter-decli , hparameter-listi hparameter-decli ::= identifier : htypei htypei ::= string | number hblocki ::= hexpression-listi hexpression-listi ::= hexpressioni hexpression-listi | ε hexpressioni ::= hfunction-calli ; | hmethod-calli ; | hvariable-accessi ; | hstring-concati ; hclass-decli ::= class identifier hextend-clausei { hfield-listi hmethod-listi } hextend-clausei ::= extends identifier | ε hfield-listi ::= hfieldi hfield-listi | ε hfieldi ::= hvisibilityi identifier ?: htypei ; hvisibilityi ::= private | ε hmethod-listi ::= hmethod-decli hmethod-listi | ε hmethod-decli ::= identifier ( hparametersi ){ hblocki } hfunction-calli ::= identifier ( hargumentsi ) hargumentsi ::= hargument-listi | ε hargument-listi ::= hvariable-accessi | hvariable-accessi , hargument-listi hmethod-calli ::= this . identifier ( hargumentsi ) hvariable-accessi ::= identifier | this . identifier hstring-concati ::= hconcati | hconcati + hstring-concati hconcati ::= string literal | hvariable-accessi

Listing 3.2.1: Grammar of the subset of TypeScript we have speciﬁed using Alloy. 3.3 REFACTORINGS 13

1 fact ValidVariableAccess{

2 all f: FunctionDecl, v: VariableAccess{ 3 (v in f.body.expression.*(FunctionCall<: arguments+ ,→ MethodCall<: arguments+ concat)) implies

4 (v.variable in f.parameters.name)

5 }

6 all m: MethodDecl, v: VariableAccess{ 7 (v in m.body.expression.*(MethodCall<: arguments+ ,→ FunctionCall<: arguments+ concat)) implies (

8 (v.variable in m.parameters.name) or

9 (v.variable in m.~methods.fields.name) or

10 (v.variable in m.~methods.^extend.fields.name and no

,→ v.variable.~name.visibility))

11 }

12 }

Listing 3.2.2: Fact that expresses what is a valid variable access.

3.3 Refactorings

Our goal with TSDolly is not only to generate TypeScript programs, but generate TypeScript programs in which a given target refactoring can be applied. To accomplish that, we specify the conditions that a program must satisfy to be accepted as input by a refactoring. We express these conditions using Alloy predicates, one predicate for each TSDolly-supported refactoring. For instance, if a refactoring acts on function declarations, then we have a corresponding predicate stating that our program instance must have a function declaration signature element. Each refactoring predicate has also a run command that runs the predicate for a fixed scope. The run command tells the Alloy Analyzer to find all the instances, up to a certain size, satisfying that predicate. The run commands associated with the refactoring predicates are used by the instance generation step of TSDolly described in the following section. We currently support five out of the nine different refactorings implemented by the Type- Script compiler project version used by TSDolly, 3.9.5. In what follows, we describe the supported refactorings and their associated Alloy predicates.

3.3.1 Convert parameters to destructured object The “Convert parameters to destructured object” refactoring has the goal of transforming functions to emulate named parameters in TypeScript. Since JavaScript, and therefore, TypeScript, do not have builtin support for named parameters, the way to emulate this feature is to refactor a function with a list of parameters to instead take a single parameter object. This new parameter object will have one property for each of the old parameters. We can see an example of this 3.3 REFACTORINGS 14

refactoring in Listing 3.3.1.

1 function sum(a: number, b: number){

2 return a+ b;

3 }

4 sum(1,2);

1 function sum({ a, b }: { a: number; b: number; }) {

2 return a+ b;

3 }

4 sum({ a: 1, b: 2 });

Listing 3.3.1: Example of the “Convert parameters to destructured object” refactoring. At the top is the code before the refactoring. At the bottom is the code that results from applying the refactoring to the sum function.

This refactoring can be applied to function and method declarations that have more than one parameter. It can also be applied to constructors and arrow functions or function expressions with more than one parameter, but those are not modeled in our Alloy speciﬁcation. To capture this condition, we have the ConvertParamsToDestructuredObject predicate, shown in Listing 3.3.2. The predicate states that our program instance should have either a function declaration element with more than one parameter or a method declaration element with more than one parameter, which is the necessary condition for applying the refactoring.

1 pred ConvertParamsToDestructuredObject{

2 (some f: FunctionDecl|#f.parameters>1) or (some m:

,→ MethodDecl|#m.parameters>1)

3 }

Listing 3.3.2: Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Convert parameters to destructured object” refactoring.

3.3.2 Convert to template string The “Convert to template string” refactoring converts a string concatenation expression to a template literal (also called a template string). Listing 3.3.3 shows an example of what this refactoring does. 3.3 REFACTORINGS 15

1 const name= "Alice";

2 const greeting= "Hello, "+ name+ ". Welcome!";

1 const name= "Alice";

2 const greeting= `Hello, ${name}. Welcome!`;

Listing 3.3.3: Example of the “Convert to template string” refactoring. At the top is the code before the refactoring. At the bottom is the code that results from applying the refactoring to the string concatenation expression on the right-hand side of the greeting variable declaration.

A template literal is a string literal that allows string interpolation, that is, it is a string literal that can have placeholders for expressions in it. At runtime, a placeholder in the template literal is replaced by the string representation of the value of the expression inside the placeholder. For example, a template literal such as `One plus two equals ${1+2 }` evaluates to "One plus two equals 3". Note that in the template literal, a placeholder has the form ${ } with an expression inside. We can think of a string concatenation as a sequence of subexpressions (operands) joined by the + operator, where the whole expression evaluates to a string at runtime. Since an expression with the + operator could also stand for addition of numbers and evaluate to a number, the refactoring implementation needs evidence that the expression is in fact a string concatenation and results in a string at runtime. To this end, the refactoring is applicable to an expression with the + operator only if at least one of the operands is a string literal, since it can then be sure the expression evaluates to a string. The refactoring is applicable to programs that have string concatenation expressions, and to express that we have the ConvertToTemplateString predicate, shown in Listing 3.3.4.

1 pred ConvertToTemplateString{

2 one StringConcat

3 }

Listing 3.3.4: Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Convert to template string” refactoring.

We also specify the condition that a string concatenation must have at least one string literal as operand using the StringConcatLiteral fact shown in Listing 3.3.5. This fact asserts that at least one operand of the string concatenation expression must be a string literal, otherwise we risk having an expression which uses the + operator but is not actually a string 3.3 REFACTORINGS 16

concatenation.

1 fact StringConcatLiteral{

2 all s: StringConcat{

3 some l: StringLiteral|l in s.concat

4 }

5 }

Listing 3.3.5: Fact that guarantees a concatenation expression is in fact a string concatenation by having a string literal operand.

3.3.3 Generate ‘get’ and ‘set’ accessors The “Generate ‘get’ and ‘set’ accessors” refactoring, also known as the “Encapsulate Field” or “Encapsulate Variable” refactoring [1, Chapter 6], generates a getter and a setter function, also called accessors, for a class field. A getter is a function that is bound to an object’s field, such that when that field is read, the getter function is called to compute that field’s value. The field value is then the return of the getter function. Similarly, a setter is a function that is bound to an object’s field, but it is used to set the field’s value instead of reading it. When the field is set, instead of writing the new value directly to the field, the setter is called with the new value as argument. Listing 3.3.6 illustrates an example of this refactoring. This refactoring can be applied to a class field, for private, protected or public fields. Note that our Alloy specification does not include the protected field modifier, only the private and public modifiers. So, for this refactoring to be applicable to a program, that program should

have a class declaration that declares at least one ﬁeld. The GenerateGetAndSetAcces c sors predicate in Listing 3.3.7 expresses this condition.

3.3.4 Extract Symbol The “Extract Symbol” refactoring extracts certain expressions into a new symbol in some scope. This description might seem vague, but this is because this refactoring provides different actions that can be applied to different types of expressions and scopes. To illustrate some actions that this refactoring offers, we can look at Listing 3.3.8. In this example, there are “Extract Symbol” actions available to the sayHello("Charlie") function call in line 6. The available actions are “Extract to function in global scope”, whose

result is shown in the bottom part of the example, and “Extract to inner function in ‘talkTo c Charlie’ function”, whose result is not shown but differs only in the scope where the newly created function (sayHelloToCharlie) is placed. Those actions of the “Extract Symbol” refactoring implement what is commonly referred to as the “Extract Function” refactoring [1, Chapter 6]. 3.3 REFACTORINGS 17

1 class Person {

2 public name: string;

3 }

1 class Person {

2 private _name: string;

3 public get name(): string {

4 return this._name;

5 }

6 public set name(value: string){

7 this._name= value;

8 }

9 }

Listing 3.3.6: Example of the “Generate ‘get’ and ‘set’ accessors” refactoring. At the top is the code before the refactoring. At the bottom is the code that results from applying the refactoring to the name ﬁeld.

1 pred GenerateGetAndSetAccessors{

2 some c: ClassDecl|#c.fields>0

3 }

Listing 3.3.7: Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Generate ‘get’ and ‘set’ accessors” refactoring. 3.3 REFACTORINGS 18

1 function sayHello(name: string){

2 console.log("Hello, "+ name+ "!");

3 }

5 function talkToCharlie() {

6 sayHello("Charlie");

7 console.log("How was your day?");

8 }

1 function sayHello(name: string){

2 console.log("Hello, "+ name+ "!");

3 }

5 function talkToCharlie() {

6 sayHelloToCharlie();

7 console.log("How was your day?");

8 }

10 function sayHelloToCharlie() {

11 sayHello("Charlie");

12 }

Listing 3.3.8: Example of the “Extract symbol” refactoring. At the top is the code before the refactoring. At the bottom is the code that results from applying the “Extract to function in global scope” action of this refactoring to the sayHello("Charlie") function call in line 6. 3.3 REFACTORINGS 19

In the same example code, we could also apply the “Extract Symbol” refactoring to the " c Charlie" string literal passed as argument to the sayHello call in line 6. Here, the actions available are “Extract to constant in enclosing scope” and “Extract to constant in global scope”. If we apply the “Extract to constant in enclosing scope” action to the "Charlie" string literal, the result is the code shown in Listing 3.3.9. Those actions of the “Extract Symbol” refactoring implement what is commonly referred to as the “Extract Variable” refactoring [1, Chapter 6].

1 function sayHello(name: string){

2 console.log("Hello, "+ name+ "!");

3 }

5 function talkToCharlie() {

6 sayHello("Charlie");

7 console.log("How was your day?");

8 }

1 function sayHello(name: string){

2 console.log("Hello, "+ name+ "!");

3 }

5 function talkToCharlie() {

6 const charlie= "Charlie";

7 sayHello(charlie);

8 console.log("How was your day?");

9 }

Listing 3.3.9: Example of the “Extract symbol” refactoring. At the top is the code before the refactoring. At the bottom is the code that results from applying the “Extract to constant in enclosing scope” action of this refactoring to the "Charlie" string literal in line 6.

Limiting our analysis to the subset of TypeScript modeled in our specification, this refactoring can be applied to function and method calls, string literals and field accesses. To capture this, we have the ExtractSymbol predicate, shown in Listing 3.3.10. The last disjunction of the predicate expresses the condition that there exists a field access. More specifically, it states that there must be some field identifier that is used by some variable access. variable relates a variable access to an identifier, so ~variable is the inverse relation and relates an identifier to a variable access. 3.4 INSTANCE GENERATION 20

1 pred ExtractSymbol{

2 some (FunctionCall+ MethodCall) or some StringLiteral or {

3 some f: FieldIdentifier| some f.~variable

4 }

5 }

Listing 3.3.10: Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Extract Symbol” refactoring.

3.3.5 Move to a new file The “Move to a new file” refactoring moves a top-level declaration (i.e. a declaration on file scope) to a new file. Listing 3.3.11 shows an example of applying this refactoring. The condition needed for applying the refactoring is the presence of a top-level declaration. Considering the language features modeled in our specification, this means that the program should contain a function declaration or a class declaration to be refactored by the “Move to a new file” refactoring. We express this condition in the MoveToNewFile predicate shown in Listing 3.3.12.

3.4 Instance generation

Given an Alloy specification and specific predicates for a particular refactoring, the first step of TSDolly is instance generation. This step uses the Alloy API [13] to generate Alloy instances based on our TypeScript specification. These instances represent TypeScript programs that TSDolly later converts into actual TypeScript programs. An example of an instance generated by Alloy can be seen in Figure 3.3. This example instance represents a TypeScript program that contains a single top-level class declaration, and this class has a single method declared, with no parameters and an empty implementation block. In the instance generation step, we start by taking as arguments the path of the Alloy specification file that contains our TypeScript specification, and the name of the target refactoring. With those arguments in hand, we call the Alloy API to load our specification file. Once it is loaded, we then locate the run command associated with the target refactoring and use the Alloy API to call the Alloy Analyzer to execute that command. As a result of running that command, the Alloy API yields a Solution object. This object is an instance that satisfies the constraints specified by our facts and the predicate associated with the run command. The solution object also functions as an iterator, and we call the next method of that solution object to get the next solution object that represents a different instance. We keep calling the next method to obtain all solution objects, until the call to the satisfiable method returns false, indicating that there are no more solutions possible. The Alloy API is implemented in Java. For this reason, we implement the instance gen- 3.4 INSTANCE GENERATION 21

File: file.ts

1 function sum(a: number, b: number){

2 return a+ b;

3 }

5 function product(a: number, b: number){ 6 return a * b; 7 }

File: file.ts File: newfile.ts

1 function product(a: 1 function sum(a: number, b:

,→ number, b: number){ ,→ number){ 2 return a * b; 2 return a+ b; 3 } 3 }

Listing 3.3.11: Example of the “Move to a new file” refactoring. At the top is the code before the refactoring, on file file.ts. At the bottom is the code that results from applying this refactoring to the sum function declaration. This results in removing the sum function from the file.ts file and placing it in a newly created file called newfile.ts.

1 pred MoveToNewFile{

2 some (FunctionDecl+ ClassDecl)

3 }

Listing 3.3.12: Predicate that expresses the necessary conditions a program must satisfy to be refactored by the “Move to a new ﬁle” refactoring. 3.4 INSTANCE GENERATION 22

Figure 3.3 An instance generated by Alloy that represents a TypeScript program. The program represented contains a single top-level class declaration, with a single method declared with no parameters and an empty implementation block. 3.5 BUILDING TYPESCRIPT PROGRAMS 23 eration step in Java as well. However, the TypeScript compiler project is implemented in TypeScript, and we would like to leverage the compiler project APIs to build our TypeScript programs and to call TypeScript refactorings. For that reason, we implement the other TSDolly steps in TypeScript. The instance generation step must communicate the generated Alloy instances to the remaining steps implemented in TypeScript. To that end, in this step we must serialize the generated Alloy instances in a way that the rest of TSDolly’s pipeline can understand. We use JSON as a serialization format. We serialize each Alloy instance into a JSON object. The JSON object that represents an Alloy instance resembles a TypeScript AST, with the exceptions men- tioned in Section 3.2. We encode each atom in an Alloy instance as a JSON object. The JSON object has a property for each field that the atom has. Those properties contain other JSON objects or JSON arrays representing other atoms or sets of atoms. Each JSON object also has a type property, indicating the signature/type of the atom it represents, and an id property with a string that uniquely identifies that atom. At the end we build a JSON array containing all of the instances. We convert this JSON array into a string and write it to a file that the next TSDolly step will read. The instances are serialized to JSON by a class we implemented, called Program. The constructor of this class receives the solution object that represents an instance, and collects information about what elements of each signature are present in the instance and what tuples of each relation are present. We obtain this information by calling methods of the solution object. Once the Program object is constructed, we call its toJson method, which uses the information collected to build a JSON object that represents the instance.

3.5 Building TypeScript programs

Once we have the Alloy generated instances, the next step is to convert them into actual Type- Script programs. In this step, we use the serialized instances produced in step 1 to build Type- Script ASTs. We build these ASTs using the TypeScript compiler API. Once we have an AST, we transform it into textual code, and we also compile this code to check for compilation errors. Because we are using TypeScript compiler project APIs, from this step on our code is implemented in TypeScript. We start this step by reading the serialized instances produced by the previous step. Since the instances are serialized as a JSON string, we use the JSON.parse method from JavaScript’s standard library to parse the string into an array of JavaScript objects. Each object is a JavaScript representation of an Alloy generated instance. To make sure that the TypeScript code, the Alloy speciﬁcation and the instance generation code all agree on the shape of an instance, we validate the JavaScript object produced from the serialized instances. To accomplish this, we write in TypeScript the expected type of the object we obtain from reading the serialized Alloy instance. We then generate a JSON schema [14] from that type and validate the object using this schema. This guarantees that the object conforms to the expected type, otherwise we throw an exception. Once we validate the shape of the object obtained from an Alloy instance, we need to build a TypeScript AST from it. To do this, for each object, we recursively visit its nested objects (i.e. 3.5 BUILDING TYPESCRIPT PROGRAMS 24

the objects pointed to by the initial object’s properties) and call functions from the TypeScript compiler node factory. For each TypeScript AST node type, there is a node factory function used to build a node of that type. When recursively visiting the objects to build our AST, we call the node factory function that corresponds to that object’s type. For instance, when visiting an object that corresponds to a function declaration (which in the Alloy instance would be an element of the FunctionDecl signature), we call the createFunctionDeclaration function from the TypeScript compiler. The process of building a TypeScript AST is straightforward, but there are some details we need to consider. We need to be mindful of differences between our Alloy specification of TypeScript and the TypeScript compiler node types. When visiting a JSON object that corresponds to an instance’s class declaration atom, we consider the fact that the extend field in our Alloy model relates two class declarations, while in TypeScript compiler node types, a class declaration node has an extends property that points to an identifier, and not to another class declaration. To account for that, when we are visiting a class declaration object whose extend property points to another class declaration object, we extract the name property (an identifier) from this other object and ignore the rest to build the corresponding class declaration node from our TypeScript AST. For the same reason, we map all of the identifier objects to a TypeScript identifier node, regardless of its type as present in our Alloy specification (e.g. ClassIdentifier, FunctionIdentifier). Since our Alloy specification has no concept of ordering (for simplicity, we only use sets), we need to be careful when building lists of AST nodes where the order matters. In our code, this happens when building a list of declarations. Our specification of TypeScript models class declarations and inheritance between classes, and when a class B extends (inherits from) a class A, the declaration for class A must appear before the declaration for class B, otherwise we get a compilation error. To solve this, we topologically sort our declarations based on this dependency between class declarations. Another detail in our translation implementation is that, when creating an AST node for a class’s field, we mark that field as optional. This avoids a compilation error for uninitialized fields, since we don’t include field initialization in our Alloy specification of TypeScript. Once we have a TypeScript AST, we create a TypeScript program. A TypeScript program encapsulates the information that the TypeScript compiler needs to provide diagnostics, such as compilation errors and warnings. A program can comprise some source files, a compiler configuration, an instance of the type checker, among other things. In our case, we need a TypeScript program for type checking and for applying refactorings. We create a TypeScript program object using the TypeScript compiler API, passing the TypeScript AST as argument.

We use the TypeScript program to collect compilation errors. To do this, we call the g c etPreEmitDiagnostics method of the TypeScript program object. This method returns an array of Diagnostics objects that represent diagnostics generated before the JavaScript- emitting part of the compilation process, which serves our purposes because we are not interested in code emission. A Diagnostics object has a category, which can be Warning, Error, Suggestion, or Message. Following our principle of trying to generate compilable programs, we want to collect compilation errors. So we ﬁlter the diagnostics array to obtain an array containing only error diagnostics, which are the compilation errors that are 3.6 APPLYING REFACTORINGS 25

relevant to our context. It is in this step of TSDolly’s pipeline that we implement our skip-based sampling strategy. TSDolly receives a positive integer called skip, which indicates how we should sample the Alloy generated solutions. For a skip value of n, the sampling strategy consists in splitting the array of Alloy instances into contiguous chunks of size n, and randomly selecting one of them. Only the instances we select for sampling are converted into a TypeScript program in the way we described above, and only those programs are used by the next TSDolly step. The remaining instances are ignored. An implementation detail pertaining to this step is that we have organized the processing of Alloy instances in a streaming manner. Due to the large number of generated Alloy instances, the JSON string representing the array of all serialized instances can be large. We are using the Node.js runtime [15] to run TSDolly, and Node.js imposes a limitation on the size of strings that we can load in memory. A string cannot have more than 536,870,888 characters, otherwise the runtime throws an exception. Our JSON string can exceed that imposed character limit. To overcome this limitation, we used Node.js’s Stream API [16]. Our functions receive a chunk of data at a time, instead of reading the whole string at once. That means we read and process one instance object at a time to avoid having to read all of them into memory as one large string.

3.6 Applying refactorings

Once we have a TypeScript program produced by the previous step, we can use it to test a TypeScript refactoring. In this step, we apply the target refactoring provided as argument to the TypeScript programs that TSDolly generated. The TypeScript compiler project provides a language service API that contains functions that are typically used by code editors to provide functionalities like completion suggestions and, more relevantly, automated refactoring implementations. To refactor our generated Type- Script programs, we use this API to ﬁnd out if the target refactoring is available to be applied to a program and, if so, to apply the refactoring. The language service API contains two refactoring-related methods that are relevant to us:

• getApplicableRefactors: this method returns a list of which refactorings can be applied to a certain part of the code. To call it, we must provide a ﬁle from our program, and a position or range in that ﬁle where we would want to apply a refactoring.

• getEditsForRefactor: this method returns the changes (edits) that result from applying a given refactoring to our program. To call it, we must provide a ﬁle, a position or range (similar to getApplicableRefactors), and the name and action of the refactoring we want to apply.

For each generated TypeScript program, we ﬁrst call the getLanguageService method of the TypeScript program object to obtain an object of type LanguageService. This object implements the language service API and provides the methods listed above. Once we have that object, the next step is to ﬁnd out if the refactoring we are targeting is available at a given portion of our generated program’s code. 3.7 TSDOLLY OUTPUT 26

Since the getApplicableRefactors method expects a position or range of our code, we need to have a way of deciding what portions of our code are relevant for a refactoring. We could have a naive implementation that tests every position and every range. However, that would be inefﬁcient, as we know that only some parts of our code are relevant for applying a refactoring. Our implementation is based on our knowledge of which AST node types a refactoring could be applied to. For each refactoring that TSDolly supports, we have a predicate function that, given an AST node, returns true if we should consider that node when looking for places to apply a refactoring and false otherwise. We do a tree walk on the program’s AST and, whenever we visit a node that the predicate function considers relevant, we call getApplicableRefactors with the code range obtained from that node’s range property. This method returns an array of objects, each containing the name of the refactoring available and that refactoring’s action that is applicable at that range of the code. We ﬁlter this array to obtain the objects that correspond to the refactoring we are currently testing. With the array containing the available actions of the target refactoring, we call getEditsForRefactor for each action in that array to get the edits for that action, passing as argument the same code range of the node we’re visiting. Once we visit all of the nodes of a program’s AST, we have an array of refactoring edits. Since we can apply a refactoring at different nodes and obtain the same set of edits, we remove the duplicate elements of this array of edits. These edits are then used to build refactored versions of the original generated program. To do this, we clone the original TypeScript program and apply the edits to the text of the program. Note that we can end up with more than one program as a result of applying the target refactoring, because there can be different actions of that refactoring available to a program, and the refactoring might be applicable to different parts of a program in different ways. After applying the target refactoring, for each program we have a set of TypeScript programs corresponding to applying refactoring actions to the original program. In addition to producing refactored versions of the programs, we also check if those programs have compilation errors, using the same methodology described in Section 3.5.

3.7 TSDolly output

After we run TSDolly, we obtain an output that encompasses all of the information collected in TSDolly’s steps. We have information about each generated TypeScript program, such as the program code and its compilation errors. We also have, for each program, information about whether the target refactoring is applicable to the program, and a set of programs that result from applying the refactoring to the original program. Note that TSDolly only analyzes and produces that information for the sampled Alloy instances. We encode the TSDolly output as an array of objects, each object containing the information for a generated TypeScript program. We serialize this array of objects as a JSON string and store it in a ﬁle so that it is available for inspection later if we need it. The result object generated for each program is described by the Result interface shown in Listing 3.7.1. A result has a Program, which contains information about the generated TypeScript program, and an array of RefactorInfo objects, one for each distinct way the 3.7 TSDOLLY OUTPUT 27

target refactoring can be applied to the program. A Program has an array of files, each file here represented by its path (fileName property) and its content (text property). A program also has an array of compilation errors. The hasError property is a boolean flag that is true when the errors property array is non- empty, that is, when the program has compilation errors, and that flag is there just to make it easier to look for programs that don’t compile when examining the results. A RefactorInfo object contains information about a particular refactoring applied to a program. This object has the refactoring name (name property), the name of the refactoring action that was applied (action property), and the range of the original program that triggered

that refactoring action, that is, the range that was used as an argument to the getApplica c bleRefactors and getEditsForRefactor methods described in Section 3.6. It also contains a RefactorEditInfo object (editInfo property), which is how the TypeScript

compiler project encodes the effects a refactoring has on a program’s text. The result c ingProgram property is the Program that results from applying the refactoring changes described by the RefactorEditInfo object to the original program. The introduc c esError property is a boolean ﬂag that is true if the refactoring introduced a compilation error, that is, if the original program had no compilation errors but the resulting program has compilation errors. 3.7 TSDOLLY OUTPUT 28

1 export interface Result {

2 program: Program;

3 refactors: RefactorInfo[];

4 }

6 export interface Program {

7 files: File[];

8 errors: CompilerError[];

9 hasError: boolean;

10 compilerOptions: ts.CompilerOptions;

11 }

13 export interface File {

14 fileName: string;

15 text: string;

16 }

18 export interface RefactorInfo {

19 name: string;

20 action: string;

21 triggeringRange: ts.TextRange;

22 editInfo: ts.RefactorEditInfo;

23 resultingProgram?: Program;

24 introducesError?: boolean;

25 }

27 export interface CompilerError {

28 file?: string;

29 code: number;

30 line?: number;

31 start?: number;

32 length?: number;

33 category: ts.DiagnosticCategory;

34 messageText: string;

35 }

Listing 3.7.1: Interfaces that describe the result object produced by TSDolly for each program it analyzed. CHAPTER 4 Evaluation

As we’ve seen in Chapter 3, TSDolly works in three steps. In step 1, TSDolly takes as input a target refactoring that we are going to test, and calls the Alloy API to generate instances of our TypeScript metamodel. Then, in step 2, TSDolly converts those instances into actual Type- Script programs. Finally, in step 3, TSDolly applies the target refactoring to those generated programs, resulting in new TypeScript programs. This chapter describes how we evaluated TSDolly. We start by describing the main metrics we collected, then we describe the experiments we executed and, ﬁnally, the obtained results.

4.1 Metrics

TSDolly outputs information about each program that was sampled and analyzed. From that information, we collect aggregate metrics that allow us to evaluate TSDolly, such as the number of analyzed programs that had compilation errors. More speciﬁcally, we collect the following metrics:

• Number of Alloy instances generated by step 1.

• Number of instances that were sampled for analysis and converted into TypeScript programs.

• Number of TypeScript programs that compiled successfully.

• Percentage of successfully compiling TypeScript programs from total TypeScript programs.

• Number of TypeScript programs to which the target refactoring could be applied (at least once). In other words, the number of refactorable programs.

• Percentage of refactorable TypeScript programs from total TypeScript programs.

Those metrics reﬂect our goal of generating programs that can be used to test if refactorings preserve behavior. The percentage of programs that are refactorable tells us how many of the generated programs can serve as input for the supported refactorings. The percentage of programs that compile without errors tells us how many of the generated programs can be used to test behavior preservation of refactorings, since we typically need compilable programs to deﬁne behavior preservation.

29 4.2 EXPERIMENTS 30

4.2 Experiments

In order to evaluate TSDolly, we ran it with a set of parameters and collected the previously described metrics. We examined the compilation errors of the generated programs and on the refactored programs. We also measured the execution time of each TSDolly run. The parameters with which we ran TSDolly are divided into variable parameters and ﬁxed parameters. Variable parameters are those that changed between each TSDolly execution, and the ﬁxed parameters did not change but are still relevant to our experiment.

4.2.1 Variable parameters For each execution of TSDolly, the parameters that changed were the target refactoring and the skip sampling argument. Since TSDolly targets a single refactoring at a time, we separately ran TSDolly for each of the five refactorings it supports (described in Section 3.3). For each of those refactorings, we ran TSDolly twice: once with a skip value of 25 and once with a skip value of 50. Choosing a skip 1 1 value of 25 and 50 means that only 4% ( 25 ) and 2% ( 50 ) of the Alloy instances generated in step 1 were sampled and processed by further steps (i.e. transformed into a TypeScript program and used as a refactoring input). Although we would like to process all of the generated instances, we chose those values because we had limited time to run TSDolly. So, we needed to sample the generated instances, and we chose the specific values of 25 and 50 because the execution time of TSDolly could then fit our time constraint. Moreover, the value of 25 has been used in a previous study that used JDolly [17]. Since we ran TSDolly twice for each refactoring, each time changing only the skip value, we could reuse the instances that TSDolly generates in step 1. This is possible because the instance generation step only depends on the target refactoring and on the Alloy specification file (which is fixed). For each refactoring, we first ran the instance generation (step 1) of TSDolly. The generated instances were serialized and saved, and then used as input when we ran the rest of TSDolly’s pipeline (steps 2 and 3) twice, once with skip 25, once with skip 50.

4.2.2 Fixed parameters For each execution of TSDolly, there were parameters that did not change but influence our experiments and results. One fixed parameter is the scope defined for the Alloy run commands we used in TSDolly. We have described in Section 3.4 that TSDolly generates instances for a target refactoring by telling the Alloy Analyzer to execute a run command associated with a predicate for the target refactoring. The Alloy Analyzer then finds instances that satisfy our Alloy specification and the run command’s predicate. Since the Alloy Analyzer is a bounded model finder, it only find instances up to a certain size. This maximum size is determined by the scope argument of the run command. This number is an upper bound on the number of atoms of each signature that can be present in an instance. We ran all run commands with a scope of two. This means that there can only be at most two atoms of each signature. We chose a scope of two because a scope of one generates 4.3 RESULTS 31 programs that are too trivial (for instance, there cannot be two classes where one extends the other), but a scope higher than two generated a number of instances in the order of millions, far more than we had the resources to process and analyze in the rest of TSDolly’s pipeline. When using the Alloy Analyzer API in step 1, we used the MiniSat [18] SAT solver. Some of our fixed parameters are TypeScript compiler and language configurations. The first of those parameters is the TypeScript compiler options. The TypeScript compiler can be configured with options [12] that impact project structure, type checking, code emitting, etc. In our experiments, we only explicitly specified two options:

• “strict”: we set this option to “true”. This enables the strictest level of type checking rules.

• “noEmit”: we set this option to “true”. This disables the code emitting portion of the TypeScript compiler, allowing us to use it only for type checking and for its refactoring API.

We left the other options unspeciﬁed, therefore, using their default values.

The other fixed parameter is TypeScript’s user preferences. The getApplicableRef c actors and getEditsForRefactor functions from TypeScript’s language service API, used in step 3 of TSDolly, both expect an argument of type UserPreferences. This type represents user configurations that the editor passes as a parameter when calling the language service API. In our experiments, we only explicitly specified one option: the allowTextC c hangesInNewFiles option is set to “true”. This is necessary because one of the supported refactorings, “Move to a new file”, needs this option to be true so that the refactoring can be applied. To build and run the Java part of TSDolly (step 1), we used OpenJDK version 11.0.8. For the TypeScript part of TSDolly (steps 2 and 3), we compiled the code to JavaScript using the TypeScript compiler version 4.0.2 and then used the Node.js runtime version 14.9.0. Lastly, we ran the experiments on a computer with the following configuration:

• Operating System: Microsoft Windows 10 Home

• Processor: Intel®CoreTM i7-7700HQ @2.80GHz

• RAM: 16GB, 2400 MHz

4.3 Results

As we described in the previous section, we ran TSDolly twice for each of the ﬁve supported refactorings, once with skip value of 25 and once with skip value of 50. For each run, we collected the metrics described in Section 4.1, and we also measured execution time and inspected the output data that TSDolly produced. Table 4.1 shows the metrics we obtained. The results show that all generated programs that we analyzed could be refactored. One reason for that is that our speciﬁcation of the TypeScript language describes a small subset of 4.3 RESULTS 32

Refactoring Generated Sampled Compilable Refactorable instances Skip 25 Skip 50 Skip 25 Skip 50 Skip 25 Skip 50 Convert 315,310 12,612 6,306 12,379 6,183 12,612 6,306 parameters (4%) (2%) (98.15%) (98.05%) (100%) (100%) to destructured object Convert to 239,399 9,576 4,788 9,491 4,748 9,576 4,788 template (4%) (2%) (99.11%) (99.16%) (100%) (100%) string Generate 546,968 21,879 10,939 21,342 10,660 21,879 10,939 ‘get’ (4%) (2%) (97.55%) (97.45%) (100%) (100%) and ‘set’ accessors Extract 406,096 16,244 8,122 15,854 7,935 16,244 8,122 Symbol (4%) (2%) (97.60%) (97.70%) (100%) (100%) Move to 499,038 19,962 9,981 19,522 9,755 19,962 9,981 a new ﬁle (4%) (2%) (97.80%) (97.74%) (100%) (100%)

Table 4.1 Metrics collected during TSDolly experiments. ‘Generated instances’ is the number of instances generated in step 1. ‘Sampled’ is the number of instances sampled for the next steps. ‘Compi- lable’ is the number of programs produced from the sampled instances that compile. ‘Refactorable’ is the number of programs that are refactorable at least once by the target refactoring. the language features. This means that the programs we generated are simple, so they do not trigger preconditions that prevent a refactoring from being applicable. Looking at the percentage of compiling programs, we see that it was similar between the skip values used. The percentage of compiling programs is high, above 97%. The same reason- ing of generating simple programs applies here: although we do not specify all type checking rules, our programs are simple enough that the rules we do specify lead to such a high compilation rate. To see why the percentage of compiling programs is not 100%, we inspected the compilation errors in the programs generated by TSDolly. There were two kinds of compilation error. The first kind consists of errors with the code 2416, which refers to method overriding. When a class A extends another class B, A can over- ride B’s methods, but the type of the overriding method must be assignable to the type of the overridden method. Our Alloy specification tries to express this constraint through a fact that says that two method declarations that share the same name and belong to the same class hier- archy must have compatible types. In this fact, method declarations have compatible types if 4.3 RESULTS 33 their parameters’ types form the same set. However, in TypeScript, the order of the parameters matter, so this fact is an approximation of the real TypeScript type checking rule. For some of the generated programs, even though the parameters of the two methods have the same set of types, their order is incompatible. For instance, one method could have parameters of type number and string, and the overriding method could have parameters of type string and number, in that order, making the overriding method’s type not assignable to the overridden method’s type. This kind of error happens because in our Alloy specification we don’t use ordered se- quences, but instead use sets, since sets are built into Alloy and easier to use. Therefore a method declaration’s parameters are simply a set, instead of a sequence of parameters. The other kind of errors are code 2345 errors. These are triggered when the type of an argument of a method or function call is not assignable to the declared type of the corresponding parameter. This compilation error happens in TSDolly-generated programs when a field is used as argument in a function or method call. Function and method parameters can be annotated with one of two types: string and number, since those are the ones present in our specification. Fields can be annotated with one of those same two types. However, fields are also marked as optional, to avoid a compilation error triggered by uninitialized fields. The actual type of a field is then a union type of the annotated type and the undefined type, where undefined is the type a variable has if it was not initialized. When a function or method is called with a field access as an argument, it triggers a compilation error because there is a mismatch between the type of the parameter as specified in the function or method declaration, which is string or number, and the type of the field used as argument, which is string | undefined or number | undefined (the | operator indicates type union). The error then happens because type string | undefined is not assignable to type string. There were no runtime exceptions when calling the TypeScript refactoring API, which means that checking the availability of refactorings and applying refactorings did not crash for the programs TSDolly analyzed. We used the TypeScript compiler as a test oracle on our experiments for comparing the analyzed program with their refactored versions. Using the TypeScript compiler, we checked if applying a refactoring to a program introduced a compilation error. Our evaluation discovered that the “Generate ‘get’ and ‘set’ accessors” refactoring introduced compilation errors. The compilation error introduced happened because the refactoring generates a getter functions with an inappropriate return type when applied to a field marked as optional. Since all of the fields are marked optional in TSDolly, this refactoring introduced a compilation error in all of the generated programs it was applied to.

To understand the error, consider the example in Listing 4.3.1. The FieldIdentifie c r_1 field is annotated with type string and is marked as optional (it has a question mark at the end). To the compiler, this field has type string | undefined (a union of the str c ing and undefined types). When we apply this refactoring, it generates a getter function, called FieldIdentifier_1, and modifies the field name to _FieldIdentifier_1. The return type of the getter function is string, the same as the type annotated in the field. However, the generated getter function returns the field, whose type is actually string | undefined due to it being optional. The compiler then produces an error of code 2322 with 4.3 RESULTS 34

the message “Type ‘string | undeﬁned’ is not assignable to type ‘string’, because the type s c tring | undefined of the ﬁeld access expression returned is not assignable to the type string declared as the getter function’s return type”.

1 class ClassIdentifier_0 {

2 private FieldIdentifier_1?: string;

3 }

1 class ClassIdentifier_0 {

2 private _FieldIdentifier_1?: string;

3 public get FieldIdentifier_1(): string {

4 return this._FieldIdentifier_1;

5 }

6 public set FieldIdentifier_1(value: string){

7 this._FieldIdentifier_1= value;

8 }

9 }

Listing 4.3.1: Example of the error introduced by the “Generate ‘get’ and ‘set’ accessors” refactoring. At the top is the code before the refactoring, with no compilation errors. At the bottom is the refactored code, and line 4 (highlighted) triggers the compilation error of code 2322.

We reported this behavior to the TypeScript team1 and they conﬁrmed it was a bug in the “Generate ‘get’ and ‘set’ accessors” refactoring implementation. We also measured the time to run TSDolly in our experiments. The results are in Table 4.2. We observe that generating the instances (running step 1 of TSDolly) only takes a few minutes, while analyzing even just a sample of those generated instances (running steps 2 and 3) takes hours. Steps 2 and 3 dominate the execution time of the TSDolly pipeline. We also see that the average execution time per program varies between refactorings. This is explained by some factors. Each refactoring implementation may take a different amount of time to execute. The programs we generate for each refactoring are also different, which can inﬂuence the time it takes to compile them. Lastly, some refactorings, such as “Extract Symbol”, offer more than one action to be applied per program, which increases the execution time of step 3 (applying refactorings).

1Bug report: https://github.com/microsoft/TypeScript/issues/40994. 4.3 RESULTS 35

Instance generation (step 1) Other steps (steps 2-3) Skip 25 Skip 50 Refactoring Total Avg. Total Avg. Total Avg. Convert 4min26s 843µs 16h50min 4.81s 8h13min 4.69s parameters to destructured object Convert 3min7s 783µs 12h19min 4.63s 5h59min 4.50s to template string Generate 9min57s 1092µs 32h50min 5.40s 13h36min 4.48s ‘get’ and ‘set’ accessors Extract 6min38s 980µs 59h32min 13.20s 29h27min 13.06s Symbol Move to 11min32s 1386µs 30h52min 5.57s 14h52min 5.37s a new ﬁle

Table 4.2 Execution time of TSDolly experiments. In ‘Instance generation’, ‘Total’ is the time to generate all instances for a refactoring. ‘Avg‘ is the average time per instance generated. In ‘Other steps‘, ‘Total’ is the time to execute steps 2 and 3 (building programs and applying refactorings) for all programs. ‘Avg’ is the average time per program built and refactored. CHAPTER 5 Conclusion

In this work, we presented a technique for generating TypeScript programs that can be used as input for testing TypeScript refactorings. This technique is implemented in the TSDolly tool. TSDolly uses the Alloy Analyzer and a specification of the TypeScript language written in Alloy to generate instances that represent TypeScript programs. Then, TSDolly builds Type- Script programs from those generated instances. Those programs are used as input for the five TypeScript refactorings that TSDolly currently supports. The results from our evaluation show that our technique indeed produces programs that can serve as inputs to the TypeScript refactorings we evaluated. All of the generated programs we analyzed could be refactored by their target refactoring. This means that TSDolly can successfully be used to generate programs that satisfy a refactoring’s preconditions. The majority of the analyzed generated programs compiled without errors (at least 97.45%). Those programs therefore could be used as test input in association with a test oracle for determining behavior preservation, since behavior preservation commonly assumes programs have no compilation errors. Even though the TSDolly-generated programs use only a small subset of the TypeScript language, we were able to find a bug in a TypeScript refactoring. The “Generate ‘get’ and ‘set’ accessors” refactoring introduced a compilation error when applied to some previously compilable programs. We found the bug by using the TypeScript compiler as a test oracle and the TSDolly-generated programs as test input. This suggests that TSDolly can generate useful programs for finding bugs in TypeScript refactoring implementations.

5.1 Related work

Our work is based on JDolly [3], a tool that generates Java programs for testing Java refactorings. JDolly uses an Alloy speciﬁcation for the Java language and the Alloy Analyzer to generate bounded-exhaustive instances that are then transformed into Java programs. The programs generated by JDolly have been successfully used to detect bugs in refactoring implemented by Eclipse, NetBeans, and the JastAdd Refactoring Tools [3, 17]. JDolly’s technique has also been adapted through CDolly [4], a tool that generates C programs that are used to test C refactorings. Our work differs from JDolly in two aspects, other than the fact we target the TypeScript language instead of Java. In TSDolly, we had to implement the instance generation step in Java and the rest of the steps in TypeScript, whereas JDolly is fully implemented in Java. In doing so, we had to implement a solution for communicating the Java program output to

36 5.1 RELATED WORK 37 the TypeScript program. Another difference is that the Java refactorings targeted by JDolly are focused on refactoring classes, such as “Rename class” and “Push down method”. The TypeScript refactorings we support in TSDolly, and in general the refactorings provided by the TypeScript compiler project, are not as focused on classes, due to the different nature of the language. Among the 5 refactorings we support, only the “Generate ‘get’ and ‘set’ accessors” refactoring is focused on classes (fields more specifically). Daniel et al. [19] proposed a different approach for program generation for refactoring testing. They implemented a Java library called ASTGen. ASTGen provides generators and combinators that are used to generate Java programs. ASTGen uses a generative approach, which means that users must write how their ASTs should be constructed, in an imperative fashion. Another similar Java library used for program generation is UDITA [20]. UDITA extends ASTGen, and its users must also use Java as the language to specify how ASTs should be built. However, in addition to supporting a generative approach like ASTGen, UDITA also supports a filtering approach. Users can write predicates that specify what should be generated, and the generators are responsible for searching for satisfying combinations. Both ASTGen and UDITA require the programmer to specify programs through Java, an imperative language. In contrast, TSDolly and JDolly use Alloy, which is declarative. An in-depth comparison of strengths and weaknesses of each specification style remains as future work. Since our focus is on generating programs that are suitable for testing refactorings, an important goal is that programs successfully compile (or, in the case of a dynamically-typed language, that programs are semantically well-formed), because we can define behavior preservation for such programs. Among other approaches for program generation that aim to generate semantically well-formed programs, Kreutzer et al. [21] propose a language-agnostic technique that takes semantic rules into consideration and generates valid, compilable programs. Their technique consists of *Smith, a test program generation framework, and LaLa, a language based on attribute grammars. To generate programs, users use LaLa to specify both the syntactic and semantic rules of a language. Their goal is broader than ours, since they are interested in evaluating compilers. We take a focused approach as we model a subset of the TypeScript syntax and add predicates to constrain program generation for a particular refactoring, to generate suitable programs. CodeAlchemist [22] is a fuzz testing tool that generates JavaScript programs while avoid- ing semantics errors in addition to syntax errors. It works by taking existing JavaScript programs (seeds) and breaking them into fragments (code bricks). CodeAlchemist then generates JavaScript programs by combining code bricks. To avoid semantics errors, CodeAlchemist uses a new technique, called semantics-aware assembly, where each code brick has conditions that specify when they can be combined with other code bricks to generate semantically valid programs. Those conditions refer, for instance, to variable use and types. When combining code bricks, CodeAlchemist uses both static and dynamic analysis to determine what are the conditions for a code brick and if a code brick combination satisfies them. CodeAlchemist was used to test JavaScript engines, and was able to find bugs in V8, ChakraCore and JavaScriptCore. A different approach used to test refactorings is to use a corpus of existing programs as test 5.2 FUTURE WORK 38 input, instead of program generation. Gligoric et al. [23] have applied this method to test Java refactorings. They collected eight open-source projects and used them as input for C and Java refactorings implemented by Eclipse, successfully finding new bugs. Using a corpus of programs has the advantage that there’s no need to write a program generator. The bugs found also have the advantage of occurring in real programs, whereas bugs found by program generators might be rarely-occurring corner cases that refactoring developers do not need to prioritize. However, using real programs can increase the cost of checking refactoring correctness, especially if they are large. For JavaScript and TypeScript, a further complication is figuring out how to run a real program due to the diversity of tools and stan- dards in the ecosystem. For instance, we need to be mindful of what environment a project targets (e.g. browser or Node.js), the environment version, the language version, dependency management, and tools such as the Babel transpiler. Those aspects can be controlled if we use generated programs.

5.2 Future work

As we have seen in Chapter 3, TSDolly generates programs based on a specification of a small subset of the TypeScript language. As future work, we could increase the subset of TypeScript specified in Alloy, adding new language features. A related future improvement would be to extend TSDolly to support more TypeScript refactorings. Those improvements would allow the TSDolly-generated programs to be used for (possibly) finding more bugs. We could also extend TSDolly to support testing TypeScript’s codefixes, which are program transformations whose goal is to fix a compilation error or compilation warning. Another direction for future work is to increase TSDolly’s efficiency. As we see in our results, even with a reduced scope, the number of Alloy instances generated for our TypeScript specification is in the order of hundreds of thousands, and this number could drastically increase as more language features are added. We could reduce the number of generated instances by carefully selecting which language features should be present in programs generated for testing a given refactoring, as not all language features may be relevant to that refactoring. A second improvement to TSDolly’s efficiency could be achieved by finding what is the optimal skip number that allows us to only test a small rate of the generated programs without losing the ability to find bugs. We could implement test oracles for testing refactoring properties, for instance, a behavior preservation oracle. A test oracle would allow us to use TSDolly generated programs as input for testing more properties of refactoring implementations. As another future work, we could further investigate the quality of the programs generated by our technique. We would like to use metrics other than compilation rate and refactorability rate, such as program similarity, to increase our confidence that the generated programs are suitable for testing refactorings. Lastly, in the future, we would like to compare our work with other approaches used for program generation and refactoring testing. We would like to compare our use of Alloy and Alloy Analyzer with other space search algorithms for the task of generating programs suitable for testing refactorings. We would also like to compare aspects such as ability to find bugs, 5.2 FUTURE WORK 39 ease of use for developers and execution time. Bibliography

[1] Martin Fowler. Refactoring: improving the design of existing code. Addison-Wesley Professional, 2018.1, 3.3.3, 3.3.4, 3.3.4

[2] William F Opdyke. Refactoring object-oriented frameworks. 1992.1

[3] G. Soares, R. Gheyi, and T. Massoni. Automated Behavioral Testing of Refactoring En- gines. IEEE Transactions on Software Engineering, 39(2):147–162, 2013.1, 5.1

[4] M. Mongiovi. Scaling Testing of Refactoring Engines. In 2016 IEEE/ACM 38th Interna- tional Conference on Software Engineering Companion (ICSE-C), pages 674–676, 2016. 1, 5.1

[5] Daniel Jackson. Software Abstractions - Logic, Language, and Analysis, Revised Edition. The MIT Press, 2012.1, 2.1.1

[6] Gustavo Soares, Rohit Gheyi, Dalton Serey, and Tiago Massoni. Making program refactoring safer. IEEE software, 27(4):52–57, 2010.1

[7] Alloy tools. URL: https://alloytools.org. Accessed: 15 September 2020. 2.1, 2.1.2

[8] Alexandr Andoni, Dumitru Daniliuc, Sarfraz Khurshid, and Darko Marinov. Evaluating the “small scope hypothesis”. In Popl, volume 2. Citeseer, 2003. 2.1.2

[9] TypeScript. URL: https://www.typescriptlang.org/. Accessed: 16 Septem- ber 2020. 2.2

[10] Gilad Bracha. Pluggable type systems. In OOPSLA Workshop on Revival of Dynamic Languages, 2004. 2.2

[11] TypeScript Compiler Project. URL: https://github.com/microsoft/ TypeScript. Accessed: 26 October 2020. 2.2

[12] TSConﬁg. URL: https://www.typescriptlang.org/tsconfig. Accessed: 20 October 2020. 3.2, 4.2.2

[13] Alloy API documentation. URL: http://alloytools.org/documentation/ alloy-api/index.html. Accessed: 15 September 2020. 3.4

40 BIBLIOGRAPHY 41

[14] JSON Schema. URL: https://json-schema.org/. Accessed: 22 October 2020. 3.5

[15] Node.js. URL: https://nodejs.org/en/. Accessed: 26 October 2020. 3.5

[16] Node.js Stream API. URL: https://nodejs.org/docs/latest-v14.x/api/ stream.html. Accessed: 17 September 2020. 3.5

[17] M. Mongiovi, R. Gheyi, G. Soares, M. Ribeiro, P. Borba, and L. Teixeira. Detecting Overly Strong Preconditions in Refactoring Engines. IEEE Transactions on Software Engineering, 44(5):429–452, 2018. 4.2.1, 5.1

[18] MiniSat. URL: http://minisat.se/Main.html. Accessed: 27 October 2020. 4.2.2

[19] Brett Daniel, Danny Dig, Kely Garcia, and Darko Marinov. Automated testing of refactoring engines. In Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, pages 185–194, 2007. 5.1

[20] Milos Gligoric, Tihomir Gvero, Vilas Jagannath, Sarfraz Khurshid, Viktor Kuncak, and Darko Marinov. Test Generation through Programming in UDITA, page 225–234. Asso- ciation for Computing Machinery, New York, NY, USA, 2010. 5.1

[21] P. Kreutzer, S. Kraus, and M. Philippsen. Language-Agnostic Generation of Compil- able Test Programs. In 2020 IEEE 13th International Conference on Software Testing, Validation and Veriﬁcation (ICST), pages 39–50, 2020. 5.1

[22] HyungSeok Han, DongHyeon Oh, and Sang Kil Cha. CodeAlchemist: Semantics-Aware Code Generation to Find Vulnerabilities in JavaScript Engines. In NDSS, 2019. 5.1

[23] Milos Gligoric, Farnaz Behrang, Yilong Li, Jeffrey Overbey, Munawar Haﬁz, and Darko Marinov. Systematic testing of refactoring engines on real software projects. In Giuseppe Castagna, editor, ECOOP 2013 – Object-Oriented Programming, pages 629–653, Berlin, Heidelberg, 2013. Springer Berlin Heidelberg. 5.1