PORTING 100K LINES OF CODE TO TYPESCRIPT MILLIE MACDONALD tiny.cloud SOFTWARE ENGINEER @ TINY The Talk ● About Tiny and me ● The Conversion Project: ○ Motivation ○ Why TypeScript? ○ The conversion ○ Benefits, lessons learned and ongoing work TL;DR: Types help. Gradual typing really helps. About Tiny ● TinyMCE is a popular open source project By the numbers: ● A rich text editing platform that helped launch ● 1 million+ users on Cloud and scale the adoption of Medium, WordPress, ● 125,000 downloads/week Shopify, Tumblr, Atlassian, EventBrite & more ● 1,200 paying customers ● In 2015, we launched a startup with offices in Palo Alto, Australia and Sweden ● 270 contributors About Me ● From Brisbane, Australia ● Bachelor of Engineering (Honours), Software Engineering ● QA Intern then QA Engineer then Software Engineer at Tiny ● @metricjs on Github and Twitter “100k Lines of Code”? At the time of the conversion, TinyMCE was: ● Core = ~30k LoC JS -> TS ● User Interface = ~30k LoC ● 44 core plugins + several premium plugins = ~40k LoC Plus various other projects and libraries that get bundled in… The Conversion Project Before ● We tried and rejected many frameworks ● We use functional programming ● So we wrote our own libraries and frameworks + Avoided upgrade problems and framework churn + Easier for us to fix problems − FP + JS means defensive code -> code size and runtime performance issues ● Then ES6 was announced... Motivation for the Conversion Project ● Decided to convert to ES6 in 2017 ● Had been discussing alternative JS languages for a while ● Decided to do both conversions at once ● Our original wish list for an alt JS was: ○ static typing ○ pattern matching Why TypeScript? ● We looked at ReasonML, PureScript and TypeScript ● For each language we considered: ○ how much of the feature set we needed ○ the learning curve ○ the upfront development cost to switch Why TypeScript? ● ReasonML ○ Trialed it by rewriting our PowerPaste plugin for TinyMCE in ReasonML, replacing complicated regex with strong types and pattern matching + Syntax is similar to JS so easy to learn − Requires a full rewrite ● PureScript: ○ Was already used for some of our internal prototypes + Side-effect tracking speeds up code reviews and training, and optics improve runtime efficiency − Lots of FP features we don’t need on the client side Why TypeScript? ● Typescript − No pattern matching + Vanilla JS can be valid TypeScript + ES6 modules == TypeScript modules + We were already using our own module system with a strict syntax so conversion could be done with a script + Types! Even better: gradual types! What is Gradual Typing? ● Start with minimal types and add more over time ● Requires static and dynamic types - like TypeScript ● Minimal initial cost of conversion ● Flexible ongoing time/dev investment ● Can start with simple types and improve later The How Conversion Process for each Library 1. Run script to convert modules to ES6/TypeScript syntax 2. Manually check the script converted each file correctly 3. Update with changes from already-converted dependencies 4. Fix TypeScript problems found 5. Run and fix tests 6. Update build process The Conversion Script 1. Transpile modules using an Abstract Syntax Tree (AST) a. Change import syntax b. Format body of the file correctly c. Change export syntax and apply export default <any> 2. Move files and rename to .ts 3. Copy templates for configuration files 4. Generate Main.ts from api folder 5. Update package.json and .gitignore Module Conversion Example define( import { Dependency } from 'example'; 'Example Module', [ ‘example.Dependency' ], var bar = 0; function (Dependency) { var foo = function () { … }; var bar = 0; var foo = function () { … }; export default <any> { return { foo, foo, bar bar }; }; } ); Adding Types ● The script doesn’t add types ● We gradually started adding types to libraries ● We add types with each PR ○ New code must be at least partially typed ○ “If you touch it, type it” was the rule ● Invested time to fully type our most used libraries ● Now, many of our libraries are fully or at least partially typed Benefits of TypeScript and Gradual Typing Types ● Gained type-based tools e.g. find all usages and automated refactoring ● Compile time type checking ● Even simple types caught some serious bugs ● Typing our most used dependencies caught many more Example Bugs Missing comment resulted in accidental global Incorrect function argument list length Gradual Typing ● Allowed the team to go back to normal tasks quickly ● Able to type modules slowly and methodically by gradually removing placeholder types ● Can leave adding types until we have spare time ● Can type new code without having to type the old code around it ● Type inference tools allow for some automated typing ● Types make it easier to read and debug code, and to train new devs Lessons Learned Convert Common Libraries First ● Our modular architecture means we have a lot of libraries and dependencies ● We mostly converted the most used dependencies first which allowed us to cascade their types and bug fixes through other libraries ● But some other libraries were done standalone and they have duplicate type definitions, clashing types, etc. Gradually Introduce Compiler Features A couple times we enabled multiple new compiler features without testing them properly on the libraries... Write a Script to Test Everything ● Write a script that does every kind of check it can - types, linting, build, tests, etc. ● Run it when you make a big change ● Ours is called the Really Cool ScriptTM ○ Contains a list of all our libraries ○ Clones each one, installs its dependencies, runs tsc and npm test Developer Adjustments ● Discuss changes in coding style, tools and processes ● Communicate changes and their impact ● Add scripts that check types, etc. without devs having to remember complicated commands Continuing Work Continuing Work ● More types! ● Enabling more compiler features ● Building a type definition file for TinyMCE’s editor API After Typescript ● Typescript is great but it is still mostly Javascript ● Sometimes you really need strict types, pattern matching, etc. to ensure code quality ● So... RTC and ReasonML ● Tiny is working on Real Time Collaboration! ● Client-side is entirely in ReasonML ● More info coming soon! Check out TinyMCE! ● tiny.cloud ● github.com/tinymce/tinymce Keep in touch: ● metricjs on Twitter and Github Thank you! Questions?.