To ESM is human The Journeyman’s Guide to Modules in Node.js Ujjwal Sharma @ryzokuken!1 Ujjwal Sharma (@ryzokuken) •Node.js – Core Collaborator •Electron – Maintainer (Upgrades WG) •V8 Contributor •TC39 Contributor •Google Summer of Code •Student •Speaker !2 @ryzokuken What is a module? !3 @ryzokuken module noun. /ˈmɒdjuːl/ Modules are the fundamental building blocks of the code structure. !4 @ryzokuken But how did we get here? Let’s talk history. !5 @ryzokuken March JavaScript 1.0 released 1996 !6 @ryzokuken Era of the <script> tags begins. !7 @ryzokuken JavaScript code too long? Try breaking up your code into multiple files. !8 @ryzokuken Benefits: • Basic Modularity Drawbacks: • No Dependency Resolution • Namespace Pollution !9 @ryzokuken Feb 2005 PrototypeJS !10 @ryzokuken Aug 2006 !11 @ryzokuken Oct 2009 !12 @ryzokuken Apr 2012 !13 @ryzokuken 999+ global variables Use IIFEs? 1 global variable !14 @ryzokuken Benefits: • Basic Modularity • (Less) Namespace Pollution Drawbacks: • No Dependency Resolution • Namespace Pollution • Messy Syntax? !15 @ryzokuken !16 2009 “JavaScript on the server” !17 @ryzokuken ServerJS ➡ CommonJS !18 @ryzokuken Benefits: • Proper Modularity • Independent of HTML Drawbacks: • Synchronous require !19 @ryzokuken Fact Node.js � CommonJS Frontend Implementations: •Browserify •Webpack !20 @ryzokuken CommonJS � Frontend JavaScript Idea Let’s make a proposal! Module/Transfer/C !21 @ryzokuken circa Asynchronous Module Definition 2010 !22 @ryzokuken Oct 2010 !23 @ryzokuken Oct 2010 starts using AMD !24 @ryzokuken James Burke, dissatisfied with how Dojo worked, created 2010 !25 @ryzokuken All problems are solved! � Let’s go home. !26 @ryzokuken Not quite. !27 @ryzokuken Problems before AMD and CJS: •No dependency resolution •Namespace pollution !28 @ryzokuken Problems after AMD: •Verbosity •Strictness pertaining to order •#justHTTP1.1Things !29 @ryzokuken Nobody: JS Developers: We will just use CJS on the browser. !30 @ryzokuken AMD Creators: *Create AMD* JavaScript Developers: !31 @ryzokuken !32 @ryzokuken Rise of Module Bundlers Write CJS, get a single bundle file !33 @ryzokuken Now everyone uses CJS on both the server and the browser? !34 @ryzokuken Congratulations! Order has been restored to the galaxy. !35 @ryzokuken Module Authors: But how do I enable people to use my module without worrying about which format? !36 @ryzokuken Late Universal Module Definition 2011 !37 @ryzokuken !38 @ryzokuken TC39 !39 @ryzokuken July ECMAScript Modules 2014 !40 @ryzokuken � Static Imports � ESM ⚔ CJS ESM ⚔ Node.js � Support for Dynamic Imports � !41 @ryzokuken � Static Imports � 1. Static Analysis 2. Tree Shaking Dynamic variant !42 @ryzokuken Q: So everything was perfect, right? A: Not Quite. !43 @ryzokuken � ECOSYSTEM DISRUPTION � !44 @ryzokuken !45 @ryzokuken Now let’s see how it all works behind the scenes !46 @ryzokuken Fun Fact The modules systems in Node.js are mostly written in JavaScript !47 @ryzokuken VM Fundamentals !48 @ryzokuken Isolate A VM instance with its own heap. Electron Chrome Node.js Chrome Node.js V8 V8 V8 !49 @ryzokuken Q: What’s the job of the Isolate? A: Heap allocation and garbage collection, mainly. !50 @ryzokuken Aside: How does it all work? !51 @ryzokuken Isolate C++ Stack Heap Object Object Object Handle Scope Pointer?Handle Handle Handle Handle ??? JS Land !52 @ryzokuken Context A sandboxed execution context with its own set of built-in objects and functions. Isolate JavaScript Context Code Context !53 @ryzokuken Process WorkerThread WorkerThread WorkerThread Eventlibuv Loop Eventlibuv Loop Eventlibuv Loop Heap+GCIsolateV8 Heap+GCIsolateV8 Heap+GCIsolateV8 Node.jsnode Environment stdlib Node.jsnode Environment stdlib Node.jsnode Environment stdlib JSContext builtins JSContext builtins JSContext builtins JSContext builtins JSContext builtins JSContext builtins JSContext builtins JSContext builtins JSContext builtins !54 @ryzokuken How do Loaders work? •Fetch •Transform •Evaluation Hook !55 @ryzokuken Resolve Cached? Instantiate Return Cache !56 @ryzokuken CommonJS Loader !57 @ryzokuken CommonJS has Synchronous Load + Inline Execution !58 @ryzokuken require � Module.prototype.require !59 @ryzokuken Module.prototype.require � Module._load !60 @ryzokuken 1.filename := Module._resolveFilename(…) !61 @ryzokuken 1.filename := Module._resolveFilename(…) 2.cachedModule := Module._cache[filename] 3.if cachedModule: return cachedModule.exports !62 @ryzokuken 1.filename := Module._resolveFilename(…) 2.cachedModule := Module._cache[filename] 3.if cachedModule: return cachedModule.exports 4.[if nativeModule: return mod.compileForPublicLoader(...)] !63 @ryzokuken 1.filename := Module._resolveFilename(…) 2.cachedModule := Module._cache[filename] 3.if cachedModule: return cachedModule.exports 4.[if nativeModule: return mod.compileForPublicLoader(...)] 5.module := new Module(filename, …) !64 @ryzokuken 1.filename := Module._resolveFilename(…) 2.cachedModule := Module._cache[filename] 3.if cachedModule: return cachedModule.exports 4.[if nativeModule: return mod.compileForPublicLoader(...)] 5.module := new Module(filename, …) 6.Module._cache[filename] = module !65 @ryzokuken 1.filename := Module._resolveFilename(…) 2.cachedModule := Module._cache[filename] 3.if cachedModule: return cachedModule.exports 4.[if nativeModule: return mod.compileForPublicLoader(...)] 5.module := new Module(filename, …) 6.Module._cache[filename] = module 7.module.load(filename) 8.if throw, delete from cache 9.otherwise return module.exports !66 @ryzokuken 1.filename := Module._resolveFilename(…) 2.cachedModule := Module._cache[filename] 3.if cachedModule: return cachedModule.exports 4.[if nativeModule: return mod.compileForPublicLoader(...)] 5.module := new Module(filename, …) 6.Module._cache[filename] = module 7.module.load(filename) 8.if throw, delete from cache 9.otherwise return module.exports !67 @ryzokuken Module.load 1. Find appropriate extension. 2. Take appropriate action. !68 @ryzokuken And what exactly does Module._compile do anyway? !69 @ryzokuken ❌ !70 @ryzokuken ✅ vm.compileFunction ✅ !71 @ryzokuken ES Modules Loader !72 @ryzokuken ES Modules have Asynchronous Load + Synchronous Execution !73 @ryzokuken !74 @ryzokuken And guess what? Loader.getModuleJob is pretty much exactly inline with CJS !75 @ryzokuken 1.this.resolve(…args) 2.job := this.moduleMap.get(url) 3.If job: return job 4.job = new ModuleJob(…) 5.moduleMap.set(url, job) 6.return job !76 @ryzokuken And what’s inside ModuleJob? � !77 @ryzokuken Interop in a nutshell • Dynamic Imports • Cyclic Imports • … !78 @ryzokuken !79 @ryzokuken Special Thanks • Myles Borins (@MylesBorins) • Anna (@addaleax) • Shelley Vohr (@codebytere) • Dmitry Makhnev (@DmitryMakhnev) • The Organizers !80 @ryzokuken Спасибо! !81.