Home , SpiderMonkey

Jalangi: A Dynamic Analysis Framework for JavaScript

Koushik Sen University of California, Berkeley Joint work with Tasneem Brutch, Sash Chandra, Simon Gibbs, Liang Gong, Simon Jenson, Swaroop Kalasapur, Michael Pradel, Manu Sridharan Why JavaScript? • The RedMonk Programming Language Rankings (Popularity): January 2015 – Based on projects hosted at GitHub and quesons posted at StackOverﬂow Why JavaScript? Growth in popularity (based on jobs available) from 2012 – 2013

Source: http://blog.learntoprogram.tv/five-resons-javascript-important-programming-language-learn/ Why JavaScript? • Client-side JavaScript in Rich Web Applicaons

• Desktop Apps (Windows 8 and Gnome), Firefox OS, Tizen OS • Server-side (node.js) – Paypal, Ebay, Uber, NYmes, Linkedin, and many more • Assembly Language for the Web: emscripten, coﬀeescript, TypeScript • A language to implement DSL frameworks – Angular.js, Knockout.js, React.js

Why JavaScript? • Huge ecosystem of libraries and frameworks • JavaScript has low learning curve – people can start coding and get results quickly • No special installaon/execuon environment – Just use a modern browser • JavaScript supports funconal programming – higher order funcons • Modern JavaScript VMs are fast

Atwood’s Law

“Any applicaon that can be wrien in JavaScript, will eventually be wrien in JavaScript.” Why Tools for JavaScript? • JavaScript has its quirks (many)

1096 pages 176 pages Why Tools for JavaScript? var x = “1”; var x = “1”; ++x; x += 1; console.log(x); console.log(x);

Why Tools for JavaScript? var x = “1”; var x = “1”; ++x x += 1; console.log(x); console.log(x);

// prints 2 // prints 11 Why Tools for JavaScript?

• Easy to introduce bugs: correctness, performance, memory – Degrees of equality == vs. === • Loosely-typed – forgiving: implicit type conversion – tries hard to execute without throwing excepon • Like HTML • Highly reﬂecve – eval any dynamically created string • Asynchronous programming

Motivating Example

4 Motivating Example Motivating Example

Arithmetic operations on non-numbers may yield NaNs

4 Tools for Bug Finding and Security Analysis

• Remarkable progress in program-analysis and constraint solving – Commercial tools: Coverity, Klocwork, Grammatech, TotalView, Parallocity, Stac Device Veriﬁer from Microso, WALA at IBM – Open-source tools: GDB, lint, FindBugs, Valgrind – Academic tools: SLAM, BLAST, ESP, JPF, Bandera, Saturn, MAGIC, DART, CUTE, jCUTE – Mostly focused on C/C++ and Java programs • Hardly any soware quality tool for JavaScript and HTML5 – Stac analysis is diﬃcult for dynamic languages Jalangi

A powerful browser-independent (dynamic) analysis framework for JavaScript hps://github.com/Samsung/jalangi2

• Jalangi: A selecve record-replay and dynamic analysis framework for JavaScript. Koushik Sen, Swaroop Kalasapur, Tasneem Brutch, and Simon Gibbs. In ESEC/FSE, 2013.

Jalangi: Goals and Requirements

• Framework for Dynamic and hybrid Stac/Dynamic analysis – supports symbolic execuon, memory analysis, runme type analysis, value tracking, taint tracking, performance analysis • Handle ALL dynamic features – not OK to ignore eval, new Funcon • Independent of browser – source-to-source code instrumentaon – instrumented program when executed performs analysis • Easy Implementaon of Dynamic Analysis – Observe an execuon passively: (convenonal dynamic analysis) – Modify semancs/values – Repeatedly execute arbitrary paths within a funcon Why not Modify a Browser? • Hard to keep up with browser development • Harder to get people to use of customized browser Jalangi 1 and 2 • Jalangi 1: – hps://github.com/SRA-SiliconValley/jalangi – record execuon and replay to perform analysis – Shadow values (wrapped objects) – No longer supported • Jalangi 2: – hps://github.com/Samsung/jalangi2 – no record/replay or shadow values – acve development How Jalangi Works?

JavaScript Jalangi User Written and HTML Runtime Analysis

Jalangi

Analysis Writer

Intermediate How Jalangi Works?

JavaScript Jalangi User Written and HTML Runtime Analysis

Jalangi Source Instrumentor Information

Instrumented Files

Jalangi

Analysis Writer

Intermediate How Jalangi Works?

JavaScript Jalangi User Written and HTML Runtime Analysis

Jalangi Source Instrumentor Information

Instrumented Output Execute in Files Data Browser/Node.js

Trace Jalangi

Analysis Writer

Intermediate How Jalangi Works?

JavaScript Jalangi User Written Final and HTML Runtime Analysis Output

Jalangi Source Visualize Instrumentor Information Output

Instrumented Output Execute in Files Data Browser/Node.js

Offline Trace Analysis Jalangi

Analysis Writer

Intermediate Jalangi Instrumentaon (simpliﬁed) x = y + 1 ) x = Write(“x”, Binary(‘+’,Read(“y”, y), Literal(1), x) a.f = b.g ) PutField(Read(“a”, a), “f”, GetField(Read(“b”, b), “g”)) if (a.f()) … ) if (Branch(Method(Read(“a”, a), “f”)())) … Jalangi Runme funcon Binary(op, le, right, ...) {

result = le op right;

return result; } Jalangi Runme funcon Binary(op, le, right, ...) {

var aret = analysis.binaryPre(op, le, write, ...);

result = le op right; aret = analysis.binary(op, le, right, result, ...);

return result; } Jalangi Runme funcon Binary(op, le, right, ...) { var skip = false; var aret = analysis.binaryPre(op, le, write, ...); if (aret) { op = aret.op; le = aret.le; right = aret.right; skip = aret.skip; }} if (!skip) result = le op right; aret = analysis.binary(op, le, right, result, ...);

funcon invokeFunPre (iid, f, base, args, isConstructor, isMethod, funconIid); funcon funconEnter (iid, f, dis, args); funcon invokeFun (iid, f, base, args, result, isConstructor, isMethod, funconIid); funcon funconExit (iid, returnVal, wrappedExceponVal); funcon literal (iid, val, hasGeerSeer); funcon scriptEnter (iid, instrumentedFileName, originalFileName); funcon forinObject (iid, val); funcon scriptExit (iid, wrappedExceponVal); funcon declare (iid, name, val, isArgument, argumentIndex, isCatchParam); funcon binaryPre (iid, op, le, right, isOpAssign, isSwitchCaseComparison, isComputed); funcon getFieldPre (iid, base, offset, isComputed, isOpAssign, isMethodCall); funcon binary (iid, op, le, right, result, isOpAssign, isSwitchCaseComparison, isComputed); funcon getField (iid, base, offset, val, isComputed, isOpAssign, isMethodCall); funcon unaryPre (iid, op, le); funcon putFieldPre (iid, base, offset, val, isComputed, isOpAssign); funcon unary (iid, op, le, result); funcon putField (iid, base, offset, val, isComputed, isOpAssign); funcon condional (iid, result); funcon read (iid, name, val, isGlobal, isScriptLocal); funcon instrumentCodePre (iid, code); funcon write (iid, name, val, lhs, isGlobal, isScriptLocal); funcon instrumentCode (iid, newCode, newAst); funcon _return (iid, val); funcon endExpression (iid); funcon _throw (iid, val); funcon endExecuon(); funcon _with (iid, val); funcon runInstrumentedFunconBody (iid, f, funconIid); funcon onReady (cb);

• Each analysis needs to implement a subset of these callbacks. • Mulple analyses classes can be chained funcon binaryPre (iid, op, le, right, isOpAssign, isSwitchCaseComparison, isComputed); funcon binary (iid, op, le, right, result, isOpAssign, isSwitchCaseComparison, isComputed);

Sample analysis: check if undeﬁned is concatenated with a string

(funcon (sandbox) { funcon MyAnalysis () {

this.binary = funcon(iid, op, le, right, result){ if (op === '+' && typeof result==='string' && (le===undefined || right===undefined)) console.log(“Concatenated undefined with string at ”+ J$.iidToLocaon(J$.sid, iid)); } } sandbox.analysis = new MyAnalysis(); }(J$));

Sample analysis: count branches (funcon (sandbox) { this.endExecuon = funcon () { var trueBranches = {}; print(trueBranches, “True”); var falseBranches = {}; print(falseBranches, “False”);

funcon MyAnalysis() { } } this.condional(iid, result) { sandbox.analysis = new MyAnalysis(); }(J$)); var id = J$.getGlobalIID(iid); if (result) funcon print(map, str) { trueBranches[id]++; for (var id in map) else if (map.hasOwnProperty(id)){ falseBranches[id]++; console.log(str+ “ branch taken at ” + } J$.iidToLocaon(id)+ “ ” +maps[id] + “ mes”; } } Sample analysis: count number of objects allocated at each site

(funcon (sandbox) { this.endExecuon = funcon () { var allocCount= {}; print(allocCount); funcon MyAnalysis() { } this.literal = funcon (iid, val) { } var id = J$.getGlobalIID(iid); sandbox.analysis = new MyAnalysis(); if (typeof val === ‘object’) }(J$)); allocCount[id]++; }; funcon print(map) { this.invokeFunPre = funcon (iid, f, for (var id in map) base, args, isConstructor) { if (map.hasOwnProperty(id)){ var id = J$.getGlobalIID(iid); console.log(“ Object allocated at ” + if (isConstructor) J$.iidToLocaon(id)+“=”+maps[id]); allocCount[id]++; } }; } Sample analysis (modify semancs): interpret ‘-’ as ‘*’

(funcon (sandbox) { funcon MyAnalysis() { this.binaryPre = funcon (iid, op, le, right) { if (op === ‘-’) return {op: op, le: le, right: right, skip: true}; };

this.binary = funcon (iid, op, le, right, result) { if (op === ‘-’) return {result: le * right}; }; } sandbox.analysis = new MyAnalysis(); }(J$));

Sample analysis (modify semancs): skip execuon of an evil funcon

(funcon (sandbox) { funcon MyAnalysis() {

this.invokeFunPre = funcon (iid, f, base, args) { if (f === evilFuncon) return {f: f, base: base, args: args, skip: true}; }; } sandbox.analysis = new MyAnalysis(); }(J$));

Sample analysis (modify semancs): loop a funcon body

funcon loop(n) { var ret = ret? ret-1: n; // do something console.log(ret); return ret; } loop(10);

Sample analysis (modify semancs): loop a funcon body

funcon loop(n) { var ret = ret? ret-1: n; // do something console.log(ret); return ret; } loop(10);

Sample analysis (modify semancs): loop a funcon body (funcon (sandbox) { funcon MyAnalysis() { this.funconExit = funcon (iid, rv, ex) { return {returnVal: rv, wrappedExceponVal: ex, isBacktrack: rv?true:false}; }; } sandbox.analysis = new MyAnalysis(); }(J$)); ------Program ------funcon loop(n) { var ret = ret? ret-1: n; // do something console.log(ret); return ret; } loop(10);

Sample analysis (modify semancs): MulSE: Mul-Path Symbolic Execuon using Value Summaries • Symbolic execuon • Explore all paths in a funcon – but merge state from all paths before exing the funcon • Override default semancs to perform symbolic evaluaon • Backtrack within a funcon unl all paths are explored • Custom semancs and backtracking – for simple abstract interpretaon – for simple dataﬂow analysis Jalangi 2 Summary

• Observe an execuon and collect informaon • Change values used in an execuon • Change semancs of operators • Explore arbitrary path in a funcon • Re-execute the body of a funcon repeatedly • Maintain your own (abstract) state and call stack • 3x-100x slowdown Serious Analyses with Jalangi

• "TypeDevil: Dynamic Type Inconsistency Analysis for JavaScript,” – Michael Pradel and Parker Schuh and Koushik Sen (ICSE'15) • "JITProf: Pinpoinng JIT-unfriendly JavaScript Code," – Liang Gong and Michael Pradel and Koushik Sen (To appear at ESEC/FSE'15) • "MemInsight: Plaorm-Independent Memory Debugging for JavaScript,” – Simon Jensen and Manu Sridharan and Koushik Sen and Sash Chandra (To appear at ESEC/FSE'15) • "DLint: Dynamically Checking Bad Coding Pracces in JavaScript," – Liang Gong and Michael Pradel and Manu Sridharan and Koushik Sen (To appear at ISSTA'15) • "MulSE: Mul-Path Symbolic Execuon using Value Summaries,” – Koushik Sen and George Necula and Liang Gong and Wontae Choi, (To appear at ESEC/FSE'15) • "The Good, the Bad, and the Ugly: An Empirical Study of Implicit Type Conversions in JavaScript," – Michael Pradel and Koushik Sen (To appear at ECOOP'15) • "EventBreak: Analyzing the Responsiveness of User Interfaces through Performance-Guided Test Generaon," – Michael Pradel and Parker Schuh and George Necula and Koushik Sen (OOPSLA'14) Serious Analyses with Jalangi

Michael Pradel and Parker Schuh and Koushik Sen (ICSE'15) TYPEDEVIL: DYNAMIC TYPE INCONSISTENCY ANALYSIS FOR JAVASCRIPT (JALANGI POWERED) Observaons

• Most code follows implicit type rules – A single type per variable – A single type per object property – Funcons have ﬁxed signatures • Many bugs are violaons of these rules

TypeDevil JavaScript program • Goals: ﬁnd inconsistent types of Gather type observaons – local variables – properes of objects Summarize observaons into – funcon signatures type graph

Find, merge, and ﬁlter inconsistencies

Warnings about inconsistent types

Evaluaon

• Setup: – Sunspider, Octane, and 7 web apps • Main results: – Finds relevant problems: • 33 warnings, 15 are relevant – Pruning and merging is crucial: • 578 warnings → 33 warnings Example

• SunSpider’s regexp-dna:

var dnaOutputStr; for (i in seqs) { dnaOutputStr += seqs[i].source; }

Problem: Incorrect string value “undeﬁnedGTAGG...” Liang Gong, Michael Pradel, Koushik Sen (To appear at ESEC/FSE 2015) JITPROF: PINPOINTING JIT- UNFRIENDLY JAVASCRIPT CODE (JALANGI POWERED) JIT-Unfriendly Code Paerns

• JIT can compile JavaScript programs to efficient code – based on the premise that dynamic features are used in a regular and systemac way • Programmers may use dynamic features in unexpected ways – prohibits profitable JIT opmizaons – poor performance – no JIT compilaon feedback to programmers • Need a tool that can idenfy JIT-unfriendly code • JIT-aware profiler A Movang Example

function Thing(flag) { if (!flag) { this.b = 4; this.a = 3; } else { this.a = 2; this.b = 1; } }

for (var i = 0; i < 1000000; i++) { runtime var o = new Thing(i%2); 1.38s result += o.a + o.b; }

49 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. A Movang Example

function Thing(flag) { if (!flag) { this.b = 4; this.a = 3; } else { this.a = 2; this.b = 1; } }

for (var i = 0; i < 1000000; i++) { runtime var o = new Thing(i%2); 1.38s result += o.a + o.b; }

50 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. A Movang Example

function Thing(flag) { if (!flag) { this.b = 4; this.a = 3; } else { 20% performance boost this.a = 2; this.b = 1; } }

for (var i = 0; i < 1000000; i++) { runtime var o = new Thing(i%2); 1.03s result += o.a + o.b; }

51 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. A Movang Example

for (var i = 0; i < 1000000; i++) { var o = new Thing(i%2); result += o.a + o.b; }

52 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. A Movang Example

for (var i = 0; i < 1000000; i++) { var o = new Thing(i%2); result += o.a + o.b; }

53 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. A Movang Example

54 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. JIT Engines: Common Assumpons • objects have fixed layout • undefined properes or array elements are not accessed • non-numeric values are not stored in mostly numeric arrays • conguous indices are used in arrays • all properes of an object are inialized in constructors • … and many more JIT Engines: Common Assumpons • objects have fixed layout • undefined properes or array elements are not accessed • non-numeric values are not stored in mostly numeric arrays • conguous indices are used in arrays • all properes of an object are inialized in constructors • … and many more JIT Engines: Common Assumpons • objects have fixed layout • undefined properes or array elements are not accessed • non-numeric values are not stored in mostly numeric arrays • conguous indices are used in arrays • all properes of an object are inialized in constructors • … and many more JITProf Overview

• Checks seven paerns – one analysis class for each paern • Each analysis associates and updates meta- data with each object and each locaon – e.g. counters with each locaon – hidden class with objects • Ranks warnings based on counters – number of me a code locaon involved in a paern got executed Evaluaon

• Setup – JavaScript benchmarks: Octane and SunSpider – Top 50 websites (Alexa) • Results – opmizaon opportunies in 15 out of 39 benchmark programs – 1.1% to 26.3% speedup on Firefox and Chrome crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

60 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var bin = Array(); ...

for(var i = 0; i < str.length * chrsz; i += chrsz) bin[i>>5] |= ... return bin; }

61 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var bin = Array(); ...

for(var i = 0; i < str.length * chrsz; i += chrsz) bin[i>>5] |= ... return bin; } Operations on undefined value

62 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var len = str.length * chrsz, len1 = len>>5; var bin = []; bin.length = len1; for(var i=0;i>5] = 0;} ... for(var i = 0; i < len; i += chrsz) bin[i>>5] |= ... return bin; }

Pre-allocate array space to eliminate loading undeclared array elements

63 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var len = str.length * chrsz, len1 = len>>5; var bin = []; bin.length = len1; for(var i=0;i>5] = 0;} ... for(var i = 0; i < len; i += chrsz) bin[i>>5] |= ... return bin; } Operations on numbers

Pre-allocate array space to eliminate loading undeclared array elements

64 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var len = str.length * chrsz, len1 = len>>5; var bin = []; bin.length = len1; Performance boost: for(var i=0;i>5] = 0;}

... for(var i = 0; i < len; i += chrsz) 1.5% bin[i>>5] |= ... return bin; 31% } Operations on numbers

Pre-allocate array space to eliminate loading undeclared array elements

65 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Liang Gong, Michael Pradel, Manu Sridharan, Koushik Sen (To appear at ISSTA 2015) DLINT: DYNAMICALLY CHECKING BAD CODING PRACTICES IN JAVASCRIPT (JALANGI POWERED) State of theState Art of the Art State of the Art Coding Problem Detecon Lint-like static checkersLint-like static checkers Lint-like static checkers ⌅ Rule-based checking • Lint-like stac checkers ⌅ Rule-based checking ⌅ Rule-based checking ⌅ Detect common– Rule-based checking mistakes ⌅ Detect⌅ Detect common common mistakes mistakes ⌅ Enforce coding– Detect common mistakes conventions ⌅ Enforce⌅ Enforce coding coding conventions conventions – Enforce coding convenons

5 5 5 The Problem

• Stac checking is limited – Approximates behavior – Lint tools favor precision over soundness • Common problems – Unknown types

– Unknown aliases – Unknown code Example • www.google.com/chrome, included from Modernizr:

for (i in props) { // props is an array prop = props[i]; before = mStyle.style[prop]; ... }

• Rule: Don’t iterate over arrays with for-in. • hps://github.com/Modernizr/Modernizr/pull/1419 DLint

• DLint: Dynamic lint-like checking of code quality rules for JavaScript • Formalized and implemented 28 rules – Counterparts of stac rules – Addional rules

• Analyses wrien in Jalangi • Empirical study: Comparison between stac and dynamic checking DLint Checkers DLint Checkers

DLint framework

Uncommon Types (6) values (3)

Inheritance (5) API misuse (8)

Language misuse (6) 12 Type-Related

• Avoid accessing the undeﬁned property.

var x; // undeﬁned var y = {}; y[x] = 23; Language Misuse

• Avoid seng properes of primives, which has no eﬀect.

var fact = 42; fact.isTheAnswer = true; console.log(fact.isTheAnswer); // undeﬁned Evaluaon

• Quesons – Warnings that are missed stacally? – Addional warnings for stacally checked rules? – Warnings vs. popularity? • Setup – 200 web sites (top 50 Alexa + others) – Standard benchmarks: Octane, SunSpider – Comparison to JSHint Summary: Stac vs. Dynamic

• DLint complements stac checkers: – 49 warnings per site that are missed stacally – Addional warnings for stac rules – Addional warnings from rules that cannot be checked stacally Problems Detected by DLint

• From craigslist.org:

if (document.body.style === "width:100\%") { ... } • Never succeeds because style is an object Problems Detected by DLint

• From Octane GB Emulator benchmark: var decode64 = ""; if (dataLength > 3 && dataLength % 4 == 0) { while (index < dataLength) { decode64 += String.fromCharCode(...) } if (sixbits[3] >= 0x40) { decode64.length -= 1; } } • No eﬀect because decode64 is a primive Problems Detected by DLint Problems Detected by DLint

26 Simon Jensen and Manu Sridharan and Koushik Sen and Sash Chandra (To appear at ESEC/FSE'15)

MEMINSIGHT: PLATFORM-INDEPENDENT MEMORY DEBUGGING FOR JAVASCRIPT (JALANGI POWERED) Memory Issue: Drag or Staleness

• Staleness: long gap between last use and garbage collecon • A high number of stale objects indicates a potenal problem • Staleness can be avoided by ensuring objects become unreachable • Typical causes of staleness: Closures, caches, and event listeners Other Memory Issues

Churn

Bloat Exisng Tools: Heap Snapshots

• Chrome developer tools offers a heap snapshot capture and diff funconality • Potenally more efficient, but, – No informaon on staleness – Can miss excessive churn – In general, can’t handle fine-grained me-varying properes

Memory Leak! Memory Leak - Details jQuery issue (A)$ (B)$ (C)$ (D)$ Life>me$ Client$ JavaScript$ Instrumentor$ Instrumented$ Run$ Trace$ Enhanced$ GUI$ code$ JavaScript$ analysis$ Trace$ analyses$ code$

Figure 5: MEMINSIGHT toolJalangi chain Trace Generaon 2. We show that the detailed information collected by DECLARE x,y,m; LASTUSE 2 at 5; ALLOCOBJ 2 at 1; RETURN at 7; MEMINSIGHT is useful for diagnosing and ﬁxing mem- 1 var x = {}; ory issues in real-world web applications. 2 var y = {}; WRITE x,2 at 1; LASTUSE 4 at 7; The rest of the paper is organized as follows. After outlin- 3 function m(p,q) ALLOCOBJ 3 at 2; WRITE x,0 at 8; 4 { WRITE y,3 at 2; UNREACHABLE ing the different phases of MEMINSIGHT in Sections 2–4 as 5 p.f = q; ALLOCFUN 4 at 3; 2 at 8; described above, Sections 5 and 6 respectively present a quan- 6 }; WRITE m,4 at 3; UNREACHABLE titative evaluation of MEMINSIGHT and case studies showing 7 m(x,y); CALL 4 at 7; 3 at end; its usefulness. Finally, Section 7 discusses related work. 8 x=null; DECLARE p = 2, UNREACHABLE 2. Trace Generation q = 3; 4 at end; PUTFIELD 2,"f",3 In principle, our memory analysis framework could be imple- at 5; mented in an entirely “online” fashion, with client analyses running while the target program is being exercised. How- Figure 6: A simple code example and the corresponding trace. ever, this approach could have very high analysis overhead, Red entries are added in the enhanced trace. adversely affecting the usability of the target program. Hence, our framework divides the work into two phases. A trace 1 var elem = document.createElement("div"); generation phase runs along with the target program, record- 2 div.innerHTML = "

Hello World!

"; ing relevant memory operations into a trace file. Then, client 3 document.getElementById("x").appendChild(elem); analyses run in an offline mode, based on the recorded trace. Here we first discuss the design of our trace format, crafted Figure 7: Example to illustrate handling of DOM-related code. to balance detail with analysis overhead. Then, we discuss our handling of uninstrumented code and the DOM in particu- cation entries introduce a unique identifier used to name the lar. We defer discussion of certain challenges in handling the corresponding object throughout the trace. We use a distinct JavaScript language to Section 3.3. entry type to identify function object allocation, used to enable proper handling of closures (see below). In our implementa- 2.1. Trace Design tion, LASTUSE entries include a timestamp and all appear at the end of the generated trace (since the last use is only known To enable client analyses like leak detection, we require that at the end of the program); a separate post-processing phase traces be sufficient to reconstruct object lifetimes, i.e., when inserts the entries at the appropriate slots. objects are created and become unreachable. Hence, traces must include records of each object allocation and each mem- 2.2. Uninstrumented Code ory write, both to variables and to object fields (“properties” in JavaScript parlance). As an optimization, we avoid logging MEMINSIGHT works robustly in the presence of uninstru- writes when the old and new values are both primitive, as mented JavaScript code or native code from the environment, such writes are irrelevant to a memory analysis. A delete e.g., DOM functions. Here, we detail our strategies for han- operation on an object property is modeled as a write of null.3 dling uninstrumented code and the DOM. To handle functions, the generator logs calls and returns, Uninstrumented Code In principle, uninstrumented code and also logs declarations of local variables to enable proper could arbitrarily mutate any memory locations to which it has scope handling. For leak detection, we also log the last use of access. Attempting to discover all such behavior via code each object, where an object is used when it is dereferenced or, instrumentation alone would be difficult or impossible, partic- for function objects, when it is invoked. We only log the last ularly since invocations of uninstrumented code may not be use of each object since we found that logging all uses was observable (e.g., a browser invoking an uninstrumented event prohibitively expensive, and last use information is sufficient handler). Furthermore, such conservative detection would for computing object staleness. require frequent traversals of the full heap visible to uninstru- Figure 6 shows the generated trace for a simple example. mented code, a very costly operation. Most entries includes a source location at the end. The allo- In practice, we have found a policy of only tracking refer- 3We do not yet model the effect of delete on the shape of the object, or ences created in instrumented code to strike a good balance physical object sizes in general; see “Limitations” in Section 5.1. between coverage of relevant behaviors and analysis overhead.

4 Object Lifemes

• From trace, model runme heap – Including call stack and closures • Reference counng to compute unreachability me – Handle cycles with Merlin algorithm [Hertz et al. ASPLOS’06] • Insert unreachability mes in the enhanced trace (A)$ (B)$ (C)$ (D)$ Life>me$ Client$ JavaScript$ Instrumentor$ Instrumented$ Run$ Trace$ Enhanced$ GUI$ code$ JavaScript$ analysis$ Trace$ analyses$ code$

Figure 5: MEMINSIGHT tool chain 2. We show that the detailed information collected by DECLARE x,y,m; LASTUSE 2 at 5; ALLOCOBJ 2 at 1; RETURN at 7; MEMINSIGHT is useful for diagnosing and ﬁxing mem- 1 var x = {}; ory issues in real-world web applications. 2 var y = {}; WRITE x,2 at 1; LASTUSE 4 at 7; The rest of the paper is organized as follows. After outlin- 3 function m(p,q) ALLOCOBJ 3 at 2; WRITE x,0 at 8; 4 { WRITE y,3 at 2; UNREACHABLE ing the different phases of MEMINSIGHT in Sections 2–4 as 5 p.f = q; ALLOCFUN 4 at 3; 2 at 8; described above, Sections 5 and 6 respectively present a quan- 6 }; WRITE m,4 at 3; UNREACHABLE titative evaluation of MEMINSIGHT and case studies showing 7 m(x,y); CALL 4 at 7; 3 at end; its usefulness. Finally, Section 7 discusses related work. 8 x=null; DECLARE p = 2, UNREACHABLE 2. Trace Generation q = 3; 4 at end; PUTFIELD 2,"f",3 In principle, our memory analysis framework could be imple- at 5; mented in an entirely “online” fashion, with client analyses running while the target program is being exercised. How- Figure 6: A simple code example and the corresponding trace. ever, this approach could have very high analysis overhead, Red entries are added in the enhanced trace. adversely affecting the usability of the target program. Hence, Handling DOM our framework divides the work into two phases. A trace 1 var elem = document.createElement("div"); generation phase runs along with the target program, record- 2 elem.innerHTML = "

Hello World!

"; ing relevant memory operations into a trace file. Then, client 3 document.getElementById("x").appendChild(elem); analyses run in an offline mode, based on the recorded trace. Here we first discuss the design of our trace format, crafted • FigureHandle various DOM modificaon events and 7: Example to illustrate handling of DOM-related code. to balance detail with analysis overhead. Then, we discuss generate puield/geield events our handling of uninstrumented code and the DOM in particu- cation– uses HTML5 “mutaon observer” entries introduce a unique identifier used to name the corresponding object throughout the trace. We use a distinct lar. We defer discussion of certain challenges in handling the – also handles appendChild JavaScript language to Section 3.3. entry type to identify function object allocation, used to enable proper– Special handling of handling of closuresinnerHTML (see below). provides line In our implementa- 2.1. Trace Design tion,numbers LASTUSE entries include a timestamp and all appear at the end of the generated trace (since the last use is only known To enable client analyses like leak detection, we require that at the end of the program); a separate post-processing phase traces be sufficient to reconstruct object lifetimes, i.e., when inserts the entries at the appropriate slots. objects are created and become unreachable. Hence, traces must include records of each object allocation and each mem- 2.2. Uninstrumented Code ory write, both to variables and to object fields (“properties” in JavaScript parlance). As an optimization, we avoid logging MEMINSIGHT works robustly in the presence of uninstru- writes when the old and new values are both primitive, as mented JavaScript code or native code from the environment, such writes are irrelevant to a memory analysis. A delete e.g., DOM functions. Here, we detail our strategies for han- operation on an object property is modeled as a write of null.3 dling uninstrumented code and the DOM. To handle functions, the generator logs calls and returns, Uninstrumented Code In principle, uninstrumented code and also logs declarations of local variables to enable proper could arbitrarily mutate any memory locations to which it has scope handling. For leak detection, we also log the last use of access. Attempting to discover all such behavior via code each object, where an object is used when it is dereferenced or, instrumentation alone would be difficult or impossible, partic- for function objects, when it is invoked. We only log the last ularly since invocations of uninstrumented code may not be use of each object since we found that logging all uses was observable (e.g., a browser invoking an uninstrumented event prohibitively expensive, and last use information is sufficient handler). Furthermore, such conservative detection would for computing object staleness. require frequent traversals of the full heap visible to uninstru- Figure 6 shows the generated trace for a simple example. mented code, a very costly operation. Most entries includes a source location at the end. The allo- In practice, we have found a policy of only tracking refer- 3We do not yet model the effect of delete on the shape of the object, or ences created in instrumented code to strike a good balance physical object sizes in general; see “Limitations” in Section 5.1. between coverage of relevant behaviors and analysis overhead.

4 Success Stories

• Leaks – Fixed in one Tizen app (patch accepted) – Conﬁrmed exisng patch ﬁxes leak in dataTables – Leaks found by internal users in other apps • Churn – Fixed in one Tizen app for 10% speedup (patch accepted) – 10X speedup for escodegen (patch accepted) • Bloat – Found object inlining opportunity in old esprima version Final slide • Jalangi 2 is a powerful dynamic analysis tool – much powerful than tradional dynamic analyses tools – can change semancs/values – can repeatedly execute arbitrary paths in a funcon – portable and browser agnosc – simple API – under acve development and acvely maintained – 10X to 100X slowdown, but barely noceable on event- driven apps • Give it a try today! hps://github.com/Samsung/jalangi2

Extra slides Rule #1: Use Consistent Object Layout

function access_field(o) { return o.a; }

access_field({a:1, b:2}) access_field({b:2, a:1}) access_field({a:1, b:2})

93 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Rule #1: Check Consistent Object Layout

function access_field(o) { Meta info. per location Cache Locaons return o.a; loc1: Layouts Misses } [a,b] loc1 2 [b,a] access_field({a:1, b:2}) … … …

access_field({b:2, a:1}) … … … access_field({a:1, b:2})

• Override analysis.getField and analysis.putField to create the table. • Rank locaons by number of cache misses 94 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Rule #2: Check if Arrays are Purely Numeric For each assignment locaon count number of mes non- numeric value has been assigned to a numeric array

var a = new Array(); a[0] = 77; a[1] = 88; a[2] = {};

95 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Rule #2: Check if Arrays are Purely Numeric For each assignment locaon count number of mes non- numeric value has been assigned to a numeric array Meta info. per location # of bad Locaons var a = new Array(); assignments loc1: a[0] = 77; loc1 0 loc2: a[1] = 88; loc2 0 loc3: a[2] = {}; loc3 1

• Override analysis.getField and analysis.putField to create the table • Rank locaons by number of bad assignments 96 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Rule #3: Check Access of Uninialized Elements

Do not load uninialized (or deleted) array elements

var array = [0]; var c; for (var i=0;i<100000000;i++){ c = array[1]; }

Accessing uninialized element takes 4.907s

97 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Rule #3: Check Access of Uninialized Elements

Do not load uninialized (or deleted) array elements

var array = [0, 1]; var c; for (var i=0;i<100000000;i++){ c = array[1]; }

Accessing inialized element takes 0.121s 40X speedup!

98 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Rule #3: Check Access of Uninialized Elements

Do not load uninialized (or deleted) array elements

var array = [0, 1]; var c; for (var i=0;i<100000000;i++){ loc1: c = array[1]; Meta info. per location } Locaons # of bad reads loc1 108 … … Rank based on # of bad reads … …

SplayTree.prototype.insert = function(key, value) { ... var node = new SplayTree.Node(key, value); if (key > this.root_.key) { node.left = this.root_; node.right = this.root_.right; Performance boost: ... } else { 15% node.right = this.root_; node.left = this.root_.left; ... 6.7% } this.root_ = node; }; 101 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. date-format-tofte.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (date-format-tofte.js:274:12)] <- No. usages: 1000

var ia = input.split(""); var ij = 0; while (ia[ij]) { ... ij++; }

102 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. date-format-tofte.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (date-format-tofte.js:274:12)] <- No. usages: 1000

var ia = input.split(""); var ij = 0; while (ia[ij]) { ... ij++; } refactor

var ia = input.split(""); var ij = 0; var len = ia.length; while (ij < len) { ... ij++; } 103 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley.

date-format-tofte.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (date-format-tofte.js:274:12)] <- No. usages: 1000

var ia = input.split(""); var ij = 0; while (ia[ij]) { ... ij++; } refactor Performance boost:

var ia = input.split(""); var ij = 0; 2.05%

var len = ia.length; 5.33% while (ij < len) { ... ij++; } 104 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley.

crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

105 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var bin = Array(); ...

for(var i = 0; i < str.length * chrsz; i += chrsz) bin[i>>5] |= ... return bin; }

106 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var bin = Array(); ...

for(var i = 0; i < str.length * chrsz; i += chrsz) bin[i>>5] |= ... return bin; } Operations on undefined value

107 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var len = str.length * chrsz, len1 = len>>5; var bin = []; bin.length = len1; for(var i=0;i>5] = 0;} ... for(var i = 0; i < len; i += chrsz) bin[i>>5] |= ... return bin; }

Pre-allocate array space to eliminate loading undeclared array elements

108 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

Pre-allocate array space to eliminate loading undeclared array elements

109 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. crypto-sha1.js in SunSpider

Report of loading undeclared or deleted array elements: 1st [location: (crypto-md5.js:206:5)] <- No. usages: 3956 [location: (crypto-md5.js:43:3)] <- No. usages: 1

function str2binl(str) { var len = str.length * chrsz, len1 = len>>5; var bin = []; bin.length = len1; Performance boost: for(var i=0;i>5] = 0;}

... for(var i = 0; i < len; i += chrsz) 1.5% bin[i>>5] |= ... return bin; 31% } Operations on numbers

Pre-allocate array space to eliminate loading undeclared array elements

110 © Liang Gong, Electric Engineering & Computer Science, University of California, Berkeley. Benchmark: CPR JITProf Ch. Avg. improvement SunSpider (SS) & FF CH Rank LOC (stat. significant) (function| (statement Octane (Oct) Firefox Chrome level) level) 1inUAE, SS-Crypto-SHA1 2 5+ 6 3.3 0.9% 26.3 0.4% | PO, BOU ± ± SS-Str-Tagcloud 5 - 1inIOL 15 - 11.7 0.7% | 1inUAE, ± SS-Crypto-MD5 3 5+ 6 - 24.6 0.1% | PO, BOU ± SS-Format-Tofte 2 1 1inUAE 2 - 3.4 0.2% | ± SS-3d-Cube 5+ 5 1inNCA 1 - 1.1 0.1% SS-Format-Xparb 4 | 1 1inPO 2 19.7 0.5% 22.4± 0.3% SS-3d-Raytrace 5 | 5 1inNNA 4 ±- 2.6 ±0.2% Figure 9: Prevalence of JIT-unfriendly code on web SS-3d-Morph 1 | 1 1inGA 1 - 1.5±0.3% sites. SS-Fannkuch 1 | 1 1inGA 3 8.3 0.9% 5.4±2.3% | ± ± Oct-Splay 5 5+ 1inIOL 2 3.5 0.9% 15.1 0.3% | ± ± JavasScript files on each of those website are minified and Oct-SplayLatency 5 5+ 1inIOL 2 - 3.8 0.6% | ± uglified and because we cannot easily change the code on Oct-DeltaBlue 5+ 5+ 2inIOL 6 1.4 0.2% - | ± Oct-RayTrace 5+ 1 1inIOL 18 - 12.9 1.9% the server which responds to the AJAX requests. | ± Oct-Box2D 5+ 5+ 2inIOL 1 - 7.5 0.6% | ± To assess whether a change improves the performance, we Oct-Crypto 5+ 5+ 1inGA 1 13.8 4.9% 3.3 0.4% compare the execution time of the original and the mod- | ± ± ified program in two popular browsers, Firefox 31.0 and CPR means CPU Profiler Rank. FF means Firebug Profiler, CH means Google Chrome’s profiler. Ch. LOC is the number of changed Chrome 36.0. To obtain reliable performance data [13, 26, LOC. Short names (e.g., IOL) in the third column refers to the 9], we repeat the following steps 50 times: (1) Open a fresh profilers defined in Table 2. - means no ranking or no statistically browser instance and run the original benchmark. (2) Open significant di↵erence. Confidence intervals of improvements of Firefox a fresh browser instance and run the modified benchmark. and Chrome in the last two columns are at 95% confidence level [13]. Each run yields a benchmark score that summarizes the performance. Given these scores, we use the independent T-test Table 4: Performance improvement achieved by (95% confidence interval) to check whether there is a statisti- avoiding JIT-unfriendly code patterns. cally significant performance di↵erence between the original and the modified program. All performance di↵erences are statistically significant. Experiments are performed on Mac statement that retrieves a property frequently alternate be- OS X 10.9 using a 2.40GHz Intel Core i7-3635QM CPU tween key|value|left|right and key|value|right|left. machine with 8GB memory. The problem boils down to the following code, which initial- izes the properties left and right in two possible orders: 4.2 Prevalence of JIT Unfriendliness 1 var node = new SplayTree.Node(key, value); 2 if (key > this.root_.key) { Figure 9 illustrates the prevalence of JIT-unfriendly code 3 node.left = this.root_; patterns on the top 50 most popular web sites. The figure 4 node.right = this.root_.right; shows the total number of JIT-unfriendly code locations 5 ... 6 } else { reported by JITProf (vertical axis), depending on the mini- 7 node.right = this.root_; mum unfriendliness counter required to consider a location 8 node.left = this.root_.left; as JIT-unfriendly (horizontal axis). The results show that 9 ... JIT-unfriendly code patterns are prevalent in practice and 10 } that some patterns are more common than others. These We swap the first two statements in the else branch results provide guidance on which patterns to focus on. so that the object layout is always key|value|left|right, which improves performance by 3.5% and 15.1% in Firefox 4.3 Profiling JIT-unfriendly Code Locations and Chrome, respectively. Polymorphic Operations in SunSpider-Format-Xparb. 4.3.1 JIT-unfriendly Code Found by JITProf JITProf reports a code location that frequently performs a JITProf detects JIT-unfriendly code that causes easy to polymorphic plus operation. Specifically, the analysis ob- avoid performance problems in 15 of the 39 benchmarks. serves operand types string + string 699 times and operand Table 4 summarizes the performance improvements achieved types object + string 3,331 times. The behavior is due by avoiding these problems. The “JITProf Rank” column to the following function, which returns either a primitive indicates which analysis detects a problem and the position string value or a String object, depending on the value of of the problem in the ranked list of reported code locations. val: The table also shows the amount of changes to avoid the 1 String.leftPad = function (val, size, ch) { 2 var result = new String(val); problem. The last two columns of the table show the perfor- 3 if (ch == null) { ch = " ";} mance improvement achieved with these changes, in Firefox 4 while (result.length < size){ and Chrome, respectively. 5 result = ch + result; All JIT-unfriendly code locations detected by JITProf and 6 } 7 return result; their refactorings are documented in our technical report [14]. 8 } Due to limited space, we only discuss a few representative To avoid this problem, we refactor String.leftPad by examples in the following. replacing line 2 with: Inconsistent Object Layouts in Octane-Splay. JITProf reports a code location where inconsistent object layouts 1 var result = val + ’’; 2 var tmp = new String(val) + ’’; occur a total of 135 times. The layout of the objects at a

8 Example 2 • www.repl.it, included from jQuery: globalEval: function(code) { var script, indirect = eval; code = jQuery.trim( code ); if (code) { if (code.indexOf("use strict") === 1) { ... } else { indirect(code); // indirect call of eval } } } • Rule: Avoid eval(). (Nothing wrong with this code.) Code Quality Rules

• Informal descripon of a paern of code or execuon behavior that should be avoided or that should be used in a parcular way • Increase correctness, maintainability, code readability, performance, or security Inheritance-Related

• Avoid shadowing a prototype property with an object property.

function C() {}; C.prototype.x = 3; var obj = new C(); obj.x = 5; console.log(obj.x); DLint vs. JSHint

• DLint • JSHint – 28 rules ︎ – 150 rules – 9018 warnings ︎ – 580k warnings – Focus for comparison: 9 rules DLint Warnings

• 53 warnings per page • 49 are missed by JSHint DLint vs. JSHint Warnings

• Some sites: One approach ﬁnds all • Most sites: Beer together