Automated Conformance Testing for JavaScript Engines via Deep Compiler Fuzzing Guixin Ye Zhanyong Tang Shin Hwei Tan
[email protected] [email protected] [email protected] Northwest University Northwest University Southern University of Science and Xi’an, China Xi’an, China Technology Shenzhen, China Songfang Huang Dingyi Fang Xiaoyang Sun
[email protected] [email protected] [email protected] Alibaba DAMO Academy Northwest University University of Leeds Beijing, China Xi’an, China Leeds, United Kingdom Lizhong Bian Haibo Wang Zheng Wang Alipay (Hangzhou) Information &
[email protected] [email protected] Technology Co., Ltd. University of Leeds University of Leeds Hangzhou, China Leeds, United Kingdom Leeds, United Kingdom Abstract all tested JS engines. We had identified 158 unique JS en- JavaScript (JS) is a popular, platform-independent program- gine bugs, of which 129 have been verified, and 115 have ming language. To ensure the interoperability of JS pro- already been fixed by the developers. Furthermore, 21 of the grams across different platforms, the implementation ofa JS Comfort-generated test cases have been added to Test262, engine should conform to the ECMAScript standard. How- the official ECMAScript conformance test suite. ever, doing so is challenging as there are many subtle defini- CCS Concepts: • Software and its engineering ! Com- tions of API behaviors, and the definitions keep evolving. pilers; Language features; • Computing methodologies We present Comfort, a new compiler fuzzing framework ! Artificial intelligence. for detecting JS engine bugs and behaviors that deviate from the ECMAScript standard. Comfort leverages the recent Keywords: JavaScript, Conformance bugs, Compiler fuzzing, advance in deep learning-based language models to auto- Differential testing, Deep learning matically generate JS test code.