AN APPROACH FOR AUTOMATED VERIFICATION OF WEB APPLICATIONS USING MODEL CHECKING AND REPLAYING THE SCENARIOS OF COUNTEREXAMPLES A THESIS SUBMITTED TO THE GRADUATE SCHOOL OF INFORMATICS OF THE MIDDLE EAST TECHNICAL UNIVERSITY BY YUDUM PAÇİN IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN THE DEPARTMENT OF INFORMATION SYSTEMS SEPTEMBER 2015 AN APPROACH FOR AUTOMATED VERIFICATION OF WEB APPLICATIONS USING MODEL CHECKING AND REPLAYING THE SCENARIOS OF COUNTEREXAMPLES Submitted by YUDUM PAÇİN in partial fulfillment of the requirements for the degree of Master of Science in Information Systems, Middle East Technical University by, Prof. Dr. Nazife Baykal Director, Graduate School of Informatics Prof. Dr. Yasemin Yardımcı Head of Department, Information Systems Assoc. Prof. Dr. Aysu Betin Can Supervisor, Information Systems Examining Committee Members: Assoc. Prof. Dr. Altan Koçyiğit Information Systems, METU Assoc. Prof. Dr. Aysu Betin Can Information Systems, METU Assoc. Prof. Dr. Banu Günel Kılıç Information Systems, METU Assoc. Prof. Dr. Halit Oğuztüzün Computer Engineering, METU Assoc. Prof. Dr. Vahid Garousi Yusifoğlu Software Engineering, Haccettepe University Date: 03/09/2015 I hereby declare that all information in this document has been obtained and presented in accordance with academic rules and ethical conduct. I also declare that, as required by these rules and conduct, I have fully cited and referenced all material and results that are not original to this work. Name, Last Name : Yudum Paçin Signature : iii ABSTRACT AN APPROACH FOR AUTOMATED VERIFICATION OF WEB APPLICATIONS USING MODEL CHECKING AND REPLAYING THE SCENARIOS OF COUNTEREXAMPLES Paçin, Yudum M.S., Department of Information Systems Supervisor: Assoc. Prof. Dr. Aysu Betin Can September 2015, 137 pages The increase in the use of web applications in various domains, raised the importance of the methodologies for verification of web applications. We propose a framework for the verification of web applications with respect to access control, link consistency and reachability properties using model checking. In this approach, users define the properties by explanatory guidance of user interface. The execution traces that lead to a property violation is translated to a script that automates the replaying of the counterexample scenarios on a web browser. This facility enables the user to observe incorrect behaviors of the web application with respect to specified properties so that the user is released from the tedious task of understanding and interpreting the counterexamples generated by the model checker. In addition, to automate this verification iv process, we need to automate the model extraction of a web application to be given to the model checker as an input. To this purpose, we use two dynamic web application crawlers and automatically transform their models to an intermediate web model we have developed. This intermediate web model both enables model extraction tool independence and gives the user to edit the model manually to increase precision of verification process. In order to evaluate the tool we developed for this purpose, we conducted a user study and the participants reported our tool to be useful for detecting and visualizing errors. We also evaluated the effectiveness on real web applications and observed that the tool can reveal real faults. Keywords: Model checking, Web software verification, Counterexample animation v ÖZ WEB UYGULAMALARININ MODEL DENETLEME KULLANILARAK OTOMATİK DOĞRULANMASI VE KARŞI ÖRNEK SENARYOLARININ OYNATILMASI İÇİN BİR YAKLAŞIM Paçin, Yudum Yüksek Lisans, Bilişim Sistemleri Bölümü Tez Yöneticisi: Doç. Dr. Aysu Betin Can Eylül 2015, 137 sayfa Web uygulamalarının kullanım alanının gittikçe artması, doğrulama yöntemlerinin önemini de arttırmaktadır. Bu çalışmada, model denetleme kullanılarak web uygulamalarının erişim kontrolü, bağlantı tutarlılığı ve erişilebilirlik özelliklerinin doğrulanması için bir yöntem öneriyoruz. Bu yaklaşımda, kullanıcılara sorgulanacak özellikleri ara yüz yardımıyla tanımlama imkanı verilmektedir. Sorgulanan özelliklerin ihlaline neden olan yürütme adımları, model denetleyicinin ürettiği karşı örneklerin web tarayıcısı üzerinde oynatılmasını sağlayan betiklere çevrilmektedir. Bu sayede, kullanıcıların web uygulamasında bulunan hatalı davranışları gözlemlemelerine olanak verilmekte, kullanıcıya model denetleme aracının ürettiği anlaşılması zor ve oldukça detaylı olan karşı örneklerin çözümlenmesinde yardımcı olunmaktadır. Buna ek olarak, doğrulama sürecini vi otomatikleştirmek için, model denetleme aracına girdi olarak belirlenen modelin web uygulamasından elde edilmesinin de otomatikleştirilmesi gerekmektedir. Biz bu amaçla, var olan iki dinamik web arama robotunu kullanmakta ve çıkan modelleri otomatik olarak kendi geliştirdiğimiz ara web modeline çevirmekteyiz. Bu ara web modeli, hem model çıkarma aracından bağımsızlık sağlamakta hem de doğrulama sürecinin kesinliğini arttırmak için kullanıcıya çıkan modeli manuel biçimde düzenleme imkanı sunmaktadır. Ayrıca, bu amaçla geliştirdiğimiz aracın değerlendirilmesi kapsamında bir kullanıcı çalışması yürütülmüştür. Katılımcılar, hata tespiti ve hataların görselleştirilmesini yararlı bulduklarını bildirmişlerdir. Ayrıca, aracın hata tespitindeki etkililiği gerçek web uygulamalarıyla değerlendirilmiş ve aracın hataları tespit edebildiği gözlemlenmiştir. Anahtar Kelimeler: Model denetleme, Web yazılımı doğrulama, Karşı örnek oynatımı vii To my family, Yusuf, Nuran & Doğukan Paçin viii ACKNOWLEDGEMENTS I would like to thank my advisor Assoc. Prof. Aysu Betin Can for making this experience possible for me. I thank her for her great patience and the time that she spent with me for discussing the research and suggesting new ideas. I also would like to thank for her kind and quick responses, which helped me in every stage of this work. I also want to thank my friends, Murathan Kurfalı and Ece Kamer Takmaz for their friendship and valuable comments, which light my way most of the time. I would like to thank also Ezgi Arslan for her great support in my first year at the graduate school, which gave me the strength to continue to my education. I would like to thank my family for supporting me to pursue my education since I was a little child. My father, Yusuf Paçin, my mother Nuran Paçin and my brother Doğukan Paçin provided me with their unconditional support and love which become the motivation of my thesis. ix TABLE OF CONTENTS ABSTRACT .................................................................................................................................. iv ÖZ.................................................................................................................................................. vi ACKNOWLEDGEMENTS .......................................................................................................... ix TABLE OF CONTENTS ............................................................................................................... x LIST OF TABLES ...................................................................................................................... xiii LIST OF FIGURES ..................................................................................................................... xiv LIST OF ABBREVIATIONS AND ACRONYMS .................................................................... xvi CHAPTER 1. INTRODUCTION .................................................................................................................. 1 2. BACKGROUND .................................................................................................................... 5 2.1 Model Checking ............................................................................................................... 6 2.2 Web Crawlers Used for Modelling................................................................................... 8 2.2.1 Crawljax .................................................................................................................... 9 2.2.2 Micro-Crawler ......................................................................................................... 10 2.3 Counterexamples ............................................................................................................ 11 3. LITERATURE REVIEW ..................................................................................................... 15 3.1 Verification of Web Applications with Model Checkers ............................................... 15 3.1.1 Classification of the Related Studies ....................................................................... 19 3.1.1.1 Categorization Criteria ..................................................................................... 19 3.2 Interpreting Counterexamples ........................................................................................ 23 x 3.3 Testing Web Applications with Model Checkers .......................................................... 24 4. METHODOLOGY .............................................................................................................. 27 4.1 Running example ........................................................................................................... 28 4.2 Modelling Web Application as Intermediate Web Model ............................................. 30 4.3 Generation of Model in NuSMV ..................................................................................