University of Calgary PRISM: University of Calgary's Digital Repository Graduate Studies The Vault: Electronic Theses and Dissertations 2014-10-01 Dependency Detection and Migration in Software Systems and Libraries Cossette, Bradley Edward Cossette, B. E. (2014). Dependency Detection and Migration in Software Systems and Libraries (Unpublished doctoral thesis). University of Calgary, Calgary, AB. doi:10.11575/PRISM/25551 http://hdl.handle.net/11023/1902 doctoral thesis University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission. Downloaded from PRISM: https://prism.ucalgary.ca UNIVERSITY OF CALGARY Dependency Detection and Migration in Software Systems and Libraries by Bradley E. Cossette A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF COMPUTER SCIENCE CALGARY, ALBERTA September, 2014 © Bradley E. Cossette 2014 Abstract Software systems must change over time or risk becoming obsolete, but direct changes can im­ pact dependent functionality. Software developers perform change impact analysis and redress using automated tools to identify dependency relationships affected by change, and to recommend adaptations. However, these tools are restricted in their application: dependency analysis tools are language-specific but many systems are implemented using multiple languages, while recom­ menders are poor at identifying adaptations for change impacts from external libraries. This work proposes using semi-automated approaches for supporting change impact analysis and redress, in which the developer is relied upon to provide key details of dependency syntax or examples of correct adaptations. From such information, tool support can be generated that is appropriately configured for the developer’s software, and that can be further refined by the developer through additional details or examples until it is sufficiently accurate for their needs. Four studies validate this thesis. The first involves the semi-automated DSKETCH tool for polylingual dependency analysis, which requires developers to detail only key syntax using a sim­ plified notation; it then uses this syntax to identify where potential dependencies exist. Devel­ opers were able to successfully configure and use DSKETCH on polylingual systems with only a short period of training. The second study examines how software libraries and their application programming interfaces (APIs) evolve over successive versions. The study found that existing recommenders are generally unsuccessful, and that most observed changes could not be automati­ cally migrated. The third study introduces the UMAMI tool that detects correspondences between the syntactic structure of the old API functionality and possible replacements in the new library version to recommend adaptations for API changes. The fourth study examines how change rec­ ommenders could be hybridized in a flexible fashion, by relying on developer-provided examples of correct redress of API changes to tailor recommendations to a library’s particular characteristics. ii Acknowledgments We cannot tell the precise moment when friendship is formed. As in filling a vessel drop by drop, there is at last a drop which makes it run over; so in a series of kindnesses there is at last one which makes the heart run over. James Boswell, The Life of Samuel Johnson Completing this dissertation has been the story of many kindnesses from friends, family, and God, that helped me on the way. First, I am grateful to my supervisor Robert Walker, for his mentorship, support, and especially his friendship over the last eight years. I could write much about his character, and integrity that I now try to emulate in my own work, but I appreciate most his years of encouragement, and his constant goal of training me to be his colleague, and not just his student: I see echoes of his writing and perspective in my work every day. Many lab-mates have come and gone over the years. Rylan Cottrell, who started his Master’s and PhD degrees with me, and without whom I would have never survived Cryptography, is the first and foremost I must thank. For eight years you were my friend and co-conspirator, and I will miss your boundless willingness to talk about anything: whether God and religion, relationships, research, or the importance of Dr. Who to science fiction. Reid Holmes still acts as my big brother, introducing me to the research community at large, and always willing to offer advice. Puneet Kapur, who is the first programmer and computer scientist I know to also be a medical doctor, was a font of friendship, humour, insight, and wisdom. Thanking the cast of characters in our lab would not be complete without mentioning Craig Schock, Soha Makady, Hamid Baghi, Elham Moazzen, and Mostafa Sayed. Thank you as well to Leanne Wu, Tyson Kendon, Sean Nichols, and everyone else who also shared the joys and travails of grad life with me at Tuesday Tea. I could not have prouder parents in Bernard and Diana Cossette, and an even prouder grand­ mother in Grace Kine, who have supported me financially, emotionally, and through prayer. My grandparents and great-grandparents have loved books, knowledge, and teaching, and it was a spe­ cial moment when I realized that my father was also a published author of technical papers in his iii field; I follow in all their footsteps. My brothers and their wives—Brandon and Christine Cos­ sette, Aaron and Alicia Cossette—have been a comfort and encouragement without fail, and your joy over my accomplishments is a shadow of how proud I am of you and your children (Kael, Courtney, and Isaiah). I am also grateful for my wife’s family: Kent and Susan Dunfield, Michael, Susan, and Jacob Dunfield, and Robert Dunfield, who treated me as one of their own from the day we met, even trying to read my research papers to show their support. Denis and Laura Perron, thank you for your prayers, hospitality, and most of all for introducing me to my wife. Justin and Kellie Agnew Clark, thank you for being so accepting and gracious: your company is always treasured. I am especially grateful for the family of friends and mentors I had at Bethany Chapel in Calgary, who showered me with kindness and grace throughout my studies. But I have saved my deepest thanks for my greatest supporter, my wife Charlotte who fell in love with this PhD student. You chose to walk with me the whole way on this adventure, following me to Calgary from Vancouver, sharing your amazing circle of friends with me, embracing my family, and now following me again to Montreal´ and McGill University. I love you, and you are the only one who knows every joy and sorrow I faced during this degree, because you chose to share for better and worse with me. Sharing this with you is the greatest joy of this degree. Finally, I want to thank my God, and my Saviour Jesus Christ, who has been faithful to carry me through this entire journey, even when I doubted I would ever finish. It was a message from Jeremiah 32:1–15 six years ago that convinced me that God would see me through my PhD; may I always remember His faithfulness to bring me through the impossibilities in life. BRADLEY COSSETTE September 22, 2014 Montreal,´ Quebec´ iv For Charlotte, who took my hand, and walked the way with me. v Table of Contents Abstract ........................................... ii Acknowledgments ...................................... iii Table of Contents ...................................... vi List of Tables ........................................ x List of Figures ........................................ xii 1 Introduction ...................................... 1 1.1 Overview of Related Work .............................. 3 1.2 Semi-Automating Developer Configured Solutions . 6 1.3 Thesis Statement ................................... 8 1.4 Organization ...................................... 9 1.5 Publications ...................................... 10 2 Motivation ....................................... 11 2.1 Proactive Change: Polylingual Dependency Analysis . 11 2.2 Reactive Change: API Migration and Redress . 14 2.3 Summary: Motivation ................................. 18 3 Related Work ..................................... 20 3.1 Change Impact and Dependency Analysis ...................... 21 3.1.1 Formulating Change Impact ......................... 22 3.1.2 Reverse Engineering and Tools For Dependency Analysis . 24 3.1.3 Summary: Change Impact and Dependency Analysis . 30 3.2 Polylingual Dependency Analysis .......................... 31 3.2.1 Lexically-Aware Tools in Polylingual Contexts . 33 3.2.2 Syntactically-Aware Tools in Polylingual Contexts . 34 3.2.3 Semantically-Aware Tools in Polylingual Contexts . 37 3.2.4 Partial-Language Recognition Approaches . 39 3.2.5 Summary: Polylingual Dependency Analysis . 42 3.3 Library Evolution and API Migration ........................ 43 3.3.1 Proactive Library Migration ......................... 45 3.3.2 Reactive Library Migration .......................... 47 3.3.3 Recommending appropriate reactions . 50 3.3.4 Redressing library change impact ...................... 55 3.3.5 Summary: Library Migration and API Evolution . 57 3.4 Empirical Studies of API Change ........................... 58