Contributions for Improving Debugging of Kernel-level Services in a Monolithic Operating System Tegawendé F. Bissyandé To cite this version: Tegawendé F. Bissyandé. Contributions for Improving Debugging of Kernel-level Services in a Mono- lithic Operating System. Operating Systems [cs.OS]. Université Sciences et Technologies - Bordeaux I, 2013. English. tel-00808877 HAL Id: tel-00808877 https://tel.archives-ouvertes.fr/tel-00808877 Submitted on 8 Apr 2013 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. No d’ordre: 4768 THÈSE Présentée devant L’UNIVERSITÉ DE BORDEAUX École Doctorale de Mathématiques et Informatique pour obtenir le grade de : DOCTEUR DE L’UNIVERSITÉ DE BORDEAUX Mention INFORMATIQUE par Tegawendé François d’Assise BISSYANDÉ Équipe d’accueil : PROGRESS École Doctorale : Mathématiques et Informatique Composante universitaire : LABRI Titre de la thèse : Contributions for Improving Debugging of Kernel-level Services in a Monolithic Operating System Contributions à l’Amélioration du Débogage des Services Noyau dans un Système d’Exploitation Monolithique Soutenue le 12 Mars 2013 devant la commission d’examen M. : Xavier BLANC Président MM. : Yves LEDRU Rapporteurs Olaf SPINCZYK MM. : Julia LAWALL Examinateurs David LO Laurent RÉVEILLÈRE M. : Gilles MULLER Invité à Josy pour avoir été là de bout en bout, à tonton Michel pour le soutien “paternel”, et à ma maman Lucie pour son courage et son abnégation. Acknowledgements This dissertation would not have been possible without the support, help or guidance of several indi- viduals who in one way or another contributed and extended their valuable assistance in the prepara- tion and completion of my university studies. Among those individuals are Dr. Michel M.D. Nikiéma (tonton Michel) who stood as a father to me. I also acknowledge the courage of my mother, Lucie Bis- syandé, who, in uncertain times, did not stop fighting for our interests. Ouindpouiré Josiane Sedogo supported me throughout this PhD experience and I shall remain grateful to her. I would also like to mention my brothers and sister, especially Flore Bissyandé who was always supportive. In the research community, I am grateful to several researchers and teachers who guided me towards the realization of this dissertation. I am specially thankful for meeting Olivier Beaumont, Director of Research at INRIA, who was the best teacher I got in engineering school. I would like to express the deepest appreciation to Professor Xavier Blanc, Professor at the Uni- versity of Bordeaux, who gracefully accepted to chair my defense committee. Second, a special thank to Professors Yves Ledru from the University of Grenoble, and Olaf Spinczyk from the University of Berlin, who accepted to review my thesis document before the committee can sit. I would also like to thank other committee members, Julia L. Lawall, Director of Research at INRIA, David Lo, Assistant Professor at the Singapore Management University, and Laurent Réveillère, my thesis supervisor, Associate Professor at the Bordeaux Institute of Technology. My gratitude goes also to Gilles Muller, Director of research at INRIA whose guidance is embedded in the entire dissertation. Next, I would like to acknowledge the friends I have made here in France: • friends in CVT at LaBRI: Rémi Laplace (and JDB), Jonathan Ouoba, Jérémie Albert, Damien Dubernet, Hugo Balacey, Daouda Ahmat, Cyril Cassagnes, Jigar Solanki, Vincent Autefage, Télesphore Tiendrebeogo and Sebastien Bindel. • fellow PhD students at LaBRI: Thomas Morsellino, Florent Foucaud, Vincent Filou, Vincent Rabeux and Yi Ren. Finally, I am as ever indebted to God, who made all things possible. Abstract Despite the existence of an overwhelming amount of research on the quality of system software, Operating Systems are still plagued with reliability issues mainly caused by defects in kernel-level services such as device drivers and file systems. Studies have indeed shown that each release of the Linux kernel contains between 600 and 700 faults, and that the propensity of device drivers to contain errors is up to seven times higher than any other part of the kernel. These numbers suggest that kernel- level service code is not sufficiently tested and that many faults remain unnoticed or are hard to fix by non-expert programmers who account for the majority of service developers. This thesis proposes a new approach to the debugging and testing of kernel-level services focused on the interaction between the services and the core kernel. The approach tackles the issue of safety holes in the implementation of kernel API functions. For Linux, we have instantiated the Diagnosys automated approach which relies on static analysis of kernel code to identify, categorize and expose the different safety holes of API functions which can turn into runtime faults when the functions are used in service code by developers with limited knowledge on the intricacies of kernel code. To illustrate our approach, we have implemented Diagnosys for Linux 2.6.32 and shown its bene- fits in supporting developers in their testing and debugging tasks. The contributions of this thesis are: • We identify the interface of kernel exported functions as a sweet spot at which it is possible to interpose the generation of debugging information, in a way that improves debuggability but does not introduce an excessive runtime overhead. • We identify safety holes as a significant problem in the interface between a service and the kernel. Indeed, of the 703 Linux 2.6 commits for which the changelog refers explicitly to a function exported in Linux 2.6.32, 38% corrected faults that are related to one of our identified safety holes. Thus, although we may assume that in-tree kernel code is much more thoroughly tested than new service code under development, violations of these safety holes have still caused numerous crashes and hangs. In this thesis, we propose an extended characterization of these safety holes for the Linux kernel. • We propose an approach to allow a service developer to seamlessly generate, integrate, and exploit a kernel debugging interface specialized to the service code. This approach has a low learning curve, and in particular does not require any particular Linux kernel expertise. • Using fault-injection experiments on 10 Linux kernel services, we demonstrate the improve- ment in debuggability provided by our approach. We find that in 90% of the cases in which a crash occurs, the log contains information relevant to the origin of the defect, and in 95% of these cases, a message relevant to the crash is the last piece of logged information. We also find that in 93% of the cases in which a crash or hang occurs, the log information reduces the number of files that have to be consulted to find the cause of the bug. • We show that the generated debugging interface incurs only a minimal runtime overhead on service execution, allowing it to be used up through early deployment. Beyond operating systems software, the Diagnosys approach described in this thesis can be applied to any software based on the plug-in model, where extension code is written to interact and comple- ment a core software through an ever-expanding interface. The proposed solution thus opens up new possibilities for improving the debugging of such software. Key words Diagnosys, Debugging, Wrappers, Linux, Device Drivers, Software Engineering, Reliability, Testing. VIII Résumé Alors que la recherche sur la qualité du code des systèmes a connu un formidable engouement, les systèmes d’exploitation sont encore aux prises avec des problèmes de fiabilité notamment dûs aux bogues de programmation au niveau des services noyaux tels que les pilotes de périphériques et l’implémentation des systèmes de fichiers. Des études ont en effet montré que que chaque version du noyau Linux contient entre 600 et 700 fautes, et que la propension des pilotes de périphériques à contenir des erreurs est jusqu’à sept fois plus élevée que toute autre partie du noyau. Ces chiffres suggèrent que le code des services noyau n’est pas suffisamment testé et que de nombreux défauts passent inaperçus ou sont difficiles à réparer par des programmeurs non-experts, ces derniers formant pourtant la majorité des développeurs de services. Cette thèse propose une nouvelle approche pour le débogage et le test des services noyau. Notre approche est focalisée sur l’interaction entre les services noyau et le noyau central en abordant la question des “trous de sûreté” dans le code de définition des fonctions de l’API du noyau. Dans le contexte du noyau Linux, nous avons mis en place une approche automatique, dénommée Diagnosys, qui repose sur l’analyse statique du code du noyau afin d’identifier, classer et exposer les différents trous de sûreté de l’API qui pourraient donner lieu à des fautes d’exécution lorsque les fonctions sont utilisées dans du code de service écrit par des développeurs ayant une connaissance limitée des subtilités du noyau. Pour illustrer notre approche, nous avons implémenté Diagnosys pour la version 2.6.32 du noyau Linux. Nous avons montré ses avantages à soutenir les développeurs dans leurs activités de tests et de débogage. Les contributions de cette thèse sont les suivantes: • Nous identifions l’interface des fonctions exportées du noyau comme un endroit opportun où il est possible d’interposer la génération des informations de débogage. Cette interposition est réalisée de façon à améliorer le débogage sans introduire un surcoût d’exécution excessif.