Mapping breakpoint types: an exploratory study Eduardo Andreetta Fontana Fabio Petrillo SmArtSE Group SmArtSE Group University of Quebec at Chicoutimi University of Quebec at Chicoutimi Saquenay - QC, Canada Saquenay - QC, Canada [email protected] [email protected] Abstract—Debugging is a relevant task for finding bugs dur- In academic literature, we found some studies that try to ing software development, maintenance, and evolution. During define or classify some breakpoint types, but there are no debugging, developers use modern IDE debuggers to analyze studies that map the breakpoint types among different IDEs variables, step execution, and set breakpoints. Observing IDE debuggers, we find several breakpoint types. However, what are and different technologies. For instance, Petrillo et al. [2], the breakpoint types? The goal of our study is to map the [17] and Johnson [18] classify some breakpoint types as breakpoint types among IDEs and academic literature. Thus, Static Breakpoint and Dynamic Breakpoint; and Keppel [19], we mapped the gray literature on the documentation of the nine Vasudevan [20], and Spinellis [21] use the name Code Break- main IDEs used by developers according to the three public point, but IDEs documentation do not use these categorization rankings. In addition, we performed a systematic mapping of academic literature over 72 articles describing breakpoint types. names. Arya et al. [22], Spinellis [21], Kumar et al. [23], Finally, we analyzed the developers understanding of the main Copperman et al. [24], and Zhao [25] present breakpoint breakpoint types through a questionnaire. We present three main types with the names of Watchpoint or Data Breakpoint at the contributions: (1) the mapping of breakpoint types (IDEs and software level, while Chew et al. [26], Sommer et al. [27], literature), (2) compiled definitions of breakpoint types, (3) a and Jang et al. [28] present the same breakpoint type names, breakpoint type taxonomy. Our contributions provide the first step to organize breakpoint IDE taxonomy and lexicon, and but at the hardware level or with hardware support. support further debugging research. The goal of our study is to map the breakpoint types among Index Terms—debugging, mapping, IDE, breakpoint, break- different IDEs and academic literature. Our survey provide point type, watchpoint, taxonomy to developers a common lexicon to IDEs. In addition, our study contributes to avoid misleading breakpoint names among I. INTRODUCTION academic literature and IDE documentation. Furthermore, as Debugging is a relevant task for finding and resolving bugs suggested by Aniche et al. [29], our study can support future during software development, maintenance, and evolution [1]. debugging research. Finally, our study provides a discussion During debugging, developers use modern interactive debug- that establishes characteristics and the state of the art of the gers with Integrated Development Environments (IDEs), such main existing breakpoint types and support further debugging as Eclipse, NetBeans, IntelliJ IDEA, and Visual Studio, to de- research. tect, locate, and correct faults in software systems [2]. Among We mapped the gray literature on the documentation of the the many features provided by IDEs, breakpoints are one of nine main IDEs used by developers according to the Stack the most used features by developers to initiate a debugging Overflow Survey 2019 [30], Google Trends 2020 [31], and G2 session [3]. Observing IDEs documentation, we found several 2020 [32] rankings. In addition, we performed a systematic breakpoint types, each one has particular functionalities that mapping of academic literature over 72 articles describing could be explored by developers [4]–[8]. breakpoint types. Finally, we analyzed the developers under- arXiv:2109.00917v1 [cs.SE] 31 Aug 2021 However, what are the breakpoint types? There is a lack standing of the main breakpoint types administering a ques- correspondences related to breakpoint type names and de- tionnaire. From the collected data, we applied The Grounded scriptions in IDEs documentation and academic literature. We Theory method [33] to map and organize the definitions and found some IDEs documentation using different names for the characteristics of the main existing breakpoint types. similar breakpoint types. For example, Eclipse [4], [5] and As a result, we built a breakpoint type mapping table of IntelliJ [6] call data-sensitive breakpoints as Watchpoint or the main IDEs, as well as a breakpoint type mapping table as Field Watchpoint, but Visual Studio [7] or Visual Studio for academic papers. Furthermore, we present breakpoint type Code [8] call them Data Breakpoint. Furthermore, the IDEs definitions structured according to attributes mapped. Finally, documentation information is not clear or sufficient to support we present a taxonomy diagram with two breakpoint type developers understanding of breakpoint types. For example, categories. there are discussions in gray literature about the difference We present three main contributions: (1) the mapping of between Breakpoint, Watchpoint, Method Breakpoint [9]– the existing breakpoint types in the main IDEs and academic [12], Static Breakpoint, Dynamic Breakpoint [13], Conditional literature, (2) compiled definitions of breakpoint types, and (3) Breakpoint, and Data Breakpoint [14]–[16]. a breakpoint type taxonomy. 1 This paper is outlined as follows: in Section II, we provide a the current execution state. The developer adds this signaling, Background with a historical view of breakpoint. Followed by and it can be added multiple times in the system source code. that, we present the Related Work in Section III. The study Using breakpoints, developers explore programs (memory, call design is presented in Section IV as Methodology, whereas stack, states, logs, files, and others) to understand failures, find the results are elaborated in Section V, VI, and VII. Threats their cause (i.e., faults) and fix them. Furthermore, developers to validity are described in Section VIII. With Section IX, we start an interactive debugging session on IDEs by setting a close the paper and discuss future work. breakpoint at a specific line in the source code, then executing the code step-by-step. II. BACKGROUND The use of breakpoint originates in the 1940s during discov- III. RELATED WORK ery attempts of program problems running on ENIAC (Elec- In this section, we present some academic literature where tronic Numerical Integrator and Computer), the first digital we found some works that try to define or classify breakpoint computer in history [34] [35]. In the initial design of the types. These related works are not intended to map and ENIAC, the program flow was set by plugging cables from one organize the existing breakpoint types, but rather to describe unit to another. To make the program stop at a certain point, a some breakpoint types or some breakpoint categories in a cable was simply removed. The removal of such cable midway generic way to distinguish them and to contextualize the through the program execution was named as breakpoint. The problem this study is presenting. Furthermore, we consider term breakpoint was coined by Frances Elizabeth Holberton, in the mapping only the studies that present breakpoint types one of the six woman programmers that worked at ENIAC within the context of debugging performed in IDEs for high- [36]. level software, using the respective debuggers. We did not “Well you know, the thing is we did develop the consider studies that describe debugging directly on hardware. one word that’s in the language today, which is Petrillo et al. [2], [17], Johnson [18], and Chern et al. [41] ‘breakpoint’, at that time. Because we actually did classify some breakpoint types as Static Breakpoint, and Dy- pull the wire to stop the programs so we could read namic Breakpoint. These are broad classifications that describe the accumulators off. This was, I mean we actually trigger criterion, that is, static breakpoint is for breakpoint broke the point, and that was where the word came triggered when executing the line of code, and dynamic being from.” - Frances Elizabeth Holberton [36]. when a condition is satisfied. However, our research reveals Iterative breakpoints were initially used in the 1970s, during that there are other criteria that determine the breakpoint type the era of mainframe computers, over OLIVER debugger groups, such as how the breakpoint is added and the effect (CICS interactive test/debug). OLIVER was a proprietary when it is triggered. Furthermore, the classification proposed testing and debugging toolkit for interactively testing programs by the authors does not mention how the other breakpoint designed to run on IBM Customer Information Control System types found in IDEs could be grouped into these two categories (CICS) on IBM System/360/370/390 architecture. It provided of static and dynamic. instruction step, conditional program breakpoint, and storage Vasudevan [20] describes a generic way classification of alteration features for programs written in Assembly, COBOL, breakpoint types through Software Breakpoint and Hardware and PL/I. HLL (high-level language) users were able to see Breakpoint categories. Software Breakpoint is a category that and modify variables directly at a breakpoint [37]. includes all breakpoint types provided by IDEs through their The first use of breakpoints in IDEs (Integrated Devel- respective debuggers. Hardware Breakpoint is a category that opment