Mapping breakpoint types: an exploratory study

Eduardo Andreetta Fontana Fabio Petrillo SmArtSE Group SmArtSE Group University of Quebec at Chicoutimi University of Quebec at Chicoutimi Saquenay - QC, Canada Saquenay - QC, Canada [email protected] [email protected]

Abstract—Debugging is a relevant task for finding bugs dur- In academic literature, we found some studies that try to ing software development, maintenance, and evolution. During define or classify some breakpoint types, but there are no debugging, developers use modern IDE to analyze studies that map the breakpoint types among different IDEs variables, step execution, and set breakpoints. Observing IDE debuggers, we find several breakpoint types. However, what are and different technologies. For instance, Petrillo et al. [2], the breakpoint types? The goal of our study is to map the [17] and Johnson [18] classify some breakpoint types as breakpoint types among IDEs and academic literature. Thus, Static Breakpoint and Dynamic Breakpoint; and Keppel [19], we mapped the gray literature on the documentation of the nine Vasudevan [20], and Spinellis [21] use the name Code Break- main IDEs used by developers according to the three public point, but IDEs documentation do not use these categorization rankings. In addition, we performed a systematic mapping of academic literature over 72 articles describing breakpoint types. names. Arya et al. [22], Spinellis [21], Kumar et al. [23], Finally, we analyzed the developers understanding of the main Copperman et al. [24], and Zhao [25] present breakpoint breakpoint types through a questionnaire. We present three main types with the names of Watchpoint or Data Breakpoint at the contributions: (1) the mapping of breakpoint types (IDEs and software level, while Chew et al. [26], Sommer et al. [27], literature), (2) compiled definitions of breakpoint types, (3) a and Jang et al. [28] present the same breakpoint type names, breakpoint type taxonomy. Our contributions provide the first step to organize breakpoint IDE taxonomy and lexicon, and but at the hardware level or with hardware support. support further debugging research. The goal of our study is to map the breakpoint types among Index Terms—debugging, mapping, IDE, breakpoint, break- different IDEs and academic literature. Our survey provide point type, watchpoint, taxonomy to developers a common lexicon to IDEs. In addition, our study contributes to avoid misleading breakpoint names among I.INTRODUCTION academic literature and IDE documentation. Furthermore, as Debugging is a relevant task for finding and resolving bugs suggested by Aniche et al. [29], our study can support future during software development, maintenance, and evolution [1]. debugging research. Finally, our study provides a discussion During debugging, developers use modern interactive debug- that establishes characteristics and the state of the art of the gers with Integrated Development Environments (IDEs), such main existing breakpoint types and support further debugging as , NetBeans, IntelliJ IDEA, and Visual Studio, to de- research. tect, locate, and correct faults in software systems [2]. Among We mapped the gray literature on the documentation of the the many features provided by IDEs, breakpoints are one of nine main IDEs used by developers according to the Stack the most used features by developers to initiate a debugging Overflow Survey 2019 [30], Google Trends 2020 [31], and G2 session [3]. Observing IDEs documentation, we found several 2020 [32] rankings. In addition, we performed a systematic breakpoint types, each one has particular functionalities that mapping of academic literature over 72 articles describing could be explored by developers [4]–[8]. breakpoint types. Finally, we analyzed the developers under- arXiv:2109.00917v1 [cs.SE] 31 Aug 2021 However, what are the breakpoint types? There is a lack standing of the main breakpoint types administering a ques- correspondences related to breakpoint type names and de- tionnaire. From the collected data, we applied The Grounded scriptions in IDEs documentation and academic literature. We Theory method [33] to map and organize the definitions and found some IDEs documentation using different names for the characteristics of the main existing breakpoint types. similar breakpoint types. For example, Eclipse [4], [5] and As a result, we built a breakpoint type mapping table of IntelliJ [6] call data-sensitive breakpoints as Watchpoint or the main IDEs, as well as a breakpoint type mapping table as Field Watchpoint, but Visual Studio [7] or Visual Studio for academic papers. Furthermore, we present breakpoint type Code [8] call them Data Breakpoint. Furthermore, the IDEs definitions structured according to attributes mapped. Finally, documentation information is not clear or sufficient to support we present a taxonomy diagram with two breakpoint type developers understanding of breakpoint types. For example, categories. there are discussions in gray literature about the difference We present three main contributions: (1) the mapping of between Breakpoint, Watchpoint, Method Breakpoint [9]– the existing breakpoint types in the main IDEs and academic [12], Static Breakpoint, Dynamic Breakpoint [13], Conditional literature, (2) compiled definitions of breakpoint types, and (3) Breakpoint, and Data Breakpoint [14]–[16]. a breakpoint type taxonomy.

1 This paper is outlined as follows: in Section II, we provide a the current execution state. The developer adds this signaling, Background with a historical view of breakpoint. Followed by and it can be added multiple times in the system source code. that, we present the Related Work in Section III. The study Using breakpoints, developers explore programs (memory, call design is presented in Section IV as Methodology, whereas stack, states, logs, files, and others) to failures, find the results are elaborated in Section V, , and VII. Threats their cause (i.e., faults) and fix them. Furthermore, developers to validity are described in Section VIII. With Section IX, we start an interactive debugging session on IDEs by setting a close the paper and discuss future work. breakpoint at a specific line in the source code, then executing the code step-by-step. II.BACKGROUND The use of breakpoint originates in the 1940s during discov- III.RELATED WORK ery attempts of program problems running on ENIAC (Elec- In this section, we present some academic literature where tronic Numerical Integrator and Computer), the first digital we found some works that try to define or classify breakpoint computer in history [34] [35]. In the initial design of the types. These related works are not intended to map and ENIAC, the program flow was set by plugging cables from one organize the existing breakpoint types, but rather to describe unit to another. To make the program stop at a certain point, a some breakpoint types or some breakpoint categories in a cable was simply removed. The removal of such cable midway generic way to distinguish them and to contextualize the through the program execution was named as breakpoint. The problem this study is presenting. Furthermore, we consider term breakpoint was coined by Frances Elizabeth Holberton, in the mapping only the studies that present breakpoint types one of the six woman that worked at ENIAC within the context of debugging performed in IDEs for high- [36]. level software, using the respective debuggers. We did not “Well you know, the thing is we did develop the consider studies that describe debugging directly on hardware. one word that’s in the language today, which is Petrillo et al. [2], [17], Johnson [18], and Chern et al. [41] ‘breakpoint’, at that time. Because we actually did classify some breakpoint types as Static Breakpoint, and Dy- pull the wire to stop the programs so we could read namic Breakpoint. These are broad classifications that describe the accumulators off. This was, I mean we actually trigger criterion, that is, static breakpoint is for breakpoint broke the point, and that was where the word came triggered when executing the line of code, and dynamic being from.” - Frances Elizabeth Holberton [36]. when a condition is satisfied. However, our research reveals Iterative breakpoints were initially used in the 1970s, during that there are other criteria that determine the breakpoint type the era of mainframe computers, over OLIVER groups, such as how the breakpoint is added and the effect (CICS interactive test/debug). OLIVER was a proprietary when it is triggered. Furthermore, the classification proposed testing and debugging toolkit for interactively testing programs by the authors does not mention how the other breakpoint designed to run on IBM Customer Information Control System types found in IDEs could be grouped into these two categories (CICS) on IBM System/360/370/390 architecture. It provided of static and dynamic. instruction step, conditional program breakpoint, and storage Vasudevan [20] describes a generic way classification of alteration features for programs written in Assembly, COBOL, breakpoint types through Software Breakpoint and Hardware and PL/I. HLL (high-level language) users were able to see Breakpoint categories. Software Breakpoint is a category that and modify variables directly at a breakpoint [37]. includes all breakpoint types provided by IDEs through their The first use of breakpoints in IDEs (Integrated Devel- respective debuggers. Hardware Breakpoint is a category that opment Environment) was observed in in the describes breakpoints supported directly by the CPU, with 1980s. The Turbo Pascal IDE provided several debugging or without extra hardware support. Spinellis [21] classifies facilities, including single stepping, examination and changing software breakpoints by the term Code Breakpoint and clas- of variables, single breakpoints, and conditional breakpoints. sifies the breakpoints with related data as Data Breakpoint, In later versions, assembly language blocks could be stepped with or without hardware support. Beller et al. [42] present through in debugging process. The user could add breakpoints categories closer to those found in IDEs, using terms such as on variables and registers in an IDE window [38]. Line Breakpoint, Exception Breakpoint, Method Breakpoint, Currently, the use of breakpoint is an integral part of most Watchpoint, and others. However, we cannot find details of IDEs and one of the main resources used during the debugging each of the categories in the Beller’s study. In general, these process. Breakpoints are also features of symbolic debuggers articles describe generic categories and do not provide details like GBD [39] and WinDbg [40], so they are part of IDE or attributes of each category. Our study extends the categories instrumentation. Some popular IDEs that use breakpoints are: described by these authors, detailing them, describing subcat- Eclipse, IntelliJ, NetBeans, Visual Studio, , egories and possible relationships among them. and others. Moritz et al. [42] provide an attempt to group breakpoint In an elementary definition, breakpoint is a signaling per- types with the same nomenclature used by IDEs. Robert formed on the IDE that references a line of the source code in- Wahbe [43], Zhao et al. [25], and Copperman et al. [24] dicating to the debugger to suspend the system execution at the provide definitions for Data Breakpoints and Watchpoints to indicated point [18], [39], thus allowing the developer to see expand the boundaries on these breakpoint types. Dupriez

2 et al. [44] and Corrodi [45] define a new generation of aim to map the breakpoint types investigated by academic debuggers, which uses the concept of context breakpoint, literature. that is, breakpoints centered on objects already instantiated RQ3 - What are the breakpoint types according to in memory. practitioners? By answering this research question, we aim Kumar et al. [23] introduces the Watchpoint category as to discover what breakpoint types are according to software a subcategory of Software Breakpoint. The author introduces developers. this category to describe the specific breakpoint type called Behavioral Watchpoint and does not describe other possible B. Scope data-related breakpoint categories. Copperman et al. [24] and Zhao [25] present specific implementations for the breakpoint The scope of our research consisted of three data sources. type called Watchpoint and mention that this type is a synonym The first data source was the official documentation of the for the term Data Breakpoint. Wahbe [43] introduces the nine main IDEs used for developing high-level software. To Data Breakpoint category to describe data-related breakpoint select IDEs, we considered the crossing of the IDE rankings implementation strategies with or without hardware support. according to websites: Stack Overflow Survey 2019 [30], Our article extends these classifications and details Data Google Trends 2020 [31], and G2 2020 [32], as shown in Breakpoint and Watchpoint as synonyms within the context Figure 2. of data-related breakpoint. The second data source was four specialized academic libraries in software engineering, in which we performed IV. METHODOLOGY a systematic mapping, as shown in Figure 3. From these In this section, we describe the details our study design. libraries, we found the total of 931 candidate articles. Over We performed systematic mapping of academic literature and candidate articles, we applied some filters and selections gray literature, i.e., IDEs documentation. After that, we ad- resulting in 72 articles with breakpoint type descriptions. From ministered a survey questionnaire. From those instruments, we these 72 articles we performed the extraction of breakpoint analyzed and mapped breakpoint type definitions applying The type definitions. Grounded Theory method [33], achieving the main objective The third data source was the questionnaire administered in our study. Our methodology protocol steps is presented in to debugging practitioners, who are software developers. The Figure 1. The steps of the methodology are focus on answering questionnaire has 30 questions divided into three sections. We research questions RQ1, RQ2, and RQ3. collected 29 valid responses.

...... . Protocol Step 1 - IDEs ...... Documentation Data Selective ... Extraction - RQ1 ... In this subsection, we describe in detail the activities per- Open Step 4 - Mapping formed in each step of our study design, as shown in Figure Step 2 - Academic Analysis and Literature Data Mapping of 1. Definitions Extraction - RQ2 1) Step 1 - IDEs Documentation Data Extraction - RQ1: We searched breakpoint type descriptions in IDEs documen- Step 3 - Questionnaire Axial Data Extraction - RQ3 tation. For that, we indexed the main IDEs considering the The Grounded Theory (GT) Taxonomy crossing of the IDE rankings according to the Stack Overflow Survey 2019 [30], Google Trends 2020 [31], and G2 2020 Fig. 1. Methodology Protocol Steps – We extracted Breakpoint type definitions from three data sources, after it we processed it through The [32] websites, as presented in Figure 2. From the IDEs Grounded Theory method which generated as output a Mapping and a documentation, we extracted the breakpoint type definitions. Taxonomy. We describe below the five steps performed in this process. The Section IV is divided into three subsections. The first subsection IV-A, we present the three research questions. Initial Select Impurity and search top 10 removal duplicates The second subsection IV-B, we present the researched data removal IDE Stack Overflow sources. The third subsection IV-C, we describe in detail the 20 10 10 documetation Survey 2019 - data extraction activities performed in each step of our study design. IDE Ranking

Google Trends 30 10 9 9 9 A. Research Questions 2020 - IDE We divided our research goal into three research questions: Ranking IDEs breakpoint 10 10 RQ1 - What breakpoint types are found in IDEs? By G2 2020 - IDE 7 definitions answering this research question, we aim to map the break- Ranking point types used by gray literature as IDEs documentation Fig. 2. IDEs selection process and definition extraction – We selected nine from different IDEs and different technologies. IDEs by crossing the IDE rankings according to the Stack Overflow Survey RQ2 - What breakpoint types are studied in the aca- 2019, Google Trends 2020, and G2 2020 websites. From the selected IDEs, demic literature? By answering this research question, we we extracted breakpoint type definitions in IDEs documentation.

3 Initial search: first, we selected the IDEs from the rankings Initial Impurity Merge and search removal duplicates removal of three websites. The Stack Overflow platform offers a rank- Selection Snowballing ing of IDEs that was created through surveys made directly to cirteria 254 ACM Digital Library 160 Articles define Article data developers, conducted in 2019. The website presents more than breakpoint extraction types one ranking of IDE usage: all respondents, web developers, 192 80 Web of Science mobile developers, and SRE/DevOps. We considered these 281 170 63 9 72 four rankings and we unified them with simple arithmetic Scopus 430 189 Articles average. We did not find the IDE ranking related to the breakpoint definitions 2020 survey. We extracted Google Trends 2020 from the Computers & Applied Sciences Complete 15 Pypl website, which uses the search term metrics provided (EBSCOhost) 55 by Google Trends to generate a ranking of the most searched IDEs until November 2020. We selected the G2 website Fig. 3. Articles selection process and definition extraction – We selected 72 articles of journals and conference proceedings from specialized academic because it presents a ranking of the most widely used IDEs in libraries in software engineering. From the selected articles, we analyzed and the market according to the analysis of technical experts from extracted breakpoint type definitions. the G2 itself. Select top 10: we noticed that in all three rankings, approx- imately only the first five or six IDE tools were widely used keywords. In this initial search, we had a total of 931 articles or had easily accessible documentation. In this way, we chose found. not to extract the definitions from all of them, but to select (breakpoint* OR break-point* the top ten in each ranking. OR watchpoint* OR watch-point* Impurity removal: some of the development tools presented OR tracepoint* OR trace-point* by the websites consulted are not native IDEs, but text editors OR logpoint* OR log-point* that support the coupling of debuggers through plugins or OR catchpoint* OR catch-point*) extensions. In our study, we are only considering tools that are AND debug* IDEs, that is, that have the debugger and breakpoint features natively. Thus, at this step, we removed from the tools dataset Impurity removal: from the initial search, we found many those that are text editors. articles that are related to private software tools or articles that Merge and duplicates removal: we combined the top ten describe breakpoints in the context of hardware component IDEs in each ranking by removing duplicates and obtained a debugging. In this way, we applied an impurity removal new set of nine IDEs. reading the abstract and manually removing those articles. IDE documentation data extraction: finally, we analyzed After applying impurity removal, we had a total of 444 articles. the documentation available on the manufacturer’s website for Merge and duplicates removal: we combined all studies into each of the nine IDEs and we extracted the descriptions of a single dataset. Duplicated entries have been matched by title, the breakpoint functionalities, i.e., breakpoint type definitions, authors, year, and type of publication. After applying merge characteristics, and variations. The extracted data were stored and removal duplicates, we had a total of 281 articles. in a spreadsheet. Selection criteria: over the 281 articles, we applied some 2) Step 2 - Academic Literature Data Extraction - RQ2: We other selection criteria for removing the less significant arti- indexed 72 articles from journals and conference proceedings cles. We considered only journals and conference articles that that partially classify or partially define breakpoint types, as are a full paper with eight pages or more. We removed books, presented in Figure 3. To search for these articles, we used the PhD thesis, and short papers. We also removed full paper keyword search method, considering the following keywords: articles that are published from private universities libraries breakpoint, watchpoint, tracepoints, logpoint, catchpoint, and and that we could not have access to the entire text. After debug. In addition, we performed the study search method applying these selection criteria, we had a total of 170 articles. based on the reference to and from other studies, called Articles define breakpoint types: we performed a systematic snowballing. Finally, from the selected articles, we extracted search over the entire paper texts by the key-words: breakpoint, the breakpoint type definitions. We describe below the seven watchpoint, tracepoints, logpoint, and catchpoint. With this steps performed in this process. search, we selected the articles with breakpoint type descrip- Initial search: first, we selected the four main scientific re- tion, partially or completely. Furthermore, we selected the search specialized academic libraries in software engineering: article that describes functionalities and features of breakpoint ACM Digital Library, Web of Science, Scopus, The Collection types. This systematic search implied to partially read the of Computer Science Bibliographies and Computers & Ap- article. After applying this systematic search, we had a total plied Sciences Complete (EBSCOhost). The search query used of 63 articles. is presented below. After that, we applied the search request Snowballing: from 63 selected articles, we complemented to find all studies that deal with breakpoint and its variations the systematic search with a snowballing search. The main within the context of software debugging. The search request goal of this stage was to enlarge the set of potentially relevant was applied to the libraries considering title, abstract, and studies by considering each study selected in the previous

4 stages, and focusing on those papers either citing and cited and was initially proposed by Glaser et al. [33]. The set of by it. More technically, we performed a closed recursive procedures are composed by three steps: Open coding, Axial backward and forward snowballing activity. With snowballing, coding, and Selective coding. The steps are complementary to we selected nine more articles that have descriptions of break- each other. They are executed in an iterative way, so that the point types. In total, we selected 72 articles from which we concepts emerge from the analysis of data collection. performed data extraction. In the Open coding step, we analyzed definitions, features, Article data extraction: finally, we analyzed the articles properties, and characteristics of the breakpoint types de- and extracted breakpoint type definitions, i.e., the breakpoint scribed in the bibliography. We defined attributes, categories, descriptions, breakpoint type descriptions, and breakpoint and subcategories, and we use them to classify the definitions functionality descriptions. The extracted data were stored in of the concepts found. In the Axial coding step, we mapped a spreadsheet. We collected a total of 243 text extracts from each breakpoint type description classifying by the attributes. 72 articles, which we analyzed together with the text extracts We organized the attributes hierarchically to form a logical from the IDEs documentation and from the questionnaire data structure of relationships and similarity among the extracted in subsection IV-C4. concepts. In the Selective coding step, based on the feature 3) Step 3 - Questionnaire Data Extraction - RQ3: We similarities and relationships, we refined the categorization, administered a questionnaire for breakpoint practitioners to and we formulated theories to characterize each of the cate- gather opinions and understanding of the breakpoint types used gories and subcategories. Afterwards, we refined the theories, by software developers in the IDEs. and we created the main theory. Questions: the questionnaire has 30 questions divided into The steps are incremental, so the analysis must be done three sections: 1) general information about the respondent; iteratively among the steps, complementing as needed [33], 2) discursive questions about the developers understanding of [46]. In this way, we applied The Grounded Theory method the different breakpoint types; and 3) objective questions about on our context. Details of the result of each Grounded Theory understanding the definitions of the different breakpoint types step are described in section V. used in the IDE documentation. The sections have six, 18, and six questions, respectively. Section number two has discursive V. BREAKPOINT MAPPING questions, and section number three has objective questions. In this section, we present the artifacts and results from Participants: the participants of our questionnaire were The Grounded Theory data analysis and mapping performed. included by the following criteria: 1) adult people who have This section is outlined as follows: in subsection V-A, we experience using IDE; 2) and adult people who have expe- present the attributes and attribute groups emerged from Open rience using breakpoints. The participants that did not meet coding. Followed by that, we present the breakpoint types the following criteria were not included: 1) anyone who had mapped from Axial coding in subsection V-B. The main no experience using IDE; 2) anyone who had no experience breakpoint types emerged from Selective coding are presented using breakpoints; 3) and anyone who refused to answer the in subsection V-C. The research questions are answered in questionnaire. subsection V-D. Ethics: the questionnaire was evaluated and consented on February 5, 2021 by the Research Ethics Committee of the A. Open coding output University of Quebec at Chicoutimi (CER-UQAC) and cer- tificated by the Ethics Certificate Reference No: 2021-690. In The Grounded Theory Open coding step, we analyzed the Before being released, the questionnaire was run to two pilot definitions, the features, the properties, and the characteristics respondents for testing and verification purposes. of the breakpoint types described both in the IDEs documen- Dissemination: we disseminated the questionnaire through tation, selected articles, and from the questionnaire. From this, social media, software development communities on Discord we extracted 39 attributes grouped into four categories, as servers, and directly to software developers. It is not possible presented below: to measure the number of people reached by the disclosure. • Defined at: line of code, statement in a line of code, All responses were anonymous. by configuration, over object, over property object, field, Responses: we got a total of 70 responses, 29 of which were global variable, local variable, over a message, written in valid and 41 were invalid. Within the 29 valid, ten were partial code, block of memory; and 19 were complete. Only the valid, partial, and complete • Triggered by: line / statement hit, data changes, memory ones were considered for analysis in subsection IV-C4, that is, address changes, expression satisfied, expression value a total of 29 responses. change, expression satisfied an object id, conditional 4) Step 4 - Analysis and Mapping of Definitions: In the hit, hit count, event triggered, method / function called, data analysis subprocess, we applied The Grounded Theory method enter, method exit, class is loaded, exception (GT) method. The Grounded Theory is an interpretive and throw, field read, field write, test failure, message reaches qualitative research method that uses a systematic set of the head queue, thread start / terminated; procedures to develop an inductively derived theory about a • Effect: does not stop execution, stop execution, show log phenomenon. It is widely used in many social science fields message;

5 TABLE I BREAKPOINT TYPESON IDES

Breakpoint Terms NetBeans Eclipse Visual Studio Visual Studio Code IntelliJ PhpStorm PyCharm Total Breakpoint X X X X X X X X X 9 Exception Breakpoint X X X X X X X 7 Line Breakpoint X X X X X 5 Method Breakpoint X X X X X 5 Tracepoint X X 2 Object Breakpoint X X 2 Function Breakpoint X X 2 Data Breakpoint X X 2 Breakpoint Log X X 2 Breakpoint Stack Trace X X 2 Class Breakpoint X X 2 Watchpoint X X 2 Data Breakpoint C++ X 1 Logpoint X 1 Conditional Breakpoint X 1 Inline Breakpoint X 1 Field Watchpoint X 1 Field Breakpoint X 1 Variable Breakpoint X 1 Thread Breakpoint X 1 Logging Breakpoint X 1 Event Breakpoint X 1 Symbolic Breakpoint X 1 Test Failure Breakpoint X 1 Total 9 7 6 6 6 6 6 5 3

TABLE II BREAKPOINT TERMSON ARTICLES

Article Year Total Line Data Field Code Class Static Source Method Stateful Symbol Variable Software Selective Message Dynamic Hardware Temporal Simulated Exception Tracepoint Declarative Breakpoint Conditional Watchpoint Data Access Method Mes- sage Message con- ditional Reverse Watchpoint Reverse Expression Watchpoint Conditional Watchpoint Conditional line Local variable Watchpoint Conditional Data Expression Watchpoint Transition Watchpoint [42] 2018 X X X X X X X X 8 [44] 2017 X X X X X 5 [45] 2016 X X X X X 5 [18] 1982 X X X X X 5 [2] 2017 X X X X X 5 [47] 2012 X X X X X 5 [48] 1999 X X X X 4 [49] 2003 X X X X 4 [50] 2012 X X X X 4 [20] 2009 X X X X 4 [17] 2019 X X X X 4 [51] 1989 X X X X 4 [24] 1995 X X X 3 [52] 1996 X X X 3 [53] 2014 X X X 3 [21] 2006 X X X 3 [54] 2014 X X X 3 [55] 2012 X X X 3 [56] 2020 X X X 3 Total 11 11 6 6 5 3 3 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

6 • Complementary: conditional as derivative action, has The attribute category Triggered By aggregates the group of derivative actions, is a derivative action, has hardware attributes that characterize how the breakpoint is triggered support, multithread, not enough information. after being added to the source code. That is, triggered by the execution of a tuple of code, by data changing, by triggering an

code breakpoint breakpoint log event or other way. The attribute category Effect aggregates the Line breakpoint Tracepoint debugger breakpoint breakpoint stack trace source breakpoint logpoint group of attributes that characterize the behavior presented by inline breakpoint logging breakpoint

Thread breakpoint the debugger and by the IDE once the breakpoint is triggered. message breakpoint conditional line breakpoint Conditional breakpoint conditional data breakpoint In other words, if it interrupts the system execution or not or conditional watchpoint exception breakpoint message conditional breakpoint Event breakpoint if it presents a log message. The attribute category Comple- method breakpoint class breakpoint mentary aggregates the group of attributes that complement data breakpoint function breakpoint variable breakpoint Watchpoint field watchpoint the characteristics of each breakpoint and that do not fit into field breakpoint object breakpoint the three previous categories. For example, if the breakpoint type has: conditional, derived actions, hardware support, and Fig. 4. The Grounded Theory Open Output – The main breakpoint types others. and the terms groups – in the boxes – described by the data sources. With the attributes defined, we related each attribute to each of the breakpoint type description taken from the documen- tation, articles, and questionnaire. As a resulting artifact, we obtained a Mapping table with the relation of each attribute to each breakpoint type extracted from each of the data source descriptions. The Mapping table could be consulted by checking the dataset used in the search in the Zenodo repository [57]. Breakpoint Types

B. Axial coding output 0 10 20 30 40 50

Definition attempts by data source To relate the attributes to each of the breakpoint type descriptions, we extracted the terms of the breakpoint types Fig. 5. The Grounded Theory Axial Output – Number of times that all used by the data sources, thus contemplating The Grounded data sources presented some description of each breakpoint type. Theory Axial coding step. Each breakpoint type term described by the data source is an attempt by the IDEs documentation, academic articles, or questionnaire respondents to explain the TABLE III behavior and characteristics of each breakpoint type. THE GROUNDED THEORY AXIAL OUTPUT – CONTINUATION In Figure 5 and in Table III, we present a compilation of the Described class breakpoint, debugger breakpoint, event breakpoint, field breakpoint type terms used by the three data sources. The chart 3 times watchpoint, hardware breakpoint, simulated breakpoint, soft- in Figure 5 shows the number of times that all data sources ware breakpoint presented some description to try to define each breakpoint Described breakpoint log, breakpoint stack trace, conditional data break- type. For breakpoint types that data sources define three or 2 times point, field breakpoint, function breakpoint, logpoint, object breakpoint, source breakpoint less times, we group them in Table III, i.e., the table is the Described behavioral watchpoint, collective breakpoint, concurrent break- chart continuation. 1 time point, conditional line breakpoint, conditional watchpoint, In addition, we created two breakpoint type mapping tables: control breakpoint, data access breakpoint, data breakpoint c++, declarative breakpoint, declarative tracepoint, expression 1) Table I: breakpoint types on the IDEs, and 2) Table II: watchpoint, expression-transition watchpoint, inline break- breakpoint types on articles. For both tables, the total column point, lambda breakpoint, local variable watchpoint, logging is the count of incidences that a given term is described by breakpoint, message conditional breakpoint, meta-breakpoint, method message breakpoint, pseudo-breakpoint, reverse ex- the IDE or article as a breakpoint type. The total line is the pression watchpoint, reverse watchpoint, scan-points, selective count of breakpoint types that each IDE or article describes. breakpoint, shared breakpoint, stateful breakpoint, statement Considering the incidence of descriptions by terms pre- breakpoint, symbol breakpoint, symbolic breakpoint, temporal breakpoint, test failure breakpoint, thread breakpoint, transition sented in the compilation in Figure 5 and considering the watchpoint, unconditional breakpoint Mapping table in the dataset in the Zenodo repository [57], aThis table is the continuation of the chart in Figure 5. we identified that some terms have similar distribution of bThis table contains breakpoint types that have been described by data attributes. In ther words, the data sources give different names sources three times or less. for practically identical breakpoint types. Thus, analyzing the attribute distribution that characterize each term, the terms The attribute category Defined At aggregates the group of with the highest incidence and degree of similarity among attributes that characterize the place or way in which the the characteristics of each term, we found that the terms breakpoint is added in the source code. That is, added in a that represent the main breakpoint types described by the line of code, through configuration, in a variable or others. data sources are: Line Breakpoint, Conditional Breakpoint,

7 Thread Breakpoint, Event Breakpoint, Watchpoint, Tracepoint. to the IDEs analyzed. We present the distinct terms found In Figure 4, we present how the terms were grouped. crossing with each IDE in Table I. In Figure 4, we do not include the term Breakpoint as a breakpoint type because, despite being cited by all IDEs, RQ2 - What breakpoint types are studied in the this term is used both by the IDEs documentation and by academic literature? To answer Research Question 2, we the articles as a concept of breakpoint in a generic way. filtered and extracted from the Mapping table in dataset [57] Thus, to represent the breakpoint type, we observed all distinct breakpoint type names related to the academic that the term Line Breakpoint is the second one in the literature analyzed. The distinct terms found are listed in Table Table I most used. The terms Software Breakpoint, Hardware II. Breakpoint, Static Breakpoint, and Dynamic Breakpoint are RQ3 - What are the breakpoint types according to prac- not breakpoint types, but rather intermediate categories that titioners? To answer Research Question 3, we filtered from suggest the grouping of some breakpoint types. For example, the Mapping table in dataset [57] all distinct breakpoint type Software Breakpoint is a synonym for the term Breakpoint, names related to the questionnaire analyzed. The breakpoint while Hardware Breakpoint groups specific breakpoint types type terms brought by the questionnaire were the same as those for direct debugging in hardware. Complementarily, the term found in RQ1 and RQ2. The new terms we found through the Dynamic Breakpoint groups breakpoint types that depend on questionnaire are: events or conditions to be triggered, such as Conditional Breakpoint, Event Breakpoint, and Watchpoint, while the term Static Breakpoint groups breakpoint types that are triggered • Debugger Breakpoint: command debugger; written in without dependence on other factors. the source code, as the language’s own command, to force The term Conditional Breakpoint is sometimes described as the system to stop running. a breakpoint type or at other times as a complementary action • Statement Breakpoint: similar to Lambda or Inline of some other breakpoint type. We discuss this issue further breakpoint. A breakpoint over part of the line of code, in Section VII. The term Watchpoint is the grouping of all or execution tuple. terms that define breakpoint types related to data change, either • Lambda Breakpoint: similar to Inline breakpoint. A directly in memory or object instance. The term Tracepoint breakpoint over part of the line of code, or execution grouped all the terms that define breakpoint types that do tuple. not stop the system running when triggered, but with logging • Shared Breakpoint: similar to Tracepoint. A breakpoint a message. We also discuss the Tracepoint issue further in that logs messages and does not stop the system from Section VII. The term Event Breakpoint grouped all the terms running. It is suitable for debug by logging messages on that define breakpoint types that are triggered when a specific systems deployed on servers. event happens during execution, such as calling a specific method, triggering an exception, instantiating a class or calling a specific function. We selected the term Thread Breakpoint The sample of questionnaire respondents is composed of because it more accurately represents the breakpoint types 29 different respondents. The average age of respondents applied to distributed programming and multithreading, be- is 31.75 years. From 29 respondents, 67.86% hold some cause the term Message Breakpoint gets confused with the higher education degree, while 32.14% reported not holding a breakpoint type that emits log messages, such as Tracepoint. higher education degree. The average years of experience with software development was 9.4 years. From 29 respondents, C. Selective coding output 20.68% claim to be teachers, 58.62% claim to be students, The output of Selective coding step are structured defini- and 58.62% claim to be software professionals. From the tions for each of the six breakpoint types. We extracted the total sample, 68.96% reported that, according to their work definitions from relevant terms, which represents the main or university experience, they have studied or have been breakpoint types. Furthermore, we used the four attribute cat- introduced to breakpoint types. egories Defined At, Triggered By, Effect, and Complementary to structure the definitions. The definitions are presented in boxes 1, 2, 6, 3, 4, and 5. We evaluated the definitions from the Definition 1 — Line Breakpoint questionnaire given to software developers. Each respondent gave a score from one to ten for each of the definitions – being Line Breakpoint is the breakpoint type associated with a line one to totally disagree and ten to totally agree. The arithmetic of code. Where: It is usually inserted over the line of code or eventually over a code tuple contained in the line of code, or else average of the scores obtained is presented in Table IV. written directly in the code as a code stop instruction. Trigger: It D. Research questions output is usually triggered when the line of code is hit during the system execution. Effect: When triggered, it causes the system execution RQ1 - What breakpoint types are found in IDEs? To interruption. answer Research Question 1, we extracted from the Mapping table in dataset [57] all distinct breakpoint type names related

8 Definition 2 — Conditional Breakpoint TABLE IV BREAKPOINT TYPE DEFINITION SCORE Conditional Breakpoint is a breakpoint type associated with an expression. Where: It is usually inserted as a specific breakpoint Definition of Score type over a line of code, or it could be associated with other Line Breakpoint 8.3 breakpoint type such as a configuration usually called property Conditional Breakpoint 9.0 or action. Trigger: It is triggered whenever the condition related Tracepoint 8.6 to its associated expression is true. This condition could be related Thread Breakpoint 8.2 to data, a count or the existence of a specific object. Effect: When Event Breakpoint 8.2 triggered, it causes the system execution interruption. Watchpoint 8.0 aThe score represents the degree of developer agreements on each of the extracted breakpoint type definitions. Definition 3 — Thread Breakpoint Thread Breakpoint is a breakpoint type applied to distributed systems and parallel programming, associated with threads. VI.BREAKPOINT TAXONOMY Where: It is usually related to a thread or thread message through In this section, we present the compiled artifact from a configuration option. Trigger: It is usually triggered when the the mapping performed. From the grouping obtained by the thread with which it is associated starts or stops. Effect: When mapping, we were able to determine two major breakpoint sub- associated with a thread message, it is triggered when the message with which it is associated is at the top of the execution queue. categories called Static Breakpoint and Dynamic Breakpoint. When triggered, it causes the entire system execution interruption Within these two subcategories, we managed to organize the or just the thread with which it is associated. Complementary: Fur- main breakpoint types found, described in Figure 4. The result thermore, it could contain associated expressions that determines of this organization is the Taxonomy Diagram shown in Figure its activation only when the condition is true. 6.

Category Definition 4 — Event Breakpoint Breakpoint (Software Breakpoint) Event Breakpoint is a breakpoint type that is system execution event oriented. Where: It is usually inserted by configuration Subcategory option and eventually over a specific line of code. Trigger: It Static Breakpoint Dynamic Breakpoint is usually triggered when the event to which it is associated is triggered by the system, such as calling a method, creating an Breakpoint types object of a specific class, or throwing an exception. Effect: When Line Breakpoint Watchpoint Thread Breakpoint triggered, it causes the system execution interruption. Comple- mentary: Furthermore, it could contain associated expressions that Conditional Breakpoint Event Breakpoint determines its activation only when the condition is true. Fig. 6. Taxonomy Diagram – The main breakpoint types organized in one main category and two subcategories. Definition 5 — Watchpoint The first level of the Taxonomy Diagram is the Breakpoint Watchpoint is a type of data-oriented breakpoint and could be category. This category includes types of software-related directly related to a memory address. Where: It is usually inserted breakpoints, that is, those that are used in debuggers through over a line of code that contains a field or variable or through configuration options. Trigger: It is usually triggered when the IDEs for high-level software and that are not used in debug- data it is associated with is read, written or changed. It could also ging directly on hardware, i.e., low-level software. In addition, be triggered when the memory address to which it is associated considering the analysis of the descriptions extracted from is read, written or changed. Effect: When triggered, it causes the the data sources, the main characteristic of a breakpoint is system execution interruption. Complementary: Furthermore, it to cause the system execution to stop. Thus, all breakpoint could contain associated expressions that determines its activation types that do not cause the system execution to stop, such only when the condition is true. as the Tracepoint type, are not aggregated in the Breakpoint category. Finally, the Breakpoint category is often referred to Definition 6 — Tracepoint in academic literature as Software Breakpoint, to differentiate it from the Hardware Breakpoint category, so we also included Tracepoint logs a message on console output when execution the term Software Breakpoint in the diagram as a synonym for through it. Where: It is usually inserted over the line of code. Breakpoint. Trigger: It is usually triggered when the line of code with which it is associated is the next line to be executed by the system. Effect: The second level of the Taxonomy Diagram are the sub- When triggered, it displays a previously defined debug message categories Static Breakpoint and Dynamic Breakpoint. We on the console, not interrupting the system execution. Com- extended these subcategories from academic literature. Static plementary: Furthermore, it could contain associated expressions Breakpoint encompasses all breakpoint types that trigger when that determines its activation only when the condition is true. the line of code at which the breakpoint is located is the next line to be executed. Complementarily, Static Breakpoint

9 includes the breakpoints types whose triggering is uncondi- The term Conditional Breakpoint is mentioned in one Visual tional, that is, they do not depend on the evaluation of logical Studio Code IDE documentation as being a breakpoint type. expressions or extra events. Dynamic Breakpoint category is However, in the other eight IDEs, no term determines a the opposite of Static Breakpoint. This category includes all breakpoint type as Conditional, but conditionals are extra breakpoint types whose triggering does not depend only or actions that can be applied on which breakpoint type. In some necessarily on the execution of the line of code in which the articles, we find examples of using the term Conditional as: breakpoint is located, but on some extra event or evaluation of Conditional Data Breakpoint, Conditional Watchpoint, Mes- some logical expression. For example, to trigger a Dynamic sage Conditional Breakpoint and Conditional Line Breakpoint. Breakpoint, it is necessary that the condition assigned to the With this, we observe that Conditional Breakpoint should not breakpoint is true, or that the number of line executions be referred to as a specific breakpoint type, but rather as com- reach a certain number, or that the execution is the call of plementary actions or features of the breakpoint, regardless of a certain method, or that an object of a certain class or type the breakpoint type. Through this, when any breakpoint type is instantiated, and others. has conditional, it could be categorized in Dynamic Breakpoint The third level of the Taxonomy Diagram is composed by category of taxonomy in Figure 6. breakpoint types themselves. The breakpoint type Line Break- The Tracepoint and Logpoint types are characterized as point is the only one with static breakpoint characteristics that breakpoints that do not stop the system execution, but they appears more frequently according to the grouping shown in present log messages in the console output. However, the main Figure 5 and Table III. The Line Breakpoint type represents characteristic that determines a breakpoint is stop the system the traditional breakpoint, that is, the breakpoint type known running when the breakpoint is triggered. In addition, some according to the questionnaire respondents, being cited in 23 IDEs present the functionality to log messages to console of the 29 valid responses. The second term with practically output without classifying this functionality as a specific identical characteristics is Code Breakpoint, which can be breakpoint type, but as an action derived from the breakpoint. attributed as a synonym for Line Breakpoint. From this, we observe that Tracepoint should be presented by Also in the third level of Taxonomy Diagram, we find the IDEs documentation as a feature to support debugging, other breakpoint types in the Dynamic Breakpoint subcategory. and it should not presented as a breakpoint type. The Conditional Breakpoint is a breakpoint type with similar Analyzing the academic articles, we found descriptions of characteristics of the Line Breakpoint, but with associated breakpoint types related to specific technology proposed by triggering conditions. These conditions are in general logical the article. We find as well, improvements or specializations expressions that trigger the breakpoint when the line of code of the breakpoint types carried out by the study of those is the next line to be executed and the evaluated expression articles. These breakpoint type specializations are named with returns true value. The Watchpoint is a breakpoint type related specific terms as we can see in Table II. Some examples to data change, either directly in memory or object instance. of these terms are: Expression Watchpoint, Transition Watch- In other words, for the Watchpoint to be triggered, the data point, Data Access Breakpoint, Stateful Breakpoint, Temporal associated with a variable, a field or a memory address must Breakpoint, Declarative Breakpoint, Behavioral Breakpoint, be read, written or changed. In this way, this breakpoint type Expressions-Transition Watchpoint, Reverse Watchpoint, Con- is also dynamic because it depends on the data changes. current Breakpoint, Collective Breakpoint, Meta-Breakpoint, Finally, in the third level, we describe the Thread Breakpoint Simulated Breakpoint, Scan-Points, and Message Conditional and Event Breakpoint types. They are breakpoint types with Breakpoint. Based on this, we observe that the more specific similar characteristics regarding their triggers. Both are trig- is the breakpoint type, the more specific the name given in gered when an event happens. This event has no association the literature, thus distancing it from the terminology used by with a condition or evaluation of a logical expression. The the IDEs documentation and from the terminology known by difference between them is that Thread Breakpoint has features software developers. applied in distributed programming and multithreading while The analysis of academic articles also revealed that some the Event Breakpoint is applied in monothread executions. authors use the terms Watchpoint and Data Breakpoint as For example, Thread Breakpoint triggers when the thread with synonyms. Articles with technology usages based on the GDB which it is associated starts or stops or when associated with debugger use the term Watchpoint. The same happens for the thread message, it is triggered when the message with IDEs that have GDB based debuggers. For example, in Eclipse which it is associated is at the top of the execution queue. On we find the term Watchpoint. Other articles tend to use the the other hand, Event Breakpoint is triggered when a specific term Data Breakpoint. In contrast, the questionnaire revealed method or specific function is called, an exception is thrown that developers sometimes confuse the term Watchpoint with and others. an in-memory value monitoring functionality, also known as Watch windows or Watches, or confuse it with a message VII.DISCUSSION logging functionality, such as the Tracepoint breakpoint type. In this section, we discuss points observed during the anal- Based on this, we observe that the suitable term to be used, ysis and mapping that have an impact on academic literature, whether in academic literature or in IDE documentation, is for IDE development companies and for developers in general. Data Breakpoint.

10 Analyzing the terms found in the IDEs documentation, Finally, we evaluated the breakpoint type descriptions ex- which are presented in Table I, we observe the following tracted from the questionnaire administered. However, we did terms: Inline Breakpoint, Field Watchpoint, Field Break- not carry out a controlled experiment or case study with the point, Variable Breakpoint, Logging Breakpoint, and Symbolic application of breakpoint type descriptions within the software Breakpoint. Each of these terms is described by only one industry to verify that the descriptions are useful to developers. IDE. However, they have the same set of characteristics as the most widespread terms in other IDEs. For example: IX.CONCLUSION Inline Breakpoint, Field Breakpoint, Variable Breakpoint and In this article, we present a mapping of the breakpoint types Symbolic Breakpoint are equivalent to Line Breakpoint; Field at the software level among different IDEs. We mapped the Watchpoint is equivalent to Data Breakpoint or Watchpoint; gray literature on the documentation of the nine main IDEs Logging Breakpoint is equivalent to Tracepoint. We observe used by developers according to the three public rankings. that IDE development companies should use the same terms In addition, we performed a systematic mapping of academic and not create new terms to define their own breakpoint types. literature over 72 articles describing breakpoint types. Finally, The analysis of the questionnaire revealed that break- we analyzed the developers understanding of the main break- point practitioners, that is, software developers, use some point types through a questionnaire. instructions added to the source code to stop the system We present three main contributions: (1) the mapping of the execution. These instructions are, for example: debugger and breakpoint types in the main IDEs and academic literature, (2) System.Diagnostic.Debugger.Break(). Although this type of compiled definitions of breakpoint types, and (3) a breakpoint code statement-based breakpoint does require source code taxonomy. As results, we built a breakpoint type mapping table change and is a feature of the language rather than of the of the main IDEs, as well as a breakpoint type mapping table IDE, we understand that it should be considered a breakpoint for academic papers. Furthermore, we present breakpoint type type. In addition, the questionnaire reveals that practitioners definitions structured according to attributes mapped in The understand that breakpoint type based on code instruction is Grounded Theory method. Finally, we present a taxonomy called Code Breakpoint. However, the term Code Breakpoint diagram with two breakpoint type categories. We also bring found in academic literature is used as a generic term or the discussion about terms used, such as Conditional Break- intended to describe Line Breakpoint. Therefore, academic point and Tracepoint, should not necessarily be presented as literature should avoid using the term Code Breakpoint to breakpoint types on IDEs documentation. With the results, describe breakpoint or Line Breakpoint. we were able to describe the breakpoint types found in the Finally, analyzing the score attributed by the questionnaire IDEs (RQ1), in the academic literature (RQ2), and by the respondents to each of the breakpoint type definitions, as practitioners (RQ3). shown in Table IV, we observed that the degree of agree- Our breakpoint definitions provide to developers a common ment is between grade eight and grade nine, in a scale lexicon to IDEs. In addition, our study contributes to avoid from one to ten. This degree of agreement represents that misleading breakpoint names among academic literature and the compiled descriptions are readable and understandable to IDE documentation. Furthermore, as suggested by Aniche et software developers. From this, we believe that describing the al. [29], our taxonomy and definitions can support future functioning of each breakpoint type considering answering the debugging research. four attribute categories Defined At, Triggered By, Effect, and As future work, we plan to perform an controlled experi- Complementary can be a practice to be used by the software ment to evaluate our definitions and study breakpoint types on industry and by academic literature. hardware level. Furthermore, we plan a study on breakpoint types from documentation related to standalone debuggers. VIII.THREATS TO VALIDITY REFERENCES In this study, we considered the main IDEs used in the [1] A. S. Tanenbaum and W. H. Benson, “The people’s time sharing system,” software industry and the articles from the academic literature. Software: Practice and Experience, vol. 3, no. 2, pp. 109–119, 1973. We have not reviewed all possible IDEs. In this way, this study [2] F. Petrillo, H. Mandian, A. Yamashita, F. Khomh, and Y.-G. Gueh´ eneuc,´ “How do developers toggle breakpoints? observational studies,” in 2017 is not an exhaustive research, and we may have not included IEEE International Conference on Software Quality, Reliability and some breakpoint types, mainly those of particular technologies Security (QRS), 2017, pp. 285–295. or proprietary technologies. Besides that, we searched only [3] M. Beller, N. Spruit, and A. Zaidman, “How developers debug,” PeerJ Preprints, vol. 5, p. e2743v1, 2017. documentation related to IDE debuggers, we did not include [4] “Eclipse documentation.” [Online]. Available: https://help.eclipse.org/ descriptions of breakpoint types from documentation related \2021-03/index.jsp to standalone debuggers. [5] “Debugging the eclipse ide for developers.” [Online]. Available: https://www.eclipse.org/community/eclipse newsletter/2017/ In our mapping, we only considered breakpoints related june/articl\e1. to high-level software development oriented IDEs. However, [6] “Intellij idea documentation.” [Online]. Available: https://www. we found in academic literature descriptions of breakpoint \.com/help/idea/using-breakpoints.html#set-breakpoints [7] “Use breakpoints in the visual studio debugger.” types aimed at debugging directly on hardware, i.e., low-level [Online]. Available: https://docs.micros\oft.com/en-us/visualstudio/ software. debugger/using-breakpoints?view=vs-2019

11 [8] “User guide - debugging.” [Online]. Available: https://code.v\ [34] T. Haigh, P. M. Priestley, M. Priestley, and C. Rope, ENIAC in action: isualstudio.com/docs/editor/debugging Making and remaking the modern computer. MIT press, 2016. [9] “What is the difference between watchpoints and breakpoints in [35] T. Haigh, M. Priestley, and C. Rope, “Engineering” the miracle of the eclipse debugging?” [Online]. Available: https://www.quora.com/ eniac”: Implementing the modern code paradigm,” IEEE Annals of the What-is-the-difference-between-watchpoints-and-breakpoints-in-Eclipse-debuggingHistory of Computing, vol. 36, no. 2, pp. 41–59, 2014. [10] “Difference between breakpoint on method signature [36] H. S. Tropp and F. E. Holberton, “Computer oral history collection, vs breakpoint on first line in method.” [Online]. 1969-1973, 1977,” p. 56–57, Apr 1973. [Online]. Available: https: Available: https://stackoverflow.com/que\stions/26364634/ //amhi\story.si.edu/archives/AC0196 bart730427.pdf difference-between-breakpoint-on-method-signature-vs-breakpoint-on-first-line-in[37] B. Cristobal, IBM Oliver (CICS Interactive Test/Debug). Cede [11] “Difference between watch point and break point.” Publishing, 2011. [Online]. Available: https://books.google.ca/books? [Online]. Available: https://answers.sap.com/questions/4181705/ \id=JbkeygAACAAJ difference-between-watch-\point-and-break-point. [38] D. Carroll, Programming with Turbo Pascal, ser. Byte books. [12] “Difference between watchpoint and break- Micro Text Productions, 1985, no. v. 1. [Online]. Available: point?” [Online]. Available: https://www.lessbro.com/ https://books.google.ca/books?id=jxESAQAAMAAJ Difference-between-watchpoint-and-breakpoi\nt-/49 [39] . Stallman and S. Shebs, Debugging with GDB - The GNU Source- [13] “Difference between a static breakpoint and dynamic breakpoint.” [On- Level Debugger. GNU Press, 2002. line]. Available: https://gocoding.org/difference-between-a-static-br\ [40] D. Vostokov, WinDbg: A Reference Poster and Learning Cards. Open- eakpoint-and-dynamic-breakpoint/ Task, 2008. [14] “Understand conditional breakpoints in c++.” [On- [41] R. Chern and K. De Volder, “Debugging with control-flow breakpoints,” line]. Available: https://visualstudiomagazine.com/articles/2016/09/01/ in Proceedings of the 6th International Conference on Aspect- understand-condit\ional-breakpoints.aspx oriented Software Development, ser. AOSD ’07. New York, [15] “Break when value changes: Data breakpoints NY, USA: ACM, 2007, pp. 96–106. [Online]. Available: http: for .net core in visual studio 2019.” [Online]. //doi.acm.org/10.1145/1218563.1218575 Available: https://devblogs.microsoft.com/visual\studio/ [42] M. Beller, N. Spruit, D. Spinellis, and A. Zaidman, “On the dichotomy break-when-value-changes-data-breakpoints-for-net-core-in-visua\ of debugging behavior among programmers,” in 40th International l-studio-2019/ Conference on So ware Engineering, ICSE, 2018, pp. 572–583. ACM SIGPLAN Notices [16] “Break when value changes: Data breakpoints for .net core in [43] R. Wahbe, “Efficient data breakpoints,” , vol. 27, visual studio 2019.” [Online]. Available: DataBreakpoints\OT1\ no. 9, pp. 200–212, 1992. textendashVisualStudio201715.8Update [44] T. Dupriez, G. Polito, and S. Ducasse, “Analysis and exploration for new generation debuggers,” in Proceedings of the 12th edition of the [17] F. Petrillo, Y.-G. Gueh´ eneuc,´ M. Pimenta, C. D. S. Freitas, and International Workshop on Smalltalk Technologies, 2017, pp. 1–6. F. Khomh, “Swarm debugging: The collective intelligence on interactive [45] C. Corrodi, “Towards efficient object-centric debugging with declarative debugging,” Journal of Systems and Software, vol. 153, pp. 152–174, breakpoints.” in SATToSE, 2016, pp. 32–39. 2019. [46] K. Charmaz, Constructing grounded theory: A practical guide through [18] M. S. Johnson, “A software debugging glossary,” ACM Sigplan Notices, qualitative analysis. sage, 2006. vol. 17, no. 2, pp. 53–70, 1982. [47] E. G. Boix, “Handling partial failures in mobile ad hoc network [19] D. Keppel, “Fast data breakpoints,” 1993. applications: From programming language design to tool support,” Vrije [20] A. Vasudevan, “Re-inforced stealth breakpoints,” in 2009 Fourth In- Universiteit Brussel, Faculty of Sciences, Software Languages Lab (Ph. ternational Conference on Risks and Security of Internet and Systems D. Thesis), 2012. (CRiSIS 2009). IEEE, 2009, pp. 59–66. [48] R. Lencevicius, Query-based debugging. University of California, Santa [21] D. Spinellis, “Debuggers and logging frameworks,” IEEE software, Barbara, 1999. vol. 23, no. 3, pp. 98–99, 2006. [49] K. Maruyama and M. Terada, “Debugging with reverse watchpoint,” in [22] K. Arya, T. Denniston, A. Rabkin, and G. Cooperman, “Transi- Third International Conference on Quality Software, 2003. Proceedings. tion watchpoints: Teaching old debuggers new tricks,” arXiv preprint IEEE, 2003, pp. 116–123. arXiv:1703.10864, 2017. [50] A.-M. Visan, “Temporal meta-programming: treating time as a spatial [23] A. Kumar, P. Goodman, A. Goel, and A. D. Brown, “Behave or be dimension,” Ph.D. dissertation, Northeastern University, 2012. watched: Debugging with behavioral watchpoints,” in Proceedings of [51] C. E. McDowell and D. P. Helmbold, “Debugging concurrent programs,” the 9th Workshop on Hot Topics in Dependable Systems, 2013, pp. 1–6. ACM Computing Surveys (CSUR), vol. 21, no. 4, pp. 593–622, 1989. [24] M. Copperman and J. Thomas, “Poor man’s watchpoints,” ACM SIG- [52]A.J ahne,¨ S. D. Urban, and S. W. Dietrich, “Peard: A prototype envi- PLAN Notices, vol. 30, no. 1, pp. 37–44, 1995. ronment for active rule debugging,” Journal of Intelligent Information [25] Q. Zhao, R. Rabbah, S. Amarasinghe, L. Rudolph, and W.-F. Wong, Systems, vol. 7, no. 2, pp. 111–128, 1996. “How to do a million watchpoints: Efficient debugging using dynamic [53] T. Kosar, M. Mernik, J. Gray, and T. Kos, “Debugging measurement instrumentation,” in International Conference on Compiler Construction. systems using a domain-specific modeling language,” Computers in Springer, 2008, pp. 147–162. industry, vol. 65, no. 4, pp. 622–635, 2014. [26] L. Chew and D. Lie, “Kivati: fast detection and prevention of atomicity [54] C. Seaton, M. L. Van De Vanter, and M. Haupt, “Debugging at full violations,” in Proceedings of the 5th European conference on Computer speed,” in Proceedings of the Workshop on Dynamic Languages and systems, 2010, pp. 307–320. Applications, 2014, pp. 1–13. [27] P. Sommer and B. Kusy, “Minerva: Distributed tracing and debugging in [55] B. Alsallakh, P. Bodesinsky, A. Gruber, and S. Miksch, “Visual tracing wireless sensor networks,” in Proceedings of the 11th ACM Conference for the eclipse java debugger,” in 2012 16th European Conference on on Embedded Networked Sensor Systems, 2013, pp. 1–14. Software Maintenance and Reengineering. IEEE, 2012, pp. 545–548. [28] J. Jang and B. B. Kang, “Revisiting the arm debug facility for os [56] A. Kumar, Z. Wang, S. Ni, and C. Li, “Amber: a debuggable dataflow kernel security,” in 2019 56th ACM/IEEE Design Automation Conference system based on the actor model,” Proceedings of the VLDB Endowment, (DAC). IEEE, 2019, pp. 1–6. vol. 13, no. 5, pp. 740–753, 2020. [29] M. Aniche, C. Treude, and A. Zaidman, “How developers engineer test [57] E. Andreetta Fontana and F. Petrillo, “Breakpoints into the wild: an cases: An observational study,” arXiv preprint arXiv:2103.01783, 2021. exploratory study,” Aug. 2021. [Online]. Available: https://doi.org/10. [30] “Stack overflow developer survey 2019.” [Online]. Available: https: 5281/zenodo.5346846 //in\sights.stackoverflow.com/survey/2019 [31] “Google trends 2020.” [Online]. Available: https://pypl.github.io/IDE. h\tml [32] “Best integrated development environments (ide).” [Online]. Available: https://www.g2.com/categories/integra\ ted-development-environments-ide [33] B. G. Glaser and A. L. Strauss, Discovery of grounded theory: Strategies for qualitative research. Routledge, 2017.

12