Debugging Event-Driven Programming

James Davis Virginia Tech Blacksburg, VA, USA [email protected]

ABSTRACT Event-driven programming (EDP) is a natural way to struc- The importance of the event-driven programming (EDP) ture IoT applications [4]. However, little research has been paradigm is on the rise. While EDP has been been common invested in EDP pedagogy or process. If we don’t understand in client-side applications since around 2000 (e.g. GUI de- how to teach the IoT in general and EDP in particular, or velopment, JavaScript-driven web pages), it has only recently how developers conceptualize EDP programming, the IoT vi- entered the mainstream in the settings of mobile computing sion may not be realized to its fullest potential. Techniques (Android, 2008 [15]) and server-side programming (Node.js, from Human-Computer Interaction can inform these questions 2009 [18]). The EDP paradigm is also a natural way to struc- facing EDP. ture applications written for the burgeoning Internet of Things. However, little effort has been invested in understanding the A brief introduction to event-driven programming processes programmers use to develop and debug EDP ap- Many universities dedicate much of their computer science plications or in developing effective educational approaches curricula to procedural (imperative) programming, a sequential for EDP. This research proposes to “debug event-driven pro- or multi-threaded programming paradigm in which programs gramming,” exploring the mental processes EDP developers follow a procedure (sequence of steps, “recipe”) in a relatively use to grok their code and considering contextualized educa- linear fashion until the completion of their algorithm [60]. The tional approaches based on the students’ backgrounds. As a EDP paradigm [26] differs significantly, being non-sequential result, educators and systems researchers will be able to make (asynchronous) and relational (webs of event handlers). I informed decisions about the design of curricula, languages, anticipate that these differences make EDP different to teach, development environments, and debugging aids. learn, and practice than procedural programming.

ACM Classification Keywords The EDP paradigm is coming into its own. Though historically K.3.2. Computer and Information Science Education: Com- EDP has played second fiddle to procedural programming, the puter science education; D.2.5. Testing and Debugging: Test- explosion of the event-driven Node.js server-side JavaScript ing tools (e.g., data generators, coverage testing) framework [18] has demonstrated its relevance to applications running on the modern Internet. In an EDP system, the sys- tem defines the set of possible events (e.g. “mouse click” or Author Keywords “incoming network traffic on port 1234”), and the developer IoT; Event-driven programming; Education; Debugging; supplies a set of “event handlers” (also known as “callbacks”) Tools to be executed when particular events occur [60, 24]. During execution in an EDP application, events are generated and INTRODUCTION then routed to the appropriate handlers. The Internet of Things (IoT) holds tremendous promise for human society. Thanks to low-cost sensors and ubiquitous EDP has been used at the level of hardware interrupts since the internet connectivity, IoT devices can make smart decisions early days of computing, but to the best of our knowledge it based on real-time sensor input and advised by intelligent first saw high-level use in the Mesa programming environment. services in the cloud [35]. Nascent IoT systems are already Mesa was an early GUI system, and application developers transforming cities like Barcelona and Singapore, and more could register event handlers for events like the re-sizing of a advanced IoT systems could revolutionize areas as diverse as window [63]. EDP still holds sway in GUI environments, for homes, transportation, and health care [6]. example in the obsolete Java/AWT GUI framework [69] and the now popular Java/Swing and JavaFx GUI frameworks [20, 59]. EDP was later applied in early web servers, as researchers realized that it was particularly well suited to I/O-bound work like that of an FTP server. In this setting, network traffic is translated into client requests, which are “emitted” as events to be handled. For an FTP server, client requests are for files stored in the servers file system, so the handler would submit Contact [email protected] for permission to make copies of all or part of this work. request(s) to the file system, registering yet another event CS5724’16, Fall 2016, Blacksburg, VA. Copyright © 2016 James Davis handler to be executed once the file system completed the request. Until the file system finished its request, the server Novice programmers could process other requests in a similar fashion. In this way, Research on professional programming practice has been a single-threaded event server can handle multiple clients at paralleled by research on the practices of true novice pro- once, with minimal overhead and high performance under grammers. This line of research considers novice program- certain workloads [52]. mers, which are students in their first or second computer science course (CS1 or CS2). Research in this area stud- ies the ways novice programmers interact with source code WHAT WE KNOW ABOUT PROGRAMMERS and programming aids while programming and debugging, There is a rich literature on programmers: the kinds of bugs typically relying on deep investigations of a small group of they produce, how they debug, and the differences between students. Though the programming languages have changed novice and expert programmers. Much attention has been paid from COBOL and BASIC to Java, the findings of the earlier to the kinds of bugs produced by novices (e.g. [56]), as these generations of researchers have been replicated by more recent bug classifications can be used as a feedback mechanism to work. refine the content presented in a course. Other work has gone into debugging techniques, examining debugging both as a The study of the bugs created by novice programmers was specific form of troubleshooting [40] and as a skill to be taught spearheaded by Spohrer and Soloway, who argued that an em- in its own right [5]. pirical understanding of novice bugs would lead to improved courses. They structured their paper around two pieces of folk wisdom: that most errors are due to a small number of root Expert programmers causes, and that most errors are due to misunderstandings of Research into how programmers debug began in the 1970s, the programming language. Their data agree that a few root beginning a long tradition of work on expert programmers. causes explain most errors, but basing their work in GOMS Gould and Drongowski initiated the field, assessing expert philosophy [38] they argue that programmer errors are due to programmers in industry [28, 29]. Their employer, IBM, was the programmer’s incorrect goals and methods unrelated to understandably interested in understanding how its profes- language misunderstandings [61]. sional programmers did their work. They “seeded” one-page The oft-cited work of Katz and Anderson discussed four ex- FORTRAN programs with errors and studied 30 and 10 ex- periments with between 10 and 40 participants enrolled in a perts as they debugged them. Using a combination of notes CS2 course. The most common bug location strategies these and think-aloud, they found that certain types of bugs were students used were “simple recognition”, “forward-looking” harder to identify than others. approaches like hand simulation, and “backward-looking” ap- Ten years later, Vessey compared the debugging processes of proaches like causal reasoning (reason backward from output). “novice” and “expert” professional programmers, using multi- Of particular interest were the findings of their third experi- page COBOL programs seeded with errors. Vessey found ment, which showed that students take different bug location that experts outperform novices, noting that experts seemed strategies depending on whether or not they are familiar with to be better at “chunking” a program: experts could divide a the code. Students took forward-looking approaches when program into its related components more readily than novices, working with unfamiliar code, preferring backward-looking allowing them to focus on the chunks relevant to the bugs [65]. approaches for code they understood [40]. We note that these “novices” are professional programmers, Perkins and Martin studied 20 high school students in their not to be confused with the research on true novices in the second semester of a programming course [55]. By monitoring university setting. This research is discussed below. each student’s performance as they completed a set of asterisk- Eisenstadt provided a qualitative description of debugging printing problems, they found that these novice programmers in industry [22], collecting “debugging war stories” from 78 often ran into problems attributed to their “fragile knowledge”. professionals. He reported commonalities in the stories: many Correct programming relies on the successful integration of a of these experts employed data gathering approaches (e.g. rich set of concepts, and partial, hard to access, or mis-applied printf) and hand simulation to better understand the program knowledge often tripped up these novices. and the cause of the bug, and half of the bugs were cases in Ahmadzadeh et al. innovated a two-pronged approach to study- which the professional tools (e.g. debuggers) available to the ing CS1 students: they assessed both programming practice developers were useless due to significant spatial or temporal over the course of a semester and debugging skills in a 2-hour gaps between a bug’s root cause and its symptoms. exam. From this rich dataset they showed that good program- Inspired by Eisenstadt’s discussion of the importance of data mers (as measured by course grades) are not always good gathering, and the use of information foraging in earlier work debuggers! This finding shows that students don’t always by Russell et al. [58], Lawrance et al. studied the way experts learn debugging by themselves by working through homework interacted with a large and unfamiliar software project from an assignments, and that explicitly teaching debugging might be information foraging theory perspective [42]; they found that necessary [5]. Murphy et al. agreed with this finding in their programmer behavior could be modeled using information study of 21 students enrolled in CS2 across 7 universities: foraging without needing a cognitive model. This model was their students had difficulties applying debugging techniques also confirmed in a study of end-user programmers as they consistently and effectively [46], smacking of the fragile state evaluated a large, unfamiliar spreadsheet [30]. of novice knowledge identified by Perkins and Martin [55]. Though many authors characterize the bugs introduced by their difference in the performance of males and females measured students on a small scale, two particularly significant efforts by final grade in a CS1 course [68]. have been made to identify the issues encountered by students in CS1. The first was a “critical incidents” [27] style approach WHAT WE DON’T KNOW ABOUT PROGRAMMERS that surveyed instructors at 50 colleges for the most common Despite decades of research into the practice of programming kinds of bugs introduced by CS1 students [33], while the sec- by experts and novices, many questions remain unanswered. ond painstakingly recorded each interaction between roughly We observe two fundamental gaps in the literature: the ef- 400 students and their teaching assistants at the University of fect of programming paradigm and the effect of programmer Otago over 2 years. In doing so, Robins et al. were able to background. identify (year 1) and corroborate (year 2) the most common types of bugs introduced by students in CS1 [56]. Hristova Programming paradigms et al. classified programming errors as syntax, semantic, or First, almost all of the research on novices discussed in Sec- logic, while Robins et al. favored the higher-level labels of tion has focused on the procedural programming paradigm, background, general and specific to account for the more open- in which students give the computer a sequence of steps to ended questions asked of the teaching assistants. Hristova accomplish the desired task. This decision is presumably an et al.’s error categories fall into Robins et al.’s “general” and artifact of history: these are the languages used in the CS1 and “specific” categories, while Robins et al.’s approach identified CS2 courses taught by the researchers, and therefore are the the additional category of “background” issues like problems easiest for the researchers to study. There is no reason to sus- with the operating system or understanding the assignment. pect, however, that the conclusions of these researchers about While working in pairs, Hanks’s students replicated the prob- issues encountered by novices can be generalized to other lem distribution described by Robins et al., although each pair programming paradigms. Indeed each paradigm is different, experienced fewer issues overall [31]. with its own strengths and weaknesses [64]. Exceptions to the “procedural club” are LISP [40] (functional paradigm, though paradigm was not emphasized in this study) and Excel (logic Other lines of research on programmer behavior paradigm) [30]. The rest use strictly procedural languages Other avenues of research have pursued predictors of student or are heavily reliant on the procedural components of lan- success in computer science coursework. Researchers have guages: Pascal [61], BASIC [55], and Java [56,5, 57, 31, 33, investigated the relationship between programming ability and 46, 45]. What impact might the use of different programming a variety of psychological measures of programmers. The paradigms have on novice performance? most popular psychological measures to attempt are field de- We were able to locate only two papers that explicitly consider pendence/field independence [67] and the Myers-Briggs per- the impact of different paradigms in some way. Myers et al. sonality types (MBTI) [48]. Though we presume the reader’s compared cognitive style to ability in programming paradigm familiarity with the MBTI, field dependence and field indepen- based on non-novice students taking two post-CS2 courses dence describe how readily a person can perceive a complex concurrently, with inconclusive results [47], while White and object as composed of individual parts; field dependence im- Sivitanides proposed a theory predicting performance in lan- plies difficulty in breaking down the whole into its parts, or guages falling into different paradigms as a function of cogni- to perceive an object outside of the context in which it is ob- tive development and brain hemisphere preference, but made served. While much of the research looks for relationships no attempt to validate their theory [66]. We propose to enhance between psychological measure and course outcome ( [11, 17, the research methodologies in the field of novice programming 47, 68]), a more promising approach seems to be to assess errors with techniques to give us a deeper sense of the students’ smaller aspects of the process: individual assignments [11] or cognitive processes, and to apply these methodologies in the even the individual stages of problem-solving in the context setting of disparate programming paradigms. of programming [10]. The effort to correlate programming ability and psychological measures has been largely unsuc- Programmer background cessful, as the most consistent finding has been that academic Second, the research on programming errors in Sec- success begets academic success (e.g. [47]). However, some tions and assumes the programmers are homogeneous in promising research demonstrated that field independent pro- every respect other than amount of programming experience. grammers are better at algorithmic design [10] and locating Though some list the genders or ages of the participants, none logical errors [14]. of the analyses distinguish between participants on the basis of these differences. This is a curious state of affairs. We suggest Researchers have considered still other aspects of program- that this lack of investigation is due to the relatively small mers, including verbal ability and gender. For example, in a sample sizes of many of the studies, perhaps due in turn to study of novice debugging strategies, Romero et al. found no the man-hours required to follow the verbal protocol method- relationship between verbal ability (as measured by GRE test ologies on which they rely. Does a programmer’s background questions) and debugging skill [57]. Others have investigated have an effect on his ability or on the kinds of problems he gender, e.g. finding in end-user programmers that females experiences? introduce more bugs than males while debugging a program seeded with errors [8], and that males tinker more than females One aspect of programmer background of particular import during end-user debugging, sometimes to their detriment [9]. in the modern era is cultural background. In addition to me- From another angle, Woszczynski et al. observed no statistical chanical labor like manufacturing, Western companies large and small have outsourced intellectual work like computer RESEARCH QUESTIONS programming. We have observed this phenomenon first-hand 1. RQ1 Do the problems encountered by novice programmers while working at IBM: American employees were occasion- vary across different programming paradigms? Are the ally replaced by Indian or Chinese employees. Understanding problems and their distributions a function of the program- the ramifications of changing the cultural background of their ming paradigm? programmers could help companies determine which projects are better suited to outsourcing. To understand the issues in 2. RQ2 Does a student’s background affect the kinds of issues this area, we draw on the pioneering work of Nisbett et al. and they encounter in different programming paradigms? Nisbett summarized in [50, 49]. To answer these research questions, we design our experiments to investigate the following hypotheses: Cultural background 1. H1 Novice programmers will experience different kinds Nisbett et al.’s argument is complex, drawing relationships of issues when being taught using different programming between societal structure and cognitive processes, and we paradigms. For issues common to all paradigms, we do summarize it here. First, the social organization of ancient not expect the distribution of issues to vary significantly by Greek and Chinese cultures was markedly different. Greek paradigm. This hypothesis will be tested by comparing the culture was focused on the individual, while Chinese culture event-driven paradigm to procedural programming. was focused on the group. This difference affected their per- spective on how the natural world worked (indeed, the Chinese 2. H2 Culturally Western programmers and culturally Eastern barely acknowledged the concept of “nature as distinct from programmers will differ in their ability to perform different human or spiritual entities”. Due to their focus on the indi- kinds of activities in event-driven programming. vidual, the Greeks were prone to systematizing everything, Our first hypothesis is open-ended, mirroring the fact that looking for rules and categorizations in the nature. In contrast, the literature on novice errors is exclusively focused on the the Chinese did not construct formal models of nature. Nisbett procedural programming paradigm. This aspect of the study et al. argue that the social elements of life in these ancient will be exploratory because the literature contains no hints as cultures affected their systems of thought, in turn influencing to the types of errors expected in other paradigms. their cognitive processes. Of particular relevance to our pro- posed research, the ancient Greeks became in general more We base the cultural aspect of H2 on the differences between oriented to individual parts, while the Chinese considered parts Easterners and Westerners described by Nisbett et al. [50] and inseparably from the whole. Nisbett [49], discussed earlier in more detail (Section ). We predict, in line with previous research [14, 10] that Westerners While descriptive of life in ancient Greece and China, are these will be more effective at the intra-function aspects of EDP, differences relevant today? Nisbett et al. cite anthropological while Easterners will be more effective at dealing with the evidence that the social order of ancient Greece is similar to the inter-function relationships necessary in EDP. social order of modern North America and Europe (Western- ers), while that of ancient China is reflected in the social order What is a novice programmer? of Eastern Asian countries (China, Japan, South Korea, etc.) We will study novice programmers in answering both of these (Easterners). Nisbett et al. contend that “the original differ- research questions. It is difficult to consistently define a ences in cognitive orientations were due to the social psycho- “novice programmer” across different institutions and different logical ones”, and cite a broad array of evidence demonstrating curricula. For example, the novices of Katz and Anderson these cognitive differences between Westerners and Easterners were described as only making apparently-accidental “slips”, in the modern age. We are principally concerned with the while students at other universities at a similar point in their ancient distinction between the individual (Greece) and the academic careers are described as struggling with much more whole (China) and its implications for computer programming. basic concepts at a similar point in their academic careers [33, Whether a person is more attuned to individual components or 25, 31]. Many factors can influence student ability, including to the whole can be measured using Witkin’s concept of field the skill of the educator, the caliber of the students, and any dependence/independence (FD/FI) [67]: Westerners are more prior programming experience they might have. A strong ex- field independent, identifying individual objects as they com- perimental design must control for these factors to obtain a pose a more complex whole, while Easterners are more field sound definition of a novice programmer. For ease of compar- dependent, considering the whole composed of inseparable ison, we will study only students at Virginia Tech who have parts [50]. taken the same set of computer science courses. We propose to apply Nisbett’s research on the cultural basis of cognitive processes to the area of computer program debug- RESEARCH QUESTION 1 ging, which will help educators and employers understand the We wish to understand whether novices experience different potential weaknesses of their students and employees based issues when working in different programming paradigms. We on their cultural background. Despite over 3000 citations of hypothesize that they will indeed do so, and will establish this the work of Nisbett et al. [50], it has made few inroads into by comparing the issues they encounter when being trained in computer science research. Though Pauleen et al. appealed event-driven programming compared to procedural program- for this research in 2005 [54], the computer science education ming. This research question relies on concepts from HCI community does not seem to have heeded their call. We will. paradigms I and II [32]. Experimental design determine whether code contains syntax errors or whether Studies characterizing student issues are typically run for the it fails a particular test case, but they cannot understand the entire duration of a programming course. We will therefore programmer’s intent without pseudo-scientific psychological study students taking a programming course emphasizing analysis; Spohrer and Soloway’s work acknowledged this con- the procedural paradigm and students taking a programming cern, writing that “the[ir] plausible accounts [of programmer’s course emphasizing the event-driven paradigm. The Java pro- thought process]...must be subjected to further scientific in- gramming language can be used for either paradigm, by em- quiry” [61]. As far as we are aware, no subsequent work phasizing the existing introductory material (procedural, with has identified a good quantitative scheme for determining the light training in object-oriented programming) or by orienting programmer’s thought process. We propose a two-pronged the course towards GUI programming. We propose focusing approach with a novel methodological twist that promises to Virginia Tech’s Java-based CS1054 course [2] (CS1) in these give deeper insight into the programmer’s thought process. two directions in two consecutive semesters, comparing the We agree that on-line protocols promise the best source of errors experienced by these novices under each paradigm. The scalable quantitative data about a programmer’s thought pro- current course focuses on the procedural paradigm, but the cess. However, a novice’s source code alone – often lacking same material can be presented in the event-driven paradigm comments or good variable names – leaves much to be desired as demonstrated by previous CS1 courses taught in the event- as a window into their mind. To gain this missing insight driven paradigm [13, 16]. As a programming-intensive course, into a student’s thought process, we propose modifying the CS1054 is more suitable for studying novice errors than other Java grammar to require a comment after every statement and non-major courses like CS1014 [1]. preceding every block of code. This novel twist on on-line protocols will give us a sense of why each line of code exists. Data collection protocols In this section we discuss the methodological techniques used Furthermore, we can combine this scheme with our own appli- by researchers in this area. Though other techniques are men- cation test suite based on the specifications of each program- tioned in the literature, the primary approaches whose data is ming assignment. We agree with prior work [61, 36,5] that cited during the discussion of results are verbal protocols and compilation time is a good level of granularity. By filtering for on-line protocols. After discussing the use of these protocols, programs that are syntactically correct (those that pass compi- we propose to combine them in a novel way. lation) [61], we can determine the tests failed by each “valid” program, and use recent advances in program-level root cause Verbal protocols analysis (e.g. [39]) to pinpoint the line(s) of code responsible Researchers frequently use verbal protocols [23] to get inside for each test failure. Due to our changes to the Java grammar, programmers’ heads and understand their cognitive processes. the compiler will reject any programs without comments on Ericsson and Simon observe that verbal protocols won’t sig- every line; since these programs are syntactically correct, they nificantly interfere with participants’ cognitive processes pro- will be highly commented. We can then analyze the comment vided that the participant encodes the information verbally, on each line responsible for the test failure to understand the and that if the information is encoded in some other form true root cause of the failure. We know that “the [root cause (e.g. visual data, smells) the participant will need to devote of the] error is between the chair and the computer”, but with mental resources to vocalize that information. Though some the programmer’s comments in hand we can understand the researchers uncritically use verbal protocols for tasks involv- thought process that led to the failure. ing visual representations of data [57], the vast majority of uses seem in keeping with Ericsson and Simon’s guidelines. Virginia Tech already employs an online programming en- vironment in the form of the Web-Computer Aided Testing On-line protocols (Web-CAT) environment, used e.g. in CS1114 (CS1 for ma- A promising, more data-oriented branch of research makes use jors) [3]. Since students do their programming within this of on-line protocols, introduced by Spohrer [61]. On-line pro- environment, we have the opportunity to track their behav- tocols require the student to use the researcher’s development ior at granularities ranging from key strokes to assignment environment, tracking each student’s every move. Researchers submissions. We will modify the Java compiler in the Web- typically capture coding snapshots of each programming as- CAT environment to reject programs without comments as signment each time the student compiles their code, and re- discussed above, and to record each compilation attempt for searchers subsequently categorize the syntax errors made by subsequent analysis. novices [5, 36] or identify and explain semantic and logical Example 1: Procedural programming Figures1 and2 illus- errors based on psychological analysis of the source code [61]. trate the potential explanatory power of our comment-every- On-line data can of course be used in tandem with qualita- line on-line protocol. Figure1 is pseudocode for a function tive data [19]. On-line protocols, also known as “educational that doubles the value of each number in an array, returning an data mining”, have increased in popularity in recent years array of doubled numbers. It contains an error: on line 4, the thanks to a shift towards online programming environments programmer computed the doubled value, but fails to insert it that facilitate fine-grained data collection methods [34]. into the doubleNums array defined on line 2. The result is that Our proposed protocols on line 5 the programmer always returns an empty array. We On-line protocols A shortcoming of on-line protocols is that can only guess at the gap in the programmer’s line of reason- the data do not provide explanatory power. Researchers can ing – Did they simply forget to add it to doubleNums? Having that verbal reports can provide some illumination. In addition to our commented on-line protocol, we propose the use of a retrospective verbal protocol along the lines of Flanagan’s critical incidents [27]. We will ask students for a description of how they fixed one or more of the bugs they introduced while working on each assignment. By administering these protocols as part of the assignment submission process, these Figure 1. Novice pseudo-code algorithm. Returns an empty dou- reports should be fresh enough to provide helpful insights into bleNums, but why? the minds of novice programmers.

Data analysis Data from our commented on-line protocol will give us a window into the programmer’s thought process. This will let Figure 2. Comments associated with each line of the novice’s code. We us understand the kinds of errors to which novice program- now see the programmer’s thought process, giving us an explanation for mers are prone, in both the procedural and the event-driven the error. paradigms. By understanding these errors at a cognitive level, we can more carefully restructure the presentation of material to address common misconceptions or mistakes. We can apply pseudocode that meets the comment-every-line requirement, transcript coding techniques [23] to identify the most com- shown in Figure2, gives us concrete evidence for where our mon types of errors. We anticipate that coding these source hypothetical novice programmer’s thinking failed: they falsely code transcripts will be far more feasible than attempting a assumed that calculating doubleNum would as a side effect think-aloud protocol, which requires students to program in a add it to the doubleNums array, letting us categorize this error laboratory setting and researchers to carry out a painstaking with confidence as an S11 in the notation of Robins et al. [56]. transcription and coding of more meandering and gap-filled Example 2: Event-driven programming No novice errors verbal transcripts. specifically associated with the event-driven programming Something that has troubled us about much of the research on paradigm have been identified. In Figures3 and4, we illustrate the novice programming process has been the relatively small a novice error inspired by [12]. The assignment is intended sample size: studies often use N=10-40 and are placed within a to allow a user to draw lines on the screen. The expected single university. Given the use of a labor-intensive think aloud behavior is as follows: when the user clicks (OnMouseDown) protocol, we find these small sample sizes understandable. and releases (OnMouseUp), a line L is drawn between the 1 However, we are excited by the prospect of larger sample sizes Down and Up points. When the user clicks again, a line afforded by the more compact and more readily-understood L should connect the endpoint of his previous line to the 2 data offered by the commented on-line protocol we propose. beginning of his new line, and the new line should be drawn as already described. However, the hypothetical novice code Data from our retrospective verbal protocol should further in Figure3 will only draw the L1-style between mouse clicks, inform improvements in the presentation of course material. and won’t draw any L2-style lines connecting the L1-style Interviews like those we describe have been administered in lines. From the uncommented code in Figure3, it is not clear clinical settings (e.g. [28, 29, 25]), but we are not aware of in what way the novice’s thoughts led to the error. At a high studies using longitudinal per-student interviews [44]. We level, is the problem in comprehending the assignment, or in anticipate that the interview corpus will consist of a set of implementing it? Without comments or a conversation with “Eureka!” moments from students, which can be used to help the student, we cannot know. us understand effective debugging strategies and to give future students a sense of what the debugging process feels like from Figure4 illuminates the cause. Rather than an issue in im- a qualitative perspective. plementing the assignment using EDP concepts, which might have incorrectly inspired an example of a new class of bug, the comment on line 1 provides evidence that the programmer Threats simply misunderstood the assignment: they thought they were We have identified four threats to this study. The first is practi- to draw lines only L1-style lines between mouse clicks. We cal, while the remainder are threats to validity. do not know whether the programmer would have been able First, the CS department would need to be convinced to allow to implement the assignment correctly had they understood it, us to modify the content of one of their courses. We have but the commented on-line protocol has protected us from an proposed the use of the CS1054 course [2] because it is “CS1 incorrect analysis of the programmer. for non-majors”. Modifying it will not affect the existing Verbal protocols Nearly all of the studies cited earlier re- computer science curriculum for CS majors, so we conjecture port the use of a think-aloud verbal protocol analysis. Our that the CS department will be more likely to permit us to commented on-line protocol should give us a “source code change its design. If necessary, we could scale down our transcript” of each student’s thought process, so we do not method to introduce a 2-week module emphasizing either anticipate the need for a think-aloud verbal protocol. However, procedural or event-driven programming, the use of which we recognize that not all comments are created alike, and feel might be more amenable to the course instructors. Figure 3. Novice pseudo-code algorithm. Draws disconnected lines. What went wrong in the programmer’s thought process?

Figure 4. Comments associated with each line of the novice’s code. We see the reason for the error: the class declaration does not match the assignment.

Second, students could cheat our modification to the Java comparable to prior work. In the process, however, we will grammar by ending statements with empty comments or com- have inadvertently discovered a relatively easy way to help ments containing gibberish. We believe this tendency would novice programmers write higher-quality code. We suspect be curbed by two factors. First, the instructor has both a carrot many educators would appreciate this finding. and a stick: he can emphasize the importance of comments as part of good programming practice, and he can punish this RESEARCH QUESTION 2 behavior by including the quality of comments in his grading In the spirit of the Third Paradigm of HCI [32], we argue rubric. Second, novices are not asked to write particularly long that not all programmers are created equal, and that the cul- programs, so per-line comments should not be particularly ar- tural context of a programmer may be particularly relevant to duous. However, our commented on-line protocol could be predicting their ability to debug different kinds of problems. tiresome for students in higher-level courses (i.e. those above We have predicted, based on the work of Nisbett et al. [50], CS2). We suspect that modifying the Java grammar to only that Westerners and Easterners will have different experiences require comments at the block level would still make this while working on various aspects of EDP programming. protocol useful for higher-level courses. We expect the event-driven programming errors identified by Third, the instructors might not be able to comprehend the the experiment discussed in Section to fall into two major comments added by students. A novice might not be able to categories: intra-function and inter-function. Event-driven articulate why they are doing something. However, recall that programs have a procedural component, because they include Romero et al. reported no significant relationship between a set of functions, each of which must work correctly. We refer programming ability and verbal ability (as measured by GRE to errors in this aspect of EDP programs are intra-function questions) [57]. If true, students of high verbal ability should errors. Event-driven programs have a relational component be equally prone to writing code with and without errors, im- as well: functions interact with one another as listeners for plying that at least some of the error-strewn programs will events, and must save state between events using techniques likely be authored by students with stronger writing abilities. like closure. For example, in problems like that illustrated in These same students are expected to do a good job of keep- Figure3, a correct implementation requires the programmer to ing their comments in sync with their code. We anticipate, set the oldPoint variable to point in the OnMouseUp function therefore, that at least a subset of the student programs will be for use in the OnMouseDOwn function. This principal is comprehensible and suitable for study. of course not unique to EDP, but in EDP this inter-function Lastly, it’s conceivable that being forced to justify every line reasoning is a fundamental aspect rather than an advanced of code with a comment will change the distribution of errors concept as in the procedural paradigm. Errors in this area will experienced by our participants. The distribution of errors in be referred to as inter-function errors because they require procedural programming has been reported and replicated [56, conceptualizing relationships between different invocations of 31], and we will compare the error distribution between our the same or different functions. procedural programming dataset and the expected distribution Nisbett et al. [50] argued that Westerners are stronger at ana- as reported by these other researchers. If it does not match, lytic thought and at considering each part of a whole in isola- we will still be able to make a useful summary of error distri- tion, while Easterners excel at thinking of the role each piece butions, but we will know that the results will not be directly plays in an indivisible whole. This difference manifests itself in Witkin’s concept of field dependence/field independence in which participants attempt the “first” and “second” tasks, (FD/FI) [67], with Westerners matching the description of field as well as the order of the intra-function and inter-function independence and Easterners that of field dependence. problems within each task. The remaining time will be spent in a 15-minute post-test interview, using the retrospective We therefore refine our second hypothesis further: we predict protocol discussed in Section . that Westerners will be superior at designing and debugging intra-function algorithms, while Easterners will excel at de- signing and debugging inter-function relationships. Data analysis We will compare the field dependence/independence scores Experimental design of the participants to their intra-function and inter-function Our interest is in assessing the ability of Westerners and East- debugging ability, measured by the speed and accuracy with erners to design and debug particular aspects of problems in which they identify the bugs during the six debugging tasks. EDP programs. Though the techniques of our first experiment We anticipate that introducing cultural background, measured (Section ) will give us a sense of the experience of program- by field dependence/field independence, will lead to the first mers in general when working in the EDP paradigm, the details (to our knowledge) successful demonstration of the strong of particular programming issues may be difficult to pick out. prediction of debugging ability as a function of a psychological We therefore propose a clinical experiment in the vein of the measure. The degree of field dependence/field independence studies of debugging methodology described earlier, patterned variation across cultures is greater than within a culture [37], so particularly after [10]. the differences between Easterners and Westerners should be more pronounced than those within just the American culture. Measuring field dependence/independence Table1 shows the independent and dependent variables in It is our hope, therefore, that this experiment will demonstrate this study. Though prior studies on the interaction of FD/FI that instructors must consider the cultural context of each of and computer programming ability used the Group Embed- their students. Computer science educators do not appear to ded Figures Test to measure FD/FI [14, 10], performance on have considered this possibility, possibly leading to poor learn- this instrument does not appear to be independent of culture. ing outcomes for students from cultural backgrounds different Bagley [7] observes that Easterners register as FI on EFT-style from their instructor. Though we have deliberately selected tests in contrast to their FD scores on other measures of FD/FI. the programming paradigm in which these differences seem He hypothesizes that this is because literacy in the languages the most likely to manifest, the notion of intra-function vs. of Eastern countries typically requires the memorization of inter-function errors is relevant in more complex programs, and the ability to differentiate between hundreds or thousands particularly in the popular object-oriented paradigm used in of complex symbols. Nisbett et al. [50] refer to several other many CS programs. Software companies could also use the instruments suitable for measuring FD/FI, and following Ji results of this study to assign employees to culturally appropri- and her colleagues we will use Witkin’s Rod-and-Frame Test ate tasks, for example asking Western programmers to work on (RFT) to measure FD/FI [37]. procedural aspects of projects (e.g. business logic) and Eastern programmers to work on event-driven aspects of projects (e.g. Data collection protocols the GUI). As is standard in this community, we will administer a 2-hour test to students in the EDP course discussed in Section . In Threats the first 15 minutes, we will carry out an initial assessment of First, it’s conceivable that not enough Easterners would enroll programming background and then measure the participant’s in CS1054 to provide a useful sample size. An alternative FD/FI status with the RFT instrument. We anticipate that course in which to this study would be a hypothetical graduate- students from an Eastern background (relatively common in level course that uses concepts from event-driven program- programming classes) will score more strongly as FD, while ming, e.g. a course in the IoT. From personal experience, I students from a Western background will score as FI. know that the graduate population of the “Systems” arm of Virginia Tech’s Computer Science department is largely East- We will then administer a 20-minute warm-up task, asking ern, and these students would be interested in an IoT course. students to debug two small (one-page) programs, one with Since event-driven programming is an unusual paradigm, there an intra-function error and one with an inter-function error. should not be prior learning effects on the intra-function and Once the warm-up is concluded, the remainder of the time inter-function debugging ability of many of these students. will be divided into three portions. First, participants will Furthermore, White and Sivitanides warned that not all first- spend 45 minutes completing two design tasks, one calling year college students (nor indeed all adults) have attained full for intra-function skill and the other requiring inter-function formal operational thinking, and as a result using a graduate- skill. We are not intimately familiar with the abilities of novice level course might actually be preferable for this study [66] if programmers or with appropriate expectations for their skills the results are to be generalized to professional practitioners. at EDP, so the particular tasks will need to be determined through an iterative pilot study. Second, participants will Second, it’s possible that our predictions about the implica- be asked to complete four debugging tasks in 45 minutes, tions of field dependence and field independence are only each task featuring a different one-page program with one relevant to larger programming projects. It is clear that both error. Half of the errors will be intra-function, and half will be Westerners and Easterners can be skillful computer program- inter-function. To avoid order effects, we will vary the order mers, presumably because regardless of the programming Variable Type Description Prediction Field dependence or independence Independent Measures how effectively the participant can perceive items independent of context Westerners - FI; Easterners - FD Intra-function debugging ability Dependent Measures how well the participant can debug intra-function errors Positive correlation to FI Inter-function debugging ability Dependent Measures how well the participant can debug inter-function errors Positive correlation to FD Table 1. Independent and dependent variables in Experiment 2 paradigm under consideration, both field dependent and field have on their students’ abilities. These results will be useful independent programmers can understand complex software not only for training students in the IoT, but also for other EDP system. A field dependent programmer would presumably settings like mobile development and GUI programming. understand a complex software system holistically, seeing the relationships between all of the components indivisibly, while ACKNOWLEDGMENTS a field independent programmer might understand it as the sum We thank Professor Steve Harrison for his guidance on this of the interactions between its constituent parts. The effect of project. We thank our advisor, Dr. Dongyoon Lee, for his field dependence or independence might not manifest at the tolerance of our interest in Human-Computer Interaction and scale of a one-page program, and a study like that of Lawrance Computer Science Education. et al. [42], observing professional programmers as they exam- ine a large open-source code base, might be needed to truly REFERENCES observe any differences between individuals with these oppos- 1. 2016a. CS1014: Introduction to Computational Thinking. ing traits. However, the relative ease of studying students in (2016). https://sites.google.com/a/vt.edu/ the university setting compared to professional programmers computational-thinking/home means that a “students first” approach is desirable. 2. 2016b. CS1054: Introduction to Programming in Java. (2016). CONCLUSION http://www.cs.vt.edu/undergraduate/courses/CS1054 EDP emerged as an important paradigm in the CS curriculum in the 1990s and early 2000s as GUIs became commonplace. 3. 2016c. CS1114: Introduction to Software Design. (2016). Early explorations of EDP coursework focused on GUIs [62, http://www.cs.vt.edu/undergraduate/courses/CS1114 13, 16, 12]. While Stein argued that EDP should be included 4. Asanka Abeysinghe. 2016. Event-Driven Architecture: as one of several programming paradigms [62], others [13, 16] The Path to Increased Agility and High Expandability. argued that it could be used as the focal paradigm of a CS1 (2016). http://wso2.com/whitepapers/ course. There was some debate as to whether EDP was suitable event-driven-architecture-the-path-to-increased-agility-and-high-expandability/ for “CS1”-style classes, or if the concepts involved were too difficult. Bruce et al. argued that EDP could be used in 5. Marzieh Ahmadzadeh, Dave Elliman, and Colin Higgins. CS1 [13], and indeed that doing so simplified teaching several 2005. An Analysis of Patterns of Debugging Among key concepts [12]. However, research on EDP pedagogy has Novice Computer Science Students. In ITiCSE. 84–88. largely fallen silent: EDP did not “catch on” as a core concept DOI:http://dx.doi.org/10.1145/1151954.1067472 in CS education, remaining a specialized topic for advanced 6. Rajeev Alur, Emery Berger, Ann W Drobnis, Limor Fix, students. Kevin Fu, Gregory D Hager, Daniel Lopresti, Klara Ten years later, the IoT is increasingly imminent, and educa- Nahrstedt, Elizabeth Mynatt, Shwetak Patel, Jennifer tors have begun to explore IoT-themed classes for advanced Rexford, John A Stankovic, and Benjamin Zorn. 2015. undergraduates and graduate students. These courses are in- Systems Computing Challenges in the Internet of Things. tended for lower-level [41] or higher-level undergraduates [21, arXiv preprint arXiv:1604.02980 June (2015), 1–15. DOI: 51] or graduate students [43, 51]. Interestingly, the literature http://dx.doi.org/10.1109/MC.2011.291 on undergraduate IoT coursework does not discuss pedagog- 7. Christopher Bagley. 1995. Field Independence in ical issues related to EDP, suggesting a potential barrier to Children in Group-Oriented Cultures: Comparisons from student success: inviting students trained in one paradigm China, Japan, and North America. The Journal of Social (e.g. functional or object-oriented) into a new programming Psychology 135, 4 (1995), 523–525. DOI: paradigms (EDP) raises the specter of negative knowledge http://dx.doi.org/10.1080/00224545.1995.9712221 transfer [53]. Students may be stuck in one mode of thinking, unable to adapt to the new concepts in another paradigm, and 8. Laura Beckwith. 2005. Gender HCI issues in mis-applying concepts from the first programming paradigm problem-solving software. CHI ’05 extended abstracts on in the second [55]. Human factors in computing systems - CHI ’05 (2005), 1104. DOI:http://dx.doi.org/10.1145/1056808.1056833 We strongly feel that understanding the issues related to the EDP paradigm will inform computer science educators. This 9. Laura Beckwith, Cory Kissinger, Margaret Burnett, particular paradigm may be one of the keys to bringing the Susan Wiedenbeck, Joseph Lawrance, Alan Blackwell, promise of the Internet of Things to fruition, and as a result and Curtis Cook. 2006. Tinkering and gender in end-user understanding how to teach it is increasingly imperative. We programmers’ debugging. Proceedings of the SIGCHI believe the proposed research will inform educators as to the conference on Human Factors in computing systems - difficulties novices encounter in EDP, and that more broadly it CHI ’06 (2006), 231. DOI: will remind educators of the effect that cultural differences may http://dx.doi.org/10.1145/1124772.1124808 10. Catherine Bishop-Clark. 1995. Cognitive style and its 23. K. Anders Ericsson and Herbert A. Simon. 1980. Verbal effect on the stages of programming. Journal of Research Reports as Data. Psychological Review 87, 3 (1980), on Computing in Education 27, 4 (1995), 373. 215–251. DOI: http://dx.doi.org/10.1037/0033-295X.87.3.215 11. Catherine Bishop-Clark and Daniel D Wheeler. 1994. The Myers-Briggs personality type and its relationship to 24. Steve Ferg. 2006. Event-driven programming: computer programming. Journal of Research on introduction, tutorial, history. (2006). Computing in Education 26, 3 (1994), 358. DOI: http://parallels.nsu.ru/ http://dx.doi.org/10.1080/08886504.1994.10782096 25. Sue Fitzgerald, Gary Lewandowski, RenÃl’e McCauley, 12. Kim B. Bruce, Andrea Danyluk, and Thomas Murtagh. Laurie Murphy, Beth Simon, Lynda Thomas, and Carol 2004. Event-driven programming facilitates learning Zander. 2008. Debugging: finding, fixing and flailing, a standard programming concepts. Companion to the 19th multi-institutional study of novice debuggers. Computer annual ACM SIGPLAN conference on Object-oriented Science Education 18, 2 (2008), 93–116. DOI: programming systems, languages, and applications - http://dx.doi.org/10.1080/08993400802114508 OOPSLA ’04 (2004), 96–100. DOI: 26. David Flanagan. 2004. JavaScript: The Definitive Guide, http://dx.doi.org/10.1145/1028664.1028704 4th. (2004). 13. Kim B. Bruce, Andrea P. Danyluk, and Thomas P. 27. J. C. Flanagan. 1954. The critical incident technique. Murtagh. 2001. Event-driven programming is simple Psychological Bulletin 51, 4 (1954), 327–358. DOI: enough for CS1. Proceedings of the 6th annual conference on Innovation and technology in computer http://dx.doi.org/10.1037/h0061470 science education - ITiCSE ’01 (2001), 1–4. DOI: 28. JD Gould and Paul Drongowski. 1974. An exploratory http://dx.doi.org/10.1145/377435.377440 study of computer program debugging. Human Factors: The Journal of the 16, 3 (1974), 258–277. 14. Thomas P. Cavaiani. 1989. Cognitive styles and http://hfs.sagepub.com/content/16/3/258.short diagnostic skills of student programmers. Journal of Research on Computing in Education 21, 4 (1989), 29. John D. Gould. 1975. Some psychological evidence on 411–420. how people debug computer programs. International Journal of Man-Machine Studies 7, 2 (1975), 151–182. 15. Jacqui Cheng. 2007. It’s official: Google announces DOI:http://dx.doi.org/10.1016/S0020-7373(75)80005-8 open-source mobile phone OS, Android. Ars Technica (2007). http://arstechnica.com/gadgets/2007/11/ 30. Valentina Grigoreanu, Margaret Burnett, Susan its-official-google-announces-open-source-mobile-phone-os-android/Wiedenbeck, Jill Cao, Kyle Rector, and Irwin Kwan. 2012. End-user debugging strategies. ACM Transactions 16. Henrik Baerbak Christensen and Michael E Caspersen. on Computer-Human Interaction 19, 1 (2012), 1–28. 2002. Frameworks in CS1 - a Different Way of DOI:http://dx.doi.org/10.1145/2147783.2147788 Introducing Event-driven Programming. In ITiCSE. 31. Brian Hanks. 2008. Problems encountered by novice pair 17. Larry S Corman. 1986. Cognitive style, personality type, programmers. Journal on Educational Resources in and learning ability as factors in predicting the success of Computing 7, 4 (2008), 1–13. DOI: the beginning programming student. ACM SIGCSE http://dx.doi.org/10.1145/1316450.1316452 Bulletin 18, 4 (1986), 80–89. DOI: http://dx.doi.org/10.1145/15003.15020 32. Steve Harrison, Deborah Tatar, and Phoebe Sengers. 2007. The three paradigms of HCI. Alt. Chi. Session at 18. Ryan Dahl. Video: Node.js by Ryan Dahl. (????). the SIGCHI . . . (2007), 1–18. DOI: http://www.jsconf.eu/2009/video http://dx.doi.org/10.1234/12345678 19. Brian Danielak. 2014. How electrical engineering 33. Maria Hristova, Ananya Misra, Megan Rutter, and students design computer programs. Ph.D. Dissertation. Rebecca Mercuri. 2003. Identifying and Correcting Java University of Maryland. Programming Errors for Introductory Computer Science http://www.physics.umd.edu/rgroups/ripe/perg/ Students. In SIGCSE. dissertations/Danielak/DissertationDanielak.pdf 34. Petri Ihantola, Matthew Butler, Stephen H Edwards, 20. Stephen C. Drye and William C. Wake. 1999. Java Swing Virginia Tech, Ari Korhonen, Andrew Petersen, Kelly reference. Manning PUblications Company. Rivers, Miguel à ˛AngelRubio, Judy Sheard, Jaime 21. Shannon Duvall and Joel Hollingsworth. 2016. Creating a Spacco, Claudia Szabo, and Daniel Toll. 2015. course on the internet of things for undergraduate Educational Data Mining and Learning Analytics in computer science majors. Journal of Computing Sciences Programming : Literature Review and Case Studies. in Colleges (2016), 97–103. ITiCSE WGR’16 (2015), 41–63. DOI: http://dx.doi.org/10.1145/2858796.2858798 22. M Eisenstadt. 1993. Tales of Debugging from the Front Lines. Empirical Studies of Programmers, 5th Workshop 35. International Telecommunication Union. 2012. Overview 5365 (1993), 86–112. of the Internet of things. Technical Report. 22 pages. 36. Matthew C Jadud. 2006. Methods and tools for exploring 47. J. Myers and Brita Munsinger. 1996. Cognitive style and novice compilation behaviour. In Proceedings of the achievement in imperative and functional programming Third International Workshop on Computing Education language courses. In National Educational Computing Research. 73–84. DOI: Conference. http://dx.doi.org/10.1145/1151588.1151600 48. Isabel Myers-Briggs. 1962. Manual for The Myers-Briggs 37. L J Ji, Kaiping Peng, and Richard E Nisbett. 2000. Type Indicator. Educational Testing Service. Culture, control, and perception of relationships in the The Geography of Thought: How Asians environment. Journal of Personality and Social 49. R Nisbett. 2010. and Westerners Think Differently... and Psychologycial psychology 78, 5 (2000), 943–955. DOI: . https://books.google.com/books?hl=en http://dx.doi.org/10.1037/0022-3514.78.5.943 38. Bonnie E. John, Paul S. Rosenbloom, and Allen Newell. 50. Richard E Nisbett, Kaiping Peng, Incheol Choi, Dan 1985. A theory of stimulus-response compatibility Ames, Scott Atran, Patricia Cheng, Marion Davis, Eric applied to human-computer interaction. ACM SIGCHI Knowles, Trey Hedden, Lawrence Hirschfeld, Lijun Ji, Bulletin 16, 4 (1985), 213–219. DOI: Beom Jun Kim, Shinobu Kitayama, Ulrich Kuhnen, Jan http://dx.doi.org/10.1145/1165385.317495 Leu, David Liu, Geoffrey Lloyd, Hazel Markus, Ramaswami Mahalingam, Taka Masuda, Coline 39. Baris Kasikci, Benjamin Schubert, Cristiano Pereira, McConnell, Donald Munro, Denise Park, Winston Sieck, Gilles Pokam, and George Candea. 2015. Failure David Sherman, Michael Shin, Edward E Smith, Stephen Sketching: A technique for automated root cause Stich, Sanjay Srivastava, Timo-thy Wilson, and Frank diagnosis of in-production failures. Symposium on Yates. 2001. Culture and Systems of Thought: Holistic Operating Systems Principles (SOSP) (2015), 344–360. Versus Analytic Cognition. Psychological Review 108, 2 DOI:http://dx.doi.org/10.1145/2815400.2815412 (2001), 291–310. DOI: 40. Irvin R Katz and John R Anderson. 1988. Debugging : http://dx.doi.org/10.1037//0033-295X.108.2.291 An Analysis of Bug-Location Strategies. In 51. Evgeny Osipov and Laurynas Riliskis. 2013. Educating Human-Computer Interaction, Vol. 3. 351–399. Innovators of Future Internet of Things. IEEE Frontiers 41. Gerd Kortuem, Arosha K Bandara, Neil Smith, Mike in Education (2013). Richards, and Marian Petre. 2013. Educating the 52. Vivek S Pai, Peter Druschel, and Willy Zwaenepoel. internet-of-things generation. Computer 46, 2 (2013), 1999. Flash : An Efficient and Portable Web Server. 53–61. DOI:http://dx.doi.org/10.1109/MC.2012.390 Proceedings of the 1999 USENIX Annual Technical 42. Joseph Lawrance, Christopher Bogart, Margaret Burnett, Conference (1999), 14. DOI: Rachel Bellamy, Kyle Rector, and Scott D. Fleming. http://dx.doi.org/10.1.1.119.6738 2013. How programmers debug, revisited: An 53. John F Pane and Brad a Myers. 1996. Usability Issues in information foraging theory perspective. IEEE the Design of Novice Programming Systems. Education Transactions on Software Engineering 39, 2 (2013), August (1996), 78. http://repository.cmu.edu/isr/820/ 197–215. DOI:http://dx.doi.org/10.1109/TSE.2010.111 54. David J. Pauleen, Roberto Evaristo, Robert M. Davison, 43. Hanna Mäenpää, Sasu Tarkoma, Samu Varjonen, and Soon Ang, Macedonio Alanis, and Stefan Klein. 2005. Arto Vihavainen. 2015. Blending Problem-and Cultural bias in information systems research and Project-Based Learning in Internet of Things Education: practice: are you coming from the same place I am?. In Case Greenhouse Maintenance. SIGCSE (2015). DOI: ICIS, Vol. 17. 2–37. http://dx.doi.org/10.1145/2676723.2677262 http://aisel.aisnet.org/icis2005http: 44. RenÃl’e McCauley, Sue Fitzgerald, Gary Lewandowski, //aisel.aisnet.org/icis2005/75http: Laurie Murphy, Beth Simon, Lynda Thomas, and Carol //search.ebscohost.com/login.aspx?direct=true Zander. 2008. Debugging: a review of the literature from 55. D.N. Perkins and Fay Martin. 1986. Fragile knowledge an educational perspective. Computer Science Education and neglected strategies in novice programmers. In 18, 2 (2008), 67–92. DOI: Empirical Studies of Programmers. 213–229. http://dx.doi.org/10.1080/08993400802114581 https://books.google.com/books?hl=en 45. Laurie Murphy, Sue Fitzgerald, Brian Hanks, and Renee 56. Anthony Robins, Patricia Haden, and Sandy Garner. McCauley. 2010. Pair Debugging : A Transactive 2006. Problem distributions in a CS1 course. Conferences Discourse Analysis. Icer (2010), 51–58. DOI: in Research and Practice in Information Technology http://dx.doi.org/10.1145/1839594.1839604 Series 52 (2006), 165–173. 46. Laurie Murphy, Gary Lewandowski, RenÃl’e McCauley, 57. Pablo Romero, Benedict du Boulay, Richard Cox, Rudi Beth Simon, Lynda Thomas, and Carol Zander. 2008. Lutz, and Sallyann Bryant. 2007. Debugging strategies ˘ Debugging: the good, the bad, and the quirky âA¸Sa and tactics in a multi-representation software qualitative analysis of novicesâA˘ Z´ strategies. environment. International Journal of Human Computer Proceedings of the 39th SIGCSE technical symposium on Studies 65, 12 (2007), 992–1009. DOI: Computer Science Education (SIGCSE ’08) 40 (2008), http://dx.doi.org/10.1016/j.ijhcs.2007.07.005 163. DOI:http://dx.doi.org/10.1145/1352135.1352191 58. Daniel M. Russell, Mark J. Stefik, Peter Pirolli, and 64. Peter Van Roy and Seif Haridi. 2004. Concepts, Stuart K. Card. 1993. The cost structure of sensemaking. Techniques, and Models of Computer Programming. MIT Proceedings of the SIGCHI conference on Human factors Press. in computing systems - CHI ’93 (1993), 269–276. DOI: 65. Iris Vessey. 1985. Expertise in debugging computer http://dx.doi.org/10.1145/169059.169209 programs: A process analysis. International Journal of 59. Herbert Schildt. 2014. Java: the complete reference (9 Man-Machine Studies 23, 5 (1985), 459–494. DOI: ed.). McGraw-Hill Education. http://dx.doi.org/10.1016/S0020-7373(85)80054-7 60. Michael L. Scott. 2009. Programming Language 66. Garry L White and Marcos P Sivitanides. 2002. A Theory Pragmatics (3 ed.). Elsevier. of the Relationships between Cognitive Requirements of 61. James C Spohrer and Elliot Soloway. 1986. NOVICE Computer Programming Languages and ˘ ´ MISTAKES: ARE THE FOLK WISDOMS CORRECT? ProgrammersâAZ Cognitive Characteristics. Journal of An evaluation of two folk wisdom serves to elucidate the Information Systems Education 13, 1 (2002), 59–66. underlying or " deep-structure " reasons for novice errors. 67. Herman A. Witkin and Donald R. Goodenough. 1981. Commun. ACM 29, 7 (1986), 624–632. Cognitive Style : Essence and Origins. International 62. Lynn Andrea Stein. 1998. What We Swept Under the Universities Press. Rug: Radically Rethinking CS1. Computer Science 68. Tracy C; Shade Sherri Woszczynski, Amy B; Guthrie. Education 8, 2 (1998), 118. DOI: 2005. Personality and programming. Journal of http://dx.doi.org/10.1076/csed.8.2.118.3818 Information Systems Education 16, 3 (2005), 293–299. 63. Richard E Sweet. 1985. The Mesa programming DOI:http://dx.doi.org/10.1177/0021943604272028 environment. ACM SIGPLAN Notices 20, 7 (1985), 69. John Zukowski. 1997. Java AWT reference. O’Reilly. 216–229. DOI:http://dx.doi.org/10.1145/17919.806843