Software Debugging Using the Debugger SAM4E Xplained Pro

Faculty of Technology and Society Computer Engineering

Bachelor’s thesis 15 credits

Software debugging using the debugger SAM4E Xplained Pro

Mjukvarufelsökning med användning av felsökningsverktyget SAM4E Xplained Pro

Hamoud Abdullah Nadia Manoh

Exam: Bachelor of Science in Engineering Examiner: Ulrik Ekedahl Area: Computer Engineering Supervisor: Magnus Krampell Date of final seminar: 2018-08-21

Abstract Embedded systems are found in almost every device used in our daily lives, including cell phones, refrigerators, and cars. Some devices may be significantly more sensitive than others, meaning a bug appearing in a system could cause harm, even loss of human lives or cause no harm at all. To reduce bugs in a system, software testing and software debugging are performed.

The Computer Science program at Malmö University does not focus on teaching software debugging using a debugger. Thus, this thesis presents a debugging lab created for Computer Science students, considered to help them gain knowledge in how to use the debugger SAM4E Xplained Pro to locate bugs. As a result, four students performed the debugging lab of which 75 percent of the bugs were found and remedied.

Keywords: embedded system, software testing, software debugging, software failure, software bugs debugging techniques

Sammanfattning Inbyggda system finns i nästan alla enheter som används i vårt dagliga liv, som exempelvis mobiltelefoner, kylskåp och bilar. En del enheter kan vara betydligt känsligare än andra, vilket innebär att en bugg som existerar i ett system kan orsaka skada, till och med förlust av människoliv, eller orsakar ingen skada alls. Mjukvarutestning och mjukvarufelsökning genomförs för att reducera buggar i ett system.

Utbildningsprogrammet Datateknik och Mobil IT på Malmö universitet fokuserar inte på att undervisa mjukvarufelsökning med hjälp av felsökningsverktyg. Således presenterar denna forskning en felsökningslaboration skapat för studenter som går Datateknik och Mobil IT, som anses hjälpa studenterna att få kunskap i hur man använder felsökningsverktyget SAM4E Xplained Pro för att lokalisera buggar. Som ett resultat, utfördes felsökningslaborationen av fyra studenter varav 75 procent av buggarna hittades och åtgärdades.

Acknowledgements We would like to express our gratitude to our supervisor Magnus Krampell for all the feedback, support and patience during the work of this thesis. We would also like to thank all the students who participated in the lab assignment.

Last, but not least, we thank our examiner Ulrik Ekedahl for his valuable feedback.

Contents 1 Introduction ...... 1 1.1 Background ...... 1 1.2 Problem domain ...... 2 1.3 Research questions ...... 2 1.4 Limitations...... 3

2 Theoretical Background ...... 4 2.1 Embedded systems ...... 4 2.1.1 Real-time systems ...... 4 2.1.1.1 Hard real-time system ...... 4 2.1.1.2 Soft real-time system ...... 4 2.1.2 Architecture ...... 4 2.1.2.1 Single-unit system ...... 4 2.1.2.2 Distributed systems ...... 4 2.2 Error, fault and failure ...... 4 2.2.1 Computation ...... 6 2.2.1.1 Operator precedence ...... 6 2.2.1.2 Divide by zero ...... 6 2.2.2.2 Off-by-one ...... 6 2.2.2 Data reference (pointers) ...... 7 2.2.2.1 Uninitialized pointer ...... 7 2.2.2.2 Null pointer ...... 7 2.2.3 Redundant code ...... 7 2.2.4 Control flow ...... 8 2.2.4.1 Infinite loop ...... 8 2.3 Software testing ...... 8 2.3.1 Verification and validation ...... 8 2.3.2 Static and dynamic testing ...... 9 2.3.2.1 Black-box testing ...... 9 2.3.2.2 White-box testing ...... 9 2.3.3 Testing levels ...... 10 2.3.3.1 Unit testing ...... 10 2.3.3.2 Integration testing ...... 10 2.3.3.3 System testing...... 10 2.4 Debugging procedure ...... 10 2.5 Debugging techniques ...... 11 2.5.1 Print statements ...... 11 2.5.2 Logfile debugging ...... 11 2.5.3 Assertions ...... 11 2.5.4 Dry run debugging ...... 11 2.6 Debugger ...... 12 2.6.1 Atmel SAM4E Xplained Pro ...... 12 2.6.2 Code stepping ...... 12 2.6.3 Breakpoints ...... 12 2.6.4 Conditional debugging ...... 13 2.7 Laboratory...... 13

3 Related Work ...... 14 3.1 The impact of Software Testing education on code reliability: An empirical assessment ...... 14 3.1.1 Comments ...... 14 3.2 Debugging Students’ Debugging Process ...... 14 3.2.1 Comments ...... 15 3.3 Debugging Behaviour of Embedded-Software Developers: An Exploratory Study ...... 15 3.3.1 Comments ...... 15 3.4 WAP: Understanding the brain at software debugging ...... 15 3.4.1 Comments ...... 16

4 Methodology...... 17 4.1 Construct a Conceptual Framework ...... 17 4.2 Develop a System Architecture ...... 17 4.3 Analyze and Design the System ...... 18 4.4 Build the (Prototype) System ...... 18 4.5 Observe and Evaluate the System ...... 18

5 Result ...... 19 5.1 Construct a Conceptual Framework ...... 19 5.1.1 Literature review ...... 19 5.1.2 Software bugs ...... 19 5.1.3 Debugging techniques ...... 19 5.2 Develop a System Architecture ...... 20 5.2.1 Arduino Due ...... 20 5.2.2 AVR Dragon ...... 20 5.2.3 Atmel Power Debugger ...... 20 5.2.4 Atmel SAM4E Xplained Pro ...... 20 5.3 Analyze and Design the System ...... 20 5.3.1 Requirements ...... 20 5.3.2 Learning goals ...... 21 5.3.3 Choosing a debugger ...... 21 5.3.4 Choosing debugging techniques ...... 21 5.3.5 Choosing types of bugs ...... 22 5.3.6 Computer Science students ...... 22 5.4 Build the (Prototype) System ...... 22 5.4.1 Lab Introduction ...... 22 5.4.2 Lab guide ...... 23 5.4.3 Lab exercise ...... 23 5.4.3.1 Miscalculation - correct code ...... 23 5.4.3.2 Miscalculation - code with bug ...... 24 5.4.3.3 Off by one - correct code ...... 24 5.4.3.4 Off by one - code with bug ...... 25 5.4.3.5 Control flow - correct code ...... 26 5.4.3.6 Control flow - code with bug ...... 27 5.4.3.7 Solution to the exercises ...... 27 5.4.4 Questionnaire ...... 27 5.5 Observe and Evaluate the System ...... 28 5.5.1 Walkthrough of the lab session ...... 28 5.5.2 Observation ...... 28 5.5.2.1 Exercise One - blinkLED.c ...... 29 5.5.2.2 Exercise Two - blinkLED1.c ...... 29 5.5.2.3 Exercise Three - buttonPress.c ...... 29 5.5.3 Results of the lab exercise ...... 30 5.5.4 Evaluation ...... 31

6 Discussion ...... 33 6.1 Research methodology ...... 33 6.2 Results of the lab assignment ...... 33 6.2.1 Lemos et al...... 33 6.2.2 Böttcher et al...... 34 6.2.3 Arafa et al...... 34 6.2.4 Duraes et al...... 35 6.3 Bugs injected ...... 35 6.4 Debugging techniques ...... 36 6.5 Supervisor...... 36 6.5.1 Lab guide ...... 36 6.5.2 Lab exercise ...... 36

7 Conclusion ...... 37 7.1 Answering the research questions ...... 37 7.2 Future work ...... 38

References ...... 39

A Questionnaire ...... 43

B Solutions to exercise ...... 47

C Lab assignment ...... 71

1 Introduction Embedded systems [1] are technical innovations made from a combination of hardware and software. It is a computer system designed to perform a specific task, as for an example includes a computer keyboard, a coffee maker and an electric toothbrush. But embedded systems can also be parts built within a larger system. An automobile [2] has a number of embedded systems cooperating with each other to make the system complete. Overall, devices with embedded systems can be found in all fields [1,2,3], such as household appliances, telecommunications, medical equipment and industrial control systems. It has become a destined factor in facilitating and improving daily activities.

As the market of embedded systems keep increasing rapidly [2,3], it also implies that the level of what an embedded system can handle is constantly increasing. The embedded systems have grown larger, more complex and more integrated [5]. There is already immense market potential for new promising embedded devices. The world is facing a technical revolution [6] with self-propelled vehicles, industrial robots, medical technology, and household robots as for example.

There are devices which may be significantly more sensitive than others. A bug [3,4] that appears in a system may harm several parts of the system and may even cause the loss of human lives, or cause no harm at all. To improve quality, reliability and performance of the system, an essential process must be thoroughly performed during development. Thus, confidence will arise in the system. The process of importance is called software testing [4,5], including software debugging as a part of it.

1.1 Background Software development life cycle (SDLC) [7] is a conceptual model. The concept of the model is to define tasks to perform in the software development process. In general, SDLC follows tasks illustrated in Figure 1-1. In the model, each task [3,7] is dependent on the previous task. Only upon completion of Code & Build task, the final task of software testing will begin.

Studies have shown that bugs [3] can even appear in the first two tasks. Considering the late start of the testing phase, a bug will be identified during the latest stage in the development, near the finish line. This is usually the standard [9] process to be followed in software development. But bugs identified later in the development process tend to require more effort to be fixed rather than if they were identified at an earlier stage. Hence, the model is said to be exposed to vulnerabilities [11].

Figure 1-1 Graphical representation of SDLC [8]

Since bugs [13] often occur when developing software, it is important to identify them as soon as possible. At an early stage where it is still feasible and effective to remove existing bugs. Software testing is an essential process [5] with the intention of finding bugs and should be an iterative task starting from the beginning of SDLC, as illustrated in Figure 1-2. This is to ensure that bugs created in the first tasks can be

1 eliminated easily and not exist in the final release. There are various testing strategies and testing levels [14] performed in terms of software testing. Broadly speaking, there are two fundamental testing strategies: black-box testing and white-box testing, and three fundamental testing levels: unit testing, integration testing and system testing, as explained in Section 2.3.2 and 2.3.3

Figure 1-2 Improved graphical representation of SDLC [8]

Despite software testing being an essential process, it can only be used to show [10,13] the presence of defects in the system, but not their absence. Execution of the code may lead to a failure [10], an undesired output according to the system specifications. Based on a failure, the debugging process begins. Software testing is thus not the same as debugging, which is an integral component of the software testing process.

Debugging [10,12] occurs as a consequence of software testing and is a process to locate and further correct the encountered bug responsible for the failure. Individuals involved in debugging should, therefore, have a good understanding of the code [12]. One of the most common approaches to locate bugs is by inserting print statements [15], to print out the values of the variables involved, in the area of suspicious code. However, print statements may not be an ideal debugging technique to locate bugs in a complex system. A proper debugging technique using a debugging tool, also called a debugger, may be crucial.

The software bugs can be eliminated with both software testing and software debugging.

1.2 Problem domain The Computer Science program at Malmö University does not focus on teaching their students how to use a debugger to debug software. Since the students learn how to develop embedded software, it would be helpful to have more knowledge in debugging. Debugging software will further help students to understand their code when a system failure occurs. In association with this matter, the aim of this thesis is to create a lab assignment to teach students to use a debugger for debugging in embedded systems

1.3 Research questions To create a suitable lab assignment for Malmö University, three research questions were established.

RQ1: What types of software bugs are suitable to debug?

RQ2: Which debugging techniques can be used for debugging?

RQ3: How can a lab assignment teach students to debug?

1.4 Limitations The lab assignment is developed as a single-unit embedded system in the development environment Atmel Studio, using C programming language. A debugger that is supported in Atmel Studio is used to debug the software.

2 Theoretical Background This chapter provides a theoretical background aimed to ease the understanding of this thesis.

2.1 Embedded systems Embedded systems [2,12] have a set of characteristics that are unique to the embedded system under consideration, and thus are application specific in nature. To be application specific in nature means that embedded systems are specifically designed to perform a specific function in a certain field, such as household appliance and medical equipment mentioned in Chapter 1. Therefore, an embedded system designed with a specific function can most likely not be replaced by another embedded system with a different function.

2.1.1 Real-time systems Often an embedded system is used in a real-time constraint [17], meaning the total correctness of an executed task does not depend only upon its logical correctness, but also upon the time interval of meeting deadlines. The time interval between the start of a task and the expected time response to finish the task. Since a real-time system needs to complete certain tasks within a specified time interval, an embedded real-time system [16] must interact with its environment continuously and timely. The response requirements are what distinguishes a real-time system from a non real-time system.

Real-time systems are usually classified into hard and soft [16], depending on the consequence of missing a deadline.

2.1.1.1 Hard real-time system For a hard real-time system [16], it is important that the tasks are executed within the time interval. The deadline must be met or a system failure will occur. Flight control system and airbag deployment system are examples [2] of hard real-time systems.

2.1.1.2 Soft real-time system A soft real-time system [16] does not cause disastrous results if a deadline is occasionally missed. Instead, the value of the results rather degrades as time passes after the deadline, to a certain extent. Automatic Teller Machine (ATM) and Media Playback Systems are examples [2] of soft real-time systems.

2.1.2 Architecture Embedded systems can be classified based on their architecture [18]. Single-unit and multiple-unit, which is also called distributed, embedded systems are examples of two architectures.

2.1.2.1 Single-unit system Single-unit [18] embedded systems maintain computational capabilities and interact with the physical world via sensors and actuators which are sometimes fabricated on a single chip.

2.1.2.2 Distributed systems The term distributed [2,18] indicates that an embedded system is divided into multiple independent units, in which the units can be physically close together or be geographically distant. If the units are physically close together, they can be connected to a local network. If the units are geographically distant, they can be connected to a wide area network. If the units are networked via a wired or wireless network, they can interact with each other to achieve a common goal. With all the independent units interacting with each other, a single large embedded control unit is formed which is called a distributed system [2].

2.2 Error, fault and failure Just like any human being, software developers also make mistakes [13]. An error [10] represents a mistake or misunderstanding of the requirements of a system, produced by the software developer. The result of an error may lead software to contain a fault [10], which is an incorrect step, process or data definition [15].

A fault is also commonly referred to as defect or bug. When executing a system containing faults, it may lead to failure. However, faults do not always produce a failure [10], which is the inability of a system to perform its required functions.

There are many root causes for a code to contain bugs. Some of these are displayed in Table 1 [24], in which some are further studied in subsections.

Bugs

Data reference Data Computation Comparison Control flow Interface declaration

1 Referenced Variable is not Computations Compared variables A loop will Parameters sent variable is unset explicitly have non- have different data erroneously never to a function are or uninitialized declared arithmetic data types terminate in wrong order types

2 Array reference Declarative Computation is Mixed-mode Function will never Unit’s system of out of bounds or initialization of performed comparison with terminate parameters does by non-integer array is wrong between different different data not match (eg. data types without lengths voltage is sent to type casting function in volts but function expects millivolts)

3 Reference to Variables have Computation is On expressions A condition or loop A function has array is off by almost identical performed containing more will never execute multiple entry one names thus between the same than one Boolean due the conditions points and it increasing a data types but operator references a risk of with different data assumption about variable that is confusion lengths the order of not initialized in evaluation and all entry points execution is incorrect

4 Pointer points to Variable In division Boolean expression An iteration loop A function alters a an unreferenced assigned to computation does not state what has off-by-one-error parameter that is memory wrong data nominator can be it should state intended to be type zero only an input value

5 Value of a On expressions Operands of Loop or condition variable has a containing more Boolean operation decision is non- type or attribute than one operator are not Boolean; exhaustive (e.g. other than what assumption about comparison and expected values the compiler the Boolean operators are 1, 2 and 3, and expects order of operator are erroneously if the value is not 1 execution is mixed or 2, the program incorrect assumes it is 3)

6 Pointer points to Variable value Comparison a different data goes beyond operator is incorrect type than is meaningful range expected (e.g. probability is negative)

7 A data structure Invalid use of is referenced in integer arithmetic multiple (e.g. in expression functions, but it is using integer not defined division) identically Table 1 Certain type of bugs in software

2.2.1 Computation Computational faults [41] may involve algorithms with mathematical calculations. It may range from a computer cannot handle cases with division by zero and the computer assigns a wrong mathematical value due to round-off.

2.2.1.1 Operator precedence Logic faults such as erroneously using arithmetic operators and parentheses may be difficult to detect [30]. This due to the program executes the code without defining a failure and do not usually cause the program to terminate. An example of such a bug is shown in Figure 2-1, missing parentheses when calculating the average of two numbers [26]. The program is going to successfully execute, hence no syntax-, logic- or run-time error involved. However, the result of the calculation obtained is not correct, because of the operator precedence. The division operator has a higher precedence than the addition operator [26].

Figure 2-1 Missing parentheses

2.2.1.2 Divide by zero There are cases where computational bugs can cause the program to not execute successfully. Shown in Figure 2-2 is another example, dividing a number by zero [30]. Executing the code may cause a run-time error which in turn may cause the program to terminate.

Figure 2-2 Divide by zero

2.2.2.2 Off-by-one An off-by-one bug [41,42] may occur when a loop iterates one time too many or too few. It can be caused by starting a loop variable at 1 instead of 0 or using the operator “<=” instead of “<”, or vice versa. Figure 2-3 illustrates example of an array index out of bounds. The array (array[4]) has four elements with the indices 0, 1, 2 and 3. As j <= 4 is evaluated in the for loop, the second print statement will attempt to access a memory location where the index of the array is 4, that is nonexistent. Hence, the output value from the array is accessed from a random memory location containing a garbage value.

Figure 2-3 Off-by-one 6

2.2.2 Data reference (pointers) A pointer [37] is used to reference a memory location in which every memory location has its memory address. The value stored at the memory location may be accessed or modified using a pointer, known as dereferencing the pointer. There are some common faults related to pointers [31], studied as follows:

2.2.2.1 Uninitialized pointer An uninitialized pointer [32] occurs when a pointer is declared but not initialized. An example is shown in Figure 2-4. The pointer ptr is only declared and therefore may point to any (non-intentional) memory location such as into the system stack or the global variables. When the value of 10 is assigned to pointer, the program may write a 10 to the memory location where pointer points to. To store a value in a non-intentional memory location may have the potential to overwrite anything in the code that causes the program to behave differently than expected. It may even cause the program to terminate immediately.

Figure 2-4 Uninitialized pointer

2.2.2.2 Null pointer Unlike an uninitialized pointer which stores an undefined value in any (non-intentional) memory location, a null pointer [32] stores a defined value but does not refer to a valid memory location. That is, the null pointer is initialized but is not pointing to any memory location, see Figure 2-5. When a pointer variable does not yet have a memory location to be assigned, it is wise to use a null pointer. Having a null pointer is not going to unnecessarily harm data in the memory, unlike an uninitialized pointer.

Figure 2-5 Null pointer

2.2.3 Redundant code Unnecessary code is called redundant code [34]. This code could include unused variables, restating an expression, code that is never executed (also called unreachable code), and executed code that has no effect on the output (also called dead code). Figure 2-6 shows an example of restating an expression. The return statement has an ”x * 2” expression which is redundant code that could be replaced by variable y. Alternatively, the statement int y = x * 2 could be removed.

Figure 2-6 Redundant code

2.2.4 Control flow A control flow is defined as an execution order of function calls, statements, and instructions. Within C language, there is a control flow statement [56]. It is a statement whose execution results in the choice of choosing which of two or more paths to be followed. A loop may be involved in a control flow statement upon which the loop will never terminate or will never execute due to the conditions [24].

2.2.4.1 Infinite loop A loop is used to repeat a set of instructions. When a loop such as for loop, while loop and do...while loop, does not terminate due to no condition or having a condition that is never met, causes an infinite loop [54,55]. As a result, the program becomes unresponsive. Shown in Figure 2-7 is an example of an infinite loop. The expected output is “4 3 2 1”. Since the variable count is increasing instead of decreasing in the loop body, the while statement stays true continuously and the loop cannot terminate.

Figure 2-7 Infinite loop

2.3 Software testing Software testing [12] is a sequence of tasks applied to examine whether the system meets the end user’s specified requirements. Otherwise, the software of the system needs to be corrected according to the requirements. This is where the motive of software quality came to be introduced [19], to ensure that the released software functions as expected. As with any definitions, even the definition of software quality has many variations. Some researchers [20] state that quality cannot be defined. Some state that quality can be defined but only in a particular context. Others state that quality is lack of bugs. The definitions are diverse and debatable to prominent researchers but despite the difference in definitions [20], quality is a matter to such a degree that each and everyone is eager to strive for in their systems.

2.3.1 Verification and validation There are two terms commonly used in association with software testing, that is verification and validation [10].

• Verification [12]: Are we building the product right? • Validation [12]: Are we building the right product?

Verification [10,21] helps to ensure that the requirements of the system are fulfilled. An example [22]:

“Suppose you are building a table. Here the verification is about checking all the parts of the table, whether all the four legs are of correct size or not. If one leg of table is not of the right size it will imbalance the end product”

Though verification will give guidance in determining whether the software is of high quality, yet it does not provide that the system will fulfill its intended use in the proposed environment [21]. This is where validation is presented [10,21], a process of comparing what was built to the specified requirements.

Verification and validation of an embedded system can be achieved using static and dynamic testing [23].

2.3.2 Static and dynamic testing Static testing [12] is a process in which the code is not required to be executed. It includes reviews [10,57] to examine the program’s code and its associated documentation such as plans, requirement specifications, test cases, and user manuals. Reviews may be used to find bugs, build confidence in a component and understanding documentation.

Dynamic testing [2] is a process which involves executing the code, giving input values to examine if the output values are as expected [23]. In dynamic testing, there are two fundamental testing strategies: black- box testing and white-box testing.

2.3.2.1 Black-box testing Black-box testing [15], illustrated in Figure 2-8, is a method in which a tester interacts with the system by providing inputs and examining outputs. The entered input data is compared with the output data obtained [15] to determine whether the system being tested have run successfully or failed. The focus is on input and output, meaning the functionality of the system is examined without looking into the internal mechanism (the structure of the code) of the system. Given the circumstances, black-box testing does not require knowledge of the internal mechanism of how and where the inputs are worked upon.

Figure 2-8 Black-box testing [15]

2.3.2.2 White-box testing White-box testing [15], illustrated in Figure 2-9, is a method used to examine the functionality of the system but also ensure that all code is executed. Hence, knowledge of the code is required when performing white- box testing.

Figure 2-9 White-box testing [15]

2.3.3 Testing levels As mentioned in Section 1.2, an embedded system can be tested in different testing levels in which the three fundamental test levels are as follows:

2.3.3.1 Unit testing A unit [15] refers to the smallest piece of a testable part, which could be a function or a module. Unit testing [2,10] is the testing of each unit independently to ensure it is functioning according to its requirements.

2.3.3.2 Integration testing When a unit [10] has been completely tested and is functioning as accordingly, it is ready for integration. The concept is to test how units of the system work together. A unit integrated with another unit constitutes a subsystem [15]. Integration testing [10] works best as an iterative process, integrating one unit at a time into a subsystem.

2.3.3.3 System testing With integration test completed, each subsystem is assembled to form the system as a whole [10]. System testing [10] is performed to evaluate whether the system meets the specified requirements.

2.4 Debugging procedure When bugs are found, they need to be fixed. To resolve these bugs, software developers may need to perform debugging. Debugging is a way to identify the bug in the code, so that it can be corrected [36]. There are some debugging steps [36] which can be followed, illustrated in Figure 2-10 as a state diagram. The state diagram shows a distinctive procedure starting from a program execution to find a defect in the system, arranging bugs into different categories, trying to understand the bug behavior and then fixing the bug. In case a defect is not detected or remedied, the program will be executed for further examination.

Figure 2-10 State diagram of debugging procedures 10

2.5 Debugging techniques Many debugging techniques [38] are available to debug software. Some of these are hand tracing the code, causal reasoning (checking the output of the program and go back into the code to locate bugs), monitoring and logging. Every debugging technique does not need a debugger for debugging, in which some are introduced as follows:

2.5.1 Print statements This is one of the most used debugging techniques. In C programming, it involves calling the printf() function [39]. The software developer inserts a line of the print statement to check the control flow to locate bugs in the code, by printing out a message. If the message is printed and displayed in the console window with the expected output data, it indicates that the statement or function is bug-free.

2.5.2 Logfile debugging Bugs may be found by reading the program’s logfile, sometimes called output file. It is a file that records all events from program executions events such as input values, the value of variables, system notifications and error messages. The logfile records when and where every program execution happens. The software developer goes through the logfile attempting to find the reason why a bug occurs [44,45].

2.5.3 Assertions Assertion [44] is a predefined condition set within the code, usually written with an if statement. To set an assertion, allows the software developer to record an assumption in the code, where the assumption is automatically checked at runtime of the program. Every assertion is evaluated to be true at a specific point in the code [43]. When the condition of the assertion returns false, it may indicate that there is a bug in the code. The assertion pauses the execution of the program to allow the software developer to determine the cause of the bug.

2.5.4 Dry run debugging Dry run [63,65,67] is based on examining and tracing the source code by writing every single step of the code on a paper [64]. The software developer acts as a computer [66], running the instructions of the program and register the values of the variables on the paper as they change in the program. This type of execution is also called a mental run because the source code is executed and examined in the mind of the software developer. It is not executed in a machine or using a development platform like Atmel Studio [63]. An example of mental run is shown in Figure 2-11 and Figure 2-12.

Variable value Output i calculateSum

1 1

2 3

3 6

4 10

5 15 15

Figure 2-11 Dry run code Figure 2-12 Dry run debugging

2.6 Debugger Finding bugs in a program sometimes requires a debugging tool, known as a debugger. A debugger may be used by a software developer to examine and trace the source code to find the main cause of bugs [49]. It has the ability to show detailed information about the program being executed such as what data stored in each variable [50,58].

Most of the integrated development platforms as Atmel Studio has debugging control buttons, illustrated in Figure 2-13. These control buttons may give the debugger instructions to step through lines of code in the program and stop a debugging process [52]. Some debugging techniques using these control buttons are code stepping, breakpoint and conditional breakpoint.

Figure 2-13 Debugging control buttons in Atmel Studio toolbar

2.6.1 Atmel SAM4E Xplained Pro The Atmel Xplained Pro Evaluation Platform provides a full Atmel microcontroller experience [61], meaning the platforms are supported by Atmel Studio integrated development platform (IDP). Atmel SAM4E Xplained Pro [62] is a hardware platform provided to evaluate Atmel ATSAM4E16E microcontroller. Thus, all the features of ATSAM4E16E may be accessed by the SAM4E Xplained Pro. The hardware platform contains an on-board Embedded Debugger (EDBG) [62] and does not require an external tool to debug software.

2.6.2 Code stepping Code stepping is used to examine how a program is executed line by line, using the control commands Step Into, Step Over and Step Out [40,53] shown in Figure 2-14. Step Into conducts the debugger to examine an instruction or a particular function in details. On the contrary, Step Over conducts the debugger to only execute the line of code, but avoids to examine it in details. Step Out conducts the debugger to execute all the remaining instructions in the current function to exit and go to the next line of code.

Figure 2-14 Code stepping commands

2.6.3 Breakpoints A breakpoint stops or pauses a running program when the execution reaches the breakpoint. When the breakpoint is reached, the code may be examined [38] using code stepping commands.

2.6.4 Conditional debugging Sometimes, a breakpoint may have extra features, that is to hold a condition and an action. These features can be applied when a breakpoint is set. The condition is a boolean expression which is evaluated to true or false when the assigned breakpoint is reached [59,60]. An example is to set a condition that will make the program execute until variable i reaches the defined value of 3, see Figure 2-5.

Figure 2-15 Breakpoint condition

An action can log a message that will be displayed in the Output window. Figure 2-16 shows an example of an action.

Figure 2-16 Breakpoint action

2.7 Laboratory In the Computer Science program at Malmö University, lecturers are teaching topics as programming through lectures followed with laboratories. A laboratory, or lab, takes place in the university’s lab room with about 20 students. It is a session in total of 7 hours where the students utilize theory from the lectures to practice. A supervisor is available during the lab time in case of ambiguity.

Lab instructions of the task are provided for the students to read in advance for each lab session. This is to prepare the students for what are expected of them to be able to complete the task. Equipments such as computers, measuring instruments and wire cables are available in the lab room and may be used for the tasks.

The students are advised to work in pairs, but they are not obliged to do so. When the students have completed the task, they should report it to the supervisor. The students have to explain what has been done and how they accomplished the task, to show their understanding.

3 Related Work This chapter presents papers that are related to this thesis as they provide guidance in certain development phases. In each subsection, the paper is first summarized followed by a comments section as to what the paper provides to this thesis.

3.1 The impact of Software Testing education on code reliability: An empirical assessment Lemos et al. [27] state that software testing is the most important and the least known factor of software development. Although software testing is said to be important to produce a more reliable program, there is no empirical evidence to support it. Most Computer Science students graduate without learning about software testing. Hence, two investigations were carried out: one involving students and the other involving professors. The goal of the study is to investigate whether software testing can make an impact on programming skills toward more reliable code.

The students were asked to perform the experiment of Lemos et al. before and after gaining knowledge in software testing. But which testing techniques the students use to apply in their experiments is not focused on. The quality of the code produced before and after gaining knowledge in software testing was compared and according to the studies, gaining knowledge in software testing can improve code reliability in terms of correctness in as much as 20%, on average.

The investigation involving professors involved a survey to evaluate the professors’ knowledge in software testing. The survey was composed of 10 basic software testing questions. With 53 professors involved, the average score was 4.24 out of 10.00. It turns out professors lack understanding of basic software testing concepts. Concepts that according to the previous investigation with the students indicate to be important to produce a more reliable code.

3.1.1 Comments Even though Lemos et al. did not focus on which software testing techniques the students applied to their code, the results still show an improvement of programming skills in terms of reliable code. The students may have received better results if they knew which software testing techniques to apply in certain scenarios. Since this thesis focus on how to perform debugging, the students shall be taught that certain debugging techniques may be more suitable in certain scenarios.

3.2 Debugging Students’ Debugging Process Böttcher et al. [28] presents how novice programmers in their freshman year were asked to identify, analyze and fix defects in given code in Java programs. A common problem among students was that they had a lack of skill and knowledge of the essential base competencies. Despite that most IDEs support a debugging tool, students most likely do not use this tool but rather apply print statements for visual inspection of values of the variables.

Debugging is an essential skill for software developers. Therefore, Böttcher et al. designed an approach for systematic debugging to help to improve the students debugging skills. A lecture was made to fill in theoretical information for basic understanding and thereafter a lab exercise was given to the students. About half of the students found it difficult to follow the detailed demonstration of how to detect one of the bugs. In total, the results ranged from “completely lost in the code” to “all bugs found and fixed”. Some of the students worked in pairs, in which some of these pairs included a strong partner and a less strong partner that had difficulties in tackling the code.

According to Böttcher et al.’s teaching experience, it seemed to be difficult to learn novice programmers the process of debugging. It requires to have skill in abstract and logical thinking, and working systematically for example. The students approached to detect bugs in different ways such as visual diagnosis and randomly poking around.

3.2.1 Comments This paper contributes to this thesis by providing information that it is possible for Computer Science students at Malmö University to get guidance in learning how to debug through a lab assignment. What is different for this thesis, is that other equipment and environments are used.

3.3 Debugging Behaviour of Embedded-Software Developers: An Exploratory Study Arafa et al. [29] formed an investigation of how intermediate-level developers debug embedded software. Each developer participating in the studies must have completed a Real-time Operating Systems course due to the program to be debugged. The program contains a small real-time operating system (RTOS) implemented on a Janus MCF5307 based microcontroller board. Seven semantic low-level bugs, Bug-1 to Bug-7, are planted in the program. The bugs are categorized into incorrect hardware configuration bugs such as incorrect serial port baud rate, and memory leaks such as two lines of code are missing. The debugging activities performed by the developers are observed and compared between successful and unsuccessful attempts.

Three out of seven bugs are classified as incorrect serial port baud rate: Bug-1, Bug-2, and Bug-7. In total, 21 percent of the participants fixed Bug-1, 14 percent fixed Bug-2, and 100 percent fixed Bug-7. The results were observed via video recorded computer screen, saved compiled systems and from a pre-session and a post-session form filled out by the students

3.3.1 Comments This paper provides examples of what types of bugs may be planted into a project, which is of help to the lab assignment. Bug-1 and Bug-2 were more difficult to fix than Bug-7. The ultimate would be to find a level of bug complexity that are between the bugs Bug-1 or Bug-2 and Bug-7. Also how to observe students participating in the lab assignment is provided. The same method with video recorded computer screen may be used in this thesis since it might give an additional perspective on the analysis.

3.4 WAP: Understanding the brain at software debugging Duraes et al. [46] presented a study to analyze how the activity of the human brain is affected during the inspection of a source code. This is accomplished by using a tool which is used in cognitive neuroscience called functional magnetic resonance imaging (fMRI) scanner.

Six programs were developed, in which three of them were injected with the most common types of bugs, written in C language. The wrong expression in assignment or parameter, missing assignment and missing part of an expression, were the types of bugs injected. The programmers were given the programs one at a time, alternating between programs with bugs and programs with no bugs. The idea behind using the combination of programs with bugs and programs with no bugs was for the purpose of not making the programmers tired and lose attention.

Duraes et al. used certain guidelines in the choice of programming language and the complexity of the program. The C language was chosen due to it is a well-known language with the possibility to inject bugs in the program not detected by the compiler. Regarding the complexity of the program, they decided to have between 50-100 lines of code for it not to be too easy or too difficult to find the injected bugs.

There were 15 voluntary programmers with different level of expertise in reviewing the source code. The programmers were placed inside the fMRI scanner to regularly get their brain activities measured to examine which parts of the brain regions were most active when a bug was discovered. The results showed that brain regions associated with language processing and mathematics were active when the programmers reviewed the code, and a certain part of the brain associated with decision making was active when the programmer had a suspicion of a bug in the program. But the most important result was that the anterior insula region of the brain played an important role in identifying bugs.

3.4.1 Comments This paper contributes to this thesis in determining how small or large the debugging program to be created should be, and the choice of bugs and its complexity level to be used.

4 Methodology In order to conduct this research, the methodology System Development Research Process proposed by Nunamaker et al. [25] was used. The methodology shows a research process suitable for system development and consist of five iterative steps with descriptive research issues to be addressed, shown in Figure 4-1. Since a laboratory assignment is conducted in this research, the use of the methodology was adjusted. A mixed set of research methodologies such as literature review, case study and questionnaire were combined within Nunamaker et al.’s methodology.

Figure 4-1 Methodology: System Development Research Process [25]

4.1 Construct a Conceptual Framework As the first stage of development, it is important to gain a thorough knowledge of the problem domain to be able to state research questions, stated in Section 1.3. A literature review was thus performed, with the focus in software bugs, software testing and software debugging. Reliable sources were obtained by conducting searches in databases. The results gained from the literature review is stated in Section 5.1

4.2 Develop a System Architecture A lab assignment was created to teach students how to debug using a debugger. To create a lab assignment that is suitable for Malmö University setting, a debugger that is supported in Atmel Studio was used. This due to Computer Science students at Malmö University are familiar with developing software in Atmel Studio. A case study was performed to confirm the debugging support, stated in Section 5.2.

4.3 Analyze and Design the System To propose a set of software bugs and debugging techniques that are suitable to be used for teaching debugging, requirements for the lab assignment have to be determined. The limitations in Section 1.4 will take part in determining the requirements. Similarly, learning goals for the lab assignment have to be determined to derive the students to know what knowledge is supposed to be gained at the end of the lab assignment. The requirements and learning goals are stated in Section 5.3.1 and 5.3.2.

With requirements and learning goals determined, the choice of a debugger, software bugs and debugging techniques that are suitable to be used for the lab assignment, is explained in Section 5.3.3 - 5.3.5. The lab assignment was designed to be performed by a certain target group, stated in Section 5.3.6.

4.4 Build the (Prototype) System With requirements and learning goals, and the selection of components, bugs, and debugging techniques determined, the lab assignment was created. The composition of the lab assignment is shown in Section 5.4.

4.5 Observe and Evaluate the System Once the lab assignment was created, it was performed by the target group, stated in Section 5.5.1. The debugging activities of the lab assignment was observed by the supervisors, in which the observation of each exercise in the lab assignment are shown in Section 5.5.2. The observation was necessary to obtain a result of which bugs were found and remedied, shown as a table in Section 5.5.3. With every part of the lab assignment successfully performed, the evaluation of the lab assignment was assembled, shown in Section 5.5.4.

5 Result In the following chapter, the results from the different stages of Nunamaker et al.’s methodology are presented.

5.1 Construct a Conceptual Framework This section shows the results from the information gathering to be able to provide answers to the research questions.

5.1.1 Literature review The databases used for reliable information gathering were Google Scholar, IEEE Xplore, ACM Digital Library and ScienceDirect. But Google was also used, which is further explained in Section 6.4.

A set of software bugs, software testing, and debugging techniques were found and studied in Chapter 2. Through the information gathering, four papers were chosen as related work, stated in Chapter 3. These papers helped to strengthen this thesis by providing information such as what types of bugs could be planted in the debugging exercises and how data from the exercises may be collected as a result.

5.1.2 Software bugs Several types of bugs are studied and shown in Table 1 in Section 2.2. Considering creating a lab assignment for students to learn how to debug using a debugger, the bugs should not be too easy or too difficult to find and remedy. The authors of the papers in Section 3.3 and 3.4 used some of the common types of bugs in Table 1 in Section 2.2 to test their participants. It was a challenge to them and they gained more knowledge in debugging, which implies that these types of bugs may also be suitable to this thesis.

The chosen bugs to be planted into the lab assignment must let the program execute a run-time error that does not give compilation warnings. If compilation warnings appear, it would not require much effort to solve the bugs. In these situations, there would be no need to use a debugger which is the main element of this thesis.

Apart from the common types of bugs mentioned in Section 2.2, other types of bugs have also been studied such as memory bugs [68]. Some of these bugs are memory leak and invalid memory access in heap and stack. But through investigations, it may be too advanced to learn how to debug memory bugs. There is also another tool often used to detect memory bugs, that is the detector tool Valgrind [48].

5.1.3 Debugging techniques There are several debugging techniques studied in Section 2.5 and 2.6. The debugging techniques in Section 2.6 require the use of a debugger, but is not required for the debugging techniques in Section 2.5 and they are thus excluded from the lab assignment. It can be useful to understand the most common debugging techniques, studied in Section 2.6, before proceeding to the advanced ones that may be less common. These are not only used in C language but also in programming languages such as Java, C# and C++.

Post-mortem, program slicing and Deterministic Replay Debugging (DRD) are some of the debugging techniques studied but not mentioned in Chapter 2. These debugging techniques are advanced and made to debug more complex programs and thus are not suitable for the lab assignment. According to the information gathered, post-mortem [69] may be used to save information about memory, in which memory was explained in previous Section 5.1.3 to be advanced.

Program slicing [70] is suitable for larger programs where lines of code that do not affect the original program and the output data, may be removed. When small slice of the source code is removed, the developer could concentrate on the relevant part of the program.

Deterministic Replay Debugging [71] replays the steps that causes the bug in the program. This could help the developer to replay each step of code in the program execution prior to the program being crashed [47], which is also suitable for larger programs.

5.2 Develop a System Architecture A set of components were available to be received from the Computer Science program in which these components are further studied to find a debugger for the lab assignment.

5.2.1 Arduino Due Arduino Due board is a familiar microcontroller board to the Computer Science students at Malmö University. It would therefore be applicable for the students to learn how to debug software that is uploaded to a Due board. However, research proved that an external debugging tool must be attached to the Due board to be able to debug the software. AVR Dragon and Atmel Power Debugger are some of the debugging tools supported in Atmel Studio.

Visual Micro [72] is a plugin for Visual Studio or Atmel Studio and includes a debugger, the Visual Micro Debugger. But the debugger is used to debug Arduino sketches. Since Atmel Studio is used to create the laboratory assignment, it is meant for the source code to be written in C files (.c) and header files (.h), and not written in Arduino sketches. Therefore, Visual Micro was excluded.

5.2.2 AVR Dragon The debugging tool AVR Dragon was received from the Computer Science program. AVR Dragon [73] supports devices from the Atmel AVR families and includes debugging support for most of the devices. To debug with AVR Dragon, the Due board need to be an AVR microcontroller. However, the Due board is a microcontroller based on the Atmel ARM, meaning that AVR Dragon and Arduino Due does not support each other in debugging.

5.2.3 Atmel Power Debugger Atmel Power debugger was received as well from the Computer Science program. It is a development tool [74] compatible for debugging and programming both ARM and AVR microcontrollers. It is a tool that requires the use of target interfaces such as JTAG and aWire for debugging and programming, and is often used to measure, analyze, understand and optimize the power consumption of a microcontroller.

5.2.4 Atmel SAM4E Xplained Pro The hardware platform SAM4E Xplained Pro is studied in Section 2.6.1. Research shows that the platform acts as both a microcontroller and a debugger. C code can be uploaded onto the platform in which the code can also be debugged without the need of attaching an external tool.

5.3 Analyze and Design the System The lab assignment to be created is inspired to have a similar setup as a laboratory in the Computer Science program at Malmö University, explained in Section 2.7.

5.3.1 Requirements Following requirements in Table 5-1 must be taken into consideration when setting up the lab assignment.

Requirements

R1 The target group includes Computer Science students

R2 The lab assignment shall be divided into several parts: guide, exercise and evaluation

R3 The theoretical background shall be explained to gain fundamental knowledge that may assist to perform the tasks easier

R4 The integrated development platform Atmel Studio shall be used

R5 The debugger must be supported in Atmel Studio

R6 The debugging techniques shall be explained as to why they are useful

R7 The bugs shall be remedied using a debugger

R8 The lab assignment shall take a maximum of about 7 hours to complete Table 5-1 Requirements

5.3.2 Learning goals In addition, the target group is expected to meet the following learning goals in Table 5-2.

Learning goals

L1 The students shall gain knowledge about software debugging

L2 The students shall gain knowledge about debugging techniques

L3 The students shall be able to debug code using a debugger

L4 The students shall gain knowledge about the benefits of using a certain debugging technique in a certain scenario Table 5-2 Learning goals

5.3.3 Choosing a debugger Stated in 5.2.1 - 5.2.2 are some debugging issues that arise when choosing the Arduino Due board. The Atmel Power Debugger in Section 5.2.3 seemed to be more useful for measuring power consumption and was thus dismissed. The final decision was to use the SAM4E Xplained Pro, with the explanation in Section 5.2.4 in which R5 in Table 5-1 is also met. Since the lab assignment will be created in Atmel Studio using C files and header files, the C language would still be familiar to the students. The debugging part will not be an issue because that is what the lab assignment is used for, to learn how to debug.

External components have been selected to not be used which means the source code uploaded to SAM4E Xplained Pro will only treat functionalities as the built-in LED and the built-in button available on SAM4E Xplained Pro. The choice of omitting external components is primarily to keep the lab exercise between 50- 100 lines of code, as Duraes et al.’s research in Section 3.4. It is to make the exercises not too easy or too difficult to debug.

5.3.4 Choosing debugging techniques The debugger chosen was SAM4E Xplained Pro. It is a debugger that supports two of the most well-known debugging techniques which are code stepping and breakpoint. In addition, SAM4E Xplained Pro also supports the extra feature of a breakpoint, the conditional breakpoint.

To debug software mentioned in Section 5.3.3, advanced debugging techniques are not needed. Since the code will not be too advanced, the debugging techniques mentioned in Section 2.6.2 - 2.6.4 should be suitable. These are debugging techniques that are supported in the debugger.

5.3.5 Choosing types of bugs According to Section 3.3, Arafa et al. present seven semantic bugs of embedded software to be debugged. These bugs are categorized as incorrect hardware configuration and memory leaks. As mentioned in Section 5.1.2, memory bugs will be discarded. The incorrect hardware configuration bugs for Bug-1 resulted in showing a corrupted text on the screen, whereas Bug-2 caused the timer interval to be slower than intended. Since only 21 percent and 14 percent of the participants could fix these bugs, even the incorrect hardware configuration bugs seem to be difficult to debug.

Some of the well-known bugs are extracted from Table 1 in Section 2.2 and are studied in Section 2.2.1 - 2.2.4. In Section 3.4, it is said that Duraes et al. injected bugs such as the wrong expression in assignment or parameter, missing assignment and missing part of an expression in their programs. These bugs are similar to bugs that are often found in programs. But instead of using too similar bugs as in Duraes et al.’s research, computational bugs dealing with miscalculation and off-by-one were decided to be used in the lab assignment.

Assigning an incorrect value to a variable [47] is one of the most frequent bugs, according to Castelo et al. [46]. To make the bug not too obvious to find with just an incorrect value, a pointer was used to point to an incorrect variable to create a control flow bug.

Three bugs, whereof two computational bugs and one control flow bug mixed with a pointer, have been chosen as the bugs going to be implemented in the laboratory exercises. These bugs can be fixed using the debugging techniques supported by the debugger.

5.3.6 Computer Science students Four voluntary Computer Science students decided to be a part of this research. They are named as Student 1-4. Student 1 and Student 2 are in their first year, and Student 3 and Student 4 are in their second year. They have different levels of programming skills, in which the second-year students have a higher level of skills compared with the first-year students who are beginners.

5.4 Build the (Prototype) System To meet the requirement R2 in Table 5-1 in Section 5.3.1, the lab assignment was divided into three parts to distinguish the various elements to be performed. Each part is followed by a set of sections to make it easier to refer to parts and sections in the document of the lab assignment. The document is shown in Appendix C.

In the following subsections, each reference is referred to the lab assignment in Appendix C, except for Section 5.4.3.7 with the reference referred to this thesis document.

5.4.1 Lab Introduction The lab assignment begins with an introductory chapter that briefly explains the reason and the purpose behind the lab assignment. Followed is a description of the lab assignment and the appropriate preparation for it. The preparation is used to help the students gain knowledge of each subject that is dealt with, stated in Section 1.4 to fulfill R3. As mentioned in Section 1.1, the project code to be debugged is implemented in advanced, that is the lab guide and the lab exercise. The tools to be used for debugging are mentioned in Section 1.5 in which R4 and R5 are met.

The introduction of the lab assignment is shown in Chapter 1.

5.4.2 Lab guide The debugging guide was assigned to be Part 1 of the lab assignment. It has five sections in total, in which the first section shows how to set up the lab environment. The debugging guide consists of an Atmel project that is implemented in advance, to be followed together with the instructions in Part 1, and is found in Section 1 to be downloaded. Remaining sections shows and explains the approaches of using each debugging technique, fulfilling R6.

The debugging guide of the lab assignment is shown in Part 1.

5.4.3 Lab exercise The lab assignment consists of three debugging exercises within an Atmel project, found in Part 2 to be downloaded. Each exercise is divided into a section. There is one bug in each exercise, as shown below:

5.4.3.1 Miscalculation - correct code Figure 5-1 shows the correct state of the program in which the SAM4E Xplained Pro board’s LED will turn on and off in accordance to the result of the average calculation. In this case, the average calculation of the array number is 3, meaning the LED will blink three times.

Figure 5-1 Miscalculation - correct code

5.4.3.2 Miscalculation - code with bug Shown in Figure 5-2, average+=number[i] was changed to average+=number[i] +1 in line 32. This leads to an average calculation that is not correct, leading the LED not to blink.

Figure 5-2 Miscalculation - incorrect code

5.4.3.3 Off by one - correct code Figure 5-3 shows the correct state of the program and will blink the LED on and off three times in this program as well.

Figure 5-3 Off-by-one - correct code

5.4.3.4 Off by one - code with bug Figure 5-4 shows the incorrect state of the program, in which i=0 was changed to i=1 in line 33.

Figure 5-4 Off-by-one - incorrect code

5.4.3.5 Control flow - correct code The SAM4E Xplained Pro board’s button shall be pressed three times for the LED to blink. Figure 5-5 shows the correct state of the program.

Figure 5-5 Control flow - correct code

5.4.3.6 Control flow - code with bug Figure 5-6 shows the incorrect state of the program, in which *ptr==1 was changed to *ptr==&x in line 34.

Figure 5-6 Control flow - incorrect code

5.4.3.7 Solution to the exercises The solution to each exercise above is shown in Appendix B. These solutions are not in any way absolute, meaning the students to perform the lab assignment may approach different debugging techniques to solve the bugs.

The debugging exercise of the lab assignment is shown in Part 2.

5.4.4 Questionnaire A questionnaire was created to form a validation of the lab assignment to check if it was enough for the students to meet the needs and expectations of the learning goals. Table 5-3 shows the questionnaire to be answered.

Statements Agree Neutral Disagree Notes

Q1 Debugging using a debugger is challenging ☐ ☐ ☐

Q2 The applied debugging techniques are easy to use ☐ ☐ ☐

Q3 The lab assignment is well-designed and easy to follow ☐ ☐ ☐

Q4 The bugs were easy to find and remedy ☐ ☐ ☐

Q5 I understand the debugging techniques and they are convenient ☐ ☐ ☐

Q6 Being a good programmer makes it easier to find bugs ☐ ☐ ☐

Q7 Time is the main factor for not using a debugger ☐ ☐ ☐

Q8 A lab assignment is a good way to be taught debugging ☐ ☐ ☐

Q9 I understand why a debugger is used ☐ ☐ ☐

Q10 I understand when the debugging techniques are suitable ☐ ☐ ☐ Table 5-3 Questionnaire

The questionnaire of the lab assignment is shown in Part 3.

5.5 Observe and Evaluate the System The document of the lab assignment (Appendix C) was provided in advance for the four voluntary students to prepare for the actual lab session.

5.5.1 Walkthrough of the lab session The students were gathered in Malmö University’s lab room equipped with computers and other necessary components, as mentioned in Section 2.7, available to the students. With all the students gathered, the lab session started with a short walkthrough of the lab assignment to clarify any ambiguities. The walkthrough was held by the authors who in this case were the supervisors. The supervisors were available during the lab session to help and advise the students in case of difficulties.

After the walkthrough, the students decided to work in pairs. This was completely acceptable since they are usually advised to work in pairs in a general lab in the Computer Science program, mentioned in Section 2.7. Student 1 and Student 2 worked in pairs, and Student 3 and Student 4 worked in pairs.

5.5.2 Observation There are different approaches in debugging software, as mentioned in Section 5.4.3.7. In order to document the results of how Student 1-4 used the debugger to solve the exercises, the supervisors were observing the students’ computer screen. But to gain an understanding to the students’ approach of using the debugging techniques, there was an ongoing chat where the students had to say their thoughts aloud, using the method of think-aloud [33].

Starting with Part 1 in the lab assignment, the students followed the instructions in the lab guide with few concerns explained in Section 6.3. Regarding Part 2 in the lab assignment, the students’ approach to remedy the bugs in the lab exercises were as follow:

5.5.2.1 Exercise One - blinkLED.c The first exercise is related to the code in Section 5.4.3.1 where the board LED shall blink three times. But since a bug has been planted in the code, shown in Section 5.4.3.2, the LED will not blink.

Student 1 and Student 2 approached code stepping, using the Step Over command until the function on_off_LED() on line 19 was reached. Thereafter they used Step Into command to execute the function in detail. Step Over was used until line 35. At that line, Step Over was used once again to examine what the value of average was holding, in the Autos window. The value of average was 4 and the debugger had jumped outside of the loop, which means the switch case was not met. Student 2 realized on line 37 that there was a case with the number of 3. Both Student 1 and Student 2 then understood that the value of average should be 3 for the switch case to be met, meaning the calculation of average was wrong. The students restarted the debugging process with setting a breakpoint on line 29 and then use Step Over for further examination. After the execution of line 33, they noticed that the value of average was 2 by observing the Autos window. This made them realize that line 33 increments the indices with 1. Therefore, “+1” of the calculation was removed and the program started blinking after executing the program.

Student 3 and Student 4 started off with setting a breakpoint at the while loop on 29 to skip a few iterations of Step Over command if they were to start with code stepping. With the execution of the while loop, the debugger jumped inside the body of the loop. They observed the Autos window to see each value of the variable and immediately noticed that the value of number was 2, but the first value of number was 1. This made both of the students realize a bug with the indices of the array. Shortly, they realized it was not a bug with indices after all. On line 33, the calculation was incremented with 1 which added the value of 1 to the values in the array. Removing the extra addition by 1, fixed the bug.

All the students succeeded in finding and fixing the bug.

5.5.2.2 Exercise Two - blinkLED1.c As in Exercise One, this exercise shall blink the board LED three times when the code is correct, shown in Section 5.4.3.3. With a bug in the code, shown in Section 5.4.3.4, the LED will not blink.

Student 1 and Student 2 approached code stepping, using the Step Over command until the function on_off_LED1() on line 20 was reached. Thereafter they used Step Into command to execute the function in detail. In the body of the function on_off_LED1(), the students used Step Over until they reached line 37. Shown in the Autos window, the value of average was 2, which seemed to be wrong to the students. As the for loop has i=1, the students thought that the calculation of average on line 35 should take the value of index 1 of the array number, which is 1 and not 2.

Student 3 and Student 4 set a breakpoint on line 35 to execute the calculation on that line with Step Over. As soon as they saw that the value of average was 2, something had to be wrong with the indices. They noticed that the value of i in the for loop started with 1, meaning that the first index (that is 0) of the array number was dismissed and the calculation of average was therefore wrong. With the change of i=1 to i=0 in the for loop, the bug was fixed.

Student 1 and Student 2 could not fix the bug since they wrongly understood how indices in array works, meanwhile Student 3 and Student 4 fixed the bug.

5.5.2.3 Exercise Three - buttonPress.c The purpose of this exercise was to press the board button three times for the board LED to blink on and off three times. The correct code is shown in Section 5.4.3.5 whereas the code with the bug is shown in Section 5.4.3.6.

All the students started with code stepping to inspect the values of the variables when variable buttonPressed was equals to 3 (buttonPressed==3). Student 3 and Student 4 soon realized that they could use breakpoint to skip a few loop iterations. When having set a breakpoint, they changed their mind to conditional breakpoint would be more suitable since they wanted to examine a variable condition. Therefore, they set a breakpoint on line 37 with the condition “Conditional Expression Is True” when “buttonPressed==3”.

Student 1 used code stepping more compared to Student 3 and Student 4. But Student 1 also realized that conditional breakpoint could be used later on. Meanwhile Student 2 was stuck on using code stepping. As Student 1 and Student 2 worked in pair, Student 1 convinced Student 2 in using conditional breakpoint instead of code stepping.

Once the program was run with debugging, the conditional breakpoint paused the program when buttonPressed==3, where all the students decided to continue with code stepping. Using code stepping made the debugger jump to case One on line 54 which will blink the board LED, meaning the if statement was not met even though buttonPressed==3.

The if statement has three conditions that must be met. The students knew that buttonPressed==3 was a condition that could be met, so the next step was to examine the other two conditions. Student 4 first observed the defined and initialized variables on line 21-28. Only as a review, the student came to conclusion that the condition z==3 is also met, and started to examine the last condition.

All the students used the conditional breakpoint “Hit Point == 2” on line 37. When the breakpoint was hit, all the students observed every variable in the Autos window except Student 4. Student 4 only observed the variable ptr, since the student already came to conclusion that buttonPressed==3 and z==3 are met.

They all used Continue command to see if there are any difference in the values of the variables from when buttonPressed==2 is changed to buttonPressed==3. They realized that the variables buttonPressed and z are met, but pointer variable ptr had an odd value. It did not show a value at all, but a memory address.

Coming this far, the supervisors advised all the students to press and hold in the board button when they thought or knew the if statement was met and when they wanted to examine the body of the if statement.

Student 1 and Student 2 did not have knowledge in pointers which ended up with some guessing of what values to write instead of &x on line 38. The students kept using “Hit Count == “ with guessing values, and Student 1 at last, fixed the bug and realized that the address of x is not 1.

Student 3 and Student 4 changed from ptr==&x to ptr==*x having in mind that *x==1. Holding in the button while using Hit Point and Continue did not make the debugger go inside the body of the if statement. This meant that the if condition was not true. Since ptr==*x did not solve the problem, they wrote ptr==1 and used the same method as before. This was a success.

5.5.3 Results of the lab exercise A summary of what bugs the students found and remedied is shown in Table 5-4.

Student Exercise One Exercise Two Exercise Three

Student 1 bug + fix - bug + fix

Student 2 bug + fix - -

Student 3 bug + fix bug + fix bug + fix

Student 4 bug + fix bug + fix bug + fix Table 5-4 Bugs found and remedied

5.5.4 Evaluation With the lab guide and lab exercise successfully performed, the students answered a questionnaire in Part 3 of the lab assignment. The students’ notes of the questionnaire are summarized as follow: Q1: Debugging using a debugger is challenging Three students agreed to it being challenging and one student gave a neutral answer while explaining it depends on what debugging technique is being used.

Q2: The applied debugging techniques are easy to use Two students gave a neutral answer explaining that the guide made the debugging techniques understandable. But it was challenging to try using them for the exercises, where code stepping and breakpoint where the easier debugging techniques to use.

Two students disagreed explaining that code stepping was the easiest debugging technique meanwhile conditional breakpoint was the most difficult debugging technique.

Q3: The lab assignment is well-designed and easy to follow All students agreed and did not have any trouble following the laboratory guide. They thought it was good having instructions along with illustrated figures.

Q4: The bugs were easy to find and remedy One student agreed. One student was being neutral, and two students disagreed, in which these three students explained Exercise Two and Exercise Three were more challenging.

Q5: I understand the debugging techniques Two students agreed, understanding the debugging techniques are more convenient in certain scenarios. One student answered neutral since the debugging techniques were easy to understand from the guide. With more experience, it would have been more understandable.

One student disagreed, explaining breakpoint and conditional breakpoint was difficult to understand. It was difficult to know where to set a breakpoint.

Q6: Being a good programmer makes it easier to find bugs All students agreed that being a good programmer make it easier to debug, because fixing bugs requires the programmer to understand the code.

Q7: Time is the main factor for not using a debugger One student agreed and explained debugging techniques that require a debugger might be more difficult than using a print statement. Three students answered neutral, explaining that requires more time if the programmer is not experience in understanding code.

Q8: A lab assignment is a good way to be taught debugging All students agreed, expressing that a lab assignment is a session where learning by doing is a part of improving your knowledge and that it was convenient with a practical guide.

Q9: I understand why a debugger is used All students agreed and understood the convenience with a debugger through the lab guide and how it might be useful for larger programs.

Q10: I understand when the debugging techniques are suitable Three students agreed and one disagreed. Though different answers, they all thought it was easy to understand through the lab guide but difficult when practically the debugging techniques in the exercises.

The evaluation of the students’ answers to agree, neutral or disagree is also shown in Diagram 1.

Diagram 1 Questionnaire

For a more detailed evaluation of the students’ completed questionnaire with their additional notes, see Appendix A.

6 Discussion The choice of methodology to conduct this research and the results obtained in Chapter 5 are discussed in terms of performance.

6.1 Research methodology System Development Research Process proposed by Nunamaker et al. was the research methodology used. It is a methodology that is more appropriate for system development, hence the methodology is not entirely suitable for this research. The disadvantage of the methodology was the lack of steps in the research process to achieve the answers to the research questions in Section 1.3.

The lab assignment that was created was limited to Malmö University’s development environment. This meant that a case study had to be performed to obtain in-depth information about the components to select the most suitable one. Besides the case study, a questionnaire was also missing as a research process. Students had to perform the lab assignment in which their thoughts of the lab assignment are essential to answer RQ3. To obtain the validation of the lab assignment created, a questionnaire was necessary. Hence, these two methodologies were added as additional steps where it was needed within the research methodology.

Though a lack of steps, the research methodology was still used due to its iterative nature. Other than lack of steps, the research methodology was followed step by step with some iterations. The combination of the methodologies proposed by Nunamaker and a case study proved to be useful to answer RQ2, whereas the combination with a questionnaire was necessary to answer RQ3. RQ1 could be answered using Nunamaker et al.’s methodology.

6.2 Results of the lab assignment The most important results to be discussed are from the debugging guide and debugging exercise of the lab assignment. These results were compared to what other researchers have accomplished in their research, stated in Chapter 3. The most correlated results from Chapter 3 were extracted to be compared with the results obtained in this research.

6.2.1 Lemos et al. See Table 6-1 for the results from the related work affiliated to the results from the lab assignment.

Related work Lab assignment

1 It is possible to make developers improve the It was easier for the experienced programmers to code reliability if they have knowledge in use a debugger, resulting them to find and remedy software testing. all the bugs. Table 6-1 Lemos et al.

The results of the lab assignment are based on how well the students could find and remedy bugs using a debugger, meanwhile, Lemos et al. state that software testing can improve the reliability of code. The comparison of these results might be odd, but it certainly indicates that the more knowledge the developers have, the easier it is to obtain better code.

6.2.2 Böttcher et al. See Table 6-2 for the results from the related work affiliated to the results from the lab assignment.

Related work Lab assignment

1 Students found it difficult to follow the detailed Students found it easy to follow the laboratory demonstration of how to detect bugs. assignment with not much trouble,

2 Debugging is difficult to learn novice Experienced programmers (Student 3 and programmers because it requires skills in for Student 4) solved all the bugs, whereas the example abstract and logical thinking, and novice programmers (Student 1 and Student) had working systematically. more difficulties.

3 Some of students working in pairs included a Student 1 and Student 2 worked in pairs as first- strong partner and a less strong partner year students whereas Student 3 and Student 4 worked in pairs as second-year students

4 Debugging by visual diagnosis and randomly Debugging by guessing poking around Table 6-2 Böttcher et al.

Even though the students thought the debugging guide was easy to follow and to understand, they all still had trouble using the debugger in the practical debugging exercise. It was mostly the novice programmers who interacted with the supervisors during the lab session, which showed their interest in learning. Böttcher et al. state that it is difficult for novice programmers to learn to debug to which the results of Table 5-4 support the statement. But with a shown interest in learning, it is easier to learn regardless of being novice or experienced.

Often, the partners of a team think differently. It turned out that one student could have an idea of how to approach the bug meanwhile the other student did not have a clue yet. This was seen under the observation. There were scenarios in which the students used debugging by guessing when they did not understand the code, as in Böttcher et al.’ experiment. The benefits in working in pairs is that the students may gain better knowledge by being able to explain ambiguities. But it could also turn out that the stronger student may do all the work. For each student to understand, it might be more convenient for them not to work in pairs.

6.2.3 Arafa et al. See Table 6-3 for the results from the related work affiliated to the results from the lab assignment.

Related work Lab assignment

1 Results were observed, among others, via a Observing the students’ computer screen while they video recorded computer screen and saved used the method think-aloud to explain their thoughts. compiled systems.

2 Locating the bugs took more time than fixing Student 1 and Student 2 spent more time on locating the bugs. the bug than fixing the bug, and the opposite for Student 3 and Student 4. Table 6-3 Arafa et al.

As Arafa et al. mention in their research, it was difficult to provide accurate results to how the students were trying to debug the code when observing via video recorded computer screen. Therefore, assumptions of the students’ thought were necessary. At first, video recorded computer screen was supposed to be used for observation of the lab assignment, inspired by Arafa et al.’s research process. But regarding the assumptions that had to be made, it would be time consuming and thus the choice of thinking aloud was selected. This gave a more accurate results with no assumptions.

The experienced programmers could easily find the bugs in the lab assignment. But since they wanted to fix the bug using the debugger, they spent more time on fixing the bug. The software bugs in the lab assignment are low level compared to Arafa et al. who injected software bugs such as memory leaks, which are the types of bugs that are not suitable to the lab assignment, explained in Section 5.1.2. Another set of software bugs in the lab assignment such as uninitialized pointers, would most likely make it more difficult where even the experienced programmers would have trouble finding the bugs.

6.2.4 Duraes et al. See Table 6-4 for the results from the related work affiliated to the results from the lab assignment.

Related work Lab assignment

1 Projects with 50-100 lines of code to not make the Exercises where between 50-100 lines of code too easy or too difficult. code

2 15 test subjects 4 students

3 The programmers have one hour to find the bugs in The students have a total of 7 hours to each program perform the lab assignment. Table 6-4 Duraes et al.

Debugging exercises with the length of 50-100 lines of code were sufficient for learning debugging, but the bugs planted might have been too simple. If the second-year students did not find the bugs too difficult to find, the exercises do not seem to be suitable for a whole class with second-year students for example. It would not be much of a challenge to them. But since only four students performed the lab assignment, the 75 percentage of the bugs being solved are decent. With a greater number of students to perform the lab assignment would give a more trustworthy result to if the bugs were easy or difficult to find and fix.

The duration of the lab assignment maintained for about 5-6 hours, which is similar to a general lab in the Computer Science program. This meets the requirement R8 in Section 5.3.1. Duraes et al. have not clearly mentioned the results about bugs that are found and fixed, but instead have their focus on the results from the fMRI scanner.

6.3 Bugs injected From the observations of the exercises, stated in Section 5.5.2, Student 3 and Student 4 could find the bugs easily within just a few minutes. This indicates that the bugs injected might be too easy to solve for experienced programmers in general. An exercise with 50-100 lines of code that blinks a LED, counting the events of a button press or both combined, is not very complicated to understand. But the bugs injected were more challenging to the novice programmers, Student 1 and Student 2.

The types of bugs injected are well-known bugs that are similar to bugs that can be found in real programs. These are some of the bugs which do not give compilation warnings and were therefore utilized. If the same types of bugs were injected differently within another program, there is a possibility for it to be more challenging even for experienced programmers.

Software bug is a terminology that has several definitions and terms to some researchers, thus the process of information gathering was troublesome and confusing. There are many types of software bugs, but all of these bugs are not explained with adequate information to understand the type of bug. Sometimes, the bugs are not explained at all but only mentioned. Hence, there are possibilities of the dismissed software bugs to have been more suitable to be injected in the exercises.

6.4 Debugging techniques With this research, students were supposed to learn how to debug software using a debugger. According to the questionnaire in Section 5.5.4, it shows that all the students learnt what was required to meet the learning goals regarding the debugger and the debugging techniques to a certain degree. All the students understood how to use the debugging techniques within the guide, but it was more difficult for them to use in the exercise with no guidance.

The debugging techniques would maybe have been easier for them to use if the exercises were developed differently. For a bug that requires the use of a breakpoint, the program could bigger and nested loops for example. But to use another set of debugging technique may not be suitable for this research. As for example, program slicing that is mentioned in Section 5.1.3, is a debugging technique that does not require a debugger as SAM4E Xplained Pro but another tool.

Much time was spent on experimenting the functionalities of code stepping, breakpoint, and conditional breakpoint to design the debugging guide. The debugging techniques were not explained in detail when certain steps were more appropriate or why the debugging techniques are used. This is the main reason why Google was sometimes used as the search engine. Though much time was spent on experimenting, a lot of knowledge was gained, perhaps rather more than what would have been gained if the information was available.

6.5 Supervisor Both the lab guide and the lab exercise needed to be augmented with supervision.

6.5.1 Lab guide All the students thought that the lab guide was well-designed and easy to follow according to the questionnaire. Though none of them mentioned it as a note in the questionnaire, the students had some concerns in certain steps of the guide. The figures illustrated in the lab guide shows for example where to find a certain button on the toolbar, but sometimes the button was not placed where it was shown in the figure. It was also mentioned in the lab guide that the debugger may behave abnormally sometimes by skipping lines of code. This led the students to discuss with the supervisors if the debugger executed the line of code correctly.

6.5.2 Lab exercise Think-aloud method was used during the lab session. To hear the students’ thought made it easier for the supervisors to understand their approach to debugging, which in turn led to better discussions to hints. The supervision was needed more during the exercise than the guide.

7 Conclusion In this chapter, the answers to the research questions are provided. The future work of this thesis is also presented.

7.1 Answering the research questions These are the answers to the research question.

RQ1: What types of software bugs are suitable to debug? There are many types of bugs suitable to debug but the bugs injected in the exercises are the types of bugs dealing with mathematics, off-by-one and control flow. These bugs are not too advanced and are suitable for using a debugger. Regarding the requirement R8 in Section 5.3.1, not many bugs were injected since it was difficult to make an assumption of how fast the students were going to perform the lab assignment. But other types of bugs are suitable as well in which some of these are stated in Section 2.2, such as uninitialized pointers and incorrect expression or statements. The only requirement from choosing the bugs in Section 2.2 is to create bugs that do not give compilation warnings in Atmel Studio.

RQ2: Which debugging techniques can be used for debugging? SAM4E Xplained Pro was the debugger used in this research. Because students are supposed to learn how to debug software using a debugger, code stepping, breakpoint, and conditional breakpoint are the three debugging techniques utilized. This is because the debugging techniques are supported in the debugger and can be adapted to common types of bugs that may occur in the software.

A total of 75 percent of bugs were solved using the debugging techniques supported in the debugger. Considering the results, the use of the debugging techniques is considered to be a success and suitable for Malmö University.

RQ3: How can a lab assignment teach students to debug? To improve the students’ ability to debug software, a lab assignment was created consisting of a guide, an exercise and a questionnaire.

A well-designed guide that includes a theoretical part and a practical part is necessary to teach debugging. The students gained fundamental knowledge of debugging through the theoretical part whereas the practical part helped to ease the understanding of the theory. The practical part showed the convenience of using the debugging techniques in step by step instructions along with illustrated figures to follow.

The knowledge obtained from the guide was then utilized through performing exercises that contain software bugs. These exercises allowed the students to approach solving real bugs without guidance. Thus, the students’ knowledge of debugging was put to test. However, a descriptive guide does not fully complete a students’ understanding. Learning by doing is the concept to learn to debug. Therefore, it is important to have a supervisor attending the lab session available to discuss with, in case of ambiguities.

The results of the exercises could summarize if the students learned to debug. But with a questionnaire, it might help to obtain a better understanding of the students’ results from the exercises. According to the results from observation and the questionnaire, the students’ met the learning goals to some degree, if it would have been rated.

For the students to meet the learning goals, a lab assignment is proved to be able to teach students to debug.

7.2 Future work Since this thesis is aimed at Computer Science students at Malmö University, it would be more convenient to practice debugging on hardware that is used in their studies. A familiar hardware to the students is the microcontroller Arduino Due that does not have a built-in debugger like SAM4E Xplained Pro. A further development may thus be to attach the external SAM4E Xplained Pro to Arduino Due to debug software.

References

[1] M. Jiménez, R. Palomera, I Couvertier. Introduction to Embedded Systems: Using Microcontrollers and the MSP430. Page 1, 6-8. 2014.

[2] Shibu K.V. Introduction to EMBEDDED SYSTEMS. Page 2-4,73-74,85-86,630. 2009.

[3] Arnold S. Berger, Arnold H. Berger. Embedded Systems Design: An Introduction to Processes, Tools, and Techniques. Page xiii,xxi,xxiii,1-2. 2002.

[4] H. M. Iqbal, W. J. Lee. Data flow based integration testing for embedded system using interaction model. 2014. 21st Asia-Pacific Software Engineering Conference.

[5] B. Broekman, E. Notenboom. Testing Embedded Software. Page 14. 2003.

[6] F. Balarin, P. Giusto, A. Jurecska, M. Chiodo, H. Hsieh, C. Passerone, E. Sentovich, L. Lavagno, B. Tabbara, A. Sangiovanni-Vincentelli, K. Suzuki. HARDWARE-SOFTWARE CO-DESIGN OF EMBEDDED SYSTEMS. 2003.

[7] R. Murch. The Software Development Lifecycle - A Complete Guide. Page 13. 2012

[8] Why testing should start early in software development lifecycle?. 2018. [Online]. Available: http://www.softwaretestingclass.com/why-testing-should-start-early-in-software-development-life-cycle/ [Accessed: Aug. 21, 2018]

[9] P. P. Dey, B.R. Sinha, M. Amin, H. Badkoobehi, A. Datta. Error Flow Analysis of Abstract Models. 2011.

[10] A. Burnstein. Practical Software Testing: A Process-Oriented Approach. Page 7. 2003. [Online]. Available: https://svkarthikwinner.files.wordpress.com/2015/07/practical-software-testing.pdf [Accessed: Aug. 21, 2018]

[11] Gerald D. Everett, R. McLeod Jr, Software Testing: Testing Across the Entire Software Development Life Cycle. 2007.

[12] R. Chopra. Software Testing: A Self-Teaching Introduction. Chapter 5, page 1. 2018.

[13] A. Zeller. Why Programs Fail: A Guide to Systematic Debugging. Page xv. 2009.

[14] J. M. Zain, W. M. W. Mohd, E El-Qawasmeh. Software Engineering and Computer Systems. 2011.

[15] S. DESAI, A. SRIVASTAVA. SOFTWARE TESTING: A Practical Approach. 2016.

[16] Q. Li and C. Yao. Real-Time Concepts for Embedded Systems. 2017. Chapter 1, page 14.

[17] R. Kamal. Embedded Systems: Architecture, Programming and Design. Chapter 2, page 3. 2008.

[18] A. Munir, A. Gordon-Ross, S. Ranka. Modeling and Optimization of Parallel and Distributed Embedded Systems. 2016.

[19] AltexSoft. Quality Assurance, Quality Control and Testing — the Basics of Software Quality Management. Chapter 1, page 19. [Online]. Available: https://www.altexsoft.com/media/2016/10/Quality-Assurance-Quality- Control-and-Testing-%E2%80%94-the-Basics-of-Software-Quality-Management-Whitepaper.pdf [Accessed: Aug. 21, 2018]

[20] S. Shafi,S. Hassan, A. Arshaq, M. Khan and S. Shamai, Software Quality Prediction Techniques: A Comparative Analysis. 2008. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4777508 [Accessed: Aug. 21, 2018]

[21] P. Ammann, J. Offutt. Introduction to Software Testing. 2017. Introduction, page 11. [Online]. Available: http://www.cse.hcmut.edu.vn/~hiep/KiemthuPhanmem/Tailieuthamkhao/Introduction%20to%20Software%20Testing. pdf [Accessed: Aug. 21, 2018]

[22] What is Verification in software testing? or What is software verification?. [Online]. Available: http://istqbexamcertification.com/what-is-verification-in-software-testing-or-what-is-software-verification/ [Accessed: Aug. 21, 2018]

[23] D. Graham, E. V. Veenendaal, I. Evans. Foundations of Software Testing. 2008.

[24] J. Lepistö. Embedded Software Testing Methods. Helsinki Metropolia University of Applied Sciences Bachelor of Engineering. 2012.

[25] J. F. Nunamaker Jr, M. Chen, T. M. PURDIN. Systems Development in Information Systems Research.1990.

[26] Cambridge university press. Syntax and logic errors. Page 3. [Online]. Available: http://cambridgegcsecomputing.org/sites/94/upload/userfiles/217p_q_teachersnotes_editedct_3proof.pdf [Accessed: Aug. 21, 2018]

[27] Otávio A L, Fábio F S, Fabiano Cutigi Ferrari. The impact of Software Testing education on code reliability: An empirical assessment. 2017.

[28] A. Böttcher, V. Thurner, K. Schlierkamp, D. Zehetmeier. Debugging Students’ Debugging Process. 2016. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7757447 [Accessed: Aug. 21, 2018]

[29] P. Arafa, D. Solomon, S Navabpour, S. Fischmeister, Debugging Behaviour of Embedded-Software Developers: An Exploratory Study. 2017.

[30] J. Hanly, E. Koffman. Problem Solving and Program Design in C. 7th edition. 2012. [Online]. Available: https://doc.lagout.org/programmation/C/Problem%20Solving%20and%20Program%20Design%20in%20C%20%287t h%20ed.%29%20%5BHanly%20%26%20Koffman%202012-03-04%5D.pdf [Accessed: Aug. 21, 2018]

[31] Harsh L, Gaurav P. Root Cause Analysis of Software Bugs using Machine Learning Techniques. 2017. [Online]. Available: https://ieeexplore-ieee-org.proxy.mau.se/stamp/stamp.jsp?tp=&arnumber=7943132 [Accessed: Aug. 21, 2018]

[32] Richard Reese. Understanding and using C pointer. 2013. [Online]. Available: https://doc.lagout.org/programmation/C/Understanding%20and%20Using%20C%20Pointers%20%5BReese%202013 -05-18%5D.pdf [Accessed: Aug. 21, 2018]

[33] University of Amsterdam. THE THINK ALOUD METHOD. Page 1. 1994. [Online]. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.821.4127&rep=rep1&type=pdf [Accessed: March 07, 2018]

[34] A. M. LEITÃO. Detection of Redundant Code Using R2D2, Software Quality Journal, 12, 361–382, 2004. [Online]. Available: https://link.springer.com/content/pdf/10.1023%2FB%3ASQJO.0000039793.31052.72.pdf [Accessed: Aug. 21, 2018]

[35] H. Lal, G. Pahwa. Root Cause Analysis of Software Bugs using Machine Learning Techniques. 2017.

[36] S. Srivastva, S. Dhir, Debugging Approaches on various Software Processing Levels. 2017.

[37] B. Kernighan and D. Ritchie. C Programming Language, ANSI C Version. 1988. Chapter 5, page 83. [Online]. Available: http://117.3.71.125:8080/dspace/bitstream/DHKTDN/6554/1/The%20C%20Programming%20Language4603.pdf [Ac cessed: Aug. 21, 2018]

[38] Adragna P. Software debugging techniques. 2005. [Online]. Available: http://cds.cern.ch/record/1100526 [Accessed: Aug. 21, 2018]

[39] Debugging Techniques. 2005. [Online]. Available: https://static.lwn.net/images/pdf/LDD3/ch04.pdf [Accessed: Aug. 21, 2018]

[40] Atmel Studio. Debug Control. [Online]. Available: https://www.microchip.com/webdoc/GUID-ECD8A826-B1DA-44FC-BE0B-5A53418A47BD/index.html?GUID- 1BC19729-74D5-426D-9390-56CC86357F2F [Accessed: Aug. 21, 2018]

[41] Vipindeep V, Pankaj Jalote. List of Common Bugs and Programming Practices to avoid them. 2005. [Online]. Available: https://www.iiitd.edu.in/~jalote/papers/CommonBugs.pdf [Accessed: Aug. 21, 2018]

[42] V ipindeep V, P. Jalote, List of Common Bugs and Programming Practices to avoid them, 2005. [Online]. Available: https://www.iiitd.edu.in/~jalote/papers/CommonBugs.pdf [Accessed: Aug. 21, 2018]

[43] J.H.M. Dassen. Debugging C and C++ code in a Unix environment. 1999. [Online]. Available: http://www.socsci.ru.nl/idak/publications/debug/debug.pdf [Accessed: Aug. 21, 2018]

[44] M. Brzozowski and P. Langendoerfer. Overview and Benchmarks of Pragmatic Debugging Techniques for Wireless Sensor Networks. 2013. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6671892 [Accessed: Aug. 21, 2018]

[45] James H. Andrews. Testing using Log File Analysis: Tools, Methods, and Issues. 1998. [Online]. Available: https://pdfs.semanticscholar.org/9ec5/dfd10c091d008ab32a4b85d2f77cc74084c8.pdf [Accessed: Aug. 21, 2018]

[46] J. Duraes, H. Madeira, J. Castelhano, C. Duarte, M. Castelo Branco. WAP: Understanding the brain at software debugging. 2016. [Online]. Available: https://ieeexplore-ieee-org.proxy.mau.se/stamp/stamp.jsp?tp=&arnumber=7774510&tag=1 [Accessed: May.10, 2018]

[47] J. Duraes, H. Madeira Emulation of Software Faults: A Field Data Study and a Practical Approach. 2006. [Online]. Available: https://ieeexplore-ieee-org.proxy.mau.se/stamp/stamp.jsp?tp=&arnumber=4015509 [Accessed: Aug.12, 2018]

[48] P. Zhou, F. Qin, W. Liu, Y. Zhou, and J. Torrellas. iWatcher: Efficient Architectural Support for Software Debugging. 2004.

[49] N. Handigol, B. Heller, V. Jeyakumar, D. Maziéres and N. McKeown. Where is the Debugger for my Software- Defined Network?. 2017.

[50] N. Nitta. Experiences of Debugger-based Architecture Comprehension, 2017.

[51] T. Dupriez, G. Polito and S. Ducasse. Analysis and exploration for new generation debuggers. 2017.

[52] Fabian F. Morgan. Debugging step-back button. 2003. [Online]. Available: https://patents.google.com/patent/US7178136B2/en [Accessed: May.09, 2018]

[53] Microchip. Atmel SAM4E Xplained Pro. Embedded Debugger. [Online]. Available: https://www.microchip.com/webdoc/sam4explainedpro/sam4explainedpro.Xplained_Pro.EDBG.html [Accessed: May.01, 2018]

[54] J. ZhangA. Path-based Approach to the Detection of Infinite Looping. 2001. [Online]. Available: https://ieeexplore-ieee-org.proxy.mau.se/stamp/stamp.jsp?tp=&arnumber=990006 [Accessed: May.01, 2018]

[55]Tutorialspoint.com. C PROGRAMMING TUTORIAL. Page 57. [Online]. Available: https://phy.ntnu.edu.tw/~cchen/pdf/ctutor.pdf [Accessed: Aug.21, 2018]

[56] A. Corradini. Principles of Programming Languages, lesson 21, Department of Computer Science, Pisa. [Online]. Available: http://pages.di.unipi.it/corradini/Didattica/PLP-14/SLIDES/PLP-21.pdf [Accessed: Aug.21, 2018]

[57] Magnus Krampell. Föreläsning. F3-intro_System test_Reviews and Test reporting_MK. Page 17. 2017. [PowerPoint slides] .

[58] Welcome to Jim's Computer Science Topics Area. Debugging Programs. [Online]. Available: https://www.cs.utah.edu/~germain/PPS/Topics/debugging_programs.html [Accessed: Aug.21, 2018]

[59] C. Zhang,D. Yan,J. Zhao,Y. Chen,S. Yang. BPGen. An Automated Breakpoint Generator for Debug. 2010. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6062177 [Accessed: Aug.21, 2018]

[60] Z. Al-Sharif, C. Jeffery and M. Said. Debugging with Dynamic Temporal Assertions. 2014. [Online]. Available: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6983849 [Accessed: Aug.21, 2018]

[61] Microchip. Atmel SAM4E Xplained Pro. Xplained Pro. [Online]. Available: https://www.microchip.com/webdoc/sam4explainedpro/sam4explainedpro.Xplained_Pro.html [Accessed: Aug.21, 2018]

[62] Microchip. Atmel SAM4E Xplained Pro. User Guide. [Online]. Available: http://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-42216-SAM4E-Xplained-Pro_User-Guide.pdf [Accessed: Aug.21, 2018]

[63] InterPlanetary File System. Dry run (testing). 2017. [Online]. Available: https://ipfs.io/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Dry_run_(testing).html [Accesse d: Aug.21, 2018]

[64] What is dry run in C++, for an amateur?. 2016. [Online]. Available: https://www.quora.com/What-is-dry-run-in-C++-for-an-amateur [Accessed: Aug.21, 2018]

[65] Saurabh Shukla. #1 What Is Dry Run Part 1 of 2. 2017. [Online]. Available: https://www.youtube.com/watch?v=9JVnEcclp4Y [Accessed: Aug.21, 2018]

[66] Dry Run Testing & Trace Tables. 2015. [Online]. Available: http://www.101computing.net/trace-table/ [Accessed: Aug.21, 2018]

[67] S. Naz, S. Shirazi, T. Iqbal, D. Irfan, M. Junaid and Y. Naseer. Learning Programming through Multimedia and Dry- run. 2014. [Online]. Available: https://www.researchgate.net/profile/Saeeda_Naz/publication/262727252_Learning_Programming_through_Multimed ia_and_Dry-run/links/0deec53c1b8fb9c56d000000/Learning-Programming-through-Multimedia-and-Dry- run.pdf [Accessed: Aug.21, 2018]

[68] H. Lal and G. Pahwa. Root Cause Analysis of Software Bugs using Machine Learning Techniques. 2017. [Online]. Available: https://ieeexplore-ieee-org.proxy.mau.se/stamp/stamp.jsp?tp=&arnumber=7943132&tag=1 [Accessed: Aug.21, 2018]

[69] D. Sidler and K. Eguro. Debugging Framework for FPGA-based Soft Processors. 2016. [Online]. Available: https://ieeexplore-ieee-org.proxy.mau.se/stamp/stamp.jsp?tp=&arnumber=7929524&tag=1 [Accessed: Aug.21, 2018]

[70] F. Wotawa. On the relationship between model-based debugging and program slicing. 2001. [Online]. Available: https://ac-els-cdn-com.proxy.mau.se/S0004370201001618/1-s2.0-S0004370201001618-main.pdf?_tid=f539d58e- 53c0-4cd5-930a-47514d45a7b7&acdnat=1537085372_6bac2afe04d5ab1df655a139374cd955 [Accessed: Aug.21, 2018]

[71] S. Narayanasamy, G. Pokam and B. Calder BugNet: Continuously Recording Program Execution for Deterministic Replay Debugging. 2005 [Online]. Available: https://cseweb.ucsd.edu/~c [Accessed: Aug.21, 2018]

[72] Arduino. Visual Micro: Develop Arduino programs with Microsoft Visual Studio or Atmel Studio. 2018. [Online]. Available: https://playground.arduino.cc/Code/VisualMicro [Accessed: Aug.21, 2018]

[73] Atmel. The Atmel AVR Dragon Debugger. 2018. [Online]. Available: https://www.microchip.com/webdoc/GUID- 04E3421E-81C3-4E6C-BD29-9E1A7BAEBA7E/index.html

[74] Microchip. Programmers and Debuggers, power debugger user guide. Page 4, 10. 2016. [Online]. Available: http://ww1.microchip.com/downloads/en/DeviceDoc/Power-Debugger_UserGuide.pdf [Accessed: Aug.21, 2018]

A Questionnaire

Student 1

Statements Agree Neutral Disagree Notes

Q1 Debugging using a ☒ ☐ ☐ debugger is challenging

Q2 The applied debugging ☐ ☐ ☒ Code stepping was the easiest, but first I had techniques are easy to use trouble navigating with the commands

Q3 The lab assignment is well- ☒ ☐ ☐ designed and easy to follow

Q4 The bugs were easy to find ☐ ☐ ☒ I had trouble with Exercise Two and Three. and remedy

Q5 I understand the debugging ☐ ☐ ☒ I understand code stepping, but the techniques (conditional) breakpoint was difficult. I didn’t know where to set the breakpoint.

Q6 Being a good programmer ☒ ☐ ☐ makes it easier to find bugs

Q7 Time is the main factor for ☐ ☒ ☐ Maybe, I guess. For me, it just takes time since not using a debugger I’m not used to programming a lot of code yet.

Q8 A lab assignment is a good ☒ ☐ ☐ way to be taught debugging

Q9 I understand why a ☒ ☐ ☐ The theoretical part and the guide made me debugger is used understand the use of a debugger, though for a bigger system.

Q10 I understand when the ☒ ☐ ☐ I understand when following the guide, but it’s debugging techniques are more difficult when you have to test by yourself. suitable for real.

Student 2

Statements Agree Neutral Disagree Notes

Q1 Debugging using a ☒ ☐ ☐ debugger is challenging

Q2 The applied ☐ ☒ ☐ The guide made it easy to understand the debugging techniques techniques and the use of it. But in the exercises, are easy to use conditional breakpoint was the most difficult to use. It has too many conditions to choose between.

Q3 The lab assignment is ☒ ☐ ☐ Easy to follow with both text and figures. well-designed and easy to follow

Q4 The bugs were easy ☐ ☐ ☒ to find and remedy

Q5 I understand the ☐ ☒ ☐ I understand the techniques and when they are debugging techniques convenient to use, gained from the guide. With more and they are time adjusting to the using the techniques, it would convenient have become easier to use in the exercise.

Q6 Being a good ☒ ☐ ☐ Indeed! As a beginner, I’m not very familiar with programmer makes it code which made it more difficult for me to get an easier to find bugs idea about what technique to use.

Q7 Time is the main ☐ ☒ ☐ I think it’s important to understand the code first, factor for not using a which will make it easier to use a debugger. If you debugger don’t understand (most of) the code, you don’t know where to start and that will be time consuming.

Q8 A lab assignment is a ☒ ☐ ☐ Learning by doing! good way to be taught debugging

Q9 I understand why a ☒ ☐ ☐ But I think it’s more convenient for those with more debugger is used experience of coding.

Q10 I understand when the ☐ ☐ ☒ It’s difficult in the exercises, but understandable in debugging techniques the guide. are suitable

Student 3

Statements Agree Neutral Disagree Notes

Q1 Debugging using a ☐ ☒ ☐ It depends on what debugging technique is used. debugger is challenging Most difficult was to keep all conditions in mind when using conditional breakpoint.

Q2 The applied debugging ☐ ☒ ☐ Code stepping and breakpoint are debugging techniques are easy to techniques that I’ve heard of before. I haven’t use learnt how to use them properly, until this guide showing good examples.

Q3 The lab assignment is ☒ ☐ ☐ Easy text with great figures. well-designed and easy to follow

Q4 The bugs were easy to ☐ ☒ ☐ Last exercise was more difficult, but the first two find and remedy exercises were easy.

Q5 I understand the ☒ ☐ ☐ Very convenient especially in certain scenarios. debugging techniques and they are convenient

Q6 Being a good ☒ ☐ ☐ Easier to debug for someone who understands programmer makes it the code compared to someone who don’t easier to find bugs

Q7 Time is the main factor ☐ ☒ ☐ Better to use it for larger programs. If it’s a smaller for not using a debugger program, a debugger might not be suitable.

Q8 A lab assignment is a ☒ ☐ ☐ good way to be taught debugging

Q9 I understand why a ☒ ☐ ☐ debugger is used

Q10 I understand when the ☒ ☐ ☐ Easy to understand through the guide. debugging techniques are suitable

Student 4

Statements Agree Neutral Disagree Notes

Q1 Debugging using a ☒ ☐ ☐ debugger is challenging

Q2 The applied debugging ☐ ☐ ☒ Conditional breakpoint was the difficult one. techniques are easy to use

Q3 The lab assignment is ☒ ☐ ☐ Good to have many illustrated figures. well-designed and easy to follow

Q4 The bugs were easy to ☒ ☐ ☐ It wasn’t too challenging to find the bugs. find and remedy

Q5 I understand the ☒ ☐ ☐ debugging techniques and they are convenient

Q6 Being a good ☒ ☐ ☐ programmer makes it easier to find bugs

Q7 Time is the main factor for ☒ ☐ ☐ I think people think it’s more convenient to use not using a debugger e.g. print statements. Conditional breakpoints are probably more difficult to use than a print statement.

Q8 A lab assignment is a ☒ ☐ ☐ Yes, it’s pedagogical. And you are learning by good way to be taught doing. debugging

Q9 I understand why a ☒ ☐ ☐ debugger is used

Q10 I understand when the ☒ ☐ ☐ Easy to understand the theoretical but more debugging techniques are difficult when to use the practical suitable

B Solutions to exercise

Exercise One Students performing this exercise could solve this issue through using the SAM4E Xplained Pro On-board debugging. This could be done using a debugging technique called code stepping.

First students may first build the program and then run it without debugging. See Figure B-1 illustrating how to build the program.

Figure B-1 Build solution

A correct state of the program shall blink the board LED three times. If it is not the case, the student will use Start Debugging and Break to debug the program code. See Figure B-2.

Figure B-2 Start debugging the program

The debugging session starts on the first executable instruction in main() in line 12. See Figure B-3.

Figure B-3 Debugging start at main

The student may use step Over command and continue using it till they reach the on_off_LED() function in line 18. See Figure B-4.

Figure B-4 Debugging start at main

The debugging position (next instruction to be executed) can be seen in the Call Stack window, shown in Figure B-5. The window displays the variable name accompanied with information as the line number of the instruction to be executed. It is recommended to also observe the Call Stack window to keep track of each step you take with the control commands. See Figure B-5.

Figure B-5 Debugging position in Call Stack window and variable value in Autos window

The students may use the Step into command to go inside the on_off_LED() function and examine it in detail. They can see the value of the array number and its type in the Autos window. See Figure B-6. Stepping over line 23 using the step over command will execute line 27. The value of all variables and result of the total size of the array will be shown in the Autos window. See Figure B-6.

Figure B-6 Autos window shows value of variable

The total array size is size3= size1 / size2 → 20/4 = 5. Stepping over the code, the For loop will be executed 5 times according to the For loop condition ( size3 = 5 ) and the variable values in the first iteration will be shown in Autos window. See Figure B-7.

Figure B-7 Variables value in the for loop iteration

The Average value in the first iteration is (average+=number[i] +1) = 0 + 1 +1 = 2. See Figure B-8.

Figure B-8 For loop first iteration

In the second iteration the value of average is 2 + 2 +1 = 5. See Figure B-9.

Figure B-9 For loop second iteration

In the third iteration the value of average is = 5 + 3 +1 = 9. See Figure B-10.

Figure B-10 For loop third iteration

In the fourth iteration the value of average is 9 + 4 +1 =14. See Figure B-11.

Figure B-11 For loop fourth iteration

In the fifth iteration the value of average is 14 + 5 +1 = 20. See Figure B-12.

Figure B-12 For loop fifth iteration

The array average value is average = average / size3 → 20/5 = 4 which is not correct. The for loop after the switch case line 38 will not execute and the board led lamp will not blink because the correct average result is 3. See Figure B-13 and B-14.

Figure B-13 The average value is 4

Figure B-14 The Led lamp will not turn ON/OFF and the program will go to break

Because the average array calculation is incorrect. We think the students will notice that the problem with the code is that the statement in line 33 is incorrect and the that (+ 1) is added to the statement.

In order to solve this issue students can decrement the average value in every iteration by one. Students can do that by double clicking the average value 2 and write 1. See Figure B-15.

Figure B-15 Decrement the average value by 1

Stepping over the code the value of average will be 1 in the first iteration. See Figure B-16.

Figure B-16 The average value is 1

In the second iteration the average value will be 4 and students will change to 3 and step over the code. See Figure B-17 and B-18.

Figure B-17 The average value is 4

Figure B-18 The average value is 3

In the third iteration the average value will be 7 and students will change to 6 and step over the code. See Figure B-19 and B-20.

Figure B-19 The average value is 7

Figure B-20 The average value is 6

In the fourth iteration the average value will be 11 and students will change to 10 and step over the code. See Figure B-21 and B-22.

Figure B-21 The average value is 11

Figure B-22 The average value is 10

In the fifth iteration the average value will be 16 and students will change to 15 and step over the code. See Figure B-23 and B-24.

Figure B-23 The average value is 16

Figure B-24 The average value is 15

This is the end of the iteration. Now the array average value is 3. See Figure B-25.

Figure B-25 The array average value is 3

By stepping over the code to the switch case, the students will see that the array average value does matches the switch case condition so the board led lamp will blink. See Figure B-26.

Figure B-26 Led lamp ON

Exercise Two Students have fixed the bug in section 5.1 using the stepping debugging technique and because the off- by-one error in implemented in the same program we think that the students would not use only the stepping technique they should combine it with the breakpoint technique. From the previous exercise, students know that the problem is located around the for loop iteration.

To fix the bug in the program the students will set a breakpoint in line 33 in the code. See figure B-27 and Figure B-28

Figure B-27 Inserting breakpoint in line 33

Figure B-28 Breakpoint inserted - red point

Stepping over the code to the for loop in line 33, the initial value of (i). See the Autos window Figure B-29.

Figure B-29 The initial value of (i)

In the first for loop iteration the value of average is (average=average + number[i]) = 0 + 2 = 2. See Figure B-30. The first element of the array element (1) will not be calculated, because the initial value of i = 1 in the for loop. Hopefully the students will notice that and change the initial value of i to zero.

Figure B-30 For loop first iteration

Students can change the initial value of i in red color and confirming by pressing. See Figure B-31. Notice this will not change the actual code when the debugging is finished, it is a way to try out and see if it works.

Figure B-31 Changing the initial value of i in red

Stepping over the code will show the initial value of i which is zero. See Figure B-32.

Figure B-32 Now the initial value of i = 0

So now in the first for loop iteration the value of average is (average=average + number[i]) = 0 + 1 = 1. See Figure B-33.

Figure B-33 For loop first iteration

In the second for loop iteration the value of average is (average=average + number[i]) = 1 + 2 = 3. See Figure B-34.

Figure B-34 For loop second iteration

In the third for loop iteration the value of average is ( average=average + number[i]) = 3 + 3 = 6. See Figure B-35.

Figure B-35 For loop third iteration

In the fourth for loop iteration the value of average is (average=average + number[i]) = 6 + 4 = 10. See Figure B-36.

Figure B-36 For loop fourth iteration

In the fifth for loop iteration the value of average is (average=average + number[i]) = 10 + 5 = 15. See Figure B-37. This is the final average value which is 15, because the for loop condition is (i

Figure B-37 For loop fifth iteration

The value of average is going to be (average=average + number[i]) = 15 / 5 = 3 which is match the value in the switch case. See Figure B-38.

Figure B-38 The array average value is 3

The debugging session will stop now because of the breakpoint. The students have fixed the bug and now they have managed to learn:

First deleting the breakpoints. See Figure B-39.

Figure B-39 Deleting breakpoint

Second changing the actual code replacing the initial value of (i) in the for loop from (1) to (0). See Figure B-40.

Figure B-40 Change the i value from (1) to (0)

And finally running the program again to make sure the program works as it should.

Exercise Three After using two kind of debugging techniques, students should now be confident in using the on-board SAM4E Xplained Pro debugger. We think that students will choose to use the stepping debugging to see every code execution combine with the conditional debugging technique to check why the board led lamp will not blink. Students will start by setting a breakpoint at line 34 with condition/action and press close. If the if condition is true, this message “The if condition is true! The led lamp will blink :)“ will be shown in the output window. See Figure B-41.

Figure B-41 setting breakpoint at line 34 with condition and action

The students will start the debugging process. Because we start in state One and the button is not pressed in which the board Led lamp will be OFF. See Figure B-42.

Figure B-42 State One: button is not pressed

Stepping over the code using the step over command, we will be taken to state Two in which the button is active (pressed). See Figure B-43.

Figure B-43 State Two: button is pressed

Now students should press the board button once, hold and then step over the code. The number of time the button pressed will be shown in the Autos window. See Figure B-44.

Figure B-44 Button is pressed once

As you can see in Figure B-45, nothing is written in the Output window which means that the if condition is false. This also means that there is something wrong with the if condition!

Figure B-45 Nothing shown in the output window

Students will start looking carefully at every statements/variables in the if condition! (z==3) is correct because x=1, y=2 and z=x+y= 3. But the pointer (ptr) is pointing to the memory address of x (&x) which is not equal to x=1. If the students change the ( *ptr==&x ) to (*ptr==1) and use debugging session again, the program will execute as it should! Here in figure B-46, you see the message in the output window confirming that the if condition is true! The Led lamp will blink three times!

Figure B-46 Output message-If condition is true

C Lab assignment

Lab assignment

Practice how to debug embedded systems using a debugger

1 Introduction In software development, bugs may occur as a result of errors. There are scenarios where bugs are easy to find and remedy, while it may be significantly more difficult in other scenarios. Scenarios where a system is larger and complex may need the help of a debugger to find and remedy bugs.

1.1 Purpose The purpose of this lab assignment is to help students gain knowledge in the field of debugging embedded systems in the modern development environment Atmel Studio (Integrated Development Platform). The students will use the hardware platform Atmel SAM4E Xplained Pro board, which contains an on-board debugger, to debug a project code in Atmel Studio, implemented in advance.

1.2 Description of the lab assignment In this lab assignment, you are via a debugging guide introduced to a debugger. There are several debugging techniques used for debugging but only three of them are used here: code stepping, breakpoint and conditional breakpoint. Each debugging technique will give you an insight into when these approaches may become convenient. This lab assignment does not cover each and every scenario to when it is suitable to use these debugging techniques when you debug your code.

After being introduced to the debugger and its debugging techniques, you will perform a debugging exercise where you have to use your obtained knowledge in debugging to find and remedy bugs. In the end, there is a questionnaire to be answered about your thoughts of this lab assignment.

With all these steps, results in the lab assignment to be divided into three parts, followed by sections:

Part 1: Debugging guide 1. Lab setup 2. Debugging with code stepping 3. Debugging with breakpoint 4. Debugging with conditional breakpoint 5. Debugging with action breakpoint

Part 2: Debugging exercises 1. Exercise One 2. Exercise Two 3. Exercise Three

Part 3: Questionnaire 1. Evaluation

1.3 Requirements to pass this lab assignment To pass this lab assignment, you have to complete every part as well as possible and meet the learning goals.

L1 The students shall gain knowledge about debugging techniques

L2 The students shall gain knowledge about the debugger

L3 The students shall be able to debug code using a debugger

L4 The students shall gain knowledge about the benefits of using a certain debugging technique in a certain scenario

1.4 Appropriate preparation • This lab is based on debugging software. Read Appendix A to get an insight about bugs, debugging, debugger and debugging techniques • Read through this entire document carefully

1.5 Tools to be used The hardware and software tools required are as follow:

1.5.1 Hardware • Atmel SAM4E Xplained Pro board • USB cable (Standard-A to micro-B or micro-AB)

1.5.2 Software • Atmel Studio

PART 1: DEBUGGING GUIDE

2 Section 1: Lab setup First, you have to set up the lab environment by connecting Atmel SAM4E Xplained Pro with Atmel Studio. To confirm the connection, you are going to run a test program.

2.1 Download Sam4e16eGuide project Throughout this lab assignment, you are going to work with Sam4e16eGuide project, available at: https://drive.google.com/drive/folders/1W0YSm6dTRauaXdtst4hh1gzXpkAMZbBH. Download and save the project to your desired location in the computer.

Also, a small note to keep in mind: this project does not contain bugs planted in advance. Thus, each program execution should be working as intended. The focus of this debugging guide is to show you certain scenarios where a debugger might be useful. Showing you the mechanism of debugging techniques with the debugger, will hopefully give you the mindset to how you can debug your code for the upcoming debugging exercise.

2.2 Detect Atmel SAM4E Xplained Pro within Atmel Studio Start Atmel Studio. It will automatically open the Start Page, see Figure 2-1.

Figure 2-1 Atmel Studio: Start Page

Connect a USB cable between your PC and the DEBUG USB port on the board. A green power LED is lit on the board once the connection is complete, see Figure 2-2.

Figure 2-2 Pro board connected to PC

With the connection complete, Atmel Studio will start detecting the board. When the board is detected, SAM4E Xplained Pro - 5738 page automatically opens, see Figure 2-3.

Figure 2-3 Pro board detected in Atmel Studio

2.3 Launch Sam4e16eGuide project You will now work with Sam4e16eGuide project. To find the saved project, go to File → Open → Project/Solution, see Figure 2-4.

Figure 2-4 Open project

Find the location where you saved the project in step 3.1, choose the project and left click Open to open the saved project, see Figure 2-5.

Figure 2-5 Open saved project

When the project is successfully opened, Atmel Studio should show the connected board, see Figure 2-6. You may also close SAM4E Xplained Pro - 5738 page because it is not needed.

Figure 2-6 Pro board connected to Sam4e16eGuide project

2.4 Run without debugging The project contains C files (.c) and include files (.h) visible in the Solution Explorer window, in which you are going to work with blinkLED.c and main.c, see Figure 2-7.

Figure 2-7 Project files

Though you will work with both blinkLED.c and main.c, focus will be on blinkLED.c which contains three functions. Each function is blinking a LED, with different implemented conditions. If you left double click on blinkLED.c, it will open the C file in a new window. See Figure 2-8 to see first function, stepping_technique().

Figure 2-8 stepping_technique()

The function has a very simple code of turning board LED on and off continuously. This section and in the next section where this function will be used, does not require more complex code. When you have seen and understood the code you will be working with, it is time to run the code.

Left double click on main.c to open the C file. You can see three function calls commented out on line 15- 17, see Figure 2-9. These are the three functions in blinkLED.c, in which you have already seen stepping_technique().

Figure 2-9 Function calls

You want to execute stepping_technique(), thus you have to first uncomment line 15, see Figure 2-10.

Figure 2-10 Uncomment line 15

In this step you are going to run the code only to see that it works as it should on the board. Will the board LED turn on and off every second, as the code shown in Figure 2-8 says? You will not immerse each feature of the code in details, which can be done with debugging. Therefore, you will run the code without debugging. To run the code, go to Debug → Start Without Debugging, see Figure 2-11.

Figure 2-11 Run without debugging

When you run the code, you will see the project starts building in Output window, see Figure 2-12. Usually, you would have to first build the project (Build → Build Solution) to compile the code files. After building the

79 project, you are able to run the project. But in this case, you can start with running the project immediately due Build Solution executes automatically within Start Without Debugging function.

Figure 2-12 Build Solution

When the project is successfully built, the code that is previously loaded onto the board is being erased, see Figure 2-13. The board will be erased every time Build Solution has run successfully.

Figure 2-13 Erase previous code on Pro board

When the board has been erased completely, the new code that was compiled with Build Solution starts loading onto the board, see Figure 2-14.

Figure 2-14 Load code onto Pro board

When the new code has successfully loaded onto the board, it should say “Ready”, see Figure 2-15. With all these successful executions within Start Without Debugging function, the code should now be running on the board.

Figure 2-15 Load code onto Pro board succeeded

If everything works properly, the board LED should be turning on and off every second, see Figure 2-16 and 2-17.

Figure 2-16 Board LED on Figure 2-17 Board LED off

3 Section 2: Debugging with code stepping Up until now you have run a project without debugging only to see that the code works on the board. In this section you will be introduced to the on-board debugger EDBG (Atmel Embedded Debugger) of Atmel SAM4E Xplained Pro. With EDBG you can debug a project with several debugging techniques, in which you will learn code stepping here. Code stepping is a debugging technique used to examine how your program is executed by stepping through your code line by line.

3.1 Run with debugging You are going to examine the same code that you executed in Section 1, that is stepping_technique(). Hence, there is no need to make changes in the code. Instead of running the project without debugging as before, you are in this step going to debug the code. To run the project with debugging, go to Debug → Start Debugging and Break, See Figure 3-1.

Figure 3-1 Run project with debugging

The first steps of Start Without Debugging function used in Section 1, is also executed within Start Debugging and Break function. The project is building, followed by earlier code on the board being erased and new code loads onto the board. Does board LED turn on and off as it automatically did in Section 1?

Instead of running the code loaded onto the board, the project starts to run in debugging mode, see Figure 3-2. This will not execute the code, which is why the board LED does not turn on and off.

Figure 3-2 Debugging process starts

You have to wait until “Running” has changed to “Stopped”, see Figure 3-3. This indicates that debugging mode is ready to run, which makes it possible to debug the code.

Figure 3-3 Debugging is ready

Being able to debug the code means that you should be able to see the Atmel SAM4E Xplained Pro on- board debugger (EDBG) connected to the project, see Figure 3-4.

Figure 3-4 EDBG connected

3.2 Code stepping commands With EDBG connected, it is possible to use the three control commands of code stepping: • Step Into - check a certain instruction (line of code) in details • Step Over - executes instruction without examining it in detail, and steps to the next instruction to be called • Step Out – executes remaining instructions in the current function and return to the next instruction from where the function was called

These control commands are located as buttons in the toolbar, see Figure 3-5.

Figure 3-5 Code stepping commands from toolbar

The control commands can also be found in Debug from the menu bar, see Figure 3-6.

Figure 3-6 Code stepping commands from menu bar

Coming this far, it is time to use these control commands in the next step.

3.3 Use code stepping From here on, the first debugging procedure starts. You will use the debugging technique code stepping to examine each instruction, to when and where the board LED turns on and off. When running a project, whether it is with or without debugging, execution always begin on the first executable instruction in main(). Hence, you should see a yellow arrow in main(), see Figure 3-7. The yellow arrow indicates the position of the instruction to be executed.

Figure 3-7 Instructions to be executed

The mechanism of code stepping is to be able to examine each executable instruction when using the

84 control commands. In figure 3-7, you can see that the debugging procedure starts on line 10. But the curly bracket is not an executable instruction. For the debugger to act right, the debugging procedure should start on line 11, which has an executable instruction.

NOTE: each time you run the program with debugging, the debugging procedure will start on line 10 in main(). Keep in mind that this is not the right procedure, and that the debugger should not skip a line of code. There seems to be an abnormal behavior in the debugger.

You may also view your debugging position (next instruction to be executed) in the Call Stack window, shown in Figure 3-8. The window displays the name of the function accompanied with information as the line number of the instruction to be executed.

Figure 3-8 Debugging position in Call Stack window

Use Step Over command once. This executes line 10, even though it is not an executable instruction, and goes to the next instruction, see Figure 3-9.

Figure 3-9 Position after Step Over command

It is a good idea to also observe the Call Stack window to keep track of each step you take with the control commands, see Figure 3-10. You are in the same function, but the line number changed from 10 to 11 after using Step Over. Compare Figure 3-8 with Figure 3-10 to see the changes.

Figure 3-10 New position

Use Step Over until you reach line 13, see Figure 3-11. It may seem like the debugger only jumps to the next instruction each time you use Step Over, but both functions board_init() and sysclk_init() are executed, just not examined in detail.

Figure 3-11 Position on line 13

So how can you examine an instruction, a function as board_init() or sysclk_init(), in detail, you might wonder? When you have reached line 13, try using the Step Into command for the first time. This will take you to the first executable instruction inside the function delay_init(), see Figure 3-12.

Figure 3-12 Current position in delay_init()

The reason why the debugger goes inside the function is because of Step Into command, which is used when you want to examine an instruction in detail.

Do not forget to look at the Call Stack window to see your current position on line 12 in delay_init(), see Figure 3-13. Call Stack does not only remember your current position, but also remember your position before you decided to examine an instruction in detail using Step Into. That is, your position in main(), see Figure 3-11 if you must.

Figure 3-13 Positions in delay_init() and main()

You can now examine delay_init() to see what it does. You may use Step Over to execute an instruction without examining it in detail, or use Step Into again, to further examine another instruction. But for now, let us say that you do not want to further examine delay_init() but instead wants to exit the function. To do so, Step Out command is useful.

With Step Out, the debugger executes all remaining instructions, that is line 12 to line 16, see Figure 3-12. These instructions are executed line by line without being examined in detail, which may remind you of Step Over. But with Step Over, only one instruction at a time is executed, so you would have to use Step Over several times before you reach the last instruction in delay_init().

Use Step Out and see what happens. It leaves you the trouble of using Step Over several times and instead executes all remaining instructions sequentially in one click. When each instruction has been executed, the debugger exits delay_init() and returns to the next instruction after delay_init(), see Figure 3-14.

Figure 3-14 Positions in main() after Step Out

The reason why the debugger skips line 14 and goes to line 15 is because of its empty line that has no effect when executing a program. It can be seen as a non-executable instruction.

What you really want to examine in detail in this section, is stepping_technique() where the code of blinking board LED is. While on line 15, what do you do? Use Step Into. This takes you inside the function, in which the debugging procedure continues on line 20 because the while condition is true, see Figure 3-15.

You have had both main.c and blinkLED.c opened in different tabs. When the debugger jumped into stepping_technique(), the tab was switched from main.c to blinkLED.c due to stepping_technique() is a function from blinkLED.c.

Figure 3-15 Position in stepping_technique()

Use Step Over to execute the instruction on line 20. Do you see that the board LED is now lit? When the execution is done, the debugger jumps to the next instruction to be executed, line 21. On the next instruction, you may try using Step Into to further examine delay() and use Step Out to exit the function. If you are not further examining delay(), use Step Over.

When you use Step Over, line 21 is executed and the board LED is still on, and the debugger jumps to next instruction. A second use of Step Over will execute the instruction to turn off the board LED, and goes to next instruction on line 23, see Figure 3-16.

Figure 3-16 Position after Step Over commands

Your current position is on the last executable instruction inside the while loop. What happens with the instructions when you have an infinite loop? The instruction keeps executing over and over again. Use Step Over to see how the debugger will execute the last instruction inside the loop and then jumps to the beginning of the loop, see Figure 3-17. It takes you to line 21.

Figure 3-17 Iterate the instructions

Remember in step 4.3 where the debugger seemed to have an abnormal behavior? The abnormal behavior is also shown as you step through an infinite loop. Using Step Over when on line 23, takes you to the next instruction on line 21. This is not correct. The debugger should jump to the first instruction on line 20, as it did in Figure 3-15.

NOTE: the very first time you enter a loop, the debugger works as it should, as in Figure 3-15. As soon as you are about to repeat the execution of the loop, the debugger starts showing an abnormal behavior. It skips the first instruction and goes directly to the second instruction, as in Figure 3-17. Overall, the debugging procedure still works.

3.4 Stop debugging Keep using the Step Over command to examine the instructions inside the while loop. The board LED should be turning on and off. Do not forget to try Step Into and Step Out. You should by now have got the hang of how code stepping works. If so, you are ready to stop the debugging process.

The control command is located as a button in the toolbar, see Figure 3-18.

Figure 3-18 Stop Debugging from toolbar

The control command can also be found in Debug from the menu bar, see Figure 3-19.

Figure 3-19 Stop Debugging

The session with the debugging technique code stepping is finished.

4 Section 3: Debugging with breakpoint Next debugging technique is setting breakpoints. It is a useful technique when you want to stop (the term pause is similarly used) the execution of a program. Where you want the execution to stop, is where you set a breakpoint. The program will run continuously until it hits the line where you set a breakpoint. When the execution stops at the breakpoint, you may use code stepping commands to further inspect the code.

4.1 Code stepping is not suitable You will work with breakpoint_technique(), meaning you have to do changes in main.c to execute the right function. Open main.c window to comment out line 15 and uncomment line 16, see Figure 4-1.

Figure 4-1 Uncomment line 16

To get a better understanding of breakpoints, you should run the program using code stepping first. To use code stepping, you had to run the program with Start Debugging and Break, shown in Figure 3-1. If you do not remember the process, return to step 3.1 and follow the steps. As per usual, if everything was done right, the debugging procedure should start on line 10 in main(), see Figure 4-2.

Figure 4-2 Start position of debugging

Use Step Over to execute each instruction without examining it in detail, until you reach line 13. If you use Step Over once again, the debugger executes the instruction and then jumps to line 16, see Figure 4-3. The reason why the debugger skips the empty line, has already been explained. But why does the debugger skip line 15? It is for the same reason as for the empty line, lines that are commented out has no effect when executing a program.

Figure 4-3 Debugger skips line 14 and line 15

You want to examine breakpoint_technique() in detail. Use Step Into, which takes you to line 31 inside the function, see Figure 4-4. Line 30 is an executable instruction, but you have to remember the abnormal behavior of the debugger skipping lines. The real starting point should be on line 30.

Figure 4-4 Debugging procedure starts on the second executable instruction

Nonetheless, line 30 still executes even though the debugger skipped the line. If you examine the Locals window, you can see that line 30 was executed. The variable number has been assigned the value of 0, which is what the instruction on line 30 is supposed to execute, see Figure 4-5.

Figure 4-5 Variable assigned a value

The function breakpoint_technique() will turn board LED on and off forever. But before reaching the instructions of turning board LED on and off, there is a variable (number) that has to reach a certain value in an if statement.

Your new task will be to reach the infinite while loop on line 37 with code stepping. To reach the while loop, the if statement on line 35 must be met. Use Step Over to execute each instruction line by line. You will soon realize that you must use Step Over many times before the value of number reaches 55. Try it out. The debugger will jump between the instructions on line 31, 33 and 35.

The Locals window keeps track of the values of the variables i and number, see Figure 4-6. When the value of i is 10, the value of number is 55.

Figure 4-6 Variables shown in Locals window

With the given values of the variables, your debugging position should be on line 35, see Figure 4-7.

Figure 4-7 If statement is met

The if statement is now met. Using Step Over now will take the abnormal behavior of the debugger to jump an extra step to a non-executable instruction, an extra step to line 29, see Figure 4-8.

Figure 4-8 Abnormal step to line 29

Another use of Step Over will finally take you inside the while loop on line 39. If you keep using Step Over inside the loop, the board LED will turn on and off.

Did you find it troublesome to use Step Over several times before reaching the while loop? Not very much? Imagine if the if statement on line 35 would have the condition “number >= 500” instead, would that be troublesome? You would have to overuse the Step Over command for the if condition to become true.

Do you understand why code stepping is not always suitable, especially when it comes to a loop with many iterations? If so, you can stop the debugging procedure by using Stop Debugging command.

4.2 Set breakpoint Cases where you want to skip iterating a loop countless of times, is a good example to when breakpoints are useful. Thus, a breakpoint on line 37 in debugging_technique() must be set, either left click the gray margin along the left side of blinkLED.c window, or right click anywhere on line 37 and go to Breakpoint → Insert Breakpoint, see Figure 4-9.

Figure 4-9 Two ways of setting a breakpoint

A red dot on the gray margin indicates the insertion of a breakpoint, see Figure 4-10.

Figure 4-10 Breakpoint is set

To let a program run until it hits a breakpoint, go to Debug → Start Debugging, see Figure 4-11. Start Debugging is used for running programs until it hits a breakpoint, meanwhile Start Debugging and Break will take you to where the program execution starts, that is on the first executable instruction in main().

Figure 4-11 Run until breakpoint is hit

Whenever the breakpoint is hit, the execution stops and the instruction on that line is not executed. Since it is a while loop that is always true, the debugging procedure stops on line 40 inside the loop, see Figure 4-12. Inside the while loop, you can use stepping commands as you wish to turn board LED on and off.

Figure 4-12 Debugging position after breakpoint is hit

As demonstrated, it went faster for the if statement to become true by setting a breakpoint, rather than using code stepping line by line.

4.3 Delete breakpoint Since you are done with this technique, you may stop the debugging process. See Figure 3-19 if you do not remember the control command.

You may also remove the existing breakpoint on line 37, either left click on the red dot located on the gray margin, or right click anywhere on line 37 and go to Breakpoint → Delete Breakpoint, see Figure 4-13.

Figure 4-13 Delete breakpoint

5 Section 4: Debugging with conditional breakpoint

Within Atmel Studio, there are additional features that can be used with breakpoints. One of the features is to set a condition to a breakpoint, resulting in a debugging technique called conditional breakpoint. This technique allows you to stop the execution of a program when a specific condition that you have set, is met.

For instance, you did not want to iterate the instructions of the loop line by line, but instead used a breakpoint to execute the loop without examining it, in previous section. But sometimes, you may want to examine a loop when a variable has a certain value. That, you cannot do with breakpoints because breakpoints will execute the whole loop until it is finished. This is when conditional breakpoints are useful.

5.1 Breakpoint settings You will work with conditional_breakpoint(), meaning you have to do changes in main.c to execute the right function. Open main.c window to comment out line 16 and uncomment line 17, see Figure 5-1.

Figure 5-1 Uncomment line 17

With conditional breakpoints, you have to first set a breakpoint and then you can set a condition to that breakpoint. You will in this section examine an if statement, so set a breakpoint on line 58. To set a condition, hover over the breakpoint, and click on the Settings button, shown in Figure 5-2.

Figure 5-2 Settings button

A Breakpoint Settings window where you can choose to set a condition will open, see Figure 5-3.

Figure 5-3 Breakpoint Settings

5.2 Conditional Expression Is True It is possible to set different types of conditions on a breakpoint. If you check the Conditions box, one type of conditional breakpoint (Conditional Expression) will automatically be chosen, see Figure 5-4.

Figure 5-4 Conditional Expression

If you want to examine the execution of a program only when a variable has a certain value, you can do that by setting a Conditional Expression with the option “Is True”. Set the condition to “i == 4” to only examine the code when the value of i is 4, see Figure 5-5. Click the Close button to save the condition, and run the program with Start Debugging.

Figure 6-5 Set condition

When the breakpoint is hit, the instruction on that line is not executed, as with breakpoints in previous section. The execution will stop at the breakpoint only when the value of i is 4. You can inspect that the value of i really is 4 in the Locals window, see Figure 5-6.

Figure 5-6 Variable i == 4

With the condition met, the execution has stopped. You can now examine the execution when the value of i is 4, using code stepping. Use Step Over once. It will take you to the else statement on line 72 since the if statement is not met, see Figure 5-7.

Figure 5-7 If statement is not met

100

Use Step Over until you reach line 54. This will execute the else statement where the board LED will turn on and off. You can see in the Locals window that the value of i has not changed, which means the while statement will still be true when you use Step Over again to execute the instruction. This takes you to line 54, see Figure 5-8.

Figure 5-8 While statement is met

The next instruction to be executed will increase the value of i with 1, which means i == 5. Use Step Over to see what happens. The program will say it is “Running”, but you will soon realize that it will stay on “Running” endlessly, see Figure 5-9.

Figure 5-9 Program is Running

The condition that you set has become false. With Conditional Expression Is True, you can only examine the code when the condition (i == 4) is met. As soon as the condition is false, the debugger will stop working and is why the program stays on “Running” mode. Also, when the condition is false, you cannot see the value of the variable in Locals window.

101

5.3 Delete condition You can now stop the debugging procedure and delete your current condition. Hover over the breakpoint to open Breakpoint Settings and click the delete condition button, see Figure 5-10.

Figure 5-10 Delete condition

5.4 Hit Count = Check the Conditions box to set another type of conditional breakpoint. Left click the drop-down list to see other options. Select Hit Count, see Figure 5-11.

Figure 5-11 Choose another type of conditional breakpoint

102

The new conditional breakpoint will automatically be set to Hit Count with the option “=”, see Figure 5-12.

Figure 5-12 Hit Count

If you want to examine the code only after a statement has hit a certain value, Hit Count = is useful. Set the value to 3, see Figure 5-13. This means, when the value of i has become 3 in the if statement, you can further examine the code from there. Click the Close button to save the condition, and run the program with Start Debugging.

Figure 5-13 Set condition

The execution stops at the breakpoint only when the value of i is 3. You can inspect the value in the Locals window, see Figure 5-14.

Figure 5-14 Variable i == 3

103

With the condition met, the execution has stopped. You can now further examine the code when the value of i is 3, using code stepping. Whenever you feel like you are done using code stepping, you can delete the condition and move on to the next step.

If you by mistake uses the Continue command, the program will say it is “Running”, but you will soon realize that it will stay on “Running” endlessly. If that happens, stop debugging. The reason why the program will stop working when you use Continue command will be explained later in conjunction with another example.

5.5 Hit Count >= Check the Conditions box and select Hit Count from the drop-down list to the left. You will now change the option of how to use Hit Count when the condition is met. Instead of using the option “=” as in previous step, you will now change it to the option “>=”, in the drop-down list to the right, see Figure 5-15. These options are rather similar.

Figure 5-15 Hit Count >=

To understand the difference between the options “=” and “>=”, set the same condition as you used for the option “=”, that is 3, see Figure 5-16. Click the Close button to save the condition, and run the program with Start Debugging.

Figure 5-16 Set condition

The execution stops at the breakpoint only when the value of i is 3. You can inspect the value in the Locals window, see Figure 5-17. So far, the execution is similar to the option “=”.

104

Figure 5-17 Variable i == 3

What is different, is that you can now either use code stepping or Continue command to further examine the code. Use Continue once. This turns the board LED on and off once, and then the value of i changes to 4, see Figure 5-18.

Figure 6-18 Variable i == 4

Yet another use of Continue will turn the board LED on and off, and then the value of i increases with 1, see Figure 5-19. Instead of using Continue, you can also use code stepping if you want.

Figure 5-19 Variable i == 5

The reason why you can use Continue and execute the instructions when i == 3, i == 4, i == 5, and so on without examining it in detail, is because of the greater-than sign (>) in the option “>=”. It allows the debugger to hit the breakpoint whenever the value of i is at least 3.

In previous option with “=”, you could not use Continue. This is because you have limited the condition so that the debugger can only hit the breakpoint when i == 3. The program executed in which line 54 increased

105 the value of i to 4. When using Continue, the program will execute each instruction until it hits the breakpoint again. But since i == 4 and the breakpoint can only be hit when i == 3, the debugger does not reach the breakpoint ever again. Therefore, you can only use code stepping with the option “=”.

When you understand the difference, you can stop the debugging procedure and delete the condition.

5.6 Hit Count Is a multiple of Check the Conditions box and select Hit Count from the drop-down list to the left. Select the last option, Is a multiple of, from the drop down list to the right, and set the condition to 3 even in this example, see Figure 5-20. When the right settings are set, click the Close button to save the settings, and run the program with Start Debugging.

Figure 5-20 Set condition for Hit Count Is a multiple of

The execution stops at the breakpoint when the value of i is 3. You can inspect the value in the Locals window. Here, you can either use code stepping or Continue to further examine the code.

The option “Is a multiple of” is useful when you want to skip using Continue to iterate the instructions each time when a value has changed. Continue is used to execute the instructions until it hits the breakpoint again, where the execution stops. With this new option, the Continue command does not need to stop the execution each time when the breakpoint is hit. Instead, you can set a condition to which number of times you wish the breakpoint can be hit, before it stops the execution.

In this example, you set the condition to 3, which means that the debugger can iterate the instructions and hit the breakpoint three times. On the third time that the debugger hits the breakpoint, it stops the execution.

Use Continue. The board LED will turn on and off three times each. When the board LED has blinked the right amount of times, the value of i changes to 6, see Figure 5-21.

Figure 5-21 Execution stops when breakpoint is hit three times

106

If you use Continue once again, the execution will stop when i == 9. It is also possible to use code stepping to execute each instruction. When you feel like you understand, you can stop the debugging and delete the condition.

107

6 Section 5: Debugging with action breakpoint The other additional feature that can be used with breakpoints is to set an action to the breakpoint. With an action, you can log a message to the Output window without making a change in the code, which is why this technique is useful. Usually when a debugger is not used, you would add a print statement as printf() in the code to log a message. You would have to add and delete, or comment out and uncomment the print statements, to your own needs when executing a program. This may cause harm if for example the wrong instruction by mistake is added or deleted.

6.1 Set action You will use the same breakpoint on line 56 in conditional_breakpoint(), as in previous section. To set an action, you have to open the Breakpoint Settings windows, the same as when a condition is set. Instead to check the Conditions box, check the Actions box. This opens a message field where you can write what to be logged in the Output window, see Figure 6-1.

Figure 6-1 Message field

If you would like to log a message each time an instruction is executed, you can do that with an action. Whenever line 56 is executed, a message will be logged. You can write anything in the message field, for example “This guide has soon come to an end!” or a more suitable message, see Figure 6-2. Notice that the Continue execution box is checked.

Figure 6-2 Message to be logged

108

Click the Close button to save the action, and run the program with Start Debugging. What you will see, is that the board LED will turn on and off in a loop. But each time the board LED has turned on and then turned off, a new message will be logged in the Output window. In total, 20 messages will be logged, see Figure 6-3. Try to figure out yourself why 20 messages are logged by looking at the code.

Figure 6-3 Message in Output window

That is how you log a message each time an instruction is executed. Stop the debugging procedure and open Breakpoint Settings. You are going to uncheck the Continue execution box now to see the difference. If you want to change the message, do so by clicking on the message field, see Figure 6-4. Click the Close button to save the action, and run the program with Start Debugging.

Figure 6-4 Change message

109

The message is logged just once in the Output window, see Figure 6-5.

Figure 6-5 Message in Output window

If you look in the Local window, the value of i is 1, see Figure 6-6. The execution stopped when the breakpoint was hit and therefore, only one message was logged.

Figure 6-6 Variable i == 1

Use Continue once. The board LED will turn on and off once, the value i increases with 1, a new message is logged when the debugger hits the breakpoint and the execution stops. Look at the Output window to see a new message being logged, See Figure 6-7.

Figure 6-7 New message logged

110

Look at the Local window to see that the value of i has increased after one iteration, see Figure 6-8.

Figure 6-8 Variable i increased

Instead of using Continue, you can also use code stepping. Try it out?

This wraps up the whole debugging guide. You have now learnt the mechanism of several debugging techniques. With the knowledge you have gained, you may move on to the debugging exercise in Part 2!

111

PART 2: DEBUGGING EXERCISE

Now you will apply the knowledge that you gained from Part 1 to find bugs in the following three exercises.

7 Section 6: Download Sam4e16eExercise Throughout this lab exercise, you are going to work with Sam4e16eGuide project, available at: https://drive.google.com/drive/folders/1W0YSm6dTRauaXdtst4hh1gzXpkAMZbBH. Download and save the project to your desired location in the computer. You should now launch the project.

8 Section 7: Exercise One Exercise One represents the code in blinkLED.c which is shown in Figure 8-1. This is the first program to be debugged. SAM4E Xplained Pro board’s LED lamp will blink on and off when the bug(s) are successfully remedied.

Figure 8-1 Code of Exercise One

112

9 Section 8: Exercise Two

Same as in Exercise One, the SAM4E Xplained Pro board’s LED lamp should blink on and off when the bug(s) are successfully remedied. The code to be debugged is shown in Figure 9-1 and is found in blinkLED.c.

Figure 9-1 Code of Exercise Two

113

10 Section 9: Exercise Three

If you press the SAM4E Xplained Pro board button three times, the board LED lamp should blink on and off. The program code to be debugged is shown in Figure 10-1, found in buttonPress.c.

Figure 10-1 Code of Exercise Three

114

PART 3: QUESTIONNAIRE

We hope you have gained knowledge in the several debugging techniques using a debugger tool. We would like to have your feedback from your experience of performing this laboratory assignment.

11 Section 10: Evaluation

Please help us fill in the questionnaire in Table 1.

Statements Agree Neutral Disagree Notes

Q1 Debugging using a debugger is challenging ☐ ☐ ☐

Q2 The applied debugging techniques are easy to use ☐ ☐ ☐

Q3 The lab assignment is well-designed and easy to follow ☐ ☐ ☐

Q4 The bugs were easy to find and remedy ☐ ☐ ☐

Q5 I understand the debugging techniques and they are convenient ☐ ☐ ☐

Q6 Being a good programmer makes it easier to find bugs ☐ ☐ ☐

Q7 Time is the main factor for not using a debugger ☐ ☐ ☐

Q8 A lab assignment is a good way to be taught debugging ☐ ☐ ☐

Q9 I understand why a debugger is used ☐ ☐ ☐

Q10 I understand when the debugging techniques are suitable ☐ ☐ ☐ Table 1 Questionnaire

Thank you for your contribution to this research by completing this lab assignment!

115

Appendix A Theoretical background

1 Error, fault and failure Just like any human being, software developers also make mistakes [13]. An error [10] represents a mistake or misunderstanding of the requirements of a system, produced by the software developer. The result of an error may lead software to contain a fault [10] which is an incorrect step, process or data definition [15] in the software. Fault are also commonly referred to as defect or bug. When executing a system containing faults it may lead to failure. However, faults do not always produce a failure [10], which is the inability of a system to perform its required functions.

There are many root causes for a code to contain bugs. Some of these are:

1.2 Computation Computational faults [41] may involve algorithms with mathematical calculations. It may range from a computer cannot handle cases with division by zero and the computer assigns a wrong mathematical value due to round-off.

1.2.1 Operator precedence Logic faults as erroneously using arithmetic operators and parentheses may be difficult to detect [30]. This due to the program executes the code without defining a failure and do not usually cause the program to terminate. An example of such bug is shown in Figure 2-1, a missing parentheses when calculating the average of two numbers [26]. The program is going to successfully execute, hence no syntax-, logic- or run-time error involved. However, the result of the calculation obtained is not correct, because of the operator precedence. The division operator has a higher precedence than the addition operator [26].

Figure 2-1 Missing parentheses

1.2.2 Divide by zero There are cases where mathematical bugs do not successfully execute the program. Shown in Figure 2-2 is another example, dividing a number by zero [30]. This may cause a run-time error that may cause the program to terminate when executing the code.

Figure 2-2 Divide by zero

116

1.2.3 Off-by-one An off-by-one bug [41,42] may occur when a loop iterates one time too many or too few, either by starting a loop variable at 1 instead of 0 or using “<=” instead of “<”, or vice versa, are some examples of this type of bug. See Figure 2-3, illustrating an out of bounds example of an array. The array (array[4]) has four elements with the indices 0, 1, 2 and 3. As j <= 4 is evaluated in the for loop, the second print statement will attempt to access a memory location where the index of the array is 4, that is nonexistent. Hence, the output value from the array is accessed from a random memory location containing a garbage value.

Figure 2-3 Off-by-one

1.3 Data reference (pointers) A pointer [37] is used to reference a memory location in which every memory location has its memory address. The value stored at the memory location may be accessed or modified using a pointer, known as dereferencing the pointer. There are some common faults related to pointers [31], studied as follows:

1.3.1 Uninitialized pointer An uninitialized pointer [32] occurs when a pointer is declared but not initialized. An example is shown in Figure 2-4. The pointer (ptr) is only declared and therefore may point to any (non intentional) memory location such as into the system stack or the global variables. When the value of 10 is assigned to pointer, the program may write a 10 to the memory location where pointer points to. To store a value in a non intentional memory location may have the potential to overwrite anything in the code that causes the program to behave differently than expected. It may even cause the program to terminate immediately.

Figure 2-4 Uninitialized pointer

1.3.2 Null pointer Unlike an uninitialized pointer which stores an undefined value in any (unintended) memory location, a null pointer [32] stores a defined value but does not refer to a valid memory location. That is, the null pointer is initialized but is not pointing to any memory location, see Figure 2-5. When a pointer variable does not yet have

117

a memory location to be assigned, it is wise to use a null pointer. Having a null pointer is not going to unnecessarily harm data in the memory.

Figure 2-5 Null pointer

1.4 Redundant code Unnecessary code is called redundant code [34]. This code could include unused variables, restating an expression, code that is never executed, also called unreachable code, and executed code that has no effect on the output, also called dead code. Figure 2-6 shows an example of redundant code. The return statement has a x * 2 expression which is redundant code that could be replaced by variable y. Alternatively, the statement int y = x * 2 could be removed.

Figure 2-6 Redundant code

1.5 Control flow A control flow is defined as an execution order of function calls, statements and instructions. Within C language, there is a control flow statement [56]. It is a statement whose execution results in the choice of choosing which of two or more paths to be followed. A loop may be involved in a control flow statement upon which the loop will never terminate or will never execute to the conditions [24].

1.5.1 Infinite loop A loop is used to repeat a set of instructions. When a loop such as for loop, while loop and do...while loop, does not terminate due to no condition or having a condition that is never met, causes an infinite loop [54]. As a result, the program becomes unresponsive. Shown in Figure 2-7 is an example of an infinite loop. The expected output is “4 3 2 1”. Since the variable count is increasing instead of decreasing in the loop body, the while statement stays true continuously and the loop cannot terminate.

118

Figure 2-7 Infinite loop

2 Software testing Software testing [3,12] is a sequence of tasks applied to examine whether the product meets the end user’s specified requirements. Otherwise, the software need to be corrected according to the requirements. This is where the motive of software quality was introduced [19], to ensure that the released software do function as expected. In order to improve quality, reliability and performance of a system, an essential process must be thoroughly performed during development. Thus, confidence will arise in the system. The process of importance is called software testing [4,5], including software debugging as a part of it.

Despite software testing being an essential process, it can only be used to show [10,13] there are presence of defects in the system, but not their absence. Execution of the code may lead to a failure [10], an undesired output according to the system specifications. Based on a failure, debugging process begins.

3 Debugging procedure When bugs are found, they need to be fixed. In order to resolve these bugs, software developers need to perform debugging. Debugging is a way to identify the bug in the code, so that it can be corrected [36]. There are some debugging steps [36] which can be followed to determine the location of the bug, arranging bugs into different categories, trying to understand the bug behavior and finally fixing the bug.

Debugging is illustrated in Figure 2-10 as a state diagram. The state diagram shows distinctive debugging procedures, starting from program execution to finding defect and examining it. In case the defect is not detected or resolved, the program will be executed again for further examination.

119

Figure 2-10 State diagram of debugging procedures

4 Debugger Finding bugs in a program sometimes requires a debugging tool, known as a debugger. A debugger may be used by a software developer to examine and trace the source code to find the main cause of bugs [49]. It has the ability to show detailed information about the program being executed such as what data stored in each variable [50,58].

Most of the integrated development platforms as Atmel Studio, has debugging control buttons, illustrated in Figure 2-13. These control buttons may give the debugger instructions to step through lines of code in the program and stop the debugging process [52]. Some debugging techniques using these control buttons are code stepping, breakpoint and conditional breakpoint.

Figure 2-13 Debugging control buttons in Atmel Studio toolbar

120

4.1 Code stepping Code stepping is a debugging technique used to examine how a program is executed line by line, using the control commands Step Into, Step Over and Step Out [40,53], see Figure 2-14. Step Into constructs the debugger to examine an instruction or a particular function in details. On the contrary, Step Over constructs the debugger to only execute the line of code, but avoid to examine it in details. Step Out constructs the debugger to execute all the remaining instructions in the current function to exit and go to the next line of code.

Figure 2-14 Code stepping commands

4.2 Breakpoints A common feature in a debugger is the ability to set breakpoints which stops or pauses a running program when execution reaches the breakpoint. When the breakpoint is reached, the code may be examined [38] using code stepping commands.

4.3 Conditional debugging Breakpoint has extra features. The behavior of the program may be examined and inspected using breakpoints that holds a condition but also an action. When a breakpoint is set, a condition and an action may be applied. This condition is a boolean expression which will be evaluated to true or false when the assigned breakpoint is reached [59,60]. An example is when a variable reaches a defined value, as shown in Figure 2-15. The conditional expression will make the program execute until variable i reaches the value of 3.

Figure 2-15 Breakpoint condition

An action may be applied as well, to include a log message which can be displayed on the output window when i == 3, see Figure 2-16.

121

Figure 2-16 Breakpoint action

122