Enhancing Learnability of Pair Programming Practice When Introducing Novices

MASARYKOVA UNIVERZITA

FAKULTA INFORMATIKY

Enhancing Learnability of Pair Programming Practice when Introducing Novices

DIPLOMOVÁ PRÁCE

Pavel Januš

Brno, jaro 2014

Declaration

I declare that this paper is my original authorial work, which I have worked out by my own. All sources, references and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.

Advisor: Bruno Rossi, PhD. i

Acknowledgement I would like to express my gratitude to my supervisor Bruno Rossi for leading me through the creation of my diploma thesis. I am grateful to him for stimulating discussions about the content of the thesis and supporting developing of my own ideas. I also want to appreciate his provided time, knowledge, valuable information and his patience and willingness.

I express special thanks to my parents for their understanding, and my girlfriend for her support.

Abstract

The current thesis focuses on understanding the mechanisms that lead novice developers to better learn the Pair Programming (PP) practice, one of the pillars of Agile Metho- dologies. The main aim is to provide theoretical-grounded suggestions that could help to improve the process by which novices are introduced to PP and start mastering it.

To contextualize PP, we review the emergence of Agile development methodology compared to more heavy-weight methodologies and we evaluate the importance of PP as one of the key practices of Extreme Programing (XP). We then delve into empirical studies that investigate the efficacy of PP in software development and into the identification of the state of the art in terms of the introduction of PP to novices. In this context, we identify problematic issues that emerge during the learning process of the PP practice and that could hinder its adoption.

In the analytical part of the thesis, we identify the mechanisms that could be better applied to enhance learnability of PP. We chose an educational model, Bloom's Taxonomy (BT) of learning domains, to frame the discussion about the actions that can be applied for the improvement of PP learnability. Such framework and recommendations can be applied in future works to when introducing new developers to Agile Methodologies. Keywords

Pair Programming, Agile Methodologies, Extreme Programming, Bloom’s Taxonomy, introducing to novices, empirical studies.

iii

Content 1 Introduction ...... 1 1.1 Motivation...... 1 1.2 Objectives ...... 2 1.3 Problem Statement ...... 2 1.4 Main Thesis Contribution ...... 3 1.5 Thesis Structure ...... 4 1.6 Research Methodology ...... 5 2 Traditional Software Development ...... 7 2.1 History ...... 7 2.1.1 Software Crisis ...... 7 2.2 Life-cycle Models ...... 9 2.2.1 Code Fix Model ...... 9 2.2.2 Waterfall Model ...... 9 2.2.3 Spiral Model ...... 10 2.3 Object Oriented Approach ...... 11 2.3.1 RUP Methodology ...... 12 3 Agile Software Development ...... 14 3.1 Characteristics ...... 14 3.2 Agile Manifesto ...... 15 3.3 Traditional and Agile Comparison ...... 16 3.4 Summary of Agile Methodologies ...... 18 3.4.1 SCRUM ...... 19 3.4.2 Feature-driven Development ...... 20 3.4.3 Lean Development ...... 20 3.4.4 Test-driven Development ...... 21 3.4.5 Extreme Programming ...... 21 4 Pair Programming ...... 26 4.1 Introduction and Practices ...... 26 4.4.1 Pair Jelling ...... 28 4.4.2 Pair Pressure ...... 29 4.2 The Myths ...... 30

4.3 Empirical Studies ...... 32 4.3.1 Benefits ...... 33 4.3.2 Weakness/Problems ...... 37 4.3.3 Partner Finding ...... 40 5 Learnability of Pair Programming ...... 44 5.1 Current Problems ...... 44 5.2 Current State ...... 46 6. Bloom’s Taxonomy of Learning Domains ...... 50 6.1 Bloom’s Taxonomy in Software Engineering ...... 51 6.2 Course Improvement...... 53 6.3 Knowledge Level of Novices and Experts ...... 53 6.4 Knowledge Assessment ...... 54 6.5 Assignment Assessment and Testing ...... 58 7 Proposed Solutions ...... 60 7.1 Applying Bloom’s Taxonomy ...... 60 7.2 Recommendations ...... 63 8 Conclusion and Future Works ...... 67 Literature references ...... 70 Appendix ...... 76 A. Summary of the Cited Studies ...... 76 B. Compatibility Questionnaire ...... 89

1 Introduction Traditional software development is also called "heavyweight" of Model-driven development, because it is based on a sequential series of steps that have to be defined and docu- mented in detail, it does not support changes of requirements and the complete form of the system has to be known at the beginning. The result of such a development was usually expensive, unwanted and unreliable software. Recently, a new approach of software development in Software Engineering has emerged. This approach is called Agile software development. The main idea of Agile software development is to develop as fast as possible the functional part of the system, present it to the customer and improve it in accordance with the customer’s feedback. For Agile development human resources, fast reaction on changing requirements and project completion in time are important..

One of the best known Agile methodologies is Extreme Programming. It takes beneficial elements of traditional software development to "extreme" level. One of its key characteristics is Pair Programming (PP) practice. In PP code is created by two people working together at one workstation. PP increases software quality and does not impact on time to deliver. PP developers produce on average about 15% less functionality than two individually working developers, but with the higher quality of software [13, 22, 26]. Higher quality of software can cause big savings later in the project. PP can be used both in academic and business environment. In academic environment it can improve the final grades and retention of the students, increase confidence in the solution and also reduce instructor’s workload. Another benefit is that it is an enjoyable practice. For example, in the study [26] 95% of students using PP enjoyed work more than when programming alone. 1.1 Motivation PP is a widely used practice with several advantages both in academic and business environment. PP is a proper practice to be used in software engineering courses to learn the programming. Usage of PP in labs can increase the productivity of students, confidence in the solution and also it can reduce the instructor’s workload. In PP some problems/issues can be also identified and these problems can involve the process of learning. There exist several studies about PP, but there is not a wider study about learnability of PP. It could be really useful to find solutions for some of the problems with the learnability of PP and mi- nimize their effect.

1.2 Objectives The aim of this thesis is to review the traditional and Agile software development, compare them, show their differences and describe some of their methodologies. Furthermore the thesis aims at the description of the Extreme Programming and one of its key characteristics PP practice. One of the objectives is to analyze PP in terms of empirical studies investigating the efficacy of PP and identification of the state of the art in terms of introducing of PP to novices. PP investigation also aims at the identification of benefits and weaknesses of PP. Another objective of the thesis is to define concepts of learnability in PP, identify problems in learnability of PP and provide possible solutions for some of them. Some solutions for the problems are created in accordance with current research and some with the application of the Bloom’s Taxonomy. 1.3 Problem Statement Based on the review that was carried out on the current literature we have identified several issues/problems related to PP and its learnability. Some of the identified problems are still open, so there is an opportunity to provide some solutions for them.

The first of the open problems of PP is pair jelling. Pair jelling is the initial period when the pair is adjusting. At this period is pair less efficient.  There does not exist direct solution for the pair jelling. Pair jelling is mentioned in some studies, but not elaborated more. In studies its influence is only mentioned, but not the instructions of how to lower its effect and how to shorten the period of its influence.

The second open problem is about the scheduling. It can be described as difficult for the developers to find common time to meet and cooperate.  In the thesis we have identified some recommendations which could solve some of the problems with the scheduling, but just for the academic environment. There exists a possibility to use distributed PP which was presented to be used when the developers are at the different geographical places, but not as one of the solutions for the scheduling problems.

The last open problem is partner compatibility. This problem is one of the most common and it can damage the whole pair cooperation. Problems with the partner compatibility can be caused by personality clash, different skill level, experiences, programming style and self-esteem of the partners.  There exist studies testing compatibility in terms of personality type, programming style differences and self-esteem, but no study testing all the three aspects together and with the usage of the “test run” (first simple program to control the compatibility of the pair). 1.4 Main Thesis Contribution The thesis contributes to the improvement of the learnability of PP in several ways. In this section we will provide a list of such a improvements and recap the main thesis contribution.

The main contributions are:  Identification of PP benefits both in academic and business environment based on the review that was carried out at the current literature.  Identification of all the existing issues/problems in literature and from practice about (learnability) of PP.  Collection of the information from literature on the topic about partner finding.  Linking of problems to existing solutions and identification of open problems (defined in the previous section).  Proposal of a framework according to Bloom's taxonomy to enhance the learning process for novices beginning to work with PP.  The identification of several recommendations for improvement of the PP learning process, not strictly linked to BT.  The combination of existing literature for the definition of compatibility questionnaire to be used during the application of the proposed framework.  Definition of further future works for the applicability and validation of the framework.

1.5 Thesis Structure The thesis starts with chapter 2 and the description of the traditional heavy-weight development methodologies and the reasons, why these methodologies are not mostly suitable for today's projects. Chapter also describes the history of traditional software development and the event called software crisis. In the end of the chapter some of the life-cycle models and Object Oriented approach are briefly described.

In Chapter 3 is described the idea of Agile software development and the reasons why this approach is so popular nowadays. Agile software development is contextualized and its characteristics and Agile Manifesto are described. The chapter also provides the comparison between traditional and Agile software development to connect chapter 2 and 3. The chapter ends with the summary and brief explanation of some Agile Methodologies counted Extreme Programming practice, which is described more in detail.

Chapter 4 provides a description of the Pair Programming (PP) practice, one of the key characteristics of Extreme Programming mentioned in the previous chapter. Chapter about PP consists of introduction and practices, myths of PP and empirical studies. From the empirical studies the benefits (both in academic and business environment) and weaknesses of the PP and also the aspects of partner finding are derived.

In the chapter 5 called Learnability of Pair Programming problems of the PP identified in the chapter 4 are described. For every problem its impact on the PP learning process is identified and the existing solutions are provided.

Chapter 6 describes the Bloom's Taxonomy (BT) as a chosen educational model to frame the discussion in the next chapter. BT is also described in the software engineering context. Chapter also describes an advantage of the usage of the BT to improve software engineering course, different approach to knowledge of novices and expert developers and the assessment opportunities.

Chapter 7 deals with the open problems of PP solutions with the usage of the BT. In the second part of the chapter there are mentioned recommendations for the instructors interested in adopting PP in their labs.

The last chapter is about the conclusion and future works.

1.6 Research Methodology In this section, we will provide a logical flow (Figure 1.1) to clarify the process of data collection, analysis and elaboration. Data validation remains open for the future works. We will also describe the research questions which were used to accomplish the research work.

Figure 1.1: logical flow In the thesis we will answer the research questions to accomplish the research work. Re- search questions will be answered in accordance with the literature review of existing sources (books, studies, researches etc.) about PP. For the every question we will collect the existing data, analyze and elaborate them.

The research questions are: Q1a: What are the general benefits of PP?  This research question will enable us to find out general benefits of PP. Identified benefits will present opportunities of PP and they can change the negative viewpoint of some developers to this Extreme Programming practice. The benefits can be presented to the developers at the beginning of the PP learning process.

Q1b: What are the benefits of PP in academic environment?  This research question is connected with the previous question, but this question will explore benefits which are typical for the academic environment and that cannot be usually identified in the business environment. Identified benefits can be used as a motivation for the use of PP in academic environment. There will be also explored the opportunity of PP to improve programming courses. Benefits can be presented to the students learning the PP practice.

Q2: What are the weaknesses/problems of PP?  In this research question, we will find out the general weaknesses and problems of PP. There will be examined how these problems influence the PP development. We will also identify the least liked aspects of PP.

Q3: What is the impact of the PP problems on learning of PP of novices?  This research question depends on the problems/issues identified in the Q2. When the problems are found out, their impact on the process of PP learning can be identified. We can also identify the open problems of PP that impact the learnability. These open problems will be later in the thesis elaborated and the solution for them will be provided.

Q4: What has been done to solve the problems of PP impacting the learnability?  This research question depends on the results from the questions 2 and 3. When we will find out the problems/issues of PP and their impact on learnability we can identify what has been done to solve these identified problems.

The aim of these research questions is to allow us to use the information gathered to work towards the provision of mechanism to improve the learnability of PP.

2 Traditional Software Development Since the beginning of software development (which was in 1940’s), various types of development methodologies have gone through many stages of evolution. It was a long way from traditional software development (TSD) also called Model-driven development to Agile software development. A lot of changes have been happening over time, and this chapter will describe reasons of some of them.

The chapter starts with a brief description of the history of TSD, software crisis and the reasons of its beginning. Then there are provided descriptions of the development Life- cycle models and the evaluation of them. The mentioned models are Code Fix model, Wa- terfall model and Spiral model. Chapter ends with the specification of the Object Oriented approach and its best known methodology – Rational Unified Process. 2.1 History We will divide software engineering history into the periods [1]:  Until mid 1960s: it can be called breakthough years. Single purpose programs in suit with given computer and architecture were developed. No complex-driven software development existed. Programs were unchangeable and unmaintainable.  Late 1960s and early 1970s: software engineering came into existence. Software engineering is one of the themes in NATO conference in 1969.  1970s: software engineering domain was established. Problems with software delivery and unfinished projects. Software development wasn’t led. 1976-1977 was the start of using software engineering techniques such as specification, design, architecture, testing etc.  1980s: large progress in software engineering and its methodologies, populariza- tion of Object Oriented approach, analysis and design. Start of standardization, software interoperability and cooperation of software products.  1997: software engineering was accepted as the field of knowledge with certificate in USA. 2.1.1 Software Crisis Software crisis started at the end of 1960 and it was one of the pillars leading to creation of software engineering. Characteristic elements of software crisis were elongation and price increasing of the projects, low program quality, difficulty/impossibility of maintenance and

innovation, low productivity of the developers, non-effective development, uncertainty of the result etc. [1].

For the illustration of the alarming situation look at Figure 2.1 with a graph of government orders in the USA with the price $ 6.8 billion in the period of the software crisis.

Figure 2.1: Government software orders in USA in the period of the software crisis [2] Reasons of the software crisis were [1]:  communication problems at all levels,  wrong approach of developers to the development: often main goal for developers was “to have a fun”, the idea was that only developer knows “how to program it”,  wrong estimates: time, price, scope, efficiency,  wrong planning: developers often used “cowboy coding” – autonomy over the development process,  low productivity: developers wrote a lot of useless code, they tried to write as fast as possible, but did not write useful parts of the software,  ignoring the basic rules,  underestimation of threats and risks,  problems with the technology.

2.2 Life-cycle Models Traditional software models were originally proposed to bring order to the chaos of software development. History has indicated that these models have brought a certain amount of useful structure to software engineering work and have provided a reasonably effective road map for software teams [3]. 2.2.1 Code Fix Model Since the beginning of software development most of the software have been developed by Code and Fix model (or Build and Fix model). The model can be described easily as coding the application, delivery and bug fixing. Projects were developed without any planning, but at the time of using of this model, no large program systems were developed [1].

Figure 2.2: Scheme of the Code and Fix model [1] 2.2.2 Waterfall Model Waterfall model arises from Stagewise model, both models consist of the same phases, but Waterfall model allows feedback on the end of every phase. Feedback is used for evaluation of the phase and potential re-doing/correction. Sequential phases are characteristic (after the end of the phase the next phase starts, two phases cannot be done together). Waterfall model is a typical document driven approach.

This model was a revolutionary model, its benefits were: clear dividing of the phases, possibility of back step in development, full documentation and simplicity. Model is applicable to small and simple projects with clear requirements [1].

Disadvantages of this model are [1]:  model is too much simple – not usable for bigger and complex projects,  developments process is not flexible – if customer comes with a change to phases of analysis, design must be done again and new documents are created,  all the product is delivered at the end of the project – customer is involved only at the project opening and product delivering,

 if an analyst misunderstood the customer’s vision of the product, it is discovered at the moment of product delivering.

Figure 2.3: Scheme of the Waterfall model [4] 2.2.3 Spiral Model Spiral model was proposed by Barry Boehm in 1986 [5]. It is an evolutionary software process model that couples the iterative nature of prototyping with the controlled and systematic aspects of the waterfall model [3]. Spiral model brings to process of software development two important concepts: iterative and risk-driven approach. Development is done in repeated cycles (iterations). In every iteration there are phases of determine objectives and constraints identification, risk analysis, development/testing and planning. The output of the iteration is a prototype, which can be verified by a customer. This leads to specification of requirement and early defect detection [1].

Benefits of this model are early elimination of bad solutions, reusable components, the model is usable for complex projects and evaluation of resources and activities [1].

Disadvantages of this model are [1]:  required expertise of the people – people must be able to identify and analyze risks,  overall complexity – the model is “heavy” and there is a lot of bureaucratic tasks,  software is not released before the end of the last cycle – the model uses prototypes, but they can focus just on the part of the system,

 changes are allowed after the end of the cycle – no changes can be done in the middle of the cycle,  missing elaboration of all activities of the development – e.g. formation of a contract, specification and milestones are not defined.

Figure 2.4: Scheme of the Boehm’s Spiral model [5] 2.3 Object Oriented Approach In Object Oriented (OO) approach data and functions are inseparable parts of the object. OO approach relies on producing graphical models of the system and, in many cases, auto- matically generating code from these models. The UML has many diagram types and so supports the creation of many different types of system model. Most common diagrams are class diagram, activity diagram, use-case diagram, state diagram etc. [8].

2.3.1 RUP Methodology Rational Unified Process is an extensive, sophisticated, object-oriented iterative methodology. RUP originate from the Unified Process. The methodology was developed by Rational Software company (subsequently acquired by IBM) as a commercial product. Rational Corporation is a builder of complete environments (tools and technology) that support the process. RUP is use-case-driven approach. Use-case is defined as a sequence of actions done by the system or inside the system, which gives a value to the customer. The methodology uses visual models for the better understanding of the developed software. Visual models are based on modeling syntax of the Unified Modeling Language (UML) [1, 3].

The development is done in iterations with four phases. The phases are Inception, Elabora- tion, Construction and Transition. In the Inception phase of the project must be defined the goal and requirements, in the Elaboration phase customer and project needs are analyzed and the basic architecture is defined. The Construction phase aims at development of the application design and creation of the source code. In the last phase Transition is a project passed to the customer or to the next iteration cycle. Every phase consists of iterations (different number of iterations depending on project needs) [1, 7].

The disadvantage of RUP methodology is that it is an extensive model. It means that for the small teams and large projects a lot of time can be consumed by methodology exploration and intermediate product creation.

Figure 2.5: Scheme of the RUP methodology [6] In this chapter we have seen the description of the traditional software development with its history (with the description of the software crisis), life-cycle models such as Code and Fix model, Waterfall model and Spiral model, in the end of this chapter is described Object Oriented approach and one of its methodologies – RUP methodology. The aim of this chapter was to show, that there exist other approaches than the Agile software development. The aim of the chapter was also to show the reasons, why traditional software development is usually not appropriate for today's projects.

In the next chapter we will describe the idea of Agile software development, present its characteristics and Agile Manifesto. Furthermore we will compare Agile software development with the traditional software development. In the end of the next chapter there is a brief explanation of selected Agile methodologies such as SCRUM, Feature Driven development, Lean development, Test-driven development and Extreme Programming, which is described in greater detail.

3 Agile Software Development The chapter starts with the characteristics and principles of the Agile software development. Then there is a description of the Agile Manifesto – one of the pillars of this methodology. There is also provided the comparison with the traditional software development to connect this chapter with Chapter 2. The chapter ends with the brief descriptions of the Agile methodologies such as SCRUM, Feature-driven development, Crystal Methodology, Lean development, Test-driven development and the best known one – Extreme Program- ming, which is described more in detail. 3.1 Characteristics As was seen in the previous chapter, traditional software development (TSD) refers to the situation 10-20 years ago: TSD well fulfills that situation and its requirements. The quality was important, perhaps because of the fear of the repeat of the software crisis. Customers waited longer for the software, but they predicted a good result. But there was a change in thinking. Today’s customer does not want to wait long for the software, but customer still wants a high-quality software.

There exists an interesting paradox: a lot of customers are satisfied with a relatively unus- able product. Maybe, the reason for the paradox is: when the project starts, customer does not exactly know what should the system do [1]. Agile methodologies solve this problem with close customer’s involvement. Nowadays, the most important parameter is the veloci- ty of software development. The application can be earlier used and the functionality will be added sequentially. The flexibility to the changing requirements is important.

In accordance with research of Scott W. Ambler, the failure of software projects led by traditional approach occurs more often than with Agile software development [11]. One of the reasons is, that traditional approach believes in customer requirement specification and do not assume any change of requirements.

Basic principles of Agile methodologies are [1]:  Iterative and incremental development with short iterations: project is divided into increments. Development of every increment is iterative (particular phases of development are going on repeatedly). New functionalities are delivered frequently (even every day). The advantage of this approach is that the customer knows in which state the development is and what the team is developing. So there is not a possibility of delivering something different than was expected. Another advantage

is sequential delivering of functionality, not everything at project end (as it was in TSD).  Direct, personal communication in team and with the customer: agile attitudes try to integrate communication directly to the process (e.g. frequent meetings). The advantage is a earlier detection of the problem in the team (e.g. poor relationship of the team members). The customer is ideally the team member. He/she should communicate with the development team, co-create the design and co-decide about tests.  Repeated, continuous automated testing: in regards to frequent system changes, faster analysis and design, it is important to control continuously the system correctness. Tests should be automated and even written before implementation of the tested parts to keep quality. 3.2 Agile Manifesto In February 2001, 17 software developers and engineers met in Utah to discuss lightweight development methods. They wanted to preempt all uselessness of the development process. On the basis of their experiences they defined 12 agile principles and signed Mani- festo for Agile software development. They found The Agile Alliance – group of independent thinkers about software development. Among the best known members of the manifesto belong: Kent Beck, Mike Beedle, Alistair Cockburn, Ward Cunningham, Martin Fowler, Jim Highsmith, Ken Schwaber, Jeff Sutherland [9].

Agile Manifesto has two basic propositions [1]:  To accept and facilitate change is more effective, than to try to avoid it.  It is necessary to be ready to react on unpredictable events, because they arise in almost all projects.

Authors of the Agile Manifesto prefer [9]:  Individuals and interactions to processes and tools,  Working software to comprehensive documentation,  Customer collaboration to contract negotiation,  Responding to change to following a plan.

Twelve principles behind the Agile Manifesto [9]: 1. Our highest priority is to satisfy the customer throughout early and continuous delivery of valuable software.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. 3. To deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 4. Business people and developers must work together daily throughout the project. 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 7. Working software is the primary measure of progress. 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 9. Continuous attention to technical excellence and good design enhances agility. 10. Simplicity – the art of maximizing the amount of work not done, it is essential. 11. The best architectures, requirements, and designs emerge from self-organizing teams. 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. 3.3 Traditional and Agile Comparison For the traditional methodologies are software requirements, specified at the beginning, fixed so that their fulfillment is the main goal. Time and resources are flexible. Agile methodologies have opposite perspective. The main goal is fast product delivery. Functionality is changed in accordance with time and resources. We can see it in Figure 3.1. Left triangle represents traditional methodology, right triangle represents Agile methodology.

Figure 3.1: Traditional and Agile triple constrain comparison [6] Traditional and Agile methodologies are two most used representatives of software development. They have different processes and suit different types of projects. In Table 3.1 are comparisons of these methodologies from other point of view.

Table 3.1: Traditional and Agile methodologies comparison [10]

Agile and Model-driven methodology comparison

Model-driven methodology Agile methodology

processes, directs on explicit practices, directs on tacit knowledge, content of the knowledge, view the people as a people are considered as key factor of methodology secondary factor the success processes and activities are define so-called "barely enough metho- detail of the described in detail dology", which directs on activities that methodology create value and eliminate activities that does not create a value is focused on the quality of the is focused on value for the customer and processes, assumption that quali- high quality of the product quality ty processes lead to the quality result assumes predictability of the fu- assumes unpredictability of the future, ture, emphasizes on anticipation emphasizes on adaptation on changes predictability (requirement of gathering and (incremental requirement gathering, planning beforehand) planning after the iteration)

changes underlie the change changes are allowed and used, custom- changes control and effort is to ers have possibility to re-evaluate re- minimalize the changes quirements considering new knowledge assumption, that good processes highest priority is to satisfy the customer value for the leads to good results, too much customer aimed on own processes, not on results for the customer in starting and final phases, after the customer is a control subject customer project signing the specification docu- throughout all the project, the customer participation ment takes the control the team can change the priority of the functions of technologic staff in every iteration developers try to implement all only required functions, demand of extend of the the functions, which can minimalization solution the customer need in the future relationship provided contractually confidence and cooperation customer-developer standard individuals emphases on abilities, knowledge and people qualifications skills of the people form of the mainly written emphases on face-to-face communication communication extensive documentation documentation is not significant, but documentation understandings mainly waterfall, eventually itera- incremental development with short form of the tive and incremental with long iterations development iterations 3.4 Summary of Agile Methodologies The reason for this section is to see that Extreme Programming is not the only Agile methodology. Extreme Programming is the best known and the most used methodology, but there exist others, which are important too. We briefly describe some chosen methodologies such as SCRUM, Feature-driven development, Crystal methodology, Lean development and Test-driven development and in the end of the section we will describe the idea of Ex- treme Programming.

3.4.1 SCRUM Scrum is an iterative and incremental development framework. Development begins with the definition of the goals, which are called Product Backlog. Development is divided into iterations/sprints and each sprint has duration from 1 to 4 weeks. In every sprint there are scheduled the tasks to be finished – Sprint Backlog. On each Sprint day communication meetings are organized – Daily Scrum. Daily Scrum starts precisely on time at the same location and time every day. The length of the Daily Scrum is up to 15 minutes. Each team member answers 3 questions: What have you done since yesterday? What are you planning to do today? Any impediments? These questions can identify the threats, which can slow down the project. At the end of the sprint the functional prototype is presented to the customer and he/she provides the feedback. The last point of the sprint is the Sprint Retros- pective, where the whole sprint is evaluated [64, 65].

Two questions are answered during Sprint Retrospective [65]:  What went well during the sprint?  What could be improved in the next sprint?

In the Scrum development process three primary roles are identified [65]: Product Owner: determine what needs to be done during next Sprint. Development Team: produce what is needed in the Sprint and demonstrate what they have done, product owner determines what to produce next Sprint. Scrum Master: control and improve the process of development.

Figure 3.2: Scheme of the Scrum development process [64]

3.4.2 Feature-driven Development Feature-driven development (FDD) emerges from iterative and incremental basis. FDD is driven from a client-value functionality perspective and its main purpose is to deliver working software repeatedly in any time manner. FDD methodology brings to the Agile software development the Object Oriented modeling (objects and its relationship is described). Development consists of 5 activities: developing the overall model, building the feature list, planning by feature, designing by feature and building by feature. First 3 activities are sequential and the remaining two are iterative. The project starts with making the global model, which is converted to the feature list. Each feature list is planned and de- signed. Features are divided into smaller parts and each part is given to the suitable person (to be implemented) based on task type and personal skills. Planning is based on the full list of features and starts from the hardest estimated tasks. Iterations are short (about 2 weeks) and the preliminary solution is presented to the customer [66].

3.4.3 Lean Development Lean development methodology resulted from the Lean manufacturing, which was developed by the Toyota (Japan car producing company). Main purpose of lean development is to identify and eliminate wasting. It is typical for the Lean development that any goal other than the creation of value for the end customer is considered to be wasteful [71].

Lean development is based on 7 principles [71]:  eliminate waste (do not create what customer does not need/want),  amplify learning (without feedback we cannot learn and improve),  decide as late as possible (changing requirements are so often, that they becomes expected, majority of decisions is done as late as possible),  deliver as fast as possible (faster delivery = faster receiving of the feedback, customer can start to using some functionality),  empower the team (decisions should be made by developers, let people do their own job),  build integrity in (components work together as a whole, refactoring can be used),  see the whole (consider the entire project, create an organization culture able to constant improvement).

Lean methodology tries to deliver to the customer a perfect product, which fulfils all the requirements. This approach is usually used when developing systems with frequent/large change of requirements. Lean development is usually used as a part of other Agile methodologies [72].

3.4.4 Test-driven Development The main idea of Test-driven development (TDD) is to pay attention to the process of testing. TDD is not exactly the methodology, it is rather more a programming technique. The goal is to define precise tests that should find defects in developed software. TDD combines the test-first technique and refactoring. The test-first technique steps are: adding the test, if the test passes, than the new test is written, if the test fail, than it is changed until the test passes. Refactoring is the code editing without a change of functionality. In the refactoring process is the code cleaned up. One or more test cases are written before adding a new code. After completion of all tests the process of implementation starts [67].

TDD cycle has five steps [68]:  add a test (each feature creation starts with the test, the test will fail at this moment),  run all tests (all tests should fail at this moment),  write a code (at this moment the process of creating functionality starts),  run the tests (if the tests pass, code fulfils tested requirements, if not, the previous step is repeated),  refactor code (duplicities removing, variable and method names correction etc).

Debugging is eliminated, because all bugs are replaced by the test cases. It is important to create effective tests with the clear output data. A successful test is one step to the creation of a functional software. To the customer is delivered software, which do not need to be more tested. Disadvantages of this approach are high required level of knowledge and continuous supervision. There is a possibility to integrate TDD in another methodology [67, 68].

3.4.5 Extreme Programming Extreme Programming (XP) is one of the most famous and used Agile methodologies. It is a young methodology and it did not result from any approach used in other branch. The methodology is called "Extreme Programming" because it takes beneficial elements of traditional software development to "extreme" level. It is an iterative and incremental development methodology. Methodology is useful for projects with ambiguous/changing require-

ments. The ideal size of the team is from 2 to 10 developers, so XP is proper for small teams. XP facilitates a responds to changing requirements during the whole process of the development. XP provides frequent releases in short development cycles. XP do not specify exact steps, but it can be adapted according to the requirements and possibilities of the team. XP can be a very contributive methodology, when the team of developers has previous experience and knows well what the product owner wants. XP can be combined with other methodologies [69, 70].

3.4.5.1 Values and Drivers XP is not exactly a set of rules, rather it is a way to work in harmony with your personal and corporate values. There are 5 values of XP and they try to bring down the cost of making changes. Values are supported by 12 rules/principles (see Section 3.4.5.3) [70].

XP recognizes 5 values (initially there were 4) [70]:  communication – communication is very important (both in the team and with the customer), block in communication can cause problems,  simplicity – create the simplest possible solution which can work. Functionality can be added later (better to pay for it in the future, because it do not have to be required),  feedback – provides information about the state of the development process, feedback can be from the system (unit tests), customer (acceptance tests) and team (planning game),  courage – for example, courage to remove improper source code, also the persistence of programmers when solving complex problems etc.,  respect – respect for others and also self-respect, programmers should know what other team members do.

3.4.5.2 Roles None of the roles listed below have to be performed by just one individual. All team members contribute in any possible way.

Roles existing in XP projects [69]:  Customer is a business representative. He/she provides the requirements, sets the priorities (what to do first) and steers the project. It is beneficial, when the customer is a real end user (knows the domain).  Programmer/Developer is responsible for the creation and delivery of the software.

 Tester helps to the customer with the creation of acceptance tests.  Analyst can help to the customer with the requirement specification.  Coach helps and supports the team. Coach facilitates the whole process of software development.  Manager provides resources, ensures external communication and coordinates project activities.

3.4.5.3 Practices XP contains 12 basic principles leading to creation of the software product of a high quality.

12 principles of XP [69]:  Pair Programming – is one of the key characteristics of XP. Two programmers work together at one workstation. Pair programming is properly described in the following chapter (chapter 4).  Planning Game – the whole team is involved in creating of the plan. There are two main planning steps. The first is Release Planning. During Release Planning, the customer presents features and programmers estimate their difficulty. With the knowledge of cost estimates and importance of the features the project plan can be created. The second step of planning is Iteration Planning. In Iteration Planning, the customer presents expected features for the next iteration. Features are broken down into smaller tasks and their cost is estimated. Based on the amount of work done during previous iteration the current iteration is planned.  Short Releases – XP teams must every iteration release running, tested software delivering business value to the customer. This practice prevents the "big bang" delivery of the software, which was typical for the traditional software development. Customer can use software in any possible way (the best way is release of software to the end users).  Metaphor – XP teams develop the simple story/vision, how the whole system works. All people interested in the project understand the metaphor.  Simple Design – XP team design the system in the simplest possible way. Design suits well for the current functionality and software is ready for the next functionality addition. For the incremental iterative process the design is crucial. Design tasks are in both release and iteration planning.  Test-driven Development – XP teams practice Test-driven development (for further information see Section 3.4.4). First, they produce tests then make it work. Teams produce a code with nearly 100% coverage.

 Refactoring – is a process of removing duplicities, correction of the variable and method names etc. System is restructured, but without the change of the functionality. Refactoring reduces complexity and improves code readability.  Collective Code Ownership – any programmer can change/improve any part of a code at any time. One of the benefits for programmers is the ability to use code that was created by another programmer. Collective code ownership can cause problems, if programming work on a code they do not understand, but XP has two techniques to avoid this: pair programming and programmers testing.  Continuous Integration – the system is integrated and tested several times a day. Developers keep the system fully integrated all the time. This can prevent the integration problems when everything has broken and no one knows why.  40 Hours Working Week – overwork leads to the lassitude and lowers the productivity. People perform the best if they have had a rest.  On-site Customer – the customer is a part of the team through the development process. Customer should be on hand and available to answer the questions. There should be frequent communication between the customer and development team.  Coding Standards – all programmers keep the rules (coding standards), that should help with the communication through the source code. All the source code follows the same coding standards.

In the current chapter we have seen the description of the Agile software development with its characteristics and Agile Manifesto. Furthermore, we have compared Agile and traditional software development to show their differences. In the end of the chapter we have provided the brief description of selected Agile methodologies such as SCRUM, Fea- ture-driven development, Lean development, Test-driven development and Extreme Pro- gramming, which is described in greater detail. For the Extreme Programming we have described its values and drivers, roles and practices. The aim of this chapter was to show the advantages of Agile software development and its applicability to today's projects. Oth- er aims were to show the differences between traditional and Agile software development and also to contextualize the Pair Programming practice (in the description of the Extreme Programming practices).

In the next chapter we will describe the idea of Pair Programming (PP) practice one of the key characteristics of Extreme Programming, which was explained in the previous chapter. In the beginning of the chapter we will describe its roles and practices. Further we will explain terms pair jelling and pair pressure and mention the myths of PP and we will also

analyze the existing empirical studies and derive from them benefits and weaknesses of PP. We will describe the benefits typical for the academic environment (they cannot be identified in the business environment). In the end of the chapter, we will describe PP partner finding in terms of personality type, programming style differences and self-esteem

“The human eye has an almost infinite ca- pacity for not seeing what it does not want to see. Programmers, if left to their own de- vices, will ignore the most glaring errors in their output – errors that anyone else can see in an instant [28].”

4 Pair Programming In this chapter we will describe the idea of Pair Programming (PP), its roles and practices. Then we will briefly describe the terms of spontaneous PP, pair jelling and pair pressure. Furthermore we will mention the most typical myths of PP. Of course, we will mention empirical studies about PP benefits and weaknesses. In the end, we will show the difference between collocated and remote PP and its benefits and weaknesses. 4.1 Introduction and Practices Pair programming is the key characteristic of the Extreme Programming method and a crucial concept during the coding activity. XP recommends that two people work together at one computer workstation (one screen, keyboard and mouse for the pair) to create the design, the algorithm, the code for a story or test. This provides a mechanism for real time problem solving (two heads are often better than one) and real-time quality assurance (the code is reviewed as it is created). It also keeps the developers focused on the problem at hand [3].

In practice, each person takes on a slightly different role. One programmer driver is operating the keyboard, while the other navigator or observer is watching, learning, asking, talking, and making suggestions. In theory, the driver focuses on the "tactical" aspects of completing the current task: the syntax, semantics, and algorithm. The navigator focuses less on that, and more on the "strategic" direction of the work: the test they are trying to get to pass, the technical task to be delivered next, the time elapsed since all the tests were run, the time elapsed since the last repository commit, and the quality of the overall design. The two programmers switch roles frequently [12].

Process of knowledge sharing involves two parties, namely contributor and receiver. Contributor contributes his/her knowledge and transfers it to the receiver. The receiver receives the knowledge and tries to understand it and transform it to his/her own know-

ledge. This scenario is similar to the PP where the navigator plays the role of contributor and the driver is a receiver. Navigator provides suggestion when assisting the driver to solve the problem which is being implemented. Driver uses the given suggestion and combines it with his/her own knowledge to come out with best solution. In business environment there is the motivation to share the knowledge for the reward, in academic environment plays the role of motivation the grade. Sharing the knowledge helps to achieve a good solution [60].

During PP some explicit and mostly tacit knowledge is shared. Explicit knowledge is easy to share, because it can be expressed by words and numbers and representation of Explicit knowledge is easy to understand. On the other hand it is hard to gain tacit knowledge because it is very hard to formalize. Tacit knowledge is about judgement and strategic decision making. The main source of tacit knowledge is experience and thinking. Tacit knowledge is obtained in PP between partners to generate learning, thinking and decision making [60].

Ergonomic considerations are needed, if you want to start with PP. As a base, programmers need to sit comfortably side by side and in front of a large (17-inch minimum) monitor. Small tables are not recommended. The driver sits in front of the keyboard and mouse, navigator must have clean view of the monitor. When they switch roles, they should simply slide the keyboard and mouse from one person to the other. Many programmers prefer to have their own display, keyboard, and mouse attached to one computer. The navigator can easily and immediately take control. Effective communication within a pair and with other pairs is recommended. Programmers need to ask questions, make decisions on things such as integration issues. Workstation layout presented on Figure 4.1 supports both intra and outer pair communication [34].

Figure 4.1: Role model workstation layout [34]

The term used in literature/thesis are solo programming or individual programming which means programming itself, but each solo programming member of the team has the possibility to communicate with or ask other team members. This is the most typical way of software development used in industry.

Spontaneous PP means that developers work in pairs spontaneously, i.e., when they think that it is the most effective [38]. Spontaneous PP was explored in the study [24]: novices use PP more during their first month in the team than veterans. In the second month time spend on PP is dramatically reduced, while veterans use PP continuously about the same percentage.

There exist two types of PP. The first type of PP is traditional, called collocated, where programmers share computer and sit in the same room. The second type is called distributed or remote PP, where programmers work together, but at different geographical places. Distributed PP team can be called a virtual team. A virtual team can be defined as a group of people, who work together towards a common goal, but across time, distance, culture and organizational boundaries [15]. In a virtual team the communication is really important. Poor communication can cause problems like inadequate project visibility and problematic coordination among the team members. The technology used must be robust enough to support distributed development of software, not just to provide communications. As the business environment becomes more global and there is a need of reducing operating costs, the concept of virtual teams is of increasing importance [14].

In PP the knowledge management is one of the important factors. It is not good to have all the knowledge in any area of the system known just for one person. This is very risky, because the loss of such a person or a few individuals can be devastating. By pair rotation, the knowledge is divided and the risk is reduced. With pair rotation people have the possibility to know many of their team members in personal, this can break down many communication barriers. Pairing with different people (not the same person all the time) can improve the knowledge of the whole developed system, programming and tool techniques gained from many different people.

4.4.1 Pair Jelling Many programmers are skeptical of benefits of collaborative work. They are afraid of additional communication, different working habits, programming style and ego. Programmers go through an initial adjustment period in the transition from solitary to collaborative pro-

gramming [22]. The term pair jelling was mentioned in the study [22]. It is the initial period of time when the pair is adjusting. In this period PP is less efficient than in the following periods. In a business environment the adjustment period varies from a few hours to several days, depending on individuals. In an academic environment, the students generally adjusted in the first assignment. For the first assignment, pairs took 60% more programmer hours to complete assignment, but after the adjustment period, this 60% de- creased to a minimum of 15%. At the end of the second assignment all students reported, that they had overcome the urge to grab the mouse and keyboard from the partner. It takes a few clean compiles and pair could feel like: “We just got through our test with no defects!” Partners feel as the one.

In [52] the type of union is described as: A jelled team is, when people are so knit, that the whole is greater than the sum of the parts. The production of such a team is greater than that of the same people working in unjelled form. The enjoyment that people derive from their work is greater than when working on their own. Once a team begins to jell, the probability of success goes up dramatically.

The book [53] explains what happens when a team jells. The members of such a team learn the strength and weakness of each member, so they can prosper from strength and avoid weakness. As the result, the team productivity is higher than the sum of its individuals' productivity.

In the study [13] the problem with low initial productivity was described, when pairs have to rearrange, that slowed pairs down because the new pairs would have to spend time getting in phase with one another (pair jelling) before working effectively.

4.4.2 Pair Pressure PP puts a positive pair pressure on each of the pair. Programmers work harder, because they do not want to let their partner down. Pair programmers are motivated to complete the task during the session, because their shared time is more valuable for them. They do not want to waste each other's time. Pair pressure also improves adherence to procedures and standards. When people work alone, they can surf the web, go to the meetings and lunch when they want, so they are less productive and motivated [34].

In the study [22] one of the students stated: “When I worked on the machine as the driver, I concentrated highly on my work. I wanted to show my talent and quality work to my partner. When I was doing it, I felt more confident. In addition, when I had a person observing my work, I felt that I could depend on him, since this person took care of my work and I could

trust him. If I made any mistakes, he would notice them, and we could have a high-quality product. When I was the non-driver, I proofread everything my partner typed. I felt I had a strong responsibility to prevent any errors in our work. I examined each line of code very carefully, thinking that if there were any defects in our work, it would be my fault. Preventing defects is the most important contribution to the team, and it put a bit of pressure on me.” 4.2 The Myths In this section, based on book [34], we mention the 7 most typical myths of PP. Each myth is briefly described and there is given an argument, why it is not true.

The 7 myths of Pair Programming [34]: Myth 1: It will double the workload with two doing the work one can do. Managers usually say that PP increases the cost of development. Software engineers say, when they sign up to do their tasks, they do not leave extra time for helping other people do their work. There exists an experiment to disproof such a myth. In this experiment 41 students collaborated writing four programs over the period of 16 weeks. When solo and pair programmers completed, the pair programmers finished in about half the time as solo programmers, and there were fewer defects. As the confirmation, that Myth 1 is not true, see Section 4.3.1 Benefits of PP.

Myth 2: I'll never get to work alone. I couldn't stand that! There was a survey about usage of PP. Only 22% said, that they spend 75% or more of their day with PP. The total of 30% said, that they had spend less than half a day with PP. Pro- grammers generally work alone for a certain period of each day. In PP there are schedule problems to deal with: flex hours, sick partner, meetings etc. Advice is: If you do not want to work together all the time, work together on the most complex tasks. Have scheduled "PP hours" in the morning and/or afternoon, this time everyone does PP, no one interrupts. During other hours go to meetings, work on simple tasks, answer the emails etc.

Myth 3: It will work well only with the right partner. It was found, that it will work well with most (not all) partners. There is one problematic kind of person, someone with excess ego. Other kinds of people are able to cooperate with anyone. Exception is pairing of an expert and the novice. The expert needs to be in the role of a mentor and know, that his/her own productivity will be impacted.

Myth 4: Pair programming is good for training. But, once you know what you're doing, it is a waste of time. If you pair two experts to solve a really complex problem, they will combine their expertise and it will bring results, because they are probably experts in a slightly or very different areas. Experienced programmers will fill in many of other's knowledge gaps. This is a great knowledge management strategy, because they will pass tacit knowledge around a team.

Myth 5: I'll never get a credit for doing anything. I'll have to share all the recognition with my partner. Things work best, if each task has an owner. The owner recruits a partner to help with various parts of the task. It works well, if the partner has some related knowledge. But now, much of the time of one programmer is spend on the task, they not own. How can he/she feel good about that? We need to feel satisfaction from our team succeeding (compare to any team sport, individual success means nothing without team success). We are gaining from understanding the system much more and it can help us in our own tasks later.

Myth 6: The navigator finds only syntax mistakes. How boring is that! Compilers can do that better than the humans can anyway. The major benefit of PP is the continual code review, navigator can find syntax mistakes (it is recommended to give the driver more than a nanosecond to fix their own syntax mistakes), but the job of the navigator in much more important. Navigator holds the roadman and is moving the driver to the desired destination.

Myth 7: The only time I ever get any real work done is when I'm alone. Now, I'll never get anything done! Pair Programming would drive me crazy. In the [52] authors described mental flow as a condition of deep (meditative) involvement. You feel euphoria and do not perceive the time and the effort, the work seems to be good. Not all positions need the state of flow to be productive, but in engineering, design, development, writing etc. it is a must. It takes at least 15 minutes of concentration to descend to the flow. After every interruption (phone call, e-mail notification etc.) it takes another 15 minutes to get back into the flow. People are afraid, that they will never reach this flow state when working with another person. But the two operate as one intelligent organism and share joint flow. Pairs prevent each other from flowing down (producing wrong/low- quality core). Pairs turn off e-mail notification, do not surf the web, mute phone ringing, so they prevent the interruption. Pairs reported, that after interruption of one of the partners, the interrupted partner can rejoin the flow faster than 15 minutes. After session the pair

returns to the "normal" mode and handles e-mails, calls etc. The opposite of mental flow is the mental block. Pair partner is able to break through the mental block.

In [52] authors defined Environmental Factor or E-Factor as: 푈푛𝑖푛푡푒푟푢푝푝푒푑 ℎ표푢푟푠 퐸 − 퐹푎푐푡표푟 = 퐵표푑푦 푃푟푒푠푒푛푡 퐻표푢푟푠 The higher E-Factor is, the more productive programmers are. During PP we work very intensively, because we value our time with the partner. Others see us working with someone else, so they leave us alone. As a result, we have bigger blocks of uninterrupted hours. Pair programmers are able to silence their phone, divert their calls etc., have fewer inter- ruptions, and can be more productive (as much as 12 times more). With the pair partner depending on us, we are just less likely to "cheat". 4.3 Empirical Studies We start with the literature review of studies about benefits of PP, then we continue with the weakness of PP for the comparison of plus and minus of the PP practice. We will also describe benefits of PP which are typical for the academic environment. In the section about the weaknesses/problems of PP we provide the brief explanation of every problem and mention the least liked aspects of PP. In the end of this chapter advantages and disadvantages of distributed PP are provided.

Section is composed of empirical studies from various sources. Some of the studies are based on experiments where students are participants. Moreover, the results of such studies are sometimes contradictory. As more valuable are seen studies based on cooperation with IT companies using PP. For more information about used studies see Appendix A. Summary of the Cited Studies, this section provides a summary of the information about used studies.

There is an interesting citation about a value of PP: “Some may question the value of PP if the collaborators do not perform “twice” as well as individuals, at least in the amount of time spend. For instance, if the collaborators did not perform the task in half the time it takes an individual, it would still be more expensive to employ two programmers. However, there are at least two scenarios where some improved performance over what is expected or possible by a single programmer may be the goal: 1, speed up the development, 2, improve software quality [29].”

4.3.1 Benefits This section describes the main benefits of PP practice and is composed of literature review of existing studies. In this first part of this section we mention the best known benefits of PP, in the second part we describe benefits, which are typical for the academic environment (these benefits cannot be identified in business environment). Benefits typical for the academic environment are improved students’ retention, better final Grades, lower instructor’s workload in PP labs and higher student confidence in the solution/program.

There are typically two perspectives about the application of PP: those that are in favour and those that are against. Arguments provided against are [13]:  Managers view programmers as a key resource, and do not want to "waste" by doubling the number of people needed to develop a piece of code.  Programming has traditionally been taught and practiced as a solo activity.  Many experienced programmers are skeptical to a program with another person. Some say their code is "personal," or that another person would only slow them down. Others say working with a partner will cause trouble at coordinating work times or code versions.

Proponents of the PP practice underline instead:  Several experienced programmers prefer working in pairs, making it their preferred programming style.  Seasoned pair programmers describe working in pairs as "more than twice as fast".  Even relative novices contribute to an expert's programming, according to interviews.

In the last several years many studies emerged about benefits of PP. Studies contain claims that PP:  improves/increases productivity [19, 20, 29],  increases job satisfaction/enjoyment [13, 21, 22, 35, 42, 44, 50],  facilitates integration of newcomers/reduces training costs [13, 23, 24],  supports knowledge transfer [13, 19, 23, 27, 41],  reduces defect rate [13, 17, 22, 25],  improves quality of the solution [13, 19, 22, 26, 27, 35, 41, 50],  reduces the time needed for task completion [29, 22, 41],  improves team communication [13, 26, 41, 42],  has positive effect on students’ retention and final grades [29, 30, 32, 35, 41, 42, 50],  reduces instructor’s workload when learning in PP labs [13, 30, 41, 42],

 increases confidence of the students in the solution/program [22, 35, 50],  resource reduction (especially for the academic environment) and the increasing of the team spirit among the students [41].

Some of the benefits are unique for the academic environment and cannot be measured/controlled in business environment. For example, retention rates, grades (course completion rates, pass rates etc.), reduction of instructor’s workload and confidence in the course solution/program. All these benefits are typical for academic environment and they will be examined in this section.

In industry environment, software developers usually spend 30% of the time working alone, 50% of their time working with one another person, and 20% of their time working with two or more people [31]. However, in academic environment students must program alone, and collaboration is considered to be cheating. This time spend working alone is not consistent with the student future professional life, where the collaboration is required. Students recognized that in PP labs they improved communication skills and collaboration. PP labs suit well for real world settings, where people are often randomly matched to collaborate [42]. Engineering students will have to work in the future in a team and their performance could depend more on ability to work in the team than on their technical skills [75].

4.3.1.1 Students’ Retention and Final Grades Beneficial effect of PP on students’ retention rates and final grades were confirmed in studies [29, 30, 32, 35, 41, 42, 50].

Study [30] examined the hypothesis, that students using PP will have equal or higher % of C grade. The hypothesis was supported. Study [42] examined the similar hypothesis, and it was supported only for non computer science majors.

In the study [32] about 600 students cooperated, some used PP some solo programming. Students using PP produced better programs, completed the course with better rates and performed as well at the final exam as solo programming students.

Study [35] based on the participation of 555 students during four seasons showed, that average completion rates of PP students were 90,8% and 80,4% for solos. Pass rates 72,3% for PP and 62,8 for solos and final exams had no significant difference.

In the study [41] the short duration exercises were used (3 hours without a break and under the supervision of a laboratory instructor). With the use of short exercises the instruc-

tor had a full control of students and students. When long duration study is used, some students reported that it had been difficult to find time to meet and work together. After exercise completion students write a control test (tests were written separately, not in pairs). Study found out that PP improve design ability (11% better design marks), reduced time to finish a task (19% time reduction) and increased knowledge and programming skills (13% better test marks).

In the study [50] data was collected from 554 students. The authors found that PP improved course completion and consequently course pass rates.

4.3.1.2 Instructor’s Workload As was mentioned in Section 3.6.2.1 about the benefits of PP, one of the benefits was, that PP reduced instructor’s workload when learning in PP labs, this was supported by studies [13, 30, 41, 42].

Students are better served by the instructors in paired labs, than in solo labs. Average 1,14 “give ups” (when many students need answer the question from instructor, those who wait long would often give up and stay with an unresolved problem) in PP labs and 3,17 in solo labs. Using PP leads to reduced grading to half, students have fewer questions and better quality questions (about advanced issues, for example how to improve their algorithm) and the overall instructor effort is reduced [30].

Study [42] mentions that solo programming lab sessions were quiet and frustrating. Fre- quently students needed to wait 10-30 minutes to ask question (sometimes simple one), during waiting time were students unproductive. PP labs were vocal/interactive and students seemed to help each other resolve questions. PP students were self-sufficient, so the instructor had more time. Pairs who needed help, got help faster and had little downtime. As a result, instructors had more time and found their job satisfying and rewarding. Also student’s effort and willingness to learn seemed higher than in traditional sessions.

4.3.1.3 Confidence in the Solution/Program Confidence in the solution/program is one the important factors that influence the enjoyment of the work. Hypothesis, that PP has a positive effect on student confidence in the solution/program was supported in the studies [22, 35, 50].

In the study [35] based on student participation, one of the tested aspects was the confidence of the students in their solution. After completion of each program, students were asked to answer the questions about the program, one of the questions was: "On the scale

0-100%, how confident are you in your solution to this assignment?" Results were, that average confidence in the solution of PP students was 89,4% and of the solo programming students 72,2%.

Study [22] used online survey to evaluate the confidence of the students in their solution. 95% of the students stated, that they were more confident when they used PP. Study [50] also examined the confidence in the solution of the students, even with the respect of the gender. Students answered questions about confidence in the solution after each assignment on the scale 0-100% (same scale as in the study [35]). Results of the study are in Fig- ure 4.2.

Figure 4.2: Confidence in program solutions: closing one gender gap [50] There are also studies that provide contradictory results:  In the study [22] there was mentioned that developers using PP spend in average 40-50% development the time needed by solo programmers, but in the studies [20, 29] pair programmers spend 71-77% time in comparison with solo programming.  In studies [13, 17, 22, 25] the positive effect of PP on the reduction of defects was mentioned, but the study [33] did not supported this hypothesis. In one of the case the defect density of PP and solo programming was the same.  In the studies [19, 20] there was mentioned the positive effect of PP to improve/increase productivity, but study [33] found no indications towards the supe- rior productivity of PP.

 In study [13] and in the book [34] better following of coding standards was mentioned, but it was not supported in the study [33] based on the results of case studies.

4.3.2 Weakness/Problems In the previous section we saw that PP can bring benefits, even though the results are not definite and there are also contradictory observations. This section is about the other side of PP, about its weakness and the least liked aspects of this agile technique. We will mention selected results from various studies of PP.

In the Microsoft Corporation there was a wide survey [27] about PP with participation of Microsoft’s employees. One of the questions was about top problems of PP. Most of the respondents answered: cost efficiency, scheduling problems, personality clash, disagreements, skill differences, programming styles differences, difficulties to find a partner, personal style differences and odd number of developers. The study [18] based on students divided into pairs and solos found the least liked aspects of PP: having to share the screen, keyboard and mouse, and having to switch between the roles of driver and navigator. Another study [30] based on beginning students of introductory programming course presents PP partner compatibility as the biggest problem.

The study [31] based on students from introductory programming course found the most common reasons of stress in pairs: differences in programming experience, problems with reliability, different programming styles and problems with scheduling. Problems with scheduling in PP are mentioned in the study [37] as it was difficult to find common time for the developers to meet and cooperate.

PP problems include noise and facility considerations too. Effective pair has to communicate, silence is a signal for a dysfunctional pair. This can be seen as a problem for the solo programmers sitting near the pairs, so the facility changes are required. Facility changes are also required for the proper function of PP. Concentration in pair is difficult for some personality types. Some people prefer working alone, when they are investigating something, making a little prototype, 'operating in creative mode'. But they must be open to re- vise their ideas created in isolation with their partner [34].

There can be a feeling, that a pair cannot be wrong, you are confident to your solution so much, but the overconfidence is also one of the problems of PP, especially if your organization does not do reviews. Detailed testing can lower the danger of overconfidence. Once you start PP, you feel it is hard to program alone. One of the students said: "When I program

alone, I feel as if lost half my brain." This paradoxy can be called dependency. So what to do, if your partner is not available? Think, if there is another part of the system, you can work on with another partner. If there is anyone you can help, you have programmed alone before, so you can do it again [34].

We discussed pair pressure in Section 3.6.1, pairs are motivated to complete their tasks during the session and do not want to let their partner down. These are the benefits, but there exists a negative part of Pair Pressure, too. It is called rushing. Pairs rush to complete the task and do a poor job. They must slow down and do it correctly together [34].

Another weakness/problem of PP is, when you have an odd number of people in the team. One of the rules of XP is that all production code should be done by a pair, but many programmers feel a little claustrophobic, when they are locked in a pair all the time, so they will appreciate to work alone. When some code is produced by solo developer, you should ensure that code is reviewed before being checked in. If developers feel that they need a break, they can volunteer to be an odd person [73]. Solution for such a situation was mentioned in the book [34]. There was a team of 3 people: 2 worked in a pair and 1 person alone. Every 10 minutes the third person rolled a chair to the pair to consult about code. They had a short "triple brainstorming" and then solo developer rolled away. It was en- couraged to change a solo programming person daily or more frequent.

There is always a threat, that one of the students in pair (the better one), will do all the work and the students will obtain the same grade. Study [43] tried to avoid this, with a web-based peer evaluation tool, where students assigned 0 to 20 points to their partner for each of the five questions (see Section 5.2 What has been done to solve the problems). In accordance with assigned points, the partner will get a score of 0%–100%. This score influenced the final assigned points of the student in the whole course.

One of the problems that arise in PP is collocation. If partners cannot sit side-by-side, for example one pair programmer has home office (working from the home) [34]. The solution of such a problem can be distributed PP.

Discussing collocated and distributed PP, previous studies (see 3.6.2.1) have indicated that pair programming is better than individual programming when the pairs are physical- ly collocated. However, important questions arise: How effective is PP if the pairs are geo- graphically distributed? An experiment was conducted to compare the different working arrangements of student teams. The teams were both collocated and in distributed environments [14].

The goal of the experiment was not to test if a remote pair could be as efficient as a collocated one, but to simply see if the programming pairs could work well enough to make functional software in reasonable time. The pairs reported that after a few early sessions in which they were learning the platform behavior, they felt comfortable and naturally by coding with this technology. Experiment details: the class had 132 students, teams were composed of 2-4 students. The two metrics used for the analysis were productivity, in terms of lines of a code per hour, and quality, in terms of the grades obtained by the students for the project. Students’ feedback formed an important third input for the experiment [14].

The results of the experiment indicate that it is feasible to develop software using distributed PP, and that the resulting software is comparable to software developed in collocated or virtual teams. From this experiment we found that effective remote teaming could be done with the PC sharing software and audio support [14].

Advantages of distributed PP (dPP) over collocated PP [16]:  The visibility is improved in comparison with collocated PP at a single PC/monitor, since each dPP participant has a screen.  The navigating dPP participant can use the PC to search the Web for resources  No office changing or traveling is needed to meet one’s partner. Work on other projects can continue until dPP appointment time.  Although not tested, meetings are possible when on trips, out of town, etc.  Pairs are forced to keep electronic copies and records of their work and ideas. For example, instead of drawing on a physical whiteboard, the participants used Net- Meeting’s whiteboard. This ensured they would be able to go back and look at earlier plans.  Pair members are less likely to start conversations off topic, meetings are almost completely focused on the task. The computer is the medium for all exchanges, and participants cannot turn away from their computers and chat one-on-one.

Disadvantages of distributed PP compared to collocated PP [16]:  Users cannot point, making it difficult to describe where a problem is.  A problem with one computer forces both to stop working. This theoretically doubles the MTTF (Mean Time to Failure) over using a single computer.  Pair members cannot see facial expressions. Webcams are too small, too limited in frame rate, and too expensive in bandwidth consumption to help here.

 Passers-by often do not know a programmer is in a dPP session, and will enter an office and begin a conversation (a specific sign must be used to tell this if one does not want a shut door).  Lack of physical proximity means large amounts of time spend on verbal explana- tions that could rapidly be resolved by a visual diagrams, although NetMeeting has a whiteboard, it is cumbersome to use and does not adequately solve this problem. 4.3.3 Partner Finding One of the problems reported in many studies about PP, was partner’s compatibility, when pairs were matched randomly. Well known argument of PP is: "Two heads are better than one." But in reality it is not always true. Developers must put more effort to create mutual understanding between them to make good decisions. This is more complex, when partners have different abilities and incompatible personality types [60]. In the study [49] authors found that attitudes of female students did not differ significantly from that of male students. Authors also identified that students prefered to do PP with the partners of the same or higher academic level and with the partners of the same gender. In the study [27] developers preferred partners with following attributes (in order of popularity): complementary skills, flexibility, good communications, smart, personable, at least the same skills, strong programmer, better skills than me, able to focus, knowledge, acceptable personal hygiene and quick typing.

4.3.3.1 Personality type The Myers Briggs Type Indicator (MBTI) is often used in studies to classify people into one of the sixteen personality types based on the dimensions such as introversion (extraversion or introversion), intuition (sensing or intuition), feeling (thinking or feeling), and judgment (judging or perceiving) [46]. MBTI was proven by hundreds of studies over past 40 years that it was a valid and reliable instrument to discover personality types [74].

Study [43] examined hypothesis that pairs were more compatible if developers with different personality types were grouped. This was confiremed in CS1 class. However, this same result was not found in the SE class.

4.3.3.2 Skill level and experience Developers preferred working with a partner who matched their own qualifications and experiences. If the pairs’ skill level and experience are not in a match, the stronger partners tended to complain of carrying an unbalanced share of the work. On an open- ended survey, some of the stronger partners also suggested that such pairs were more

productive when the stronger of the pairs was the driver [47]. However, this type of pairing relationship is not advisable because the weaker developer feels disjoint and unimportant [34].

In the [48] psychologist Lev Vygotsky explained student’s preference towards working with partners of similar skill level. Vygotsky defined the Zone of Proximal Development (ZPD) as the range of ability of a student with and without the assistance of a teacher or a more capable peer. The lower end of the range is the student’s ability without assistance, the higher end of the range is the student’s ability with assistance. If the teacher/peer works too far above the student’s ZPD, the student will become confused and no intellectual growth will occur. If the teacher/peer works too far below the student’s ZPD, the student will not be challenged enough and will stagnate.

In the study [43] the students were asked to evaluate their perception of the partner’s skill level and their joint compatibility on an ordinal scale: 1. Assess the technical competency of your partner relative to yourself [better, about the same, weaker]. 2. Assess how compatible you and your partner were [very compatible, OK, not compatible].

Study [43] found that here exists a significant positive relationship between compatibility and the same perceived skill level of the partners in three tested computer science courses.

4.3.3.3 Self-esteem A survey-based study [44] with more than 60 students (first-year students of an introductory programming course that used PP) involved, examined the relationship between the self-reported self-esteem and the performance. The results showed that students with lower self-esteem liked PP more than students with higher self-esteem. There was also some evidence that high self-esteem students disliked PP the most, when paired with a low self- esteem partner. Additionally, students seemed to do their best work, when paired with students of similar self-esteem.

In the study [43] students were asked to evaluate themselves on a scale 1 (I do not like programming) to 9 (I love to program) and the compatibility of pairs was measured. There was a weak negative association between the same self-esteem of a pair and the compatibility of students.

Term code warrior comes from a paper [45]. Code warriors see themselves fighting with the enemy compiler, forcing it to assent to their glorious code and to produce a program that obeys their every desire. They are often not as advanced as they think. Weber-Wulff concentrates on how these students can be led to understanding that large software systems require documentation, deep understanding, and team programming.

Term code-a-phobe was mentioned in the study [44]. Code-a-phobes have little satisfaction from programming. They frequently report “hating the programming” or “being hope- less at programming”. These students often aspire to jobs that do not require highly developed programming skills. This seems to be an unfortunately common phenomenon among students.

Study [44] tested the collaboration and enjoyment of the work, based on warrior-phobe scale. On the study participated first year students (more than 60 students) with previous programming experience. Students were asked to complete a web questionnaire. Based on the results they were evaluated on a warrior-phobe scale from 1 to 9. Values 7-9 were for a code warrior and 1-3 for a code-a-phobe, other values were for the middle group. Students were then asked to complete a simple program using PP. They were grouped according to warrior-phobe scale. First, the same types were grouped and in the second attempt opposite types. Students were asked to write about their reaction about cooperation and enjoyment.

Students with similar attainment placed themselves at very different places on the warrior- phobe scale. Students with less self-confidence seemed to enjoy PP the most. The students identified as warriors liked PP the least. Warriors like PP even less when they are paired with phobes and those students produce their best work when paired with students of similar or not very different levels of confidence, but the results of the study are not statistically significant because of a small sample size [44].

In the current chapter we have described pair programming (PP), its practices and important terms such as driver, navigator, pair jelling, pair pressure and the difference between collocated and distributed PP. Further we have mentioned the myths of PP and analyzed the existing empirical studies and derive from them benefits and weaknesses of PP. We will mention benefits of PP for education (these benefits cannot be identified in the business environment). In the last section of this chapter we have elaborated problems with the partner finding in terms of personality type, skill and experience and self-esteem. Section about partner finding can be used both in academic and business environment. The main

aim of this chapter was to identify the weaknesses/problems of PP practice. These identified problems will be elaborated in Chapter 6 (Problems with the Learnability of Pair Pro- gramming), their effect on learnability will be analyzed and existing solutions for them will be provided.

In the next chapter we will describe the problems of PP and show their impact on the ler- nability of PP. In the end of the next chapter we will provide existing solutions for the identified problems.

5 Learnability of Pair Programming Based on the review that was carried out at the current literature we have identified several issues/problems related to PP and its learnability. The main identified problems of PP are pair jelling (initial period of the time when the pair is adjusting), problems with reliability, scheduling problems, noise and facility considerations, odd number of team members and partner compatibility. The reasons for the problems with partner compatibility are usually personality clash, skill level and experience, programming style differences and self-esteem. 5.1 Current Problems In this section we will describe the impact of the identified problems on the PP and what these problems have to do with learnability. Each of the problems will be introduced and its effect on learnability will be provided. All the problems can slow down the process of learning PP. Problems can also cause that the PP will not be as enjoyable as it can normally be.

Pair jelling is the initial period of time when the pair is adjusting. Pair jelling can cause less efficiency at the adjusting period than in the following periods. Developers are skeptic of benefits from collaborative work. They are afraid of additional communication, other’s work habits, different programming style and ego. For the first assignment pairs took approximately 60% more programming hours to complete the task [22]. When the pair is jelled, then the whole is greater than the sum of the parts, production is great, people enjoy the work and the probability of success goes up dramatically [52]. Members learn strengths and weakness of each member [53].  During pair jelling pairs are not productive at the beginning, so they cannot learn the PP effectively. If this period will be reduced to the minimum the effectivity of learning of PP at the beginning of learning process will be increased.

Problems with reliability can cause that just one of the partners is conforming to the task completion. The better one will do all the work and the students will obtain the same grade. This problem can be identified in the academic environment. Students often do not report, that the partner is not conforming and they do the whole work alone.

 There is an obvious problem with learnability of PP, because one of the students does nothing so he/she does not learn and the other student does the whole work, but does not use PP. The hardworking student will not enjoy the work and he/she can also be frustrated to use PP in the future.

Problems with the scheduling can be found both in academic and business environment. Problems with scheduling can be described as it is difficult to find common time for the developers to meet and cooperate [37]. Scheduling problems can be caused by partner sickness, meetings...  If the developers are not able to cooperate they will not improve their abilities of PP.

Effective pairs have to communicate, silence is a signal of a dysfunctional pair. But this can be seen as a problem for programmers sitting near to the speaking pair. This problem is called noise and facility considerations. Noise can cause problems with concentration.  Without the concentration pair programmers are not able to produce correct code and also to improve their programming abilities.

Odd number of team members are typical for both academic and business environment. In business environment it is typical for the small teams. It is sometimes impossible to avoid such a situation.  The effect of this problem to the learnability of PP is obvious. It is hard to learn PP, if you are programming alone.

Problems with the partner compatibility are one of the most common problems and they can damage the whole pair cooperation. One of the common reasons of partner incompatibility is personality clash. If two people with the incompatible personality type are paired, this can be a real problem. Programmers will be angry, unsatisfied and the work progress will be poor. Different skill level and experiences can cause an unbalanced share of the work [47]. The weaker student feels disjoint and unimportant [34]. Differences in programming style can cause disagreements of the pair and again lower productivity. Dif- ferent self-esteem of the partners can also influence the compatibility. High self-esteem students disliked PP the most, when paired with a low self-esteem partner. Additionally, students seemed to do their best work, when paired with students of similar levels of self- esteem [44]. In the study [43], authors found a weak negative association between the same self-esteem of a pair and the compatibility of students.  Cooperation of the incompatible personality types can lower the pair productivity, work satisfaction and also slow down the learning process. The different skill and

experience level can cause the pair incompatibility, no intellectual growth of stagna- tion. Differences in programming style can cause the ambiguity and disagreement. This will lower the productivity and cause the lowering of the learnability. Differ- ences in self-esteem of the partners influence the compatibility of the pair too. 5.2 Current State In this section we will provide existing solutions for the problems of PP involving the learnability. These solutions were derived from the literature review provided in Chapter 4.

Problems with reliability:  In the study [31] authors presented one of the possible solutions called pair within section. This can show the absence of role switching and missing partner. Also the authors recommended to make sections mandatory (if students cannot attend the lesson, they had to write email to the teacher in advance). This solution can only solve the situation in the programming labs, but not when the students are finishing their tasks out of the labs.  The study [43] tried to avoid this, with a web-based peer evaluation tool. Students assigned points to their partner on each of the five questions (see the questions below). Based on the assigned points the partner will get a score of 0%–100%. This score influenced the final assigned points of the student in the whole course.

Questions to evaluate partner were: 1. Did the student read the lab assignment and preparatory materials before coming to the schedule lab? 2. Did the student do his/her fair share of the work? 3. Did the student cooperatively follow the PP model (rotating roles of driver and navigator)? 4. Did the student make contributions to the completion of the lab assignment? 5. Did the student cooperate?

Scheduling problems:  When programmers are not able to meet, they can use distributed PP. In the study [14] authors found that development with distributed PP and collocated PP is comparable in terms of productivity and quality. Collocated teams do not produce better results than distributed ones. Students’ feedback indicates that distributed PP support teamwork and communication in a virtual team. But this was not provided as a solution for the problems with the scheduling.

Noise and facility considerations:  In the book [34] some recommendations about facility considerations are provided. Programmers need to sit comfortably side by side and in front of a large (17-inch minimum) monitor. Small tables are not recommended. The driver sits in front of the keyboard/mouse and the navigator must have a clean view of the monitor. When they switch roles, they should simply slide the keyboard and mouse from one person to the other.

Odd number of persons in the team/class:  Solution for the academic environment was provided in the study [31]: some students have a more flexible schedule than others so they could switch section (if it is necessary). Also in this study it was recommended to make sections mandatory (students must inform the instructor in advance if they will not visit the section) and to pair within the section. So if the students announce their absence in advance, instructor can plan to pair other students. If students paired within the section, the instructor saw at the beginning of the lesson that some students were missing (there is still a possibility that they are missing and do not inform the instructor) and the instructor can again create a new pair.

Partner compatibility  One of the possible solutions was provided in the book [34], where student write “test run” to control their compatibility.  Personality clash: in the study [43] the influence of personality type on the compatibility of the pair was examined. Authors used Myers-Briggs Type Indicator (MBTI) to identify personality type of developers and tried not to combine people with the same MBTI type.  Skill level and experience: in the study [43] the authors found that here exists a significant positive relationship between compatibility and the same perceived skill level of the partners in three tested computer science courses.  Programming style differences: as a solution for this problem one of the guidelines from the study [43] can be used – a set required coding standards for the PP sessions.  Self-esteem: In the study [43] students were asked to evaluate themselves on a scale 1 (I do not like programming) to 9 (I love to program). This approach can be used to prevent a combination of the two people with the different self-esteem.

 It is recommended to combine students of the same gender and academic achievement [49].

In the current chapter we have described the problems with the learnability of the PP and showed their impact on the learnability of PP. The elaborated problems were pair jelling, problems with reliability, scheduling problems, noise and facility considerations, odd number of people in the team and partner compatibility. Problems with the compatibility were elaborated in terms of personality clash, skill level and experience, programming style differences and self-esteem). In the end of this chapter we have provided existing solutions for the identified problems.

We have identified problems of PP influencing the learnability and also the open problems. Now we need to propose solutions for these open problems. It is important to have a learning model to frame a discussion and to solve the problems we are addressing. Learning model could help us to ensure that our framework presents all the necessary development for students or developers. Furthermore it could help us to think about the process of learning in a systematic way and to plan actions to be followed during the process of learning. When developing learning instructions and evaluating student’s performance, we have to take into account the existence of different levels of learning/knowledge. If instructors do not focus on all levels of knowledge, they can omit important learning aspects. For example, the instructor can teach practical information, but can never teach students how to apply them. Instructions in different levels of knowledge must be presented in different ways [77].

As an educational model we have chosen the Bloom’s taxonomy of learning domains (BT). Why to use BT? BT is quite a well understood framework, there is not much need to explain when it is applied. BT has an advantage of maturity, because it has been confirmed in the last 40 years that it is a valid educational principle [58]. It provides the structure for planning, designing, assessing and evaluating training and learning effectiveness. It can be used as a checklist to ensure that training is planned to deliver all the necessary development for students or learners. It can be also used as a template by which we can assess the validity and coverage of any training, course or a programmer [51]. The levels of the BT are hierar- chical – each level must be mastered before progressing to the next.

Original BT has some disadvantages: it was not created to be used outside of teaching environment, not all levels of BT can be applied to all SE topics, cognitive domain should be more precise at the application level (engineering knowledge must be operational) [54].

There exist solutions for these problems, because BT was reworked/adapted or combined with other methods to be compatible to the use in SE context. Further there exist several examples of the usage of BT in software engineering context.

In the next chapter we will describe the Bloom’s Taxonomy (BT) of learning domains’. BT will be used in the chapter 7 (Proposed Solution) to ensure, that training is planned to deliver all the necessary development. We will describe the idea of the BT, its domains and the cognitive domain (the most important for the use in SE context) more in detail. Fur- thermore we will describe the BT adapted to the SE context, possibilities of BT to improve the educational course, show the difference between the usage of the BT by the novices/experts and in the end, we will provide assessment possibilities of the BT in terms of individual assessment and assignment assessment.

6. Bloom’s Taxonomy of Learning Domains The taxonomy is a set of classification principles or a structure. The domain means a category. In 1956, Dr. Benjamin S. Bloom and a group of educational psychologists published the Bloom’s Taxonomy of learning domains (BT). BT has been extended/reworked by many other scientists. The model is useful for the explanation and application of learning objectives, teaching and training methods and measurement of learning outcomes. It also provides a structure for planning, designing, assessing and evaluating training and learning effectiveness. It can be used as a checklist to ensure, that training is planned to deliver all the necessary development for students or learners. It can also be used as a template by which we can assess the validity and coverage of any training, course or a programmer [51].

The taxonomy consists of three parts/overlapping domains. Each of the three domains has categories/levels ordered to a degree of difficulty. Categories are levels of learning development, levels increase in difficulty. Each category must be mastered, before progressing to the next one [51].

Bloom’s Taxonomy domains [51]: 1, Cognitive domain1:  intellectual capability, knowledge or 'think',  categories: knowledge/recall, comprehension, application, analysis, synthesis, evaluation. 2, Affective domain:  categories: feelings, emotions and behaviour, attitude or 'feel',  receive, respond, value, organize or conceptualize values, internalize or characterize values,  domain is assessed by members of the group and the facilitator during group work activities, members observe ways in which students react to another one [59]. 3, Psychomotor domain:  manual and physical skills, skills or 'do',  categories: imitation, manipulation, precision, articulation, naturalization,

1 Cognitive domain is the most important for the use in SE context, other two domains (affective and psychomotor) are usually neglected, because they are hard to assess [59].

 the physical activities/contributions of group members could be assessed [59].

The learner should benefit from the development of knowledge and intellect (Cognitive Domain), attitude and beliefs (Affective Domain), and the ability to put physical and bodily skills into effect (Psychomotor Domain) [51]. We can see BT levels at Table 6.1.

Table 6.1: Bloom’s Taxonomy domains in detail [51]

Cognitive Affective Psychomotor

knowledge attitude skills 1. Receive 1. Imitation 1. Recall data (awareness) (copy) 2. Respond 2. Manipulation 2. Understand (react) (follow instructions) 3. Value 3. Apply (use) 3. Develop Precision (understand, act) 4. Analyze 4. Organize personal 4. Articulation (structure/elements) value system (combine, integrate related skills) 5. Internalize value 5. Synthesize 5. Naturalization system (create/build) (automate, become an expert) (adopt behaviour) 6. Evaluate

(assess, judge)

There exists a debate if synthesis layer is below the evaluation layer, but that discussion is beyond the scope of this thesis. In the thesis we consider synthesis and evaluation as independent levels [61]. 6.1 Bloom’s Taxonomy in Software Engineering There are problems with the application of BT in software engineering (SE) context. Prob- lems that were recognized are: BT was not originally created to be used outside of teaching environment, taxonomy may not be suitable for the use in SE context, not all levels of BT can be applied to all topics (some topics use just basic levels), cognitive domain should be more precise at the application level (engineering knowledge must be operational to be useful) [54]. These are the reasons, why the BT was reworked/adapted or combined with other methods to be compatible to the use in SE context. BT has an important advantage such as maturity, because it has been confirmed in the last 40 years that it is a valid educa-

tional principle. Secondly, the assessment mechanism is used to identify the different types of knowledge [58].

One of the adaptations of the BT was two dimensional approach, which added another dimension to the BT cognitive domain. New dimension was a categorization of knowledge areas, namely: A. Concept, Framework and Reference model, B. Principle and Theory, C. Method and Technique, D. Measurement and Assessment, E. Application and Case and F. Tool [54].

In Table 6.2 you can see the expansion of the application level and omission of the evaluation level, because the target of the approach is to match the taxonomy for the undergraduate students. Each knowledge item can be identified by the category and required competence level. For example, in Table 6.2 Software Design Basic Concept was mapped to category A and to the level 2 (comprehension). Software Design Notations can be mapped to category C and level 3 (application) [54]. Table 6.2: Two dimensional approach, knowledge area framework example [54] Competence Level (1) (2) (3) (4) (5) (6) I can I can use for I can I I have assess I understand practical appli- research know tried I can cation & Improve teach Category RE. CO. AP1. AP2. AN. SY. Of Knowledge Area Concept Software Design A Framework Basic Concept Reference Model Principle B Modularity Theory Software Method Requirements C Design Technique Design, Test Notations Measurement D Quality Attribute Assessment Application Design E Case Pattern F Tool

In the engineering taxonomy additional parts were added about operational perspective to the original BT (not to all levels) [54]:  Level 3 Application – performs or is capable of performing basic work in an area, generally under supervision.

 Level 4 Analysis – performs effective, independent work in an area, serves as a role model for less expert professionals, and occasionally coaches others.  Level 5 Synthesis – performs exemplary work in an area. Regularly coach other professionals and provides project-level and possibly organization-wide leadership.  Level 6 Evaluation – performs reference work in an area and has deep experience across multiple projects. 6.2 Course Improvement The process of application of BT starts with the identification/selection of the topics to be covered (from collection of sources, earlier syllabus or textbooks) and decides for each topic which level of mastery should all students achieve upon completion of the course. The instructor creates a useful document which can be used to design lectures (following the needs of the students), test questions and assignments examining the student at a required level [55].

Example of application of the BT with the “for loop” ("fl.") topic. In CS1 course the instructor will want learners to recite the definition of "fl." (recall), also to explain a "fl." (comprehension) and to trace code containing a "fl." (application). Probably students should be able to decompose a "fl." to basic parts and explain how these relate to a problem specification (analysis). Also students should be able to design a "fl." from the scratch (synthesis). But students will not be expected to evaluate the quality of a "fl." (evaluation). So the CS1 course will introduce "fl." topic at the synthesis level [55]. Another example of expected level of knowledge for the topic debugging was examined in the case study [61]. The case study showed, that debugging required knowledge of all six levels of BT for students to be able to make the necessary updates [61].

To assess the cognitive difficulty of computing courses, a Bloom rating was created. BT level was assigned to each test question according to required cognitive behavior to answer it. The Bloom rating was determined from weighted average of the BT levels of all assessment materials in the course [57]. 6.3 Knowledge Level of Novices and Experts There are significant differences between the achieved level of knowledge of experts and novices. Novices spend more time in accumulating knowledge and in comprehending (two lowest levels of the BT), however experts concentrate their effort on higher levels [61]. Le-

vels 4-6 are known as the higher-level thinking skills. These levels are usually not achieved by the university software engineering curriculum [75].

Program comprehension is not a simple top-down or bottom-up process. The process depends on tasks, level of knowledge and programming experiences. Experienced programmers (have domain knowledge) prefer top-down approach. Developers with less programming knowledge prefer bottom-up approach. You can see the scheme of top-down and bottom-up approach at Figure 6.1. Main difficulties faced by novice programmers are the lack of the knowledge and strategies. Novices obtain knowledge, but do not know how to apply it. Expert programmers use many strategies, while novices suffer not only from the lack of the knowledge, but also from the lack of strategy to solve programming assignments [63].

Figure 6.1: Scheme of top-down and bottom-up approaches [63]

Novice programmers often follow the original design due to the lack of programming experience, while experts often abandon incorrect concepts. Novices create a class design at the beginning and keep it until the end, and they try to fit everything to their previous design and do not want to change it. Novices also do not have a deep discussion about the design at the beginning, while experts try to create a clear picture of the design [62]. 6.4 Knowledge Assessment In Sections 6.4 and 6.5 we will mention the BT assessment possibilities. In Chapter 7 we will propose a framework according to Bloom's taxonomy to enhance the learning process for novices during the beginning with PP. Evaluation of efficiency and testing of proposed learning model stay open for the future work and following two sections could be used.

Exams usually do not test students' knowledge at all the levels of BT, so the teachers do not assess the mastery of a given topic for each student. The solution is to have exams with questions covering each BT level for every topic. Such a type of exam should demonstrate the achieved level of mastery for all topics. Of course, it is difficult to create six different questions (one per BT level) for every topic [56]. BT can be used to assess homework, projects and all other assessment instruments. By understanding the level of expected ex-

pertise, test questions can assess the students’ expected mastery. This is very important [55].

There exist two types of questions, quantitative and qualitative. Quantitative questions can be corrected as right or wrong, so it allows easy quantification of the performance. It is, for example, multiple choice test or closed questions. In qualitative questions answers are more open and there can be multiple correct answers. Answers are based on the participant’s opinion. It is difficult to identify absolutely right/wrong answer. It is also difficult to quantify the performance. The lower levels of the BT (recall, comprehension and application) suit well for the quantitative questions. But for the higher levels of the BT qualitative questions seemed more appropriate [58]. In the Appendix B we can see the example of the questions to assess the knowledge of students at each of the BT levels. First two questions are quantitative, others are qualitative.

Students at each of the BT levels should know [58]:  Recall: student should remember, cite or recognize statements/definitions. Re- quired precision/exactness of the questions should not be too different from the original knowledge learned.  Comprehension: the student is able to translate the material into another language, reconfigure/reorder the material based on the relationship between individual ideas contained and extrapolate the material to make predictions. o Testing: student should recognize the term or association from the written definitions, even though the definition is in a different form from the learned one.  Application: the student should use information and apply it to the new/unfamiliar situations or situations with new circumstances. o Testing: questions should be new or contain new elements. Questions contain problems that require students to apply the abstraction in a practical way.  Analysis: the student should be able to do three types of analysis: the breakdown of material into parts to identify the elements of communication, find relationship among these elements and determine their interaction/connection and recognize principles, that hold these element together. o Testing: questions are about familiar or new material that is presented in a test situation.

 Synthesis: the student is able to combine the ideas to form a new whole and to re- combine parts of previous experience with new materials to create a well integrated whole. o Testing: it is recommended not to use too many controlled and detailed instructions, because it can lower the creativity.  Evaluation: the student is able to develop opinions, judgements and decisions on given topic/problem. o Testing: the student is asked to provide judgements on consistency/accuracy and overall evaluation.

Example of questions assessing each of the BT levels for the topic “computer program” and other definitions that the student should know we can see in Table 6.6. Table 6.3: Bloom’s levels, and sample questions for the topic “computer program” [55]

Level Explanation Sample Question

The student is expected to recite memorized Recall “What is a program?” information about the concept. The student is expected to explain the concept “How is a program similar to Comprehension in his/her own words. a recipe?”

The student is expected to apply the “What is the output of this Application concept to a particular situation. program?”

The student is expected to separate “Create a top-down design materials or concepts into component parts so Analysis for a program to perform a that their organizational given task.” structure may be understood.

The student is expected to put parts together “Write a program to Synthesis to form a whole, with emphasis on creating a perform a given task.” new meaning or structure.

The student is expected to make “Given two programs that Evaluation judgements about the value of idea or perform the same task, material. which one is a better?”

BT can be viewed as three meta levels: memorization and basic understanding (beginner level), use or competent application (intermediate level) and design or creation and criti- que (expert level). Each meta level has two phases: a production of an artifact of learning and an explanation or analysis of that production (see Table 6.4) [55].

Students go through each of these levels when they are introduced to a new topic. Follow- ing lines present an example with the "if statement" (“if st.”) [55]:  Beginners first reproduce a definition of "if st.", then they explain it with their own words.  At the intermediate level learners can apply (reproduce) the comprehension of the concept to specific instances (trace this "if st."), then it is followed by the ability to explain how a particular instance works (explain how this "if st." solves the problem).  At expert level learners can produce a new instance of the concept (write an "if st." that solves this problem) and in the end they are able to evaluate the quality of the instances of the concept (is this the best "if st." for solving this problem?).

Table 6.4: A Meta-level structure for Bloom’s Taxonomy [55] Meta-Level Produce Explain Beginner Recall Comprehension Intermediate Application Analysis Expert Synthesis Evaluation

Two types of assessment methods exist to assess an individual's performance in the group work. In the base mark adjustment method there are individual tests given as well as peer and self evaluation after every session. In the task splitting method group work tasks are splitted and some work is required to be done individually and some in the group [59].

Different types of assessment exist, such as summative assessment, which takes place at the end of a learning, while formative assessment takes place during the learning. In the peer assessment learners are asked to assess each other during peer activities. During self assessment, students assess their own work. Self assessment can be used to identify the skills that students need to improve [59].

Another type of assessment of PP course (in a closed computer laboratory) was used in the study [59]. Facilitator gave formative feedback to student before every new PP assignment and after completion, individual tests were written. Only cognitive domain was assessed by the individual tests. The aim of the individual tests was to determine whether both team members are able to complete the assignment individually and also to determine that both team members achieved the required cognitive level. Individual tests, also as self and peer evaluation controlled individual accountability. The scores of self, peer and facilitator were assessed during every PP assignment per individual. We can see the self, peer and facilita-

tor assessment rubric in Table 6.5. Scores were counted and converted to a pair mark out of 10. At the beginning students did not give poor marks to the students that did not contribute. However, as students became familiar with this type of assessment the marks for the individual and pair assessment became more aligned. Instructors often neglect the last two domains (affective and psycho-motor), because they are difficult to assess. In Table 6.5 we can see an example, where these two domains are assessed [59].

Table 6.5: Self, peer and facilitator assessment rubrics [59] Achieved to Achieved to Not Totally Assessment Criteria a lesser a great achieved achieved extent extent Cognitive outcomes* *Specific outcome that changed from one week to another, in accordance with: Knowledge to be attained 1 2 3 4 Comprehension of knowledge 1 2 3 4 Application of knowledge 1 2 3 4 Analysis of facts and information 1 2 3 4 Synthesis of information 1 2 3 4 Evaluation of program effectiveness 1 2 3 4 Affective outcomes Members’ contribution to collection of 1 2 3 4 data and information Interaction between partners 1 2 3 4 Responsibility of driver 1 2 3 4 Responsibility of navigator 1 2 3 4 Co-operation between members 1 2 3 4 Decision-making skills of members 1 2 3 4 Psycho-motor skills Keyboard and mouse skills 1 2 3 4 Use of shortcuts, textbooks, other 1 2 3 4 resources 6.5 Assignment Assessment and Testing Program style and design is usually assessed by independent reading of the source code. But this is not a good approach. Rather than assess that programs produce the correct output is to assess how well student demonstrate correctness of the solution through testing (how the test conforms to the problem). Some educators use automated systems to assess programming assignment and to control the program output. So the students mostly focus on the output correctness (less on the design, commenting, testing etc.). As the result stu-

dents do less testing and rely on sample data provided by the educator and automated grading [62].

Programming courses usually first focus on writing programs (application and synthesis BT levels). But Bloom’s work suggests that students must first master the previous levels of the taxonomy. Many educators try to teach comprehension and analysis abilities through code reading, but students need repeated, useful and immediate feedback about their performance while forming hypotheses and testing them. To write an effective test, students need to think about resulting behaviour that they expect [62].

Trial and error is a method of problem solving. Repeated varied attempts are continued until they success. It is unsystematic method, which not include any theory or organized methodology. This technique is widely used by beginners/students. Computer science students use trial and error to fix errors and to debugging. Reflection in action is a characte- rization of how students complete uncertain/new tasks. When a technique or solution fails, students switch to a reflective mode (they examine phenomenon and prior understanding). From this reflection students try to find a new understanding of the phenomenon and a change in the situation. This is a process of finding viable solution, when previous experiences do not work in a new context without a change. Students in computer science are more successful if they move from trial and error to reflection in action [62].

One of the possible solutions for the previously described problems can be Test-driven development (TDD), which is one of the Extreme Programming methods. For the TDD it is typical, that one or more test cases are written before adding a new code. When you write a new code, tests will show you if you have achieved your latest goal [62].

In the current chapter we have described the idea of the BT, its domains and the cognitive domain in greater detail, because it is the most important for the use in SE context. Fur- thermore we have described the BT adapted to the SE context. We have also provided the possibilities of the BT to improve the educational course and showed the difference between the usage of the BT by the novices and experts. In the end of this chapter, we have explained the assessment possibilities of the BT for assessing of individuals, tests or assignments.

In the next chapter, we will provide educational model as a solution for some of the problems with the learnability of the PP. The model was created by the combination of the BT adapted to the SE context and the existing solutions (derived from the empirical studies, which were analyzed in Chapter 4 called Pair Programming).

7 Proposed Solutions In the thesis, we have identified several problems/issues influencing the learnability of PP. Some of the problems are open and in this chapter we will provide possible solutions for these open problems. From the problems with the learnability of PP the following problems are open: pair jelling, problems with the scheduling and compatibility problems. Fur- thermore, we will provide the recommendations (not strictly linked to BT) which should be used by the instructor interested in teaching of the PP. For every recommendation we will mention its benefits. 7.1 Applying Bloom’s Taxonomy In this section, we will provide a framework according to Bloom's taxonomy to enhance the learning process for novices during the beginning with PP. In Table 7.1 we describe levels of BT, for every level we mention activities (what will the developers do), what should developers acquire by mastering this level and to which problem these activities are linked. Levels 4-6 are known as the higher-level thinking skills. These levels are usually not achieved by the university software engineering curriculum [75]. For the software engineering the application level is the most important. Based on the [54] we have divided application level into two separate levels. At the application level 1 (AP1), developers try the application of PP under supervision and they are learning the basic principles of PP practi- cally. When mastering the application level 2 (AP2) developers are able to produce complex programs without the supervision and also to use distributed PP.

In the process of learning the PP, developers are improving their abilities in all three BT domains. Improved skills in Cognitive domain are described in Table 7.1. From Affective domain developers improve interaction/cooperation between partners, responsibility of driver/navigator and decision making skill. From Psycho-motor domain developers improve keyboard and mouse skills, using of shortcut keys, textbooks and other resources.

Table 7.1 Application of BT to learning process of PP2 Link to Lvl. Explanation Activity Impact problem RE The developer is ex- Learn about: difference Developers have to Pair jelling. pected to recite memo- between solo and PP, driver learn the basic know- rized informa- and navigator, collocated ledge about the terms tion/definition about and distributed PP, Pair Ro- of PP. PP. tation, TDD and coding standards. CO The developer is ex- Developers are requested to When developers Pair jelling. pected to explain the describe terms PP, driver, master the recall level, concept of PP in his/her navigator, collocated PP, they must be able to own words. distributed PP, Test-driven understand the mean- development in their own ing, not only to recite words. learned phrases. AP1 The developer is ex-  Fill in the compatibility This level will enable Pair jelling, pected to apply the ba- questionnaire for the developers to use partner com- sic aspects of PP in a partner finding, basic aspects of PP patibility, given situation under  based on the previous re- which they have scheduling supervision. sults, create new pairs, learned in the pre- problems.  in academic environment vious levels. Develop- pair students of the same ers will also be paired gender and academic with the proper part- achievement, also the stu- ner based on the re- dents with the similar sults of the compati- schedule, bility questionnaire  perform “Test run” (short (defined in Appendix program) to control com- B). patibility,  create routine programs under supervision and practice frequent role switching, TDD and coding standards following. AP2 The developer is ex- Developers produce more Developers learn how Scheduling pected to apply the PP complex programs with- to use PP to develop problems. in a particular situation out supervision. They learn complex programs without the supervi- how to use distributed PP. without the supervision. sion and also how to use distributed PP.

2 Each level must be mastered, before progressing to the next – RE -> CO -> AP1 -> AP2 -> AN -> SY -> EV

AN The developer is Developers are asked to Developers train how expected to sepa- explore the relationship to understand the rate/decompose mate- between classes in the pro- meaning of the exten- rials or concepts into vided extensive OO program sive code created by component parts so and describe those relation- other developers. that their organization ships. At this level develop- structure may be ers can coach less expe- understood. rienced developers [76]. SY The developer is ex- Developers are asked to Developers improve pected to put parts to- write a program of their their ability to choose gether to form a whole, own choice, but it has to and think about some- with an emphasis on fulfill provided criteria thing new. creating a new meaning (for example: at least 5 or structure. classes, each class with minimum 3 methods, more than 150 lines of code, GUI...). There is a difficulty that no specific task is given. It is recommended to prefer quality over quantity [76]. With mastering this level developers can coach other professionals. EV The developer is ex- Developers are required to Developers improve pected to make judge- evaluate the quality of the their ability to eva- ments or evaluate the provided program in terms luate the code created quality. of: coding standard follow- by other developers ing, structure of the code, and to find errors in it. documentation understan- dability, user interface design and error finding [76].

At the Recall level (RE) developers learn the basic theory about the PP. At the Comprehen- sion level (CO) developers comprehend the meaning of the basic theory and they will be able to explain PP terms in their own words. At the Application level 1 (AP1) developers start to apply theory about the PP in the practice. They are paired based on the result of the compatibility questionnaire (see Appendix B), write “test run” and start to code routine programs under supervision. At the Application level 2 (AP2) developers start to code more complex tasks and after mastering this level they will be able to code complex tasks without the supervision. At this level developers also learn how to use distributed PP. At Analysis level (AN) developers train how to understand the meaning of the extensive code

created by other developers. At this level developers are able to coach less experienced developers. At the Synthesis level (SY) developers improve their ability to choose and think about something new. At the last level Evaluation (EV) developers improve their ability to make judgements about the code created by somebody else [76].

How are we improving the learnability of PP? To avoid pair jelling (low productivity at the beginning) developers will learn the theory about PP, before they start to develop code. They learn the theory at the first two levels. At the Application level 1 developers are paired according to the compatibility test and perform the “test run”, these both activities can avoid the problems with the compatibility of the pair. Also this will lower the pair jelling effect, because well paired developers are more productive from the beginning. At this level we can also avoid some problems with the scheduling (for academic environment), because we ensure that students with a similar schedule are paired. At the Application level 2 developers learn how to use distributed PP, this approach can be used to lower the problems with the scheduling, because developers do not need to meet personally and they can cooperate from different places. At the last three levels we do not avoid the open problems of the PP. At these levels developers train how to analyze, understand and evaluate the code created by other developers and also how to think and decide about the new code, so these levels are important too. 7.2 Recommendations In this section we will provide recommendation for improvement of the PP learning process which were derived from the literature review. All the provided recommendation have positive effect on the process of learning the PP and their following can lead to the more effective learning of PP, increased enjoyment of the work of developers and also more effective PP development at all. Recommendations are not strictly linked to BT.

Recommendations for instructor teaching the PP practice:  Start with the basic terminology: difference between solo programming and PP, terms of PP such as navigator, observer, role switching, distributed and collocated PP. Students are first able to cite the definitions, later they can explain terms in their own words. o This recommendation is based on the BT levels of knowledge. Before reach- ing higher levels, developers must master the basic levels of Recall and Com- prehension.

 Present to the developers the benefits of PP to motivate them to learn, o Developers are usually skeptic of benefits from collaborative work. They are afraid of additional communication, the other’s work habits, different programming style and ego [22].  Present to the developers the problems of PP, so than they can avoid them or lower their effect. o With the knowledge of problems/issues of PP it can be intuitive more for developers to avoid them. Also the instructor can show some possible solutions for these problems.  Teach Test-driven development. o TDD can avoid the problem called Trial and error – method of problem solving, where varied attempts are continued until success. Students should prefer using reflection in action – they try to find a new understanding of the phenomenon and a change in the situation. This is a process of finding a viable solution, when previous experiences do not work in a new context without a change [62].  Institute a coding standard. o Instructor-selected coding standard can remove differences in coding style of pairs. This increases the effectiveness of the pairs [31].  Create a pairing-oriented culture. o Instructor should discuss some of the ideas behind PP with developers. The discussion is about issues such as collective code ownership, maintenance goals and industry expectations [31].  Arrange the workplace to follow PP ergonomic considerations. o Programmers need to sit comfortably side by side and with a large (17-inch minimum) monitor, small tables are not recommended. The driver sits in front of the keyboard and mouse, navigator must have a clean view of the monitor. When they switch roles, they should simply slide the keyboard and mouse from one person to the other. Many programmers prefer to have his/her own display, keyboard, and mouse attached to one computer. The navigator can easily and immediately take control [34].  Combine developers in accordance with the personality type, self-esteem, skill and experience level. o Before the first pairing, developers will fill in a compatibility questionnaire (see Appendix B) and according to the result they will be paired.

 At the beginning developers write “test run” to control compatibility of the pair. o Pair is combined according to the compatibility questionnaire, but just to be sure developers will write together a very simple program, if they report problems, they can be re-paired.  Recommend to switch roles regularly and frequently. o Regular and frequent role switching supports the advancement of developers in both roles (navigator, observer). This can also avoid the problem with an unbalance a share of work.  Start with the simple short programs. o It was supported by authors in the study [18] that programmers who start PP with routine tasks were more effective than those who started with complex tasks.  Teach how to use distributed PP (developers can collaborate for example from different rooms). o When developers learn how to use PP from the beginning, they can use it to avoid the problems with the scheduling (when they cannot meet, but they can cooperate from different places).

Recommendations for the academic environment:  Make sections mandatory. o The instructor can define acceptable excuses: illness or already having task completed. Both students must inform the instructor if they will not visit the section. This helps to identify missing partner [31].  Pair within section. o It ensures that for some time students work together. This can be also used to avoid “solo” programmers, when the partner did not arrived, student can be paired with other an “solo” student or with the pair.  Assignments as a function of section time. o Instructor should create assignments that a reasonable percentage of students can finish within the section time. If assignments are longer, students cannot follow PP guidelines because of scheduling problems [31].  If there is an odd number of students in section, flexible students (students with more flexible schedule than others) can switch section if necessary. o This recommendation can avoid some problems with the odd number of students in the class.

 Combine students of the same gender and academic level. o Students prefer partner of the same gender and the same academic achievement [49].  Use peer evaluation for students – they can evaluate their partner and the instructor can control the productivity of the pair. o This can avoid the situation when just one of the students conforms to the task completion. Peer evaluation can for example be done through the web questionnaire.

In the first part of this chapter we have seen a framework according to Bloom's taxonomy to enhance the learning process for novices during the beginning with PP. Framework was provided in Table 7.1 where it was explained: each of the BT levels, activities performed, impact on the PP learning process and linking to the open problems. In the second part we have seen several recommendations (some recommendations are typical for the academic environment) for improvement of the PP learning process, not strictly linked to BT.

8 Conclusion and Future Works In the thesis we have reviewed the emergence of Agile development methodology to contextualize Pair Programming (PP) practice and compared Agile methodology with more heavy-weight methodologies. Furthermore we have evaluated the importance of PP as one of the key practices of Extreme Programming. PP was analyzed in terms of empirical studies investigating its efficacy, benefits and problems. We have identified problematic issues that emerge during the learning process of the PP and that could hinder its adoption. For each of the problems we have identified its influence on the learning process and reviewed the state of art in terms of existing solutions of these problems.

In addition this diploma thesis focused on understanding the mechanisms that lead novice developers to better learn the PP. We have chosen an educational model, Bloom's Taxono- my (BT) of learning domains, to frame the discussion. We have described the idea of BT, its adaptations to the software engineering context and assessment possibilities. Furthermore, we have identified open problems and a framework according to BT was proposed to solve these problems and to enhance the learning process for novices (beginning with PP). In the framework we used compatibility questionnaire (see Appendix B) which was created as a combination of all partner compatibility aspects. Also there were provided recommendations to improve the PP learning process (not strictly linked to Bloom's taxonomy).

Results in term of solutions for each of the research questions are listed below: Q1a: What are the general benefits of PP? Based on the literature review we have identified several benefits. Benefits of PP are improvement of productivity, job satisfaction, team communication, knowledge transfer and quality. Other benefits are reductions of defect rate and time to deliver, support of newco- mer integration and reduction of training costs. All the benefits listed above can change the negative viewpoint of developers of this Extreme Programming practice. These benefits can be presented to developers at the beginning of the PP learning process.

Q1b: What are the benefits of PP in academic environment? In the Q1a we have identified general benefits of PP, but there exist some benefits typical just for the academic environment. From the academic benefits, we can mention improvement of students’ retention and final grades, reduction of instructor’s workload and higher confidence in the solution. All these benefits suggest that PP could be used in academic environment for programming classes. Also these benefits can be presented to the students learning the PP practice and to the instructors interested in adopting this XP practice.

Q2: What are the weaknesses/problems of PP? We have identified various problems of PP such as pair jelling (initial period of time when the pair is adjusting), problems with reliability, scheduling problems, noise and facility considerations, odd number of team members and partner compatibility. We have elaborated how these problems influence the PP development. The identification of problems was the first step in the process of solution finding to improve the learning process of PP. From these problems we have identified open problems.

Q3: What is the impact of the PP problems on learning of PP to novices? To all the identified problems we have elaborated their influence on PP learning process. Each of the problems identified has some negative impact. Pair jelling causes a lower productivity at the beginning, problems with reliability can cause that just one of the partners conforms to the task completion (so the other developers do not practice PP). When there is an odd number of developers in the team, some developers lose a chance to practice PP. Problems with compatibility can damage the whole pair cooperation.

Q4: What has been done to solve the problems of PP impacting the learnability? The aim of this research question was to collect all the existing solutions for the problems identified based on the Q2 and Q3. The existing solutions can be used both in academic and business environment. All these solutions can avoid or at least lower the effect of PP problems. Identified solutions were used for the creation of the recommendations.

The result of the thesis could be applied by the instructors interested in adopting PP. In- structors should follow the proposed recommendations and teach in accordance with proposed framework (each level of the Bloom’s taxonomy must be mastered before progressing to the next one). Framework and recommendations could be also proper for novice developers or students interested in learning and mastering the PP practice. Furthermore, the identified benefits could serve as a motivation to start with this XP practice and to change the negative viewpoint of some developers/managers.

In a given thesis we have proposed a framework according to Bloom's taxonomy to enhance the learning process for novices. We have also identified several recommendations for improvement of the learning process. The extension of this thesis may concern the evaluation of proposed solutions and testing the efficiency of given framework on the class of students learning the PP practice. We provide a list of studies that can be used as a reference for the application and evaluation of the framework within practice (listed in the Ap-

pendix A). For the future evaluation of the provided framework could be used Sections 6.4 and 6.5 about the assessment possibilities of the BT.

Literature references [1] Kadlec, V. (2004). Agilní programování: metodiky efektivního vývoje softwaru. Brno: Computer press. [2] Ráček, J. (2010). Analýza a návrh systémů – Úvod (Document in PDF format). Faculty of informatics MU. Available online on: , [ret. 2013-09-14]. [3] Pressman, R. S. (2010). Software Engineering: a practitioner’s approach (7th ed.). New York: McGraw-Hill Higher Education. [4] Wideman, R. M. Expert Project Management – The Role of the project Life Cycle (Life Span) in Project Management (Online). Available online on: , [ret. 2013-09-16]. [5] Boehm, B. W. A Spiral Model of Software Development and Enhancement (Document in PDF format). Available online on: , [ret. 2013- 9-16]. [6] Ošlejšek, R. Objektové metody návrhu informačních systému – Strategie vývoje software (Document in PDF format). Faculty of informatics MU. Available online on: , [ret. 2013-09-17]. [7] Kroll, P., & MacIsaac, B. (2006). Agility and Discipline Made Easy: Practices from OpenUP and RUP. Pearson Education. [8] Sommerville, I. (2001). Software Engineering (6th ed.). Harlow: Addison-Wesley Publishing Company. [9] Agile Manifesto (Online). Available online on: , [ret. 2013-09- 19]. [10] Buchalcevová, A. (2009). Metodiky budování informačních systémů. Praha: Oeconomica. [11] Ambler S. W. (2010). 2010 IT Project Success Rates Survey Results (Online), Available online on: , [ret. 2013-09-21]. [12] Pair programming (Online). Available online on: , [ret. 2013-10-01]. [13] Cockburn, A., & Williams, L. (2001). The costs and benefits of pair programming. Extreme programming examined, Boston: Addison Wesley, pp. 223-248.

[14] Baheti, P., Williams, L., Gehringer, E., Stotts, D., & Smith, J. M. (2002). Distributed pair programming: empirical studies and supporting environments. TR02-010. University of North Carolina at Chapel Hill Dept. of Computer Science. [15] George, B., Mansour, Y. M., & Williams, L. (2002). A multidisciplinary virtual team. System- ics, Cybernetics and Informatics. [16] Stotts, D., Williams, L., Nagappan, N., Baheti, P., Jen, D., & Jackson, A. (2003). Virtual teaming: Experiments and experiences with distributed pair programming. In Extreme Programming and Agile Methods-XP/Agile Universe 2003 (pp. 129-141). Springer Berlin Heidelberg. [17] Phaphoom, N., Sillitti, A., & Succi, G. (2011). Pair Programming and Software Defects–An Industrial Case Study. In Agile Processes in Software Engineering and Extreme Program- ming (pp. 208-222). Springer Berlin Heidelberg. [18] Sajeev, A. S. M., & Datta, S. (2013). Introducing Programmers to Pair Programming: A Controlled Experiment. In Agile Processes in Software Engineering and Extreme Program- ming (pp. 31-45). Springer Berlin Heidelberg. [19] Lui, K. M., & Chan, K. C. (2003). When does a pair outperform two individuals?. In Extreme programming and agile processes in software engineering (pp. 225-233). Springer Berlin Heidelberg. [20] Nawrocki, J., & Wojciechowski, A. (2001). Experimental evaluation of pair programming. European Software Control and Metrics (Escom), pp. 99-101. [21] Succi, G., Pedrycz, W., Marchesi, M., & Williams, L. (2002, May). Preliminary analysis of the effects of pair programming on job satisfaction. In Proceedings of the 3rd International Con- ference on Extreme Programming (XP) (pp. 212-215). [22] Williams, L., Jeffries, R., Kessler, R. R., & Cunningham, W. (2000). Strengthening the case for pair programming. IEEE software, 17(4), 19-25. [23] Fronza, I., Sillitti, A., & Succi, G. (2009, October). An interpretation of the results of the analysis of pair programming during novices integration in a team. In Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement (pp. 225-235). IEEE Computer Society. [24] Coman, I. D., Sillitti, A., & Succi, G. (2008). Investigating the Usefulness of Pair-Programming in a Mature Agile Team. In Agile Processes in Software Engineering and Extreme Program- ming (pp. 127-136). Springer Berlin Heidelberg. [25] Vanhanen, J., & Lassenius, C. (2005, November). Effects of pair programming at the development team level: an experiment. In Empirical Software Engineering, 2005. 2005 Interna- tional Symposium. IEEE.

[26]Williams, L. A. (2000). The collaborative software process (Doctoral dissertation, The University of Utah). [27] Begel, A., & Nagappan, N. (2008, October). Pair programming: what's in it for me?. In Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement (pp. 120-128). ACM. [28] Weinberg, G. M. (1998). The psychology of computer programming. New York: Silver Anniversary Edition. [29] Nosek, J. T. (1998). The case for collaborative programming. Communications of the ACM, 41(3), pp. 105-108. [30] Nagappan, N., Williams, L., Wiebe, E., Miller, C., Balik, S., Ferzli, M., & Petlick, J. (2003). Pair learning: With an eye toward future success. In Extreme Programming and Agile Methods- XP/Agile Universe 2003 (pp. 185-198). Springer Berlin Heidelberg. [31] Bevan, J., Werner, L., & McDowell, C. (2002). Guidelines for the User of Pair Programming in a Freshman Programming Class, Conference on Software Engineering Education and Train- ing, Kentucky. [32] McDowell, C., Werner, L., Bullock, H., & Fernald, J. (2002, February). The effects of pair- programming on performance in an introductory programming course. In ACM SIGCSE Bul- letin (Vol. 34, No. 1, pp. 38-42). ACM. [33] Hulkko, H., & Abrahamsson, P. (2005, May). A multiple case study on the impact of pair programming on product quality. In Proceedings of the 27th international conference on Software engineering (pp. 495-504). ACM. [34] Williams, L., & Kessler, R. (2002). Pair programming illuminated. Addison-Wesley. [35] McDowell, C., Werner, L., Bullock, H. E., & Fernald, J. (2003, May). The impact of pair programming on student performance, perception and persistence. In Proceedings of the 25th international conference on Software engineering (pp. 602-607). IEEE Computer So- ciety. [36] Williams, L. A., & Kessler, R. R. (2001). Experiments with industry's “pair-programming” model in the computer science classroom. Computer Science Education, 11(1), pp. 7-20. [37] VanDeGrift, T. (2004). Coupling pair programming and writing: learning about students' perceptions and processes. ACM SIGCSE Bulletin, 36(1), pp. 2-6. [38] Fronza, I., & Succi, G. (2009). Modeling spontaneous pair programming when new developers join a team. In Agile Processes in Software Engineering and Extreme Programming (pp. 242-244). Springer Berlin Heidelberg. [39] Padmanabhuni, M. V. V. K., Tadiparthi, H. P., Yanamadala, M., & Madina, S. (2012). Effective pair programming practice-an experimental study. Journal of Emerging Trends in Compu- ting and Information Sciences, 3(4), pp. 471-479.

[40] Babb, J. S., Hoda, R., & Nørbjerg, J. (2013). Barriers to Learning in Agile Software Develop- ment Projects. In Agile Processes in Software Engineering and Extreme Programming (pp. 1-15). Springer Berlin Heidelberg. [41] Vivekanandan, K., & Kuppuswami, S. (2004). The effects of pair programming on learning efficiency in short programming assignments. Informatics in Education-An International Journal, (Vol 3_2), pp. 251-266. [42] Nagappan, N., Williams, L., Ferzli, M., Wiebe, E., Yang, K., Miller, C., & Balik, S. (2003, February). Improving the CS1 experience with pair programming. In ACM SIGCSE Bulletin (Vol. 35, No. 1, pp. 359-362). ACM. [43] Katira, N., Williams, L., Wiebe, E., Miller, C., Balik, S., & Gehringer, E. (2004, March). On understanding compatibility of student pair programmers. In ACM SIGCSE Bulletin (Vol. 36, No. 1, pp. 7-11). ACM. [44] Thomas, L., Ratcliffe, M., & Robertson, A. (2003, February). Code warriors and code-a- phobes: a study in attitude and pair programming. In ACM SIGCSE Bulletin (Vol. 35, No. 1, pp. 363-367). ACM. [45] Weber-Wulff, D. (2000, July). Combating the code warrior: A different sort of programming instruction. In ACM SIGCSE Bulletin (Vol. 32, No. 3, pp. 85-88). ACM. [46] Keirsey, D. (1998). Please understand me II: Temperament, character, intelligence. Del Mart: Prometheus Nemesis. [47] Melnik, G., & Maurer, F. (2002). Perceptions of agile practices: A student survey. In Extreme Programming and Agile Methods—XP/Agile Universe 2002 (pp. 241-250). Springer Berlin Heidelberg. [48] Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard university press. [49] Kuppuswami, S., & Vivekanandan, K. (2002). Female students’ perspective on the adoption of pair programming to conduct laboratory experiments. Proceedings of the First Annual Conference on Women in Information Technology, Annamalai University. Annamalai Nagar, India. [50] McDowell, C., Werner, L., Bullock, H. E., & Fernald, J. (2006). Pair programming improves student retention, confidence, and program quality. Communications of the ACM, 49(8), pp. 90-95. [51] Bloom’s taxonomy – learning domains (Online), Available online on: , [ret. 2014-03- 01]. [52] DeMarco, T., & Lister, T. (1977). Peopleware. Dorset House, New York.

[53] Katzenbach, J. R. & Smith, D. K. (1994). The wisdom of teams: Creating the high-performance organization. Harvard Business Press. [54] Azuma, M., Coallier, F., & Garbajosa, J. (2003, September). How to apply the Bloom taxonomy to software engineering. In Software Technology and Engineering Practice, 2003. Ele- venth Annual International Workshop on (pp. 117-122). IEEE. [55] Starr, C. W., Manaris, B., & Stalvey, R. H. (2008, March). Bloom's taxonomy revisited: specifying assessable learning objectives in computer science. In ACM SIGCSE Bulletin (Vol. 40, No. 1, pp. 261-265). ACM. [56] Scott, T. (2003). Bloom's taxonomy applied to testing in computer science classes. Journal of Computing Sciences in Colleges, 19(1), pp. 267-274. [57] Oliver, D., Dobele, T., Greber, M., & Roberts, T. (2004, January). This course has a Bloom Rating of 3.9. In Proceedings of the Sixth Australasian Conference on Computing Education- Volume 30 (pp. 227-231). Australian Computer Society, Inc.. [58] Buckley, J., & Exton, C. (2003, May). Bloom's taxonomy: a framework for assessing programmers' knowledge of software systems. In Program Comprehension, 2003. 11th IEEE International Workshop on (pp. 165-174). IEEE. [59] Hahn, J. H., Mentz, E., & Meyer, L. (2009). Assessment Strategies for Pair Programming. Journal of Information Technology Education, vol. 8, pp. 273-284. [60] Rejab, M. M., Omar, M., Ahmad M., & Ahmad, K. B. Pair programming in introducing knowledge sharing. [61] Xu, S., & Rajlich, V. (2004, August). Cognitive process during program debugging. In Cogni- tive Informatics, 2004. Proceedings of the Third IEEE International Conference on (pp. 176- 182). IEEE. [62] Edwards, S. H. (2004). Using software testing to move students from trial-and-error to reflection-in-action. ACM SIGCSE Bulletin, 36(1), pp. 26-30. [63] Xu, S., Rajlich, V., & Marcus, A. (2005, August). An empirical study of programmer learning during incremental software development. In Cognitive Informatics, 2005. (ICCI 2005). Fourth IEEE Conference on (pp. 340-349). IEEE. [64] MountainGoatSoftware: Introduction to SCRUM – An agile process (Online), Available online on: , [ret. 2014- 04-12]. [65] Schwaber, K., & Sutherland, J. (2011). The scrum guide. Scrum. org. [66] Feature Driven Development (FDD) and Agile Modeling (Online), Available online on: , [ret. 2014-04-15]. [67] Introduction to Test Driven Development (Online), Available online on: , [ret. 2014-04-18].

[68] Beck, K. (2003). Test-driven development: by example. Addison-Wesley Professional. [69] What is Extreme Programming? (Online), Available online on: , [ret. 2014-04-18]. [70] Extreme Programming: A gentle introduction (Online). Available online on: [ret. 2014-04-18]. [71] Decker, N. Lean Software Development. University of Wisconsin – Platteville. [72] Shore, J. (2007). The art of agile development. " O'Reilly Media, Inc.". [73] Davies, R. (2006 June). Pair Programming Explained (Online), Available online on: , [ret. 2014-05-04]. [74] The Myers & Briggs foundation, MBTI basics (Online). Available online on: , [ret. 2014-05- 04]. [75] Felder, R. M., & Brent, R. (2004, June). The ABC’s of engineering education: ABET, Bloom’s taxonomy, cooperative learning, and so on. In Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition. [76] Lister, R., & Leaney, J. (2003, February). Introductory Programming, Criterion-Referencing, and Bloom. In ACM SIGCSE Bulletin (Vol. 35, No. 1, pp. 143-147). ACM. [77] Vockell, E. (2004). Educational psychology: A practical approach. Purdue University.

Appendix

A. Summary of the Cited Studies This appendix provides a bulleted list of information about studies of PP benefits, weaknesses, partner finding and difference between collocated and remote Pair Programming (PP). About each used study we present relevant reference number, name of the study, authors, subject (what was study exploring) and the result of the study. Provided summary allow us to see which studies are based on student participation and which on company participation. Also it is useful review of the purpose and result of some of the used studies in a short way.

Summary of the used studies: [13] Distributed Pair Programming: Empirical Studies and Supporting Environments.  Authors: A. Cockburn & L. Williams.  Subject of the study: No subject – based on review of existing studies.  Focus of the study: Analysis of the existing studies about PP based on interviews and experimental data.  Result of the study: Pairs spend 15% more time to finish the program than solos. PP improves design quality, reduces defects, reduces staffing risk, enhances technical skills, improves team communication and is considered to be more enjoyable at statistically significant levels.

[14] Distributed Pair Programming: Empirical Studies and Supporting Environments.  Authors: P. Baheti, L. Williams, E. Gehringer, D. Stotts & J. McC. Smith.  Subject of the study: Was graduate class with 132 students (34 distance learning) of the introductory OO class. Students collaborated on 5-week long team project.  Focus of the study: Examination of the following hypotheses: Distributed synchronous teams (DST) will produce higher quality code and will be more productive (in terms of LOC/hr), than not synchronous teams (NST). DST will have comparable productivity and quality as collocated teams. DST will have better communication and teamwork than NST.  Result of the study: PP in virtual and collocated team is feasible for OO development. Development with distributed and collocated PP is comparable in terms of productivity and quality. Collocated teams do not produce better code than distri-

buted ones. Student feedback indicates that distributed PP supports teamwork and communication in virtual team.

[16] Virtual Teaming: Experiments and Experiences with Distributed Pair Programming  Authors: D. Stotts, L. Williams, N. Nagappan, P. Baheti, D. Jen & A. Jackson.  Subject of the study: Were eight graduate students (from 2 universities) who participated in a 5-week long experiment. Students were divided into 4 distributed pairs having one student from each university. Communication was done via the In- ternet. Two of the groups worked as virtual pairs, remaining two worked as traditional virtual teams (without PP).  Focus of the study: Comparison of the development with distributed PP and traditional virtual team in terms of development time, unit/acceptance tests written and passed. Also a comparison of the technical infrastructure for distributed PP.  Result of the study: Synchronous teams performed better than the non-paired teams (lower development time, 70% more unit test cases, acceptance test results were not statistically significant). Authors found that beneficial is: at least one face- to-face meeting and tools able to switch between design and code view. Developers worked better when they had good relationship. Programmers spoke regularly while working. Authors found that necessary basics are screen sharing, audio communication and file transfer.

[17] Pair Programming and Software Defects – an Industrial Case Study  Authors: N. Phaphoom, A. Sillitti & G. Succi.  Subject of the study: Was team of 17 developers from large Italian manufacturing company. They adopted XP a used PP during 14-months data collection.  Focus of the study: Examination of the following questions: o Q1: Does the code contain fewer defects when developers use PP? o Q2: Do the defect-containing (DC) methods contain fewer defects when developers use PP? o Q3: Are the defects reduced, once developers start to PP when working on DC methods? o Q4: Does the enhance knowledge gained from previous PP on methods modified during user story implementation (USI) help to prevent new defects af- terwards? o Q5: Do the methods modified during USI contain fewer defects when developers PP?

 Result of the study: Answers to the questions: o Q1 – yes o Q2 – no o Q3 – yes o Q4 – yes o Q5 – no

[18] Introducing Programmers to Pair Programming: A Controlled Experiment  Authors: A. S. M. Sajeev & S. Datta.  Subject of the study: Were 144 students divided into 50 pairs and 44 solo programmers.  Focus of the study: Investigate following questions: o Q1: What are the least liked aspects of PP? o Q2: Do new pair programmers work better as a pair in routine problems or in complex problems? o Q3: How best to prepare a pair (start with complex or routine tasks)?  Result of the study: o Q1: Sharing of screen, keyboard and mouse, role switching. o Q2: Pairs solve complex problems more effectively than solos. Team effectiveness was higher while solving routine problems. o Q3: Programmers who started PP with routine tasks and then moved to complex tasks were more effective than those, who started with the complex tasks.

[19] When Does a Pair Outperform Two Individuals?  Authors: K. M. Lui & K. C. C. Chan.  Subject of the study: Were 15 hands-on programmers from different companies. XP was introduced and developers were asked to write a practice program with respect to XP practices. After tutorial of XP developers participated in two experiments: o Experiment 1: Developers were divided into 5 individuals and 5 pairs to solve procedural algorithms. At the first completion they get knowledge how many answers were correct (no hint which questions were correctly answered). This was repeated until all questions were correct. o Experiment 2: All the developers were formed as pairs (group A) and then they were separated into individuals (group B). In both cases programmers

solved deduction questions with the same principle of answering as in the Experiment 1.  Focus of the study: Investigate quality and productivity of PP when solving procedural problems and deduction questions (quality and productivity is not affected by the knowledge of a particular computer language).  Result of the study: o Experiment 1: At the first attempt pairs spend 20,9% more time then solos. The total time to solve all algorithms right was for pairs 4,2% lower than for the solos. o Experiment 2: At the first attempt group A (pairs) spend 73,3% more time than solos. However group A achieved 85% correctness and group B only 51%. The total time to solve all questions right was 5,3% lower for group A. o From the results we can see that PP had better productivity and quality than the solo programming when completing procedural problems and deduction questions. The pair outperformed two individuals.

[20] Experimental Evaluation of Pair Programming  Authors: J. Nawrocki & A. Wojciechowski.  Subject of the study: Were 21 students divided into three groups: 6 in personal software process group (PSP), 5 in XP for single programmer group (XP1) and 5 in PP group. Task was to develop 4 programs.  Focus of the study: Compare PP with solo programming based on PSP and XP practice in terms of development time, efficiency, program size, predictability and number of re-submissions.  Result of the study: PP was less efficient than it was reported in [22]. PSP seems less efficient that XP1, but average code size was for some programs smaller. PP was more predictable in terms of development time and program size, amount of rework was slightly lower.

[21] Preliminary Analysis of the Effects of Pair Programming on Job Satisfaction  Authors: G. Succi, W. Pedrycz, M. Marchesi & L. Williams.  Subject of the study: Were 108 respondents around the world who filled a web questionnaire about PP techniques. There were two types of questionnaire, first for the PP developers and the second for the developers who do not use PP.  Focus of the study: Investigation of effects of PP on job satisfaction.

 Result of the study: PP had positive effect on the job satisfaction because of increased communication between developers, faster communication of design changes and organization of meetings.

[22] Strengthening the Case for Pair Programming  Authors: L. Williams, R. R. Kessler, W. Cunningham & R. Jeffries.  Subject of the study: Were 41 students of the same class who performed 4 tasks and participated in class discussions during the period of 6 weeks. Students were divided in two groups: 13 worked individually and 28 worked in pairs.  Focus of the study: Comparison of the two experimental groups in terms of development time, quality of the solution and job satisfaction.  Result of the study: Pairs completed their tasks 40% to 50% faster and had better percentage of test cases passed. More than 90% of students said, that they enjoyed PP more than solo programming and they feeled more confident in their work.

[23] An Interpretation of the Results of the Analysis of Pair Programming during Novice Inte- gration in a Team  Authors: I. Fronza, A. Sillitti & G. Succi.  Subject of the study: Was team of 17 developers: 15 mature team members and 2 newcomers that used spontaneous PP. The study period was 10 months.  Focus of the study: Evaluation of the monthly effort of PP by the whole team, novices, experts and the comparison of measured values. Furthermore, the exploration of the novices integration in the team.  Result of the study: The analysis identified four major steps of integration: Initia- tion, Independence, Maturity and Integration. The study also confirmed the hypotheses that PP facilitates the introduction of newcomers and knowledge transfer.

[24] Investigating the Usefulness of Pair-Programming in a Mature Agile Team  Authors: I. D. Coman, A. Sillitti & G. Succi.  Subject of the study: Was an agile team of 14 mature developers and 2 newcomers from the large Italian company for the period of 3 months.  Focus of the study: Identification of the cases, where the developers perceive PP useful and investigation of the usefulness of PP for the knowledge transfer. Data about the developer’s working time were collected.  Result of the study: Veteran developers used PP around 60% (not counted the non PP days), the overall PP was below 30%. The newcomers used PP more during the first month (PP seems to be valuable for training).

[25] Effects of Pair Programming at the Development Team Level: An Experiment  Author: J. Vanhanen.  Subject of the study: Were students divided into 5 teams (3 teams used PP 2 used solo programming) each having 4 members to accomplish the same 400-hour project.  Focus of the study: Investigate how PP influenced the quality of the solution, productivity, knowledge transfer, enjoyment of the work and the effort estimation accuracy.  Result of the study: PP teams had 29% lower productivity (because of the learning time of the PP practice and getting familiar with new people). If the learning time is not taken into account than the productivity of PP teams is equal to the solo ones. PP teams had 8% less defects, but the solo teams were more successful in finding and fixing the bugs (in the end they delivered the program with less defects). PP teams had slightly better quality, but they developed less functionality also they understood the packages more (this indicates the better knowledge transfer). Enjoyment of the work was the half for both approaches. The initial estimates of both groups were good, but solo programmers had better updated estimates.

[26] The Collaborative Software Process  Author: L. Williams.  Subject of the study: Were 40 senior students divided into two groups: two-thirds worked in pairs and used Collaborative Software Process (CSP) and the rest used Personal Software Process (PSP). Both groups developed the same tasks.  Focus of the study: Validation of the effectiveness of CSP in comparison with the PSP.  Result of the study: Pairs spend 15% more time on the task completion and had 15% less defects in their programs than solos. When considering long-term support (in terms of the higher quality of the programs) than is the PP cheaper for the organization. 95% of the PP students enjoyed work more than programming itself.

[27] Pair Programming: What’s in it for me?  Authors: A. Begel & N. Nagappan.  Subject of the study: Was the anonymous web-based PP survey (part of the bigger survey of Agile development). In experiment participated 487 developers, testers and managers from Microsoft Corporation. 89 respondents used PP in the past and 17 in their last project.

 Focus of the study: Evaluation of PP at Microsoft Corporation in terms of individual attitudes of PP, effects on work, benefits, problems and aspects of the good PP partner/team.  Result of the study: 64% said that PP worked well, 48% said that PP worked for their team and 39% said that it worked for their organization. 38% agreed that PP takes more time to finish the task, 25% disagreed. 65% of the respondents believed that PP results in fewer bugs. The biggest mentioned benefits of PP were: fewer bugs, spreading code understanding and higher quality software. The top problems of PP were: cost-efficiency, scheduling problems and personality conflicts. The proper partner could have: complementary skills, flexibility and communications skills.

[30] Pair Learning: With an Eye Toward Future Success  Authors: N. Nagappan , L. Williams, E. Wiebe, C. Miller, S. Balik, M. Ferzli & J. Petlick.  Subject of the study: Were students of the Java introductory programming course. Study was aimed at novice students for the period of three semesters. Course had sections with both PP and solo students. Students were also analyzed in follow-on course (solo programming course).  Focus of the study: Examination of the following hypotheses: o H1: PP students will have an equal or higher % of C or the better grades in comparison with solo programmers. o H2: PP students will have positive attitude towards the collaborative programming settings. o H3: Use of PP will not hamper students' performance in the future solo programming courses. o H4: Use of PP will lead to reduced workload for the course staff in terms of grading, questions answered and teaching effort in comparison with the solo programming course.  Result of the study: o H1: Yes. o H2: Yes (more than 80% of students choosed to continue in PP). o H3: Yes. o H4: Yes.

[31] Guidelines for the Use of Pair Programming In A Freshman Programming Class  Authors: J. Bevan, L. Werner & C. McDowell.  Subject of the study: Were 4 lecture sections of the introductory programming course. Fall/Winter quarter classes used PP while the Spring class did not. Each of the classes completed 9 assignments in the period of 10 weeks.  Focus of the study: Identification of the implementation guidelines for the instructors interested in adopting PP for their class.  Result of the study: Guidelines useful for adopting PP in computer labs are: o 1, Pair within sections: helps to identify missing partner. During first week instructor can oversee the introductory activities that can help the students to find a compatible partner (depend on the scheduling compatibility and skill level). Potential pair can perform "test run". o 2, Pair by skill level: skill level is not a crucial parameter, but it can avoid compatibility problems. Instructor can use the personality profiling test. o 3, Make sections mandatory: ensure that some time students work together. Instructor can define the acceptable excuses: illness or already having task completed. Both students must inform the instructor about the absence in advance. o 4, Assignments as a function of section time: the instructor should create assignments that a reasonable percentage of students can finish within the section time. If the assignments are longer, students cannot follow PP guidelines because of the scheduling problems. o 5, Institute a coding standard: instructor-selected coding standards can avoid differences in coding style of the pairs. This increases the effectives of the pairs. o 6, Create a pairing-oriented culture: the instructor should discuss some of the ideas behind PP with students. Discussion is about issues such as collective code ownership, maintenance goals and industry expectations.

[32] The Effects of Pair-Programming on Performance in an Introductory Programming Course  Authors: C. McDowell, L. Werner, H. Bullock & J. Fernald.  Subject of the study: Data were collected from the approximately 600 students at- tending introductory programming course. Some of students used PP, others were solo programming.

 Focus of the study: Investigation of the effect of PP on the student performance in the introductory programming course.  Result of the study: Students that used PP produced better programs, completed the course with better rates and performed as well on the final exam as solo programming students.

[33] A Multiple Case Study on the Impact of Pair Programming on Product Quality  Authors: H. Hulkko & P. Abrahamsson.  Subject of the study: Were four case studies. Case one involved 5-6th year students, case two involved research scientists, and cases three and four both practi- tioners and students.  Focus of the study: Summarize and organize the findings from the existing PP studies. Provide the new scientific evidence by reporting results from a multiple controlled case study on PP performed in close-to-industry settings. Multiple case study had two mail goals: first was to provide qualitative and quantitative information of how PP is used in actual software projects and second was to explore the impact of PP on software quality.  Result of the study: Summary of main findings of existing studies was created (for the further information see [33]). Effort spend on the PP is highest in the beginning of the project and in the defect correction phase. Productivity is lowest in the beginning of the project due to the pair jelling and it increases as the project progresses. Density of the coding standard deviations was in two cases higher for PP (40% and 80% more). PP had higher comment ratio (it should mean better readability as was mentioned in [29]). PP is most suitable when: o learning in the beginning of a project, o solving problems and thinking of ways to do complex tasks, o trivial mistakes identification from simple code.

[35] The Impact of Pair Programming on Student Performance, Perception and Persistence  Authors: C. McDowell, L. Werner, H. Bullock & J. Fernald.  Subject of the study: Was 555 students (413 men and 141 women) from the introductory programming course. There existed four sections of the course: one in the Fall and Spring, two in the Winter. Fall and Winter sections used PP, Spring section used the solo programming. Students answered questions about each of the programs (about the confidence: 0-100%, satisfaction: 1-7 and enjoyment: 1-7).  Focus of the study: Examination of the effectiveness of PP in four lecture sections of a large introductory programming course in terms of: completion and pass rates,

course performance, confidence in the solution, satisfaction, enjoyment and persistence in computer science. Study also examined the gender differences of PP and solo programming, but results were not statistically significant.  Result of the study: Completions rates were 90,8% for pairs and 80,4% for solos. Pass rates were 72,3% for pairs and 62,8% for solos. Course performance was measured the quality of the program, pairs had score 86,6% and solos 68,1%. Final exam results had no significant difference. Confidence in solution was 89,4% for PP and 72.2% for solos. Satisfaction was 6 for the pairs and 4,6 for the solos. Enjoyment of the work was 5,1 for the pairs and 4,7 for the solos. Percentage of the students declaring a computer science major 3 quarters after intro course was 70,8% for the PP students and 42,2% for the solos.

[38] Modeling Spontaneous Pair Programming When New Developers Join a Team  Authors: I. Fronza & G. Succi.  Subject of the study: Were 17 developers (15 veterans and 2 newcomers) from the large company that used spontaneous PP and other XP practices. Their work was analyzed for the period of 10 months.  Focus of the study: Observation of integration of the novices in the PP team.  Result of the study: Analysis identified 4 phases of novices integration: o In the “initiation” phase (first month) novices mostly work with the experts to be initiated to the new environment. o During the second phase, novices aim at achieving “independence”. For about two months novices work mostly alone, consolidating their knowledge. They work in the pairs with experts. o During the third phase novices gain a “maturity” status in the team. They start to use PP again, but mostly between themselves. This phase lasts about the period of 4 months. o The fourth phase is the complete “integration” of novices in the team. PP occurs with all the team members.

[41] The Effects of Pair Programming on Learning Efficiency in Short Programming Assign- ments  Authors: S. Kuppuswami & K. Vivekanandan.  Subject of the study: Were students who participated in short duration exercises (3 hours without a break and under the supervision of the laboratory instructor). Students were divided into two groups: PP group with 116 students and the traditional method group with 98 students..

 Focus of the study: Comparison of the learning efficiency of the students adopting PP with the traditional method (solo programming). Three parameters were controlled: quality of the design, speed of the laboratory task completion and the knowledge and programming skills gained (evaluated in the written test).  Result of the study: PP improved the design ability (design with the 11% better marks), reduced time to finish a laboratory exercise (time reduction was 19%) and increased knowledge and programming skills (the test marks were 13% better).

[42] Improving the CS1 Experience with Pair Programming  Author: N. Nagappan, L. Williams, M. Ferzli, E. Wiebe, K. Yang, C. Miller & S. Balik.  Subject of the study: Were 495 student in the period of two semesters. In the Fall participated 69 solo students and 44 PP students. Spring semester had 102 solo students and 280 PP students. Students were paired randomly. Student grades were based on two midterm exams, one final exam, lab assignments and programming projects (completed outside of the lab).  Focus of the study: Examination of the five hypotheses: o H1: Higher percentage of PP students will have C or better grade in comparison with the solo programming students. o H2: PP will lead to higher scores on the examination in comparison with the solos (the examinations were completed solo by all). o H3: PP will lead to the higher project scores on course projects in comparison with the solos. o H4: PP will lead to reduced workload in terms of grading, questions answered, and teaching effort for the course staff in comparison with the solo course. o H5: PP students will have positive attitude towards collaborative programming in comparison with the solos.  Result of the study: o H1: Results validated this claim for non computer science majors. o H2: Results have not validated this claim to a statistically significant level. o H3: Paired and solo programmers had comparable scores in the projects. In some cases PP students had marginally higher scores than the solo ones. o H4: Qualitative findings supported this hypothesis. o H5: Survey results supported this hypothesis.

[43] In Support of Pair Programming in the Introductory Computer Science Course  Authors: L. Williams, E. Wiebe, K. Yang, M. Ferzli & Carol Miller.  Subject of the study: Were 564 students (both undergraduates and graduates) in the period of two semesters in a introductory java programming course (CS1), software engineering course (SE) and OO course. Students had to first fulfill a web- based peer evaluation tool and assign 0 to 20 point to their partner. In accordance with 5 questions partner a score of 0%-100%. Score impacted a student’s grade. Students also evaluated partner’s skills.  Focus of the study: Examination of the following hypotheses: o H1: Pairs are more compatible if students with the different personality types are grouped together. o H2: Pairs are more compatible if students with the similar actual skill levels are grouped together. o H3: Pairs are more compatible if students with the similar perceived technical competence are grouped together. o H4: Pairs are more compatible if students with the similar self-esteem are grouped together.  Result of the study: o H1: Hypothesis was supported in the CS1 class. However, this same result was not found in the SE class. o H2: There was no association between the actual skill level and the compatibility of the CS1 and the SE students. However, there is a strong positive cor- relation between the actual skill level and the compatibility of the graduate OO students. o H3: There exists a significant positive relationship between compatibility and the perceived skill level of the partner in all three computer science courses. o H4: There is a weak negative association between the self-esteem and the compatibility of students.

[44] Code Warriors and Code-a-Phobes: A Study in Attitude and Pair Programming  Authors: L. Thomas, M. Ratcliffe & A. Robertson.  Subject of the study: Were first year students (more than 60 students) with prior programming experience. Student were asked to complete a web questionnaire. Based on the questionnaire, students place themselves on a scale from 1 to 9. 9 was a Code-Warrior (no trouble with completing all programming tasks, I love to program, I anticipate no difficulty with this course) and 1 was a code-a-phobe (I do not

like programming and I am not good at it, I have problems with writing new programs). After the evaluation the students were asked to develop routine PP task. They were grouped in accordance with the questionnaire. First the same types and in the second attempt opposite types. Students were asked to write reaction about cooperation and their programs were evaluated.  Focus of the study: Examination which student types suits best for the cooperation in PP, enjoyment of these combinations.  Result of the study: Students with the similar attainment place themselves at very different places on the warrior-phobe scale. Students liked PP and believed that it helps them to achieve a good solution. Students with less self-confidence seemed to enjoy PP the most. The students identified as warriors liked PP the least. There is some evidence that warriors like PP even less if they were paired with phobes. Stu- dents produced their best work when paired with the students of similar, or not very different, level of self-esteem, but the results of the study are not statistically significant because of small the sample size.

[50] Pair Programming Improves Student Retention, Confidence, and Program Quality  Authors: C. McDowell, L. Werner, H. E. Bullock & J. Fernald.  Subject of the study: Were 554 students who attempted the programming course. Authors analyzed 4 sections of the course. Some of the sections used PP, some used solo programming. Pairs were matched in accordance with the student preference list. Those, who stated no preferences were matched randomly. All students took the exams independently.  Focus of the study: Investigation of the effects of PP on the student’s performance in the introductory programming course and the persistence in computer science related majors.  Result of the study: PP improves course completion and course pass rates. PP contributes to the persistence in computer science major. Students who used PP were more likely to pass the subsequent programming course (which was solo programming). PP students produced higher quality programs in comparison with the solos and they were more confident in their work and also enjoy it.

B. Compatibility Questionnaire This appendix provides a questionnaire, which should be recommended for developers to be completed before the first pairing. In the test we are identifying developer’s personality type, self-esteem, skills and knowledge. It is recommended to combine partners with different personality types, same self-esteem and the same skills and experiences. For the academic environment, students fulfill first two questions, question 3 about skills and experiences will be skipped and students will be paired based on their academic achieved level.

Compatibility test questionnaire: 1, Encycle what is typical for you in the following 4 questions [74]: a, Do you prefer to focus on the outer world (Extraversion) or on your own inner world (Introversion)? – Extraversion – Introversion b, Do you prefer to focus on the basic information you take in (Sensing) or do you prefer to interpret and add meaning (Intuition)? – Sensing – Intuition c, When making decisions, do you prefer to first look at logic and consistency (Thinking) or first look at the people and special circumstances (Feeling)? – Thinking – Feeling d, In dealing with the outside world, do you prefer to get things decided (Judging) or do you prefer to stay open to new information and options (Perceiving)? – Judging – Perceiving

2, Evaluate yourselves on a 1 to 9 scale with the following meaning:  1 = I do not like programming, and I think I am not good at it. I can write simple programs, but have trouble writing new programs for solving new problems.  9 = I have no problems at all completing programming tasks to date, in fact they we- ren’t challenging enough. I love to program and anticipate no difficulty with this course – 1 – 2 – 3 – 4 – 5 – 6 – 7 – 8 – 9

3, Encycle your academic level, experiences and skill level. a, What is your academic level? – Undergraduate – Graduate b, What is your experience level? – Experiences with software development only from school projects. – Up to 2 years of experiences of software development. – Up to 4 years of experiences of software development. – More than 4 years of experiences of software development. c, Third part of the question 3 should evaluate skill level of the developer, but it depends on various aspects (the programming language, type of the project, etc.) in the particular IT company.