Software Development Methods and Usability : a Systematic Literature Review

Linköping University | IDA 30 hp/Master Thesis |Computer Science Autumn term 2017 | LIU-IDA/LITH-EX-A—17/055--SE

Software Development Methods and Usability : A Systematic Literature Review

Prabhu Raj Prem Kumar

Tutor, Johan Åberg, Linköpings universitet Examiner, Kristian Sandahl, Linköpings universitet

Final Thesis Software Development Methods and Usability: A Systematic Literature Review

by Prabhu Raj Prem Kumar

[email protected]

Supervisor: Johan Åberg, Linköpings universitet

Examiner: Kristian Sandahl, Linköpings universitet

Linköping, 4th December, 2017

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under en längre tid från publiceringsdatum under förutsättning att inga extra-ordinära omständigheter uppstår.

Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns det lösningar av teknisk och administrativ art.

Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart.

För ytterligare information om Linköping University Electronic Press se förlagets hemsida http://www.ep.liu.se/

The publishers will keep this document online on the Internet - or its possible replacement - for a considerable time from the date of publication barring exceptional circumstances.

The online availability of the document implies a permanent permission for anyone to read, to download, to print out single copies for your own use and to use it unchanged for any non- commercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional on the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility.

According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to its WWW home page: http://www.ep.liu.se/

Abstract

This thesis provides a tour of main software development methods and their assimilation with usability. Various software development methods have been developed since the origin of software products, and with integration of usability they have achieved great heights. Software developers and researchers are also keen to integrate usability with their methods to develop a successful software product, as their product might not lose its grip over their competitors.

This thesis is a preliminary literature review that provides solutions to key questions such as:

Which development methods are used in projects with high demands on usability? Do agile methods positively contribute to the usability of the product? Do projects with high demands on usability use usability specialists? Which types of usability are addressed in articles relating development methods and usability?

The research aim is to identify how software development methods deal with usability. This thesis clearly discusses about previous empirical studies on software development methods and usability.

In this thesis, Kitchenham’s systematic literature review was followed to extract the knowledge from the databases (Inspec and ACM). Validation on the literature was performed using Cohen’s kappa value with a researcher. Four research questions mentioned above are discussed in detail to analyse the results.

Agile software development methods have been widely deployed due to its iterative, incremental approach and positive contribution towards usability rather than traditional approach. The role of a usability expert has been analyzed in this thesis and their need in the development process was discussed. Furthermore, various usability types like satisfaction, efficiency and effectiveness have also been discussed in detail. Finally, this thesis suggests a solution of incorporating usability with software development methods.

iii

Acknowledgements

At first I would like to thank, Johan Åberg and Kristian Sandahl for giving me this opportunity. It is their vision that inspired me on this topic, and they guided me in the initial days of thesis and spending their valuable time till completion.

I like to thank all my teachers and professors who taught me knowledge, wisdom and discipline. It is them who made me the person, whom I am today. Also I like to thank all my childhood and college friends for their support, encouragement and faith they had on me. They made me believe that “You can do anything, if you started to try it”. I could not imagine my life without them.

I would also like to thank all my room-mates during my entire course period. They had been always attuned to me and very helpful to fulfill my dreams. It is them who made me stay happily for the entire period and finish this work successfully.

I love to thank my brother, who always stood by my side, and never doubted me in my doings. Without him, I could not be good as now.

At last, I thank my parents, the two persons who made me from their tears and joy, and brought me to this life with their love and gave me everything which they can ever give to anyone.

Linköping, December 2017

Table of Contents Copyright ...... i Abstract ...... ii Figures ...... viii Tables ...... viii 1. Introduction ...... 1 1.1 Research questions ...... 1 1.2 Thesis organization ...... 2 2.1 Background ...... 3 2.1.1. Usability ...... 3 2.1.1.1. Formative usability ...... 4 2.1.1.2. Summative usability ...... 5 2.2. Software development methods (SDM) ...... 6 2.2.1. Traditional methods ...... 6 2.2.1.1. Waterfall model...... 6 2.2.1.2. V-model ...... 7 2.2.1.3. Spiral model ...... 7 2.2.2. Agile methods ...... 8 3. Method ...... 10 3.1 Search strategy ...... 10 3.1.1 Keywords ...... 10 3.1.2 Search queries ...... 11 3.2 Study selection criteria and procedure ...... 12 3.3 Study quality assessment ...... 13 3.4 Validation of selection procedure ...... 13 3.5 Calculation of Kappa value ...... 15 3.6 Data extraction ...... 17 3.7 Data synthesis ...... 17 4. Results ...... 18 4.1 Research questions ...... 18 4.1.1 Which development methods are used in projects with high demands on usability? ...... 18 4.1.2 Do agile methods positively contribute to the usability of the product? ...... 22 4.1.3 Do Projects with high demands on usability use usability specialists? ...... 25 4.1.4 Which types of usability are addressed in articles relating development methods and ...... 28 usability? ...... 28

5.1 Discussion ...... 32 5.2 Limitations ...... 35 6. Conclusion ...... 36 7. Appendix ...... 37 8. References ...... 39

vii

Figures Figure 1: ISO 13 407 Model Overview…………………………………………………………...4 Figure 2: Waterfall model…………………………………………………………………..….…6 Figure 3: Screenshot of query 1…………………………………………………………….……11 Figure 4: Venn diagram for validation………………………………………………………...... 14 Figure 5: Venn diagram for validation of query 1……………………………………………….15 Figure 6: Venn diagram for validation of total queries…………………………………………..16

Tables Table 1. Inclusion and exclusion criteria………………………………………………………....12 Table 2. Strength of agreement table………………………………………………………….….14 Table 3. Data of the search queries (kappa value)…………………………………………….….16 Table 4. Data synthesis table for research question 1………………………………………….…19 Table 5. Data synthesis table for research question 2………………………………………….…22 Table 6. Data synthesis table for research question 3………………………………………….…26 Table 7. Data synthesis table for research question 4………………………………………….…28 Table 8. Combine approach of usability and Agile methods………………………………….…..34 Table 9. Study quality assessment table…………………………………..………………………37

viii

Chapter 1

1. Introduction

Software is the group of instructions that is used along with hardware components to form a computer system or an application. There is a saying that “Rome was not built in a day”, likewise software is also not designed in a day. It has taken over many decades to reach this level. The software engineering deals with software implementation using Software Development Methods (SDM). There are various SDM available which are used to develop software, and it is likely that without proper study of these methods, the product cannot be successful. Lack of user friendliness creates problem while developing software. A user who uses the application or computer must not feel any discomfort which may lead to the failure of the software. Thus, usability field has evolved along with the software development, which makes software easy to use so that users are able to develop the product with effectiveness, efficiency and satisfaction.

Many researchers have discussed the incorporation of usability in software development methods which created an interest in me to study and analyze their implications and its difficulties while developing software application. A systematic review procedure has been performed based on Barbara Kitchenham’s methods.

1.1 Research questions

RQ1. Which development methods are used in projects with high demands on usability? RQ2. Do agile methods positively contribute to the usability of the product? RQ3. Do projects with high demands on usability use usability specialists? RQ4. Which types of usability are addressed in articles relating development methods and usability?

In order to understand the empirical relationship between software development methods and usability and to identify software development methods and usability to be implemented for a successful project, the above research questions are investigated in this thesis.

1.2 Thesis organization

The report is systematized as follows.

Chapter 2: Theory

This chapter deals with preliminary background of the related areas. Firstly, overview of usability, usability engineering, types of usability data, usability evaluation and, usability testing are detailed, which is followed by discussion of software development methods like traditional and agile methods.

Chapter 3: Method

In this chapter, Kitchenham’s method for performing a systematic literature review is described. And then the implementation of the methods in this project work and their various outcomes such as search strategy, study selection criteria and procedure, study quality assessment technique, calculation of kappa value, data collection, data extraction and data synthesis are clearly shown.

Chapter 4: Results

This chapter presents the results and are tabulated by applying the systematic review protocol stated by chapter 3.

Chapter 5: Discussion

This chapter provides the discussion, limitations and results obtained from chapter 4.

Chapter 6: Conclusion

This chapter concludes the results obtained from the research questions.

Chapter 2

2.1 Background

In this Section, literature on Usability and Software development methods is presented.

2.1.1. Usability

The usability according to International Standards Organization (ISO 9241-11) “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.”[18]. In simpler terms “quality in use” [17].The challenge is to develop software or an application which works well according to software engineering. A Key methodology for carrying out usability is called User-Centered Design (UCD) [38]. Quality of a user’s experience (UX) is measured by usability when interacting with system whether a web site, a software application, or any user-operated mobile device [38]. According to Vredenburg et al., UCD is “An approach to designing ease of use into the total user experience with products and systems. It involves two fundamental elements – multidisciplinary teamwork and a set of specialized methods of acquiring user input and converting it into design”. [39]. Usability has demands based on every project; the demands are based on the user requirements and implementation of the development methods in the project.

Usability Engineering is the process of achieving usability in a system by applying certain set of methods and techniques like usability evaluation, heuristic evaluation and usability testing. Usability evaluation, is implemented in early stages of design process to find alternative user interfaces and to identify preferred design [21]. In Heuristic evaluation user interface designers study the interface and look for properties that they know lead to usability problems [24]. Another approach is usability testing, the designer studies the user performing the tasks and gathers the data on the problems that arise [40]. One of the main goals of usability engineering is to improve the user interface of the targeted systems [21, 24]. Usability Engineering is implemented on the period of time ranging from project inception to system delivery. Engineering is not the process of building a perfect system with infinite resources. Rather engineering is the process of economically building a working system that fulfills a need. If we cannot measure usability, we cannot have usability engineering [24].The Usability model consists of five Components [25] in the design process, they are

• Learnability: How easy is it for users to accomplish basic tasks the first time they encounter the system? • Efficiency: Once users have learned the system, how quickly can they perform tasks? • Memorability: When users return to the system after a period of not using it, how easily can they re-establish proficiency? • Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors? • Satisfaction: How pleasant is it to use the system? [25]

The Usability Model is comprised of five stages, four of which are implicitly joined in a loop. [23],The usability model is similar to software engineering incremental model. The usability model can be broken into single waterfall model, or a series of waterfalls, commonly known as Evolution model, until the final objective is reached. It is very important not to progress to the next step until all the objectives set out to be fulfilled.

Figure 1: ISO 13 407 Model Overview The ultimate goal of usability is how the system must be used by the user efficiently. In some situations, the systems are very complicated hence usability implementation is not easy. Usability must be implemented for every product to become a successful one. The usability study defines how the data collected from the users will ultimately be used within the product development life cycle. There are two ways to use the data formative and summative usability.

2.1.1.1. Formative usability

According to formative study, a usability expert is much like a carpenter who sporadically checks a wood while it is prepared and makes adjustments to positively impact the end result. The carpenter may add paints, varnishes, and finally providing wax coating right before selling. The same way as the carpenter, the usability expert sporadically evaluates a product or design, identifies shortcomings, makes recommendations, and then repeats the process, until the product reaches close to the perfect state. The formative usability is always done before the design has been finalized [37].

From formative approach we will be able to answer key questions regarding usability:

• What are the most significant usability issues that are preventing users from completing their goals or that are resulting in inefficiencies?

• What aspects of the product work well for the users? What do they find frustrating? • What are the most common errors or mistakes users are making? • Are improvements being made from one design iteration to the next? • What usability issues can you expect to remain after the product is launched?

Formative usability study is used to improve the design process. If there is no need to change the design, then formative usability is just a waste of time. The biggest obstacles of this method are the limited budget or time [37].

2.1.1.2. Summative usability

The goal of summative usability is to find how well a product of functionality meets its objectives. It also compares several products to each other.

From summative approach we will be able to answer key questions regarding usability:

• Did we meet the usability goals of the project? • How does our product compare against the competition? • Have we made improvements from one product release to the next?

The summative usability test always involves some follow up activities. The follow up actions must be planned well before taking the summative usability study. Formative testing focuses on identifying ways of making improvements, summative testing focuses on evaluating against a certain set of criteria.

A formative approach involves collecting data to help improve the design before it is launched. It is most suited when you have a positive impact on design of the product. A summative approach is chosen when you want to measure the extent to which target goals were achieved. Summative testing is sometimes used in competitive usability studies [37].

Planning a usability study needs to follow some key points [37],

• The first step is to decide whether we are going to take formative or summative approach. • Two main aspects of the user experience, performance and satisfaction, must be considered. • Budgets and timelines are planned well in advance in any usability studies involving metrics. • Three general types of evaluation methods are used in collection of usability data. ▪ Lab tests with small numbers of participants, ▪ Lab tests with large numbers of participants, ▪ Online studies with a very large number of participants.

• Participants recruited for usability testing must be clearly identified based on the criteria. • Plan well in advance how we are going to capture all the data you need for the study. • Data clean up involves manipulating the data in a way to make them usable and reliable.

2.2. Software development methods (SDM)

2.2.1. Traditional methods

There are various traditional software methods available in literature, some of which are discussed in this section.

2.2.1.1. Waterfall model

The waterfall model, also known as the cascade model developed in the mid of 1950’s and modified in the 70’s by Winston Royce [41], forming new set of principles which are followed even now. The method uses sequential design process.

User Requirements DesignImplementationVerificationMaintenance.

The iteration goes step by step and feedbacks are given back to levels from bottom levels. The waterfall model is a Top-Down approach. The process model cannot execute simultaneously thereby every team in the model has to wait for their superior level to finish their job and transmit it to the next stage.

User Requirements

Design

Implementation

Verification

Maintenance

Figure 2. Waterfall Model

The design process does not perform testing at each level, which can lead to a huge loss to project, or more expenses. The requirements must be specified by the user correctly otherwise it leads to unnecessary complications which lead to a failure. As the testing is the penultimate process of this method, finding errors in the implementation is alright, whereas finding in the design stages raises the quality of the product. As it has to be re-designed and implemented again, change lines of codes, etc. It is a slow process as it proceeds in sequential manner. Royce enhanced the waterfall model by providing a feedback loop so that each preceding stage could be revisited, and suggested that a prototype round is executed before the main project.

The waterfall model is most efficient way in relational databases, compliers, and secure operating systems. This method is used to create applications or software which provides back- end functionalities [30]. The B-model is an extension of waterfall model when an operational life cycle is attached to the waterfall model. Thus, creating a constant improvement of the software or system, would become part of development stages [30]. The incremental model is a three- dimensional representation of waterfall model. The z-axis contains a series of waterfall models to represent the number of iterations that would be made in order to improve the functionality of the end product incrementally. Incremental model is similar to the modification between the waterfall model and spiral model [30].

2.2.1.2. V-model

The V-model which has identified to have slight variation in comparison to the waterfall model was developed by NASA [14]. This model is implemented in a V shape. The left hand of the model shows the definition and decomposition, whereas the right hand shows the integration and verification. The V-model is mostly used in back-end functionalities and vee++ model is used in all back-end, front-end, and end-user application [30].

2.2.1.3. Spiral model

The spiral model is also the modified version of waterfall model, which makes a small start and proceeds to achieve big thing. It starts from a small point and proceeds level by level [9] . The levels are determined by Boehm’s spiral life cycle,

1.Determine objectives. 2.Identify and resolve risks. 3.Development and Test. 4. Plan the next iteration.

If all the levels are completed it starts again to iterate from the level 1 and follows the same step with different items to evaluate than the previous one, thereby producing a good project at the end. The spiral model is mostly used in back-end functionalities and it can be implemented to front-end and end-user applications with some modifications [30].

2.2.2. Agile methods

Agile approach is performed in a highly-collaborative manner with customers to produce quality products that congregate the requirement specifications with cost-effectiveness and time [16]. The agile model is iterative and incremental, which divides a project into smaller sub projects and implements the procedure. The planning and the development take place simultaneously. The Agile method will work well, when finding the right priorities. The higher priorities must be identified at first and executed in the iterative and incremental method. The developers who develop the code and testers who test the results work side by side to determine the right test code [32]. The greatest advantage to Agile is re-visiting the phases of the lifecycle, which improves the quality of the project and thereby reducing the development time. The idea of revisiting phases over and over is called “incremental and iterative development” (IID). The main aim of the agile method is to identify the right priorities [36]. The Agile method is more suited for software which provides the user interface to the end user [30].

Agile method is also divided into sub Categories like waterfall model. Extreme Programming (XP) [6], Feature Driven Development (FDD) [27], Adaptive Software Development (ADP) [15], Crystal clear [11], Dynamic System Development Method (DSDM) [35] and SCRUM [31] are a few well known agile methods. Extreme programming is based on the user requirements and their functionalities. It takes fast spiral steps and programmers work in pairs. The lean development attempts to deliver projects early with minimum functionalities. Scrum is best suited to small projects and it is based on process driven framework [30]. Scrum introduces the empirical process control concept for the complex and changing software projects [36].

Agile software development can be easily understood by twelve principles and four key values, which are designed by Agile alliance team manifesto, are [2],

• The highest priority is to satisfy the customer through early and continuous delivery of valuable software. • It has ability to welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. • Business people and developers must work together daily throughout the project. • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. • Working software is the primary measure of progress. • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

• Continuous attention to technical excellence and good design enhances agility. • Simplicity--the art of maximizing the amount of work not done--is essential. • The best architectures, requirements, and designs emerge from self-organizing teams. • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

The four key values with regard to Agile methodology are [2]:

▪ Individuals and interactions over processes and tools; ▪ Working Software over comprehensive documentation; ▪ Customer Collaboration over contract negotiation; ▪ Responding to change over following a plan

Agile software teams implement the system by breaking large tasks, into smaller sub-tasks with least amount of planning [1]. Iterations are normally constraint within short time frames, called time-boxes, a time-box is a previously agreed period of time during which a person or team works steadily towards completion of some goal. Rather than allow work to continue until the goal is reached, and evaluating the time taken, the time-box approach consists of stopping work when the time limit is reached and evaluating what was accomplished [10]. Agile also has same activities as waterfall model, planning, requirement analysis, design, coding, and testing. Agile provide projects to adapt changes quickly and the profits are reduction of total risks and time. But it is merely possible to develop a product in a single iteration which has enough functionalities, but the idea is to have an available release with minimum errors at the end of each iteration. Therefore, a product release can take multiple iterations [16].

Chapter 3

3. Method

In this section, the literature elaborates the methods implemented for producing the results. It all started from creating a review protocol based on the Kitchenham’s method. The purpose of the review protocol is to specify a plan, identify and collect proofs on the advantages and limitations on software development methods and usability [3].

The literature review was based on the following objectives of Kitchenham’s method,

• Undertake a systematic review of empirical research on the benefits and limitations of software development methods and usability. • Select a sub-set of studies to review in-depth. • Synthesize the evidence from these studies about the benefits and limitation of software development methods and usability. • Identify the research methods used. • Identify any gaps in current research in order to suggest areas for further investigation. • Provide a background in order to appropriately position new research activities.

3.1 Search strategy

3.1.1 Keywords

The following keywords were selected: based on the research questions, by learning the keywords, they had huge impact on understanding the ideas on the topic and analyse their results.

Usability, Waterfall, Software Development methods, and Agile.

The Articles in the literature review include empirical data. The search strategies have taken from electronic databases and hand searches of conference proceedings. The databases are • ACM digital library • Inspec

3.1.2 Search queries

1. agile AND usability, (English only and from 1994 ) in Inspec

Figure 3. Screenshot of query 1

2. (waterfall software development) in AND, (agile usability) in OR, and (Keywords:agile) (from1994) in ACM

3. ((Agile Software development methods) in TITLE) AND ((agile methods) in KEYWORD) AND ((software development) in KEYWORD)) (English only and from 1994) in Inspec

4. (software development)in AND, (agile) in OR, and (Keywords:sdlc) (from 1994) in ACM

5. Waterfall (title or abstract and from 1994) in ACM

6. Usability engineering lifecycle, (English only) in Inspec

7. Agile in title AND test methods, (English only and from 2004) in Inspec

3.2 Study selection criteria and procedure

When the search was made, the title and the abstract of all articles found in the digital libraries were extracted using the keywords. At first, Title and Abstract of each article from the library was reviewed against the inclusion and exclusion criteria. Duplicate references were eliminated. There were some articles that provided the background of the project and those does not satisfy the limitations was included, forming them has a candidate articles. The candidate articles were reviewed again against the inclusion and exclusion criteria. The selected articles were analyzed in detail. It went through the study quality assessment checks and if it was relevant then the article was listed in the reference.

It is not necessary that all the criteria must be satisfied for the inclusion and at least one of the criteria is enough for exclusion criteria described below,

Table 1: Inclusion and exclusion criteria table

Inclusion Criteria Exclusion Criteria

• All the articles, journals, conferences, • Articles not related to usability and books discussing usability or software software development methods in development methods in Human HCI and software engineering is Computer Interaction (HCI) and excluded from the studies. software engineering are considered. • Articles are not taken if it is published • Articles relating to both development before 1994. methods and demands on usability. • Articles presenting on usability but • Articles satisfying whether or not agile not discussing SDM are not methods contribute positively to the considered. usability of the product. • If the Articles do NOT present • Articles relating to demands on usability empirical data will be excluded. and usability specialists.

• Articles written in English language are only included. • Articles that contributes towards the origin of software development methods.

3.3 Study quality assessment

The study quality Assessment forms the list and the aggregate quality criteria. It is to provide a rough draft to the quality of the paper chosen. We have used score or values to understand the quality of the criteria and cannot compare quality of different papers as based on the score alone. The score procedure is Yes for 1, No for 0, and Moderate for 0.5. The total quality score can be calculated at last. If the first two questions receive a “No” then response will not continue with quality assessment [13]. (See in Appendix)

3.4 Validation of selection procedure

As the articles were selected based on the above search strategy and study selection methods, the validation of selection procedure is necessary. This can be achieved by evaluating by the researcher. The researcher performs the same steps described by author and provides the results; and can tabulate the results such as, by comparing both the results of author and the researcher and can calculate the reliability of the selection procedure.

For example, two researchers are being agreed to some implications before the start, researcher ‘A’ and researcher ‘B’. There are two categories selected such as “Included” and “Excluded” articles. Based on these categories, researchers can choose the value of the particular section. In this thesis researcher ‘A’ is the author and researcher ‘B’ is the researcher.

Groups are formed as shown below A= Articles included by both researcher A and B. B= Excluded articles by researcher A and included articles by researcher B. C= Excluded articles by researcher B and included articles by researcher A. D= Excluded articles by both researcher A and B.

Researcher A

Included Excluded

A B Researcher B Included

Excluded C D

Figure 4: Venn diagram for validation

N = 퐴 + 퐵 + 퐶 + 퐷 , where N is total number of articles.

퐴+퐷 Po = , where Po is proportion of units in which there is an agreement. 푁

퐴+퐵 퐴+퐶 퐵+퐷 퐶+퐷 Pc = (( ∗ ) + ( ∗ )) , 푁 푁 푁 푁

Where Pc is the proportion units for which agreement is expected by change.

Po−Pc K= , where K is the Kappa Value 1− Pc

According to the Landis et al., strength of agreement depends the kappa value (K) [29], The following table shows the kappa values for different ranges and strength of agreement.

Table2. Strength of agreement table Kappa Value Strength of agreement < 0.00 Poor 0.00 – 0.20 Slight 0.21 – 0.40 Fair 0.41 – 0.60 Moderate 0.61 – 0.80 Substantial 0.81 – 1.00 Almost perfect

We must note that both the author and the researcher calculated the kappa value for the first time and the goal of the thesis is more to demonstrate the method instead of claiming good correlation.

3.5 Calculation of Kappa value

Query 1: agile AND usability, (English only and from 1994) in Inspec, where A:5,B:2,C:7,D:21 and N:35

Figure 5: Venn diagram for validation of Query 1

Query 2:

(waterfall software development) in AND, (agile usability) in OR, and (Keywords: agile) (from1994) in ACM, where A:2,B:2,C:7,D:34 and N:45

Query 3:

((Agile Software development methods) in TITLE) AND ((agile methods) in KEYWORD) AND ((software development) in KEYWORD)) (English only and from 1994) in Inspec, where A:2,B:2,C:3,D:19 and N:26

Query 4:

(Software development) in AND, (agile) in OR, and (Keywords: sdlc) (from 1994) in ACM, where A:1,B:0,C:1,D:1 and N:3

Query 5a:

Waterfall (title and from 1994) in ACM, where A:2,B:1,C:2,D:5 and N:10

Query 5b:

Waterfall (abstract and from 1994) in ACM, where A:2,B:1,C:5,D:42 and N:50

Query 6:

Usability engineering lifecycle, (English only) in Inspec, where A:3,B:2,C:3,D:42 and N:50

Query 7:

Agile in title AND test methods, (English only and from 1994) in Inspec, where A:2,B:2,C:5,D:36 and N:45

Table 3. Data of the search queries (kappa value)

s.no. A B C D N 1 5 2 7 21 35 2 2 2 7 34 45 3 2 2 3 19 26 4 1 0 1 1 3 5a 2 1 2 5 10 5b 2 1 5 42 50 6 3 2 3 42 50 7 2 2 5 36 45 Total 19 12 33 200 264

where A:19,B:12,C:33,D:201 and N:264

Figure 6: Venn diagram for validation of total queries

Where N = 19+12+33+200

19+200 Po = => 0.83 264 Pc = ((0.12 ∗ 0.20) + (0.80 ∗ 0.88)) => 0.72

0.83−0.72 K= => 0.39 1−0.72

Thus, by tabulating the values and calculating the kappa value by the formulas described above, we have received a value of 0.39, which is fair according to the strength of agreement. So, we can proceed further with the data extraction and data synthesis processes.

3.6 Data extraction In this section, data are extracted from the selected articles. The following data are extracted and filled in the respective fields of the data extraction table.

• Title • Author • Database • Research questions • Quality • Industry • Study Aim • Development methods analyzed • Usability • Usability demands • Usability expert • Final discussion/ conclusion • Additional notes/comments

The table shows a clear view of the data extraction procedure; it contains the included papers, their title and the name of the author. Electronic databases from which article taken and the next column it produces the addressed research questions from the article. The quality column provides, the quality of the articles which are ranked, the article which has five-star rating is good and four star to two star rating is moderate and only one star rating is bad. The quality is ranked from the study quality assessment table. By adding the Industry column in that table, it provides the information that the article is used in Industry or academic purpose. The next column provides the data of the study aim of the articles. The next column provides the data of which development methods are used in the article, which parts of usability is attained, their usability demands which are ranged from low to high. Low demand in usability is that usability is not taken into consideration; medium demand in usability means usability is taken into consideration but not implemented. High demand in usability means usability is taken into consideration and implemented successfully and whether a usability expert is used or not. Final discussion/conclusion is also added to provide information about results and conclusions of the articles. At last additional notes/comments column is added to provide more data.

3.7 Data synthesis

The final stage of the systematic review protocol is data synthesis. The findings are examined and mapping between similarities and dissimilarities takes place. The collected data or information had to be matched with the specific research questions. In this thesis, software development methods and usability are organized according to research questions (discussed in the results chapter).

Chapter 4

4. Results

A systematic review has been performed based on the methods prescribed by Kitchenham. The search results provide a total of 264 articles based on the search questions, in which 200 articles are (discarded by both author and researcher) irrelevant to the topic and then a comparison is made between author and researcher. Kappa value has been calculated, a value of 0.39 has been attained. Thus, from the selected articles, which focus on empirical analysis between software development methods and usability, articles are analyzed and results are been tabulated.

4.1 Research questions

Furthermore, answers for the research questions based on the selected articles are discussed in this section

4.1.1 Which development methods are used in projects with high demands on usability?

For this research question, development methods and usability demand (3rd and 4th columns respectively) that were obtained from the included papers (1st column) which have been tabulated in Table 3.

The selected articles addressed the tensions, needs, developments, ideas, mechanisms, implementations, difficulties, improvements, architectures, standards, challenges, highlights, estimations between software development methods and usability. When the research results were analyzed out of 19 total articles, 13 articles stated that agile development methods were used in the projects with high demands of usability, whereas remaining 6 articles states traditional and agile methods were used based on their demands.

The development methods can be grouped into two basic approaches, traditional and agile. Agile methods have been widely deployed nowadays rather than the traditional approaches. As in the traditional model user requirements are to be specified at earlier stages of the development, which may lead the project to failure when additional requirements are added at the final stages of the project. As some customers are unable to specify their requirements fully at the early stages, so the developers find a new way of solving the problem by deploying agile methods [36].

Table 4. Data synthesis table for research question 1

Titles Authors Development Usability Final discussion/conclusion Notes/comments methods demand Towards Lee JC, Agile High It gives four guidelines for Revises the extreme(ly) McCrickard usability specialists and agile tension between usable DS practitioners. usability and agile software: method. Exploring tensions between usability and agile software development. Hitting the Jeff Patton Agile High By implementing Agile U-CD Development of Target: process the drawback of agile using Adding Extreme Programming is usability in Interaction eliminated. Tomax Design to Technologies Agile Software Development Delivering Christopher Agile High The MCT project has taken the Making the Software into Webster, described principles and software to certify NASA's Nija Shi and adapted the agile process to fit for operational use Mission Irene in with delivery into the of the company. Control Center Skupniewicz mission control center. using Agile Smith Development Techniques Concept Jeremy Agile High Results indicate that our Lessons learned Mapping in Barksdale concept mapping approach has from the Agile and D.Scott value as a visual tool in agile collaboration of Usability: A McCrickard usability environments. both agile and case Study usability. Easing team Jeremy Agile High Results indicate that there is Need for Politics in T.Barksdale, need for improvement in agile improvement in Agile usability Eric usability, and concept agile usability. D.Ragan, mapping approach is D.Scott promising for addressing McCrickard existing concerns. U-SCRUM : Mona Singh Agile High U-SCRUM is best suited for The architecture is An Agile (U-SCRUM) products that are complex and not used by the Methodology novel. When the topic is well agile community for promoting understood, there is less of an Usability advantage. Extreme Kathy Agile High After the panelist The panel Usability: Baxter, summarization with Agile, members share Adapting Kuldeep will lead to discussion about their challenges Research Kelkar, Lynn key challenges faced by user and successes Approaches Miller, Jeff researchers in Agile faced during Patton environments. development.

for Agile Development Delivering Christopher Agile High Transformation from Implementation of Software into Webster, traditional methods to Agile agile methods in NASA Nija Shi, methods which are made to NASA mission Mission Irene develop successfully control center. control center Skupniewicz delivered multiple customer- using Agile Smith selected releases into the Development Mission Control Center Techniques operational environment. Usability and Arlen Agile High Extreme Programming has Creating more User Interface Bankston, used for realms like user useful and usable Design in XP CC Pace interface elegance, cohesion application (White and usability. And they increases chances paper) address the importance of of success. these attributes rather than methods for attaining them. Adding Gerard Agile High A Canadian pacific railway The paper Usability Meszoras, project has been split into two prototype proved testing to an Janice Aston releases, One before usability to be useful in Agile Project testing and the second release many more ways with usability testing. than just for the usability testing.

Two Case Maryam Agile High The cases studies presented in Consistent user studies of user Najafi, Len this paper describe two testing and experience Toyoshiba examples of UED and Agile refinement of design and integration at designs ensures agile VeriSign. that the product is development developed to meet The needs of its users.

Little design Sisira Agile High We also present the results of Results show that Up-front: A Adikari,, two agile projects to validate users find Design Craig the proposition that products Science Mc’Donald, incorporating UCD developed using approach to John perspective into Agile this approach integrating Campbell Software Development easier to learn, usability into improves the design quality of easier to use and Agile software systems. require less Requirements support to be able Engineering to use.

Finding a David Kane, Agile High Discount usability The paper place for SRA engineering should be of introduces us to Discount International particular interest to the agile Discount usability Usability (White development community engineering Engineering in Paper) because the two disciplines approach for use Agile share many of the same with Scrum. Development: underlying principles. Throwing

Down the Gauntlet An Industrial Hassan Scrum Low The organization follows the Challenges faced Case Study for Hajjdiab, Al traditional waterfall software by agile which Scrum Shaima development model. Many caused adoption to Adoption Taleb, challenges were faced by the fail. Jauhar Ali agile teams which caused the adoption to fail. An Victor S Agile and Low The improvement to smaller Introduces to agile introduction to (White Waterfall projects using iterative and its methods. agile software Paper) processes as opposed waterfall development model. Agile is widely replacing waterfall model that itself will change and evolve well into the future. Transitioning Susan Agile Low The report describes the This paper a team to D.Shaye challenges and initial steps focuses on the Agile test taken to move a large changes in test methods development team from strategies traditional waterfall test necessary to phases to agile test method. support agile.

An Serena Agile and Low They purpose a methodology Transformation introduction to Software Traditional. neutral to both agile and more from Waterfall Agile Inc, traditional serial development model to Agile Software (White processes, so they can support methods. development paper) all the development activities within an enterprise. Continuous Puneet Agile Low A variant of SCRUM, Continuous SCRUM: Agarwal continuous SCRUM has been SCRUM should Agile adopted for this project has it be deployed only Management will be the best suited method. if there is of SAAS sufficient role for products frequent enhancements and requires significant involvement from business-users and product- owner. Agile Gangjun Agile Low Finally, agile industrial design The paper Industrial Yang, Sui management mode based on introduces Scrum design Huai Yu, Scrum is constructed. It has used in software Management Guoding solved the problem of rapid development and based on Chen, Jianjie response ability in meeting analyzes its Scrum Chu the quality requirements of usability in the customers. industrial design.

4.1.2 Do agile methods positively contribute to the usability of the product?

For this research question, development method and usability contribution (3rd and 4th columns respectively) that were obtained from the included papers (1st column) which have been tabulated in Table 4.

Usability is fulfilled by achieving effectiveness, efficiency and satisfaction and it works along with software development methods to make a successful project. Agile development methods with its iterative and incremental approach has been well deployed with usability.

Agile development methods positively contribute to usability only when agile developers and usability specialists share design documents and artifacts whenever possible, work hard for continuous interface improvement, integrate usability into day to day tasks, and avoid team members overspecialize in one area [22]. Agile development methods in general depend on a correct and complete understanding of customer needs of what the system is to do and how it is to be structured to support the customer organization. Whereas in usability technique, a project is broken into small blocks for implementation, but as a whole it implicates as a single system. Customer–Centered Design (CCD) approach supports agile from an in-depth understanding of their customers and incorporates the customer voice and provides room for User Interface (UI) and user interaction design as part of the agile process. And suggests how large projects could be addressed in an agile manner [8]. Agile projects need to adopt aspects of usability engineering by incorporating user stories and including usability specialists in the team [34]. Concept mapping is promising for addressing existing agile usability approach.

From the results, the selected articles address that agile methods and its positive contribution towards usability. But there are also some limitations on developing the systems. Usability cannot be directly imposed on agile methods, as the usability design process has to follow some steps which may violate many agile development values. There are articles which suggest some ideas, in which software development methods and usability proceed in parallel [16].

Table 5. Data synthesis table for research question 2

Titles Authors Development Usability Final discussion/ Notes/comments method contribution conclusion Towards Agile Usability It gives four Revises the tension extreme(ly) Lee JC, engineering guidelines for between usability and usable software: McCrickard usability specialists agile method. Exploring DS and agile tensions practitioners. between usability and agile software development.

An Agile Beyer h, Agile Contextual The agile team can Address the need for Customer- Holtzblatt k, design support from in- agile and Usability Centered and Baker L. depth understanding Method: Rapid of customer needs Contextual and provides room Design for UI Design. And also suggests how large projects addressed in agile manner. Hitting the Jeff Patton Agile Customer By implementing Development of agile Target: Adding satisfaction Agile U-CD process using usability in Interaction the drawback of Tomax Technologies Design to Agile Extreme Software Programming is Development eliminated. Integrating Sohaib O, Agile Usability The usability fits Gives the idea how to usability Khan K. engineering well with agile by implement agile in engineering and using iterative usability agile software approach and development: A testing throughout literature the project lifecycle. review. Agile User Ana M. Agile Usability The approach is still Incorporating Stories Enriched Moreno and patterns undergoing particular usability with Usability Augstin Yague validation, but mechanisms into agile preliminary results user stories. suggest workload for incorporating particular usability mechanisms using the knowledge is reasonably acceptable. Easing team Jeremy Agile Concept Results indicate that Need for Politics in Agile T.Barksdale, mapping there is need for improvement in agile usability Eric D.Ragan, improvement in usability D.Scott agile usability, and McCrickard concept mapping approach is promising for addressing existing concerns. Measuring Anirudha Agile and Usability The metrics were Agile is more used effectiveness of Joshi, traditional metrics found to be useful, than waterfall model HCI Integration N.L.Sarda, easy to use and in Contracted in Software Sanjay helpful in making software development development Tripathi the process better. service. processes. They correlate with each other well. UGAM also correlates well with

traditional usability evaluations. Extreme Kathy Baxter, Agile User centered After the panelist The panel members Usability: Kuldeep design summarization with share their challenges Adapting Kelkar, Lynn Agile, will lead to and successes faced Research Miller, Jeff discussion about during development. Approaches for Patton key challenges Agile faced by user Development researchers in Agile environments. Concept Jeremy Agile Concept Results indicate that Lessons learned from Mapping in Barksdale and mapping our concept the collaboration of Agile Usability: D.Scott mapping approach both agile and A case Study McCrickard has value as a visual usability. tool in agile usability environments. Usability Mary Frances Agile and Usability ISO 9241 provides Standards enable Standards Theofanos and traditional standards process related usability across the Brian C. guidance in the professionals to share Development Stanton human centered common deliverables Lifecycle design process and of HCI. other parameters. Global- Alvin W. Yeo global- Usability The usability of the Highlights the Software SDLC evaluation software is not importance of Development known until the usability evaluation in Lifecycle An product is tested global-SDLC. Exploratory with target culture. Study An Industrial Hassan Scrum Satisfaction The organization Challenges faced by Case Study for Hajjdiab, Al follows the agile which caused Scrum Adoption Shaima Taleb, traditional waterfall adoption to fail. Jauhar Ali software development model. Many challenges were faced by the agile teams which caused the adoption to fail. U-SCRUM : An Mona Singh U-SCRUM User needs U-SCRUM is best The architecture is not Agile and suited for products used by the agile Methodology prioritization that are complex community for promoting and novel. When the Usability topic is well understood, there is less of an advantage. A Web Agile Gladys Agile Usability The method New approaches to usability benigni, evaluation performs in parallel make web sites more Approach for Osvaldo both to the software and more attractive web site design Gervasi, development and and efficient. Francesco the usability

Luca Passeri, evaluation, and Tai Hoon activating a dialog Kim with the customer and testing it involving a group of potential users. Examining the Jason Chong Agile Usability The improved They include areas foundations of Lee, D.Scott engineering approach for such as quality Agile usability McCrickard integrating usability assurance, with extreme into an agile team. documentation and scenario-based And they present product management. Design four conclusions. Delivering Christopher Agile Customer The MCT project Making the software Software into Webster, Nija satisfaction has taken the to certify for NASA's Shi and Irene described principles operational use of the Mission Control Skupniewicz and adapted the company. Center using Smith agile process to fit Agile in with delivery into Development the mission control Techniques center. Interplay Silvia Agile and Usability In order to facilitate The Software between Abrahao, traditional evaluation the integration of Engineering usability and Natalia user experience, we Community has software Juristo, Effie have identified the recently development. L-c Law, and need for new acknowledged that Jan Stage resources with usability not only which to increase affects the design of the value of user interfaces but usability practice in also the software software development as a development. whole.

4.1.3 Do Projects with high demands on usability use usability specialists?

For this research question, usability demand and usability expert/ specialists (3rd and 4th columns respectively) that were obtained from the included papers (1st column) have been tabulated in Table 5.

From the results, total of twelve articles, eleven articles state that a project with high demands on usability uses usability specialists, except one article which states that design engineers can develop a project by using Discount usability engineering [12].

Usability demands are primarily based on the interface design and design process of the system. User interfaces (UI) are screen layouts, button labels, dialog boxes, message boxes, hyperlinks, etc. Recently agile teams have to develop unambiguously without these requirements specification, because UI is the viaduct between the user and the system’s function. As testing of system can also been done through UI [8].

Table 6. Data synthesis table for research question 3

Titles Authors Developme Usability Usability Final Notes/comments nt methods demand expert discussion/conclusion Towards Lee JC, Agile High Yes It gives four guidelines Revises the extreme(ly) McCrickard for usability specialists tension between usable DS and agile practitioners. usability and software: agile method. Exploring tensions between usability and agile software development. Hitting the Jeff Patton Agile High Yes By implementing Development of Target: Agile U-CD process agile using Adding the drawback of usability in Interaction Extreme Programming Tomax Design to is eliminated. Technologies Agile Software Development Delivering Christopher Agile High Yes The MCT project has Making the Software into Webster, taken the described software to NASA's Nija Shi and principles and adapted certify for Mission Irene the agile process to fit operational use of Control Skupniewic in with delivery into the company. Center using z Smith the mission control Agile center. Development Techniques Concept Jeremy Agile High Yes Results indicate that Lessons learned Mapping in Barksdale our concept mapping from the Agile and D.Scott approach has value as a collaboration of Usability: A McCrickard visual tool in agile both agile and case Study usability usability. environments. Easing team Jeremy Agile High Yes Results indicate that Need for Politics in T.Barksdale there is need for improvement in Agile , Eric improvement in agile agile usability. usability D.Ragan, usability, and concept D.Scott mapping approach is McCrickard promising for addressing existing concerns. U-SCRUM : Mona Singh U-SCRUM High Yes U-SCRUM is best The architecture An Agile suited for products that is not used by the Methodology are complex and novel. agile community for When the topic is well

promoting understood, there is Usability less of an advantage. Extreme Kathy Agile High Yes After the panelist The panel Usability: Baxter, summarization with members share Adapting Kuldeep Agile, will lead to their challenges Research Kelkar, discussion about key and successes Approaches Lynn Miller, challenges faced by faced during for Agile Jeff Patton user researchers in development. Development Agile environments. Delivering Christopher Agile High Yes Transformation from Implementation Software into Webster, traditional methods to of agile methods NASA NijaShi, Agile methods which in NASA mission Mission Irene are made to develop control center. control center Skupniewic successfully delivered using Agile z Smith multiple customer- Development selected releases into Techniques the Mission Control Center operational environment. Usability and Arlen Agile High Yes Extreme Programming Creating more User Bankston, has used for realms useful and usable Interface CC Pace like user interface application Design in XP (White elegance, cohesion and increases chances paper) usability. And they of success. address the importance of these attributes rather than methods for attaining them. Adding Gerard Agile High Yes A Canadian pacific The paper Usability Meszoras, railway project has prototype proved testing to an Janice Aston been split into two to be useful in Agile Project releases, One before many more ways usability testing and than just for the the second release with usability testing. usability testing. Two Case Maryam Agile High Yes The cases studies Consistent user studies of Najafi, Len presented in this paper testing and user Toyoshiba describe two examples refinement of experience of UED and Agile designs ensures design and integration at that the product agile VeriSign. is developed to development. meet the needs of its users. Finding a David Kane, Agile High No Discount usability The paper place for SRA engineering should be introduces us to Discount Internationa of particular interest to Discount Usability l the agile development usability Engineering (White community because engineering in Agile Paper) the two disciplines approach for use Development share many of the with Scrum. : Throwing same underlying Down the principles.

4.1.4 Which types of usability are addressed in articles relating development methods and usability?

For this research question, development method and usability aspect (3rd and 4th columns in table 6 respectively) that were obtained from the included papers (1st column in table 6) have been tabulated in Table 6.

When the search results were analyzed, in total of 18 articles, 16 articles address Satisfaction as the most required type of usability in both Agile and Traditional development methods. And 12 articles address about Efficiency, 8 articles address on Effectiveness, 2 articles on learnability.

Table 7. Data synthesis table for research question 4

Titles Authors Development Usability Final discussion/ Notes/comments method aspect conclusion Towards Agile Satisfaction It gives four Revises the tension extreme(ly) Lee JC, guidelines for between usability usable software: McCrickard usability specialists and agile method. Exploring DS and agile tensions practitioners. between usability and agile software development. An Agile Beyer h, Agile Satisfaction, The agile team can Address the need for Customer- Holtzblatt k, Efficiency. support from in- agile and Usability Centered and Baker L. depth Method: Rapid understanding of Contextual customer needs Design and provides room for UI Design. And also suggests how large projects addressed in agile manner. Hitting the Jeff Patton Agile Satisfaction By implementing Development of agile Target: Adding Agile U-CD using usability in Interaction process the Tomax Technologies Design to Agile drawback of Software Extreme Development Programming is eliminated. Integrating Sohaib O, Agile Learnability, The usability fits Gives the idea how to usability Khan K. Efficiency, well with agile by implement agile in engineering and Effectiveness, using iterative usability agile software Satisfaction. approach and

development: A testing throughout literature the project review. lifecycle. Agile User Ana M. Agile Satisfaction, The approach is Incorporating Stories Enriched Moreno and Efficiency. still undergoing particular usability with Usability Augstin validation, but mechanisms into Yague preliminary results agile user stories. suggest workload for incorporating particular usability mechanisms using the knowledge is reasonably acceptable. Easing team Jeremy Agile Satisfaction, Results indicate Need for Politics in Agile T.Barksdale, Efficiency that there is need improvement in agile usability Eric D.Ragan, for improvement in usability D.Scott agile usability, and McCrickard concept mapping approach is promising for addressing existing concerns. Measuring Anirudha Agile and Effectiveness, The metrics were Agile is more used effectiveness of Joshi, traditional Satisfaction found to be useful, than waterfall model HCI Integration N.L.Sarda, easy to use and in Contracted in Software Sanjay helpful in making software development Tripathi the process better. development service. processes. They correlate with each other well. UGAM also correlates well with traditional usability evaluations. Extreme Kathy Baxter, Agile Efficiency, After the panelist The panel members Usability: Kuldeep Satisfaction summarization share their challenges Adapting Kelkar, Lynn with Agile, will and successes faced Research Miller, Jeff lead to discussion during development. Approaches for Patton about key Agile challenges faced by Development user researchers in Agile environments. Concept Jeremy Agile Learnability, Results indicate Lessons learned from Mapping in Barksdale and Satisfaction that our concept the collaboration of Agile Usability: D.Scott mapping approach both agile and A case Study McCrickard has value as a usability. visual tool in agile usability environments.

Usability Mary Frances Agile and Effectiveness, ISO 9241 provides Standards enable Standards across Theofanos traditional Efficiency, process related usability the and Brian C. Satisfaction guidance in the professionals to share Development Stanton human centered common deliverables Lifecycle design process and of HCI. other parameters. Global- Alvin W. Yeo global-SDLC Efficiency, The usability of the Highlights the Software Effectiveness software is not importance of Development known until the usability evaluation Lifecycle An product is tested in global-SDLC. Exploratory with target culture. Study An Industrial Hassan Scrum Satisfaction The organization Challenges faced by Case Study for Hajjdiab, Al follows the agile which caused Scrum Adoption Shaima Taleb, traditional adoption to fail. Jauhar Ali waterfall software development model. Many challenges were faced by the agile teams which caused the adoption to fail. U-SCRUM : An Mona Singh U-SCRUM Effectiveness, U-SCRUM is best The architecture is Agile Satisfaction. suited for products not used by the agile Methodology that are complex community for promoting and novel. When Usability the topic is well understood, there is less of an advantage. A Web Agile Gladys Agile Effectiveness, The method New approaches to usability benigni, Satisfaction, performs in make web sites more Approach for Osvaldo Efficiency. parallel both to the and more attractive web site design Gervasi, software and efficient. Francesco development and Luca Passeri, the usability and Tai Hoon evaluation, Kim activating a dialog with the customer and testing it involving a group of potential users. Examining the Jason Chong Agile Efficiency, The improved They include areas foundations of Lee, D.Scott Effectiveness approach for such as quality Agile usability McCrickard integrating assurance, with extreme usability into an documentation and scenario-based agile team. And product Design they present four management. conclusions.

Delivering Christopher Agile Satisfaction, The MCT project Making the software Software into Webster, Nija Effectiveness, has taken the to certify for NASA's Shi and Irene Efficiency described operational use of the Mission Control Skupniewicz principles and company. Center using Smith adapted the agile Agile process to fit in Development with delivery into Techniques the mission control center. Interplay Silvia Agile and Satisfaction, In order to The Software between Abrahao, traditional Efficiency facilitate the Engineering usability and Natalia integration of user Community has software Juristo, Effie experience, we recently development. L-c Law, and have identified the acknowledged that Jan Stage need for new usability not only resources with affects the design of which to increase user interfaces but the value of also the software usability practice development as a in software whole. development. Finding a place David Kane, Agile Satisfaction, Discount usability The paper introduces for Discount SRA Efficiency engineering should us to Discount Usability International be of particular usability engineering Engineering in (White Paper) interest to the agile approach for use Agile development with Scrum. Development: community Throwing Down because the two the Gauntlet disciplines share many of the same underlying principles.

Chapter 5

5.1 Discussion

A Software engineer focuses on the completion of requirements, if there are usability requirements these will be taken into account. A usability engineer focuses not only on the Graphical User Interface (GUI) design, but also on the dialogue, help texts, manuals, and how the system will be used in the user’s business process. This is the place where the problem raises, where we could not satisfy both the usability and software engineers well, which creates confusion in the project and directly leads to failure of the project. Due to this both the development team and usability team has to work together [22].

Usability engineering helps developers to understand the users work and goals in an efficient manner. This interested the developers to shift their focus towards the agile methods [8, 28]. Usability engineering deals with developing a user interface that is efficient and effective. There are various issues such as learnability, efficiency, memorability, errors and user satisfaction [26]. Usability focuses on how the end-users will work with the software and Agile development methods focus on how the software should be developed [34].

Although there are various models in existence of the SDM, we are likely to consider the most important ones like Waterfall, Spiral, Incremental, V-model, Wheel and Spoke Model and Agile methods. The waterfall model is the most widely used in the software engineering until now. Many in the software industry say that the waterfall model is the dominant one until today [20], but the Agile methods is evolving method which makes the organizations to switch from traditional waterfall model to agile methods.

In early days traditional methods are used in projects with high demands on usability because; in traditional approach the customers specify the demands on the user requirements. Due to which projects created are based on the demands of the usability. But traditional software development methods are not best suited for today rapidly changing environment which leaves out of choice of moving to agile development method. But it also has some conflicts of implementing both the Agile methods and usability together [30].

Nowadays, mostly every organization is trying to switch from traditional waterfall model to agile method of software development. As the market is grown, a company has its branches spread in every corner of the world. Thereby developing a system with the waterfall model is not a good idea. As by creating a project for a company, where its clients are being all around the globe, where the company has to get test results from every client and act accordingly. This makes a more complex implementation of the product, which might cause the failure of the project. The waterfall model has also evolved into various other methods like B-model, Incremental model, and V-model, etc. [30].

The Agile methods and usability have more in common, they follow cyclic development method and human centered design and they both emphasize on team co-ordination and communication. Even though they have common interests, there are many areas which cause conflicts and provide a barrier to the development process [22]. The Agile methods thrives to achieve continuous interaction between developers and customers to develop a good software, whereas in Usability by continuous interaction between developers and users, creates a good software. Thus, when customers do not have a good understanding of the end-users, an agile method can miss many opportunities for the end-users.

As agile methods are processed, the design flaws are identified and corrected immediately, and then implementation and execution proceeds thereby making the process simple and easy to implement. In agile methods, the process has been modified continuously. The developers face various queries and design flaws from other departments and it has to be rectified and process continues. The developers and usability testers work side by side to determine the right test coverage [32]. The Agile methods can be implemented well for the projects which are implemented by other methods.

Results indicate that there is a need for improvement in agile usability, a concept mapping approach shows potential for addressing existing concerns. With fine-tuning of the method and the development of the proper tools, this approach has great potential to improve team interaction in agile usability environments [5]. Concept mapping in agile usability can validate effectiveness more easily than other methods [4].

U-SCRUM has two product owners, one focuses on usability and other focuses on more predictable functions. Developers can often estimate development cost more easily and quickly in SCRUM, than calculating the development cost by including usability in their projects. A SCRUM team cannot calculate the cost and time of the project when implemented with usability engineering. U-SCRUM yields improved usability than SCRUM [33].

USABAGILE_Web, is an iterative approach, working on single interfaces like web pages or standalone applications. The agile development method starts with the analysis phase and the purpose of such phase is to show interface behavior applying the Unified Modeling Language (UML). USCRUM performs in parallel both to the software development and the usability evaluation, activating a communication with the customer, and testing it with a group of potential users [7].

The Agile methods do not positively contribute to the usability of the product directly. The agile methods have some interest of conflicts with the usability when it is implemented directly. And this can be solved by using a combined approach given below [34].

Table8. Combined approach of usability and Agile methods

Agile Methods Usability Engineering Suggested Approach Deliver working software frequently Traditional software approach but Iterative development throughout iteration within phases the project Requires Generalists Requires Specialists Assemble a multidisciplinary team to ensure complete expertise Customer focus User focus Collaboration between customers, users, product managers, business analysts, developers, will maximize overall team efficiency for usable product. Test driven development and Contextual inquiry, field surveys, Unit testing + user Acceptance continuous integration usability inspection methods for testing + usability testing throughout testing the process Using onsite customer, functional Scenario based design for Integrate user stories with scenario requirement are encapsulated as user requirement analysis based design stories

The traditional waterfall model implements “Software Requirements Specification” (SRS). The designers implement the SRS and finish their job and they start working with new projects. Their duty is to create a design based on the user requirements due to which, customer feedbacks and their relations are being totally avoided which causes huge loss if the project becomes a failure. The waterfall model is “over the fence” approach. Where user specifies the requirements, and gets the output of the product in due time. It is hard for customers to specify software requirements without knowing the evolution and its progress [36]. Whereas in the agile methods the customers are able to add more requirements during the development of the process, the customer is at the center in agile development method which provides UCD. The customers are made available throughout the entire development lifecycle.

In some cases where the customers cannot afford for additional usability engineers for the project; it may be due to the cost or the project has been started with designers in the initial stages. Then a need for a usability engineer arises and by bringing a usability expert at that stage may create more problems. This can be solved by Discount usability approach where the developers can use usability techniques by themselves, by analyzing and not implementing all the usability analysis, this approach allows developers to choose some usability techniques and implement it like a Discount; hence it is termed as discount usability engineering [19]. Developers can make progress as they know their project well they can filter many process made by the usability expert thereby reducing cost and time [12].

The usability engineer’s work is not just analysis, but also to suggest new designs. Usability engineers design advice cannot be blindly taken into consideration, designs they suggest may not be an ideal solution for the problems they have exposed. The usability engineers should be from the start of the project has it pays more value [7].

The projects with high demands on usability use usability specialists, because in usability engineering the important focus is to develop a system based on user interface and satisfaction. The generally trained developer’s main aim is to develop software that works. In ideal cases, the software inspection is carried out with three to five usability experts. The detected problems are ranked from one to three starting from serious, intermediate, and light problems [7]. The evaluation phase is still managed by the experts, no interactions are necessary with customers and end –users. The adopted methodology for evaluation phase is the cognitive walk through method that allows studying how the user interacts with the interface. The experts could not properly identify the problems in usability interface because the physical or the psychological conditions of some members prevent from the identification of the errors [7].

5.2 Limitations

There were some limitations; some articles were helpful in understanding the background of software development methods and usability, but they are discarded due to lack of empirical. Even some of the articles might be missed due to mismatch in the keywords during the searches. The kappa value calculated can be inaccurate as they are calculated based on the articles selected on basis of selection criteria. The selection process is re-iterated again and articles relating to the research questions are being added into the analysis without changing the search queries.

6. Conclusion

As this project could inspire views on the software development methods and usability. It is formed by developing a systematic literature review protocol described by Barbara Kitchenham. In this study, it can be observed from the results and discussion that agile development methods are being deployed with usability to form a successful project. And even new usability techniques like “Discount Usability” to reduce cost of the product can be deployed.

To answer the first research question, agile development methods are being used in high demands of usability in most cases. The implementations may vary from project to project, and level of implementation of the project depends on the requirements, ideologies and factors influenced the development methods.

Agile methods positively contribute to the usability of the product with certain set of guidelines which helps to incorporate usability with agile methods. A combined approach has been promoted in this thesis, in which agile methods and usability engineering can be implemented with maximum correlation.

The third question, projects with high demands of usability use usability specialists, because they satisfy the usability goals effectively rather than using a developer who has less needs of usability and knowledge over it. By implementing a specialist in a project, makes it more understandable to the end user. As customers been the goal of every project.

Different types of usability’s are being addressed with the development methods implementations in which User Satisfaction plays a major role in developing a software product, and it is followed by Efficiency and Effectiveness.

Thus, this thesis helps the future researchers to find empirical knowledge between software development methods and usability. It even provides the suggested combined approach between agile methods and usability.

7. Appendix Table 9. Study quality assessment table

1. Is this a research paper? Consider: – Is the paper based on research □ Yes □ Moderate □ No

2. Is there a clear statement of the aims of the research? Consider: □ Yes □ Moderate □ No – Is the study’s focus or main focus on Agile Software Development and usability? – Does the study present empirical data? – Is there a clear statement of the study’s primary outcome

3. Is there an adequate description of the context in which the research was carried out □ Yes □ Moderate □ No Consider if the Author has identified: – The nature of the software development organization? Research design 4. Was the research design appropriate to address the aims of the research? □ Yes □ Moderate □ No Consider: – Has the Author justified the research design? Data collection 5. Was the data collected in a way that addressed the research issue? □ Yes □ Moderate □ No Consider: – Were all measures clearly defined? – Is it clear how data was collected? – Has the Author justified the methods that were chosen? – Has the Author made the methods explicit – If the methods were modified during the study has the Author explained how and why? – If quality control methods were used to ensure completeness and accuracy of data collection?

Data analysis 6. Was the data analysis sufficiently rigorous? Consider: □ Yes □ Moderate □ No – Was there an in-depth description of the analysis process? – If thematic analysis was used is it clear how the categories/ themes were derived from the data?

– Has sufficient data been presented to support the findings? – To what extent has contradictory data been taken into account? – If quality control methods were used to verify the results? Reflexivity 7. Has the relationship between researcher and participants been adequately considered? □ Yes □ Moderate □ No Consider: – Has the researcher critically examined their own role, potential bias and influence during formulation of research questions, sample recruitment, data collection, and analysis and selection of data for presentation? – How the researcher responded to events during the study and whether they considered the implications of any changes in the research design? Findings 8. Is there a clear statement of findings? Consider: □ Yes □ Moderate □ No – Are the findings explicit? – Has an adequate discussion of the evidence, both for and against the Author’s arguments, been demonstrated? – Has the Author discussed the credibility of their findings? – Are limitations of the study explicitly discussed? – Are the findings discussed in relation to the original research questions? – Are the conclusions justified from the results? Value of the research 9. Is the study of value for research or practice? Consider: □ Yes □ Moderate □ No – Does the Author discuss the contribution the study makes to existing knowledge or understanding? – Does the research identify new areas where research is necessary? – Does the Author discuss whether or how the findings can be transferred to other populations or considered other ways the research can be used?

8. References

[1] P.Abrahamsson, O.Salo, J.Ronkainen, and J.Warsta . “Agile software development methods: Review and analysis (No. 478)”. Espoo, Finland: Technical Research Centre of Finland, VTT Publications, 2002.

[2] J.Highsmith. (2001 February 11-13). Manifesto for Agile Software Development. Available: http://www.agilemanifesto.org. [Accessed 05-Sep-2014]

[3] B. Kitchenham. "Procedures for performing systematic reviews." Keele, UK, Keele University 33, no. 2004 (2004): 1-26.

[4] Barksdale and D. McCrickard, "Concept mapping in agile usability", Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems - CHI EA '10, 2010.

[5] Barksdale, Jeremy T., Eric D. Ragan, and D. Scott McCrickard. "Easing team politics in agile usability: A concept mapping approach." In Agile Conference, 2009. AGILE'09., pp. 19-25. IEEE, 2009.

[6] Beck, Kent, and Cynthia Andres. "Extreme Programming Explained: Embrace Change." Addison-Wesley,2nd edition (2004): 3.

[7] Benigini, G., Gervasi, O., Passeri, F. L. and Kim, T. “Usabagile _web: A web Agile usability approach for web site design”, In proceeding of ICCSA 2010, Part II , LNCS 6017,pp 422-431.

[8] Beyer, Hugh, Karen Holtzblatt, and Lisa Baker. "An agile customer-centered method: rapid contextual design." Extreme Programming and Agile Methods-XP/Agile Universe 2004 (2004): 527-554.

[9] Boehm, Barry W. "A spiral model of software development and enhancement." Computer 21, no. 5 (1988): 61-72.

[10] 12 Principles behind the agile manifesto, Agile Alliance, 2017. [online]. Available: https://www.agilealliance.org/glossary/timebox/ . [Accessed 05-Sep-2014].

[11] A.Cockburn , Eds., “Crystal Clear: A Human-Powered Methodology for Small Teams”. HARLOW: Addison Wesley, 2004.

[12] Kane, David. "Finding a place for discount usability engineering in agile development: throwing down the gauntlet." In Agile Development Conference, 2003. ADC 2003. Proceedings of the, pp. 40-46. IEEE, 2003.

[13] Dybå, T. (2012 December 9). EBSE: Review protocol – Agile Software Development. Available FTP: community.dur.ac.uk Directory: ebse/resources/studies/protocol/ File: Agile- Methods-literature-review-protocol.pdf

[14] Forsberg, Kevin, and Harold Mooz. "The relationship of system engineering to the project cycle." In INCOSE International Symposium, vol. 1, no. 1, pp. 57-65. 1991.

[15] J. Highsmith, Adaptive software development: a collaborative approach to managing complex systems. New York, N.Y.: Dorset House Pub., 2000.

[16] Humayoun, Shah Rukh. "Incorporating usability evaluation in software development environments." KI-Künstliche Intelligenz 26, no. 2 (2012): 197-200.

[17] ISO, I. Software Product Evaluation: General overview. ISO14598-1, ISO/IEC (1999).

[18] ISO, I. 9. Ergonomic requirements for office work with visual display terminals (vdts). The international organization for standardization, 1998. 2012, 28.

[19] Kane, David. "Finding a place for discount usability engineering in agile development: throwing down the gauntlet." In Agile Development Conference, 2003. ADC 2003. Proceedings of the, pp. 40-46. IEEE, 2003.

[20] Laplante, Phillip A., and Colin J. Neill. "The demise of the waterfall model is imminent." ACM Queue 1, no. 10 (2004): 10.

[21] Lecerof, Andreas, and Fabio Paternò. "Automatic support for usability evaluation." IEEE Transactions on Software Engineering 24, no. 10 (1998): 863-888.

[22] Lee, Jason Chong, and D. Scott McCrickard. "Towards extreme (ly) usable software: Exploring tensions between usability and agile software development." In Agile Conference (AGILE), 2007, pp. 59-71. IEEE, 2007.

[23] Navalkar, Anjoo. "Usability Engineering–Quality Approach (ISO 13407)." Human Factors [online] (2008). Available: http://pdfs.semanticscholar.org. [Accessed 10-Oct-2014]

[24] Nielsen, J. Usability Engineering. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA., 1993.

[25] C. Wilson, “Chapter 1- What is usability?” in User experience re-mastered. Burlington, MA: Morgan Kaufmann Publishers, 2010. pp. 3-22.

[26] Nielsen, Jakob. "The usability engineering life cycle." Computer 25, no. 3 (1992): 12-22.

[27] Palmer, Steve R., and Mac Felsing. A practical guide to feature-driven development. Pearson Education, 2001.

[28] Patton, Jeff. "Hitting the target: adding interaction design to agile software development." OOPSLA 2002 Practitioners Reports. ACM, 2002.

[29] Richard, J. L. and Gary, G. K. “The Measurement of observer Agreement for categorical data”. International Biometric Society, Biometrics, vol 33, no.1, 1977, pp. 159-174.

[30] Ruparelia, Nayan B. "Software development lifecycle models." ACM SIGSOFT Software Engineering Notes 35, no. 3 (2010): 8-13.

[31] Schwaber, Ken, and Mike Beedle. Agile software development with Scrum. Vol. 1. Upper Saddle River: Prentice Hall, 2002.

[32] Shaye, Susan D. "Transitioning a team to agile test methods." In Agile, 2008. AGILE'08. Conference, pp. 470-477. IEEE, 2008.

[33] Singh, Mona. "U-SCRUM: An agile methodology for promoting usability." In Agile, 2008. AGILE'08. Conference, pp. 555-560. IEEE, 2008.

[34] Sohaib, Osama, and Khalid Khan. "Integrating usability engineering and agile software development: A literature review." In Computer design and applications (ICCDA), 2010 international conference on, vol. 2, pp. 2-32. IEEE, 2010.

[35] Stapleton, Jennifer, ed. DSDM: Business focused development. Pearson Education, 2003.

[36] Szalvay, Victor. "An introduction to agile software development." Danube technologies (2004): 1-9. [Accessed: 12- Oct- 2014].

[37] Tullis, T. and William, A. Measuring the User Experience: collecting, analyzing, and presenting usability metrics. Morgan Kaufmann Publishers, Elsevier, Burlington, 2008.

[38] What & Why of Usability | Usability.gov", Usability.gov, 2017. [Online]. Available: http://usability.gov/basics/index.html. [Accessed: 10- Oct- 2014].

[39] Vredenburg, Karel, S. Isensee, and C. Righi. User-Centered Design. An integrated approach. Englewood Cliffs: Prentice Hall, 2001.

[40] J. Dumas and J. Redish, A practical guide to usability testing. Exeter, England: Intellect Books, 1999.

[41] Royce, Winston W. (1970): Managing the development of large software systems. In Proceedings, IEE Wescon, August 1970, pp. 1-9.