Automated Requirements Analysis Using Natural Language Processing

Home , Natural Language Toolkit

FACULDADEDE ENGENHARIADA UNIVERSIDADEDO PORTO

Automated Requirements Analysis using Natural Language Processing

Mariana Oliveira

Mestrado Integrado em Engenharia Informática e Computação

Supervisor: Prof. João Pascoal Faria - FEUP Co-Supervisor: Eng. Nuno Teixeira - Bosch Car Multimedia

July 16, 2018

Automated Requirements Analysis using Natural Language Processing

Mariana Oliveira

Mestrado Integrado em Engenharia Informática e Computação

Approved in oral examination by the committee:

Chair: Prof. Ana Cristina Paiva External Examiner: Prof. Rui Maranhão Supervisor: Prof. João Pascoal Faria July 16, 2018

Abstract

The automotive domain has been recognised as being remarkable regarding aspects like quality and safety. Behind all the technology involved in the creation of automotive software, there are several Software Engineering processes taking place. One crucial process is the production of artefacts. some examples are Use Cases, Models, Requirements and Design Documents. They help describe the features, architecture and design of the software, therefore their quality is directly linked to the quality of software systems as well as their development process. Surveys have shown that the most common reasons for a project’s lack of success are not technical but rather problems with requirements. For that reason, during this investigation, the Requirements documents and their quality will be the target. Considering that the automotive market is as demanding as ever concerning innovations of increasing complexity, the associated shorter development periods make it paramount to improve software development processes. In order to guarantee the quality of their software, automotive companies support requirements reviews which aim is to ensure these documents are abiding to a series of international quality standards. These describe various quality criteria which are then used to measure the quality of a project. Making sure these quality criteria are fulfilled during the requirements reviews is a complex task that requires effort and resources extending the time to release new products. It is a challenge to find a way to transform this task in a simpler and more efficient action. As such, the centre of this research is in identifying applicable quality criteria and their respective process metrics in the interest of creating a tool that finds, interprets and alerts the user about faults or incoherences in the requirements. As most requirements are written using Natural Language due to it being easily understandable by all stakeholders, a tool that analyses their quality should apply Natural Language Processing (NLP) techniques. Although there are several tools available in the market that support requirements analysis, none of them provide the necessary input type and integration means and that is why a tool was created to tackle that lack of suitable solutions. The tool that was developed during this investigation is called Requirements ANalysis Tool (RANT). RANT employs several NLP techniques such as Sentence Splitting, Tokenization and Part-of-Speech tagging, in order to assess the requirements quality by producing a final evaluation regarding the requirements structure and potential ambiguity. Creating a tool that was integrated in the used Requirements Management environment was an essential feature. Hence RANT was developed as an extension of IBM’s Requirements Man- agement environment called Collaborative Lifecycle Management (CLM). Alongside RANT, a checklist was created with the aim of being used by the analysts together with RANT in the interest of helping the analysts to perform a more complete and accurate requirements review.

i As a form of validation of RANT, binary diagnostic testing was performed with the main goal of obtaining precision, recall and accuracy values. The results obtained seem promising with values of 82%, 95% and 85% regarding precision, recall and accuracy respectively.

ii Resumo

O setor automóvel é reconhecido como sendo notável em aspectos como a qualidade e a se- gurança. Por detrás de toda a tecnologia envolvida no desenvolvimento de software automóvel, ocorrem vários processos de Engenharia de Software. Um processo crucial é a produção de artefactos, alguns exemplos são os Casos de Uso, Mod- elos, Requisitos e documentos de Design. Estes artefactos ajudam a descrever as características, a arquitectura e design do software. Assim, a qualidade destes documentos está directamente ligada à qualidade dos sistemas de software, assim como ao respectivo processo de desenvolvimento. Estudos mostraram que a maioria das razões comuns para o insucesso de um projeto não são por motivos técnicos mas sim por problemas com os requisitos. Por esse motivo, durante esta investigação, o foco estará nos documentos de requisitos e na respetiva qualidade. Considerando que o mercado da indústria automóvel está cada vez mais exigente em relação a inovações de complexidade crescente e os consequentes períodos de desenvolvimento mais curtos, levam a uma necessidade de melhorar os processos de desenvolvimento de software. Para garantir a qualidade do seu software, empresas do ramo automóvel fazem revisões aos requisitos, cujo objectivo é assegurar-se que estes documentos cumprem uma série de standards internacionais de qualidade. Estes standards descrevem vários critérios de qualidade que são então utilizados para medir a qualidade de um projecto. Certificar-se que estes critérios de qualidade se- jam seguidos durante as revisões de requisitos é uma tarefa que requer bastante esforço e recursos, levando a um prolongamento do tempo até ao lançamento de novos productos. É um desafio encontrar uma forma de transformar esta tarefa numa acção mais simples e mais eficiente. Como tal, o centro desta investigação é identificar critérios de qualidade aplicáveis e as respectivas métricas de processo com o objectivo de criar uma ferramenta que encontra, interpreta e alerta o utilizador acerca de falhas ou incoerências nos requisitos. Como a maioria dos requisitos são escritos utilizando Linguagem Natural devido a ser facil- mente compreensível por todos os interessados, uma ferramenta que analisa a qualidade dos requisitos deverá aplicar técnicas de Processamento de Linguagem Natural (PLN). Apesar de existirem várias ferramentas disponíveis no mercado que efectuam análises aos requisitos, nenhuma delas providencia o tipo de input necessário ou os meios de integração pretendidos e, por isso, foi criada uma ferramenta para mitigar essa falta de soluções adequadas. A ferramenta que foi desenvolvida ao longo desta investigação denomina-se Requirements ANalysis Tool (RANT). A RANT utiliza várias técnicas de PLN como Sentence Splitting, Tok- enization e Part-of-Speech tagging, para aferir a qualidade dos requisitos através da produção de uma análise que avalia a estrutura e a potencial ambiguidade que possa existir nos requisitos. A criação de uma ferramenta que estivesse integrada no ambiente de gestão de requisitos utilizado era uma característica essencial. Assim, a RANT foi desenvolvida como uma extenção do ambiente de gestão de requisitos da IBM chamado Collaborative Lifecycle Management (CLM). Em paralelo com a RANT, foi elaborada uma checklist com o objetivo de ser utilizada junto com a RANT e poder ajudar os analistas a realizar revisões de requisitos mais exatas e completas.

iii Como forma de validação da ferramenta RANT, o teste de diagnóstico binário foi o escolhido com o principal objectivo de obter os valores de precisão, revocação e exatidão. Os resultados obtidos foram promissores com valores de 82%, 95% e 85% para a precisão, revocação e exatidão, respetivamente.

iv Acknowledgements

First, I would like to thank my thesis supervisor Prof. João Pascoal Faria for his invaluable guidance and support. I am grateful to my co-supervisor Nuno for allowing me to freely pursue my research while still giving me all the guidance needed. I would like to express my appreciation to Bosch for allowing me to have access to such a great workplace and providing everything I could need to fulﬁl my goals for this research. A special thank you to my coworkers with whom I have had the pleasure to work and that always made me feel part of the team. I would like to show my greatest appreciation to the experts that were involved in the validation of this project without whom this research would not have had a happy ending. My deepest heartfelt appreciation goes to my boyfriend João for putting up with me and my terrible mood swings during this process and for always giving me the strength, love and motivation that I need to push through. Finally, my acknowledgements would not be complete without expressing my profound grat- itude to my parents for their relentless support and encouragement throughout my years of study and for their unwavering love and patience through tough moments. Thank you.

Mariana Oliveira

v vi “We can only see a short distance ahead, but we can see plenty there that needs to be done.”

Alan Turing

vii viii Contents

1 Introduction1 1.1 Context ...... 2 1.2 Motivation and Goals ...... 3 1.3 Document Structure ...... 3

2 Background5 2.1 Software Engineering ...... 5 2.2 Requirements Engineering ...... 6 2.2.1 Requirements Validation ...... 8 2.2.2 Requirements Quality ...... 9 2.3 Software Quality Management ...... 10 2.3.1 Software Quality Assurance ...... 10 2.3.2 Veriﬁcation and Validation ...... 11 2.4 Natural Language Processing ...... 12 2.4.1 Statistical Parsing ...... 15 2.4.2 Phrase Structure Parsing and Probabilistic Context-Free Grammars . . . 15 2.5 Conclusion ...... 16

3 State of the Art 19 3.1 NLP Toolkits and Libraries ...... 19 3.1.1 Stanford CoreNLP Toolkit ...... 19 3.1.2 The Natural Language Tool Kit ...... 20 3.1.3 Apache OpenNLP ...... 20 3.1.4 IBM Watson ...... 21 3.1.5 spaCy ...... 21 3.2 Tools for Requirements Analysis ...... 23 3.2.1 Requirements Analysis Tool ...... 23 3.2.2 QuARS ...... 24 3.2.3 Automated Requirements Measurement ...... 24 3.2.4 Qualicen Scout ...... 25 3.2.5 QVscribe ...... 25 3.3 Conclusion ...... 27

4 Proposed Solution - RANT 29 4.1 Challenges and Approach ...... 29 4.2 Checklist ...... 31 4.3 Tool Architecture ...... 34 4.4 RANT User Interface ...... 36

ix CONTENTS

4.5 Requirements Analysis Features - RANT_NLP ...... 37

5 Experimentation 41 5.1 Experiment Design ...... 41 5.2 Results and Discussion ...... 42

6 Conclusions and Future Work 47 6.1 Achievements ...... 47 6.2 Future Work ...... 48

References 51

x List of Figures

2.1 Reasons for project failure [Hul09]...... 7 2.2 The Requirements Engineering Process [Som10]...... 7 2.3 Quality Management and Software Development [Som10]...... 11 2.4 The software review process [Som10]...... 12 2.5 NLP Stages ...... 13 2.6 Dependency Graph [SAAA+]...... 14 2.7 Phrase Structure Tree [SAAA+]...... 14

3.1 CoreNLP system architecture [MSB+14]...... 20 3.2 spaCy’s pipeline [spab]...... 22 3.3 spaCy’s Architecture Diagram [spab]...... 22 3.4 RAT’s Analysis Overview [VK08]...... 23 3.5 QuARS High-level Architecture Scheme [Lam05]...... 24 3.6 Overall Smell Detection Process of Qualicen Scout [FFJ+14]...... 25 3.7 Example of QVscribe in Microsoft Word [QRA]...... 26

4.1 Checklist for Requirements Review ...... 32 4.2 Description of Quality Criteria (Continuation of the checklist) ...... 33 4.3 RANT Architecture Diagram ...... 35 4.4 Example of RANT integrated in the CLM RM environment ...... 36 4.5 Ambiguous words and expressions ...... 39

5.1 RANT’s Validation Test Results ...... 44

xi LIST OF FIGURES

xii List of Tables

5.1 RANT Validation Test Results ...... 43

xiii LIST OF TABLES

xiv Abbreviations

API Application Programming Interface ARM Automated Requirements Measurement CLI Command Line Interface CMMI Capability Maturity Model Integration NLP Natural Language Processing NLTK Natural Language Tool Kit PCFG Probabilistic Context-Free Grammar PoS Tagging Part-of-Speech Tagging PST Phrase Structure Parsing QuARS Quality Analyser for Requirements Speciﬁcations RANT Requirements ANalysis Tool RAT Requirements Analysis Tool REMsES Requirements Engineering and Management for software-intensive Embedded Systems SATC Software Assurance Technology Centre SE Software Engineering SPICE Software Process Improvement and Capability dEtermination SQM Software Quality Management SQuaRE Systems and software Quality Requirements and Evaluation V&V Veriﬁcation and Validation

Chapter 1

Introduction

Over the past decade there has been an increase in the complexity of software-intensive embedded systems in the automotive domain [BBH+14]. With the automotive market being as demanding as ever concerning the innovations of increasing complexity, the associated shorter development periods due to the high pressure to market a product make it a necessity to push the current software development processes to its limits. One efficient method of ensuring high quality software in time and within budget is by applying software engineering (SE) processes to a project [Som10]. An important part of these processes is the creation of artefacts, in particular requirements documents. Requirements Engineering is one of the fundamental activities of SE and consists of understanding and defining what services are required from the system and identifying the constraints on the system’s operation and development [Som10]. Low quality requirements can lead to misunderstandings which consequently may contribute to errors in the design flow that are either hard to detect or detected too late [SAAA+]. Therefore, the quality of these documents must be assured and is directly connected to the quality of the software systems as well as their development process. In fact, surveys have shown that the most common reasons for a project’s lack of success are not technical but rather requirements related (13.11%) [Hul09]. Most of these problems arise because the requirements are written in Natural Language which can lead to issues with clarity, consistency and ambiguity. When the requirements are written, it is critical to ensure that they comply with relevant quality criteria and international standards. Unfortunately, that is not always the case and that is what the verification and validation (V&V) phase in requirements engineering is for. During the V&V phase, processes such as reviews are used to check if the requirements documents abide by the quality criteria and standards related to those documents. Although helpful in finding defects in the requirements, these reviews can be tedious because reading and analysing each requirement individually takes time and requires a great effort to stay

1 Introduction alert and remember relevant information. By not performing the requirements reviews adequately, the time until new products are launched could be extended. Existing static analysis tools mainly target code ﬁles, which are of no use when analysing requirements and that is why there is a need for speciﬁc tools that analyse requirements. There are some solutions available that for reasons of adaptability or privacy could not be used. Throughout this investigation, a viable solution was developed by means of an integrated solution in IBM’s Collaborative Lifecycle Management (CLM) tool called RANT (Requirements ANalysis Tool) that makes use of Natural Language Processing (NLP) techniques. This solution is complemented by a checklist that shall be used alongside RANT, both are thoroughly described in4.

1.1 Context

Bosch Car Multimedia is a division of Bosch which develops smart embedded solutions for entertainment, navigation, telematics and driver assistance functions. They develop cutting-edge features tailored to modern mobility requirements, providing optimum driving convenience, safety and access to entertainment and information via smart networked architectures [Bos]. Their ambition is to enhance the quality of life with solutions that are both innovative and beneficial. They focus on their core competencies in automotive and industrial technologies as well as in products and services for professional and private use. A huge part of creating such technology are the processes that take place throughout the development of a project. Software Engineering processes are usually applied by companies in order to produce quality software on time and within budget. One of those processes is the elaboration of artefacts and their respective validation. In the scope of this experiment, the Software Specification, i.e., the Requirements Engineering process is the one on focus. As quality is of such importance in the automotive industry, it is essential to develop software fast but still abiding to certain quality criteria defined by a series of international standards regarding this matter. These criteria should be applied when the requirements are being written but that does not always happen. Hence, the validation of the requirements is a crucial task to ensure quality in the requirements and, consequently, of the software. During the requirements review which is part of the validation process, analysts manually go through each requirement trying to find mistakes and incoherences regarding the international quality standards. The problem urges because this task takes up a lot of time and adding the pressure to market a product to the necessity of having quality software, there comes a need for a partially automated solution regarding this review process.

2 Introduction

1.2 Motivation and Goals

Taking into consideration that the requirements review process is such a tremendously time consuming chore, it is paramount to find a solution that helps turn this task into a brisker and semi-automated process. Although there are numerous static analysis tools that can be used to scan source code files for errors, there is not extensive knowledge about tools that help analysing requirements. The reason for that lack of possible solutions is that the requirements are written in Natural Language which, in one hand is very good because it is accessible to everyone involved in the project; on the other hand, it can be incoherent, incomplete and ambiguous. A lot of time is spent reading the requirements one by one and looking for possible defects which might even result in an unsuccessful search because, as humans, staying always alert can be difficult and errors can be overlooked. The quality of the requirements is of the utmost importance to companies as errors in the requirements can cause serious consequences in the overall project leading to a delay in product releases consequently bringing costs to the company. With the aim of achieving quality requirements in the smallest period of time possible, the focus of this project is to develop a tool that assists the analysts during the review, turning it into a simpler and more efficient task. By identifying applicable quality criteria, the tool will analyse the requirement using NLP techniques providing an insight as to the existence of possible issues with the requirement, alerting the user to that situation.

1.3 Document Structure

From this point forward, this document is structured in five chapters. The first one, Chapter2 contains all the needed knowledge to understand the research done in this project. Then, there is a chapter regarding the State of the Art, Chapter3, with information about the existing tools that have similar goals as the one developed during this research along with relevant details about toolkits and libraries regarding this matter. Chapter4 describes the work that was pursued in detail, providing information about the architecture of the tool and its features. The validation process of this project is thoroughly specified in Chapter5, mentioning the tests done and their respective results. Lastly, in Chapter6, the conclusion taken from this experiment including some points concerning future work are stated.

3 Introduction

4 Chapter 2

Background

2.1 Software Engineering

Being in an era where technology has such an increasing impact in our lives, it seems impos- sible to imagine a world with no software, because it is everywhere around us: from our cars to our smart TVs, going through medicine and industry, cleaning robots and smart virtual assistants. In late 2013, a survey found that 23% of products now contain software in some form [Agi]. Software Engineering as defined in [Ref10] is the systematic application of scientific and technological knowledge, methods, and experience to the design, implementation, testing, and documentation of software; that is, the application of engineering to software. Applying software engineering to a project is considered the most effective way of getting software of high quality within schedule and budget [Som10]. Software processes are concerned with work activities accomplished by software engineers to develop, maintain and operate software, such as requirements, design, construction, testing, configuration management and other activities [BF14]. Four is the number of fundamental activities that are common to all software processes according to [Som10]. The one on focus during this investigation is the Software Validation activity. During the Software Validation activity the software is analysed to make sure it complies to its specification and meets the needs of the stakeholders. There are no universal software engineering methods that are applicable to all systems [Som10]. One of the most important points to consider when deciding which method to implement is the type of software that is being developed: stand-alone or interactive transaction-based applications, embedded control or batch processing or entertainment systems, modelling and simulation software and data systems. Embedded systems are software control systems that control and manage hardware devices. Bosch Car Multimedia develops embedded solutions for entertainment, navigation, telematics and driver assistance functions and they follow the REMsES (Requirements Engineering and Manage- ment for software-intensive Embedded Systems) approach.

5 Background

2.2 Requirements Engineering

Requirements Engineering is the process of understanding and defining what services are required from the system and identifying the constraints on the system’s operation and development [Som10]. Requirements Engineering is concerned with discovering, eliciting, developing, analysing, determining verification methods, validating, communicating, documenting, and man- aging requirements [Ref10]. Following Requirements Engineering processes ensures that the user expectations will be met and the software will be of high quality. In the automotive domain, where requirements are elaborated at an early stage in the project life cycle, errors made at this stage can be hazardous to the software design and implementation if not found in time and corrected. The later these errors are found, the higher are the costs [HDS+04]. Low quality requirements can lead to potential failures to discover the needed functionality, to understand the implications of such functionality, to properly explain the requirements to the developers and, very often, to understand what is the real problem that needs to be solved [RR]. In fact, surveys have shown that problems with the requirements are the main reason for the lack of success in a project, represented by 13.11% as shown in Fig. 2.1. Regarding the Requirements Engineering process, there are four main sub-activities (Fig. 2.2). During the Feasibility Study, it is evaluated whether the identified needs of the user are being fulfilled by the software and hardware solutions currently being used. It is also taken into consideration whether the suggested system will be profitable from a business point of view and if it can be developed within the available budget [Som10]. With the results from this study, it must be decided if the proposed solution is the best alterna- tive given the constraints in technology, resources, finances, etc. [BF14]. The Requirements Elicitation and Analysis process is fundamentally a human activity where the stakeholders are identified and relationships are established between the development team and the customer [BF14]. It may also involve the creation of one or more system models and prototypes in order to better understand the system that is to be specified [Som10]. After gathering the information from the previous activity, the Requirements Specification process is simply turning that information into a set of requirements that accurately define the customer needs. This document can also be used as the basis for developing effective verification and validation plans [BF14]. The Requirements Validation process is where it is checked if the requirements are real, complete and consistent. The goal of this activity is to find possible problems in the requirements document. If any are found, a report shall be elaborated showing the problems with the requirements and the necessary corrections so the requirements document can then be adjusted.

6 Background

Figure 2.1: Reasons for project failure [Hul09]

Figure 2.2: The Requirements Engineering Process [Som10]

7 Background

2.2.1 Requirements Validation

The requirements documents may be subject to Veriﬁcation and Validation (V & V) procedures [BF14]. V & V procedures are used to determine whether the development products of a given activity conform to the requirements of that activity and whether the product satisﬁes its intended use and user needs [10112]. The requirements may be validated to make sure that the developer has understood the requirements; it is also important to verify that a requirements document conforms to company standards and that it is correct, complete, accurate, consistent and testable [BF14]. But why is Requirements Validation so important? If errors are not discovered in the requirements document during this stage but later during project development or even after the system is being put to use, it can lead to extensive and costly rework. As mentioned in [Som10], this assurance is given by:

• Validity checks: The functions proposed by stakeholders should be consistent with what the system needs to do.

• Consistency checks: Requirements in the document should not conﬂict.

• Completeness checks: The requirements document should include all the requirements and all the constraints.

• Realism checks: Make sure the requirements can be fulﬁlled using current existing technology, within budget and schedule.

• Veriﬁability: It should be possible to write a set of tests that can prove that the delivered system meets each requirement.

In order to validate the requirements, there are various techniques that can be used individually or jointly: Requirements Reviews, Prototyping, Model Validation and Acceptance Tests [BF14]. Requirements reviews seem to be the most common means of both verifying and validating requirements documents [BF14]. During this process, a group of reviewers analyse the requirements systematically looking for errors, mistaken assumptions, lack of clarity, veriﬁability issues and deviation from standard practice [Sta11]. The output of a requirements review is a list containing reported problems with the requirements document and the actions necessary to cope with the reported issues [PEM03].

8 Background

2.2.2 Requirements Quality

The quality of requirements in the requirements document has a significant impact on the final system. Requirements of poor quality can lead to misunderstandings and further errors in the design that are usually difficult to detect or detected too late [SAAA+]. Therefore, it is paramount to ensure the quality of requirements by defining requirements in a way that their validation is objective, rather than a subjective opinion. Requirements are usually written in Natural Language, meaning there is no defined format by default, just normal plain text. Hence, requirements can be vague and ambiguous which calls for the use of certain standards and guidelines to help make the requirements clearer, easy to understand, complete and consistent. Software standards are essential regarding software quality management because they capture wisdom that is of value to the organisation, provide a framework for defining the meaning of quality and they facilitate the continuity when work started by another person is picked up and continued by another [Som10]. Concerning Requirements Quality, Bosch Car Multimedia follows the following international standards:

• Automotive SPICE [VDA15]

• ISO/IEC/IEEE 29148:2011 - System life cycle processes - Requirements Engineering [Sta11]

• ISO/IEC 25010:2011 - Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models [ISO02a]

• Capability Maturity Model Integration (CMMI) [CMM]

Automotive SPICE (Software Process Improvement and Capability dEtermination) is a process maturity framework that assesses the capability and maturity of organisational processes to develop embedded systems in the automotive industry [VDA15]. Basically, it is a derivation from the ISO 15504 created specially for the automotive industry. ISO/IEC/IEEE 29148:2011 defines the construct of a good requirement, provides attributes and characteristics of requirements and discusses the iterative and recursive application of requirements processes throughout the life cycle [Sta11]. SQuaRE defines a product quality model that comprises eight quality characteristics: Func- tional Suitability, Performance Efficiency, Compatibility, Usability, Reliability, Security, Main- tainability and Portability [ISO]. CMMI is a process model that provides a clear definition of what an organisation should do to promote behaviours that lead to improved performance. It has five Maturity Levels: Performed, Managed, Defined, Quantitatively Managed and Optimising [CMM].

9 Background

2.3 Software Quality Management

Software quality problems were initially discovered in the 1960s with the development of the first large software systems and they are still an issue nowadays [Som10]. Trying to solve this situation and inspired by the methods used in the manufacturing industry, formal techniques of software quality management were developed and adopted. The definition of quality software is, basically, fitness for purpose, i.e., conformance to requirements. It is providing something that satisfies the customer and ensures the needs of all the stakeholders are considered [Hul09]. Quality Management as defined in [Ref10] is a set of coordinated activities to direct and control an organisation with regard to quality. Every management decision is a compromise between cost, schedule and quality [Hul09]. Software Quality Management consists of four subcategories:

• Quality Planning

• Quality Assurance

• Quality Control

• Quality Improvement

Quality Planning involves deciding which quality standards will be used, defining quality goals and doing an estimation on the effort and schedule of the quality activities. Quality Assurance incorporates various steps that define and assess if the software processes are appropriate and that they produce software products of suitable quality for their intended pur- poses [BF14]. As for Quality Control, the project documents are analysed and executables are examined to determine whether they follow the standards set for the project. Regarding the Quality Improvement category, it is sought to improve process effectiveness, efficiency and other characteristics with the objective of improving software quality [BF14].

2.3.1 Software Quality Assurance

Quality Assurance as stated in [Ref10] is a set of planned and systematic activities imple- mented within the quality system and demonstrated as needed, to provide adequate confidence that an entity will fulfil requirements for quality. The purpose of the Quality Assurance Process is to provide independent and objective assurance that work products and processes comply with predefined provisions and plans and that non-conformances are resolved and further prevented [VDA15].

10 Background

Figure 2.3: Quality Management and Software Development [Som10]

2.3.2 Veriﬁcation and Validation

The Software Quality Management (SQM) process is significant when it comes to assuring quality in software. This process checks the project deliverables to make sure that they are consistent with organisational standards and goals [Som10]. Part of this SQM process is the Verification and Validation procedure. The purpose of Verification and Validation (V&V) is to aid an organisation to achieve quality in their system during the life cycle. V&V processes grant an unbiased evaluation of products throughout the life cycle. This evaluation shows whether the requirements are correct, complete, accurate, consistent and testable [BF14]. Verification is an attempt to ensure that specified requirements have been fulfilled and that they satisfy the standards, practices and conventions during the life cycle processes. Validation is the confirmation that the requirements for a specific intended use or application have been fulfilled and that they solve the right problem [ISO02b]. Both verification and validation are interrelated and complementary processes that use each other’s process results to establish better completion criteria and analysis [10112]. They take place early in the development or in the maintenance phase. As the output, the V & V plan documents describe the various resources and their roles and activities as well as the techniques and tools to be used [BF14]. Part of the V & V activities involves checking processes, such as reviews and inspections, at each stage of the software process from user requirements definition to program development [Som10] as illustrated in Fig. 2.3.

2.3.2.1 Reviews and Inspections

Requirements reviews are possibly the most used method of both verifying and validating a requirements document. Reviews and inspections are used side by side with program testing as part of the V & V processes.

11 Background

Figure 2.4: The software review process [Som10]

In order to arrange a requirements review, a group of reviewers is constituted with the purpose of looking for errors, mistaken assumptions, lack of clarity, verifiability issues and deviation from standard practice [Sta11]. Generally, the review process consists of three phases: pre-review activities, the review meeting and the post-review activities as shown in Fig. 2.4. Inspection as defined in [IEE08] is a visual examination of a software product to detect and identify software anomalies, including errors and deviations from standards and specifications. As part of the inspection process, a checklist with the most ordinary errors is used to help the inspectors focus during the search for bugs [Som10]. Inspections and reviews are part of the so-called static V & V techniques. These techniques ex- amine software documentation, including requirements, interface specifications, designs, models and source code without executing the code [BF14]. Inspections and reviews have a very positive outcome when it comes to discovering software errors. Yet, they demand a lot of time to organise and often lead to delays into the development process [Som10].

2.4 Natural Language Processing

Natural Language Processing (NLP) is a theoretically motivated range of computational techniques for analysing and representing naturally occurring texts at one or more levels of linguistic analysis for the purpose of achieving human-like language processing for a range of tasks or applications [D.L01]. The processing of natural languages is a difficult task and it needs different techniques to be used than the ones for processing artificial languages [ESW14]. Most requirements documents are written in Natural Language rather than using modelling or structured techniques because it is much easier to be understood by all the stakeholders in the project. Besides that, Natural Language allows the engineer to be as abstract or as detailed as required in a certain situation. On the other hand, Natural Language can be ambiguous, leading to misunderstandings in the requirements specification. A simplified view of Natural Language Processing emphasises four distinct stages as depicted in Fig. 2.5.

12 Background

Figure 2.5: NLP Stages

The Lexical Analysis purpose is to interpret the meaning of individual words and has five main techniques that can be used: Sentence splitting, Tokenization, Part-of-Speech (PoS) tagging, Morphological Analysis and Parsing. Sentence splitting is the process of breaking the text into separate sentences. During this process the Natural Language text is analysed to determine the sentence boundaries between the sentences. Most languages use punctuation marks to indicate the boundaries between sentences. There are, however, some instances were punctuation marks are not used for indicating boundaries. For instance, with abbreviations and titles punctuation marks are used, which do not indicate a sentence boundary. Tokenization splits the sentence into its meaningful units, named tokens. Based on the structure of the text, which is partly provided by the sentence splitting, the tokens are associated to a category. The most common categories are words, numbers, punctuation marks and symbols. The Part-of-Speech tagging process is responsible for tagging each token with its grammatical category, based on its definition and context. Each token is then identified with a tag, such as noun, verb, adjective or determiner [Are16]. The Morphological Analysis is the preliminary stage that takes place before syntactic analysis. The purpose of this stage is to identify the root of compound words. This can be accomplished by using stemming and lemmatization. Stemming is a technique that reduces an inflected word to its stem, usually by removing their suffixes and Lemmatization is a technique that finds the root form of a word [Are16].

13 Background

Figure 2.6: Dependency Graph [SAAA+] Figure 2.7: Phrase Structure Tree [SAAA+]

Parsing is a technique that consists in analysing a sentence by taking each word and determining its structure from its constituent parts. In order to parse a piece of text, it is necessary to have two components: a parser and a grammar. The grammar for natural languages is ambiguous and typical sentences have multiple possible analyses [BSPM16]. There are two primary types of parsing: Dependency Parsing and Phrase Structure Parsing. The ﬁrst one focuses on the relations between words in a sentence as shown in 2.6 while the second one focuses on building the Parse Tree, usually using a Probabilistic Context-Free Grammar (PCFG) as it can be seen in 2.7. The output of the Lexical Analysis is the input to the Syntactic Analysis. This process performs an analysis of the words in a sentence in order to reveal the grammatical structure of the sentence. This requires both a grammar and a parser. The output of this level of processing is a representation of the sentence that reveals the structural dependency relationships between the words [D.L01]. Semantic processing determines the possible meanings of a sentence by focusing on the inter- actions among word-level meanings in the sentence [D.L01]. It builds up a representation of the objects and actions that a sentence is describing and includes the details provided by adjectives, adverbs and propositions [RMP13]. Categorisation aims at automatically nominating new documents to categories that are already established [YT06]. In Requirements Engineering, categorising deals with classifying requirements for a certain purpose which can be helpful for the development of software. Classiﬁed requirements can be assigned to teams that each focus on a particular class of requirements.

14 Background

2.4.1 Statistical Parsing

The statistical approach to NLP has become more and more important in recent years though its application started back in the 1980s [YT06]. Statistical parsing consists in methods for syntactic analysis that are based on statistical inference from samples of natural language text [YT06]. Statistical inference may be applied for various features of the parsing process but is mainly used for disambiguation. A possible solution to the problem is a probabilistic parser as it computes the probability of each interpretation and chooses the most probable one [PES17]. The set of possible syntactic representations is usually deﬁned by a particular theoretical framework but normally takes the form of a complex graph or tree structure. The most common type of representation is a phrase structure [YT06]. The idea behind a statistical parser is that it assigns sentences in natural language to their favourite syntactic representations, either by providing a ranked list of possible analyses or by selecting a single optimal analysis [YT06]. There are two types of probability models: Discriminative and Generative.In a Discriminative model, the condition probability P(y|x) is modelled [YT06]. In the Generative model what is modelled is the conditional probability of an input x given a certain label y. The simplest Generative Statistical Parsing model is the Probabilistic Context-Free Grammars (PCFGs).

2.4.2 Phrase Structure Parsing and Probabilistic Context-Free Grammars

Phrase Structure Parsing focuses on identifying phrases and their recursive structure originating in a Phrase Structure Tree (PST). A PST contains structural information about a sentence where the root node represents the whole sentence and the non-terminal nodes represent the syntactic grammar structure in terms of constituents, while the terminal nodes are the atomic words of the sentence [SAAA+]. The analysis of the sentence and annotation into a PST is performed by structural parsers such as the one contained in the Stanford CoreNLP natural language processing toolkit [Sta17]. Natural Language can be ambiguous which can lead to originating multiple PSTs for the same sentence. How the PSTs are generated depends on the grammar employed by the parser. The simplest augmentation of the context-free grammar is the Probabilistic Context-Free Gram- mar (PCFG). Besides the elements contained in a context-free grammar, the set of terminal symbols, nonterminal symbols, the start symbol and a set of rules, PCFGs add a probability function. As shown in Algorithm1, this probability function takes each grammar rule and and associates it with the probability value of each rule [Tha17]. A PCFG can be used to estimate a number of useful probabilities concerning a sentence and its parse-tree(s) which can be useful in disambiguation [JM09].

15 Background

Algorithm 1 PCFG’s Formal Deﬁnition [Tha17] 1: G = (T,N,S,R,P) 2: T is a set of terminal symbols 3: N is a set of nonterminal symbols 4: S is the start symbol ( S ∈ N) 5: R is a set of rules/ productions of the form X → γ 6: P is the probability function 7: P is |R → [0,1] 8: ∀ X ε N, ∑X→γεR P(X → γ) = 1

The probability of an ambiguous sentence is the sum of the probabilities of all the parse trees for that sentence [JM09]. The PST with the highest PCFG score has the best probability of being the correct one.

2.5 Conclusion

Applying software engineering to a project is paramount in order to get high quality software within schedule and budget. The focus of this investigation is on one of the four main activities of software engineering: Requirements Engineering. Following Requirements Engineering processes increases the chances that user expectations are met and the software is of high quality. Mistakes made during this stage can be harmful in the future if they are not found in time and corrected. Requirements Engineering also has four fundamental sub-activities. This research is based on the Requirements Validation stage. Requirements documents shall be subject to validation and verification processes. These are important to make sure errors are corrected so there is less probability of having to do extensive and costly rework ahead in the project development. One of the most used techniques to validate requirements are the Requirements Reviews. As requirements are usually written in Natural Language and due to its ambiguous nature, it is not an easy task to evaluate their quality so it is important that requirements are written in a specific way. That is when international standards and guidelines come to aid. Software Quality Assurance is one of the sub-categories of Software Quality Management which is responsible for providing independent assurance that the products and processes comply with the predefined plans and that non-conformances are solved. Verification and Validation (V&V) methods such as Reviews and Inspections are some of the various processes of Quality Assurance. Besides being one of the most common means of validation, reviews are a time consuming and very tedious task. As most Requirements Documents are written using Natural Language, in order to be able to partially automate the process of reviewing requirements, it is necessary to process those requirements using Natural Language Processing (NLP) techniques.

16 Background

The one on target during this research is the Statistical approach. This approach computes the probability of each interpretation and chooses the most probable one. The most ordinary type of representation is a phrase structure and the simplest statistical parsing model is the Probabilistic Context-Free Grammars (PCFGs).

17 Background

18 Chapter 3

State of the Art

3.1 NLP Toolkits and Libraries

In this section, some of the existing NLP tool kits and libraries will be described, namely their main functionalities as well as their structure.

3.1.1 Stanford CoreNLP Toolkit

The Stanford CoreNLP Toolkit is an NLP toolkit developed by The Stanford Natural Language Processing Group. It offers Java-based modules for the solution of a range of basic NLP tasks as well as the means to extend its functionalities with new ones. CoreNLP is an annotation pipeline framework which provides most of the common core NLP processes. Some of those are Tokenization, Sentence Splitting, Part-of-Speech tagging, Lemmati- zation and Syntactic Analysis based on a probabilistic parser. The overall system architecture of CoreNLP, as illustrated in 3.1, consists on the input of raw text into the Annotation Object, then a sequence of Annotators add information in the analysis pipeline which then results in an Annotation object containing all the analysis information added by the Annotators [MSB+14]. This result can be in XML format or plain text. The annotators provided in Stanford CoreNLP work with any character encoding, the default being the UTF-8 encoding. While most users employ the annotators already present in the toolkit, it is also possible to add additional custom annotators to the system [MSB+14]. Stanford CoreNLP can be used directly from the command-line, via its original Java API, via the object-oriented simple API, via third party APIs for most major modern programming languages like JavaScript or Python or via a web service.

19 State of the Art

Figure 3.1: CoreNLP system architecture [MSB+14]

3.1.2 The Natural Language Tool Kit

The Natural Language Toolkit (NLTK) is a suite of program modules, data sets, tutorials and exercises, covering symbolic and statistical natural language processing [BL]. It is written in Python and is distributed under the GPL open source license. The NLTK provides simple interfaces to over 50 corpora and lexical resources alongside a series of libraries for classiﬁcation, tokenization, stemming, tagging, parsing and semantic reasoning [NLT18]. The aim of the NLTK is to assist research and teaching in NLP areas such as empirical linguistics, cognitive science, artiﬁcial intelligence, information retrieval and machine learning [BL].

3.1.3 Apache OpenNLP

The Apache OpenNLP library is a machine learning based toolkit for the processing of natural language [Apa]. The Apache OpenNLP toolkit consists of several components making it possible to build a full natural language processing pipeline, namely a sentence detector, a tokenizer, a name ﬁnder, a document categoriser, a PoS tagger, a chunker and a parser. Components contain parts which enable one to execute the respective natural language processing task, to train a model and often also to evaluate a model. Each of these facilities is accessible via its application program interface (API). In addition, a command line interface (CLI) is provided for convenience of experiments and training [Apa].

20 State of the Art

3.1.4 IBM Watson

IBM Watson is a powerful tool that uses Artiﬁcial Intelligence techniques applied to business. One of those techniques is the Natural Language Understanding service. This service analyses text to extract meta-data from content such as concepts, entities, key- words, categories, sentiment, emotion, relations, semantic roles, using natural language understanding []. Natural Language Understanding uses NLP to analyse semantic features of any text. Provided plain text, HTML or a public URL and Natural Language Understanding returns results for the features speciﬁed. The service cleans HTML before analysis by default, which removes most advertisements and other unwanted content. It is possible to interact with the Natural Language Understanding service by means of a REST API.

3.1.5 spaCy

spaCy is a library for advanced Natural Language Processing in Python and Cython. It comes with pre-trained statistical models and word vectors and currently supports tokenization for more than twenty languages. It features the fastest syntactic parser in the world, convolutional neural network models for tagging, parsing and named entity recognition and easy deep learning integration [spab]. It is commercial open-source software, released under the MIT license. Unlike NLTK, which is widely used for teaching and research, spaCy focuses on providing software for production usage. spaCy provides the means for the usual NLP techniques such as tokenization, lemmatisation, PoS tagging, entity recognition, dependency parsing, sentence recognition, word-to-vector trans- formations and many methods for cleaning and normalising text [spaa]. While some of spaCy’s features work independently, others require statistical models to be loaded, which enable spaCy to predict linguistic annotations – for example, whether a word is a verb or a noun. spaCy currently offers statistical models for eight languages, which can be installed as individual Python modules [spab]. When the NLP model is loaded and then called on a text, spaCy ﬁrst tokenizes the text to produce a Doc object. The Doc is then processed in several different steps – this is also referred to as the processing pipeline. The pipeline used by the default models consists of a tagger, a parser and an entity recogniser. Each pipeline component returns the processed Doc, which is then passed on to the next component as it can be seen in Fig. 3.2. The architecture of spaCy has essentially two central structures: Doc and Vocab [spab]. As shown in Fig. 3.3, the Doc object owns the sequence of tokens and all their annotations and the Vocab object owns a set of look-up tables [spab]. The Doc is made by the Tokenizer and then modiﬁed by the components of the pipeline. The Language object takes raw text and sends it to the pipeline and then returning an annotated document.

21 State of the Art

Figure 3.2: spaCy’s pipeline [spab]

Figure 3.3: spaCy’s Architecture Diagram [spab]

22 State of the Art

Figure 3.4: RAT’s Analysis Overview [VK08]

3.2 Tools for Requirements Analysis

In this section, a few existing tools for requirements analysis and their respective major functionalities will be described. These tools present a similar goal as the one of this project but are not a viable solution either for adaptability or privacy issues.

3.2.1 Requirements Analysis Tool

The Requirements Analysis Tool (RAT) performs a range of analysis on requirements documents based on industry best practices while allowing the user to write the documents in a stan- dardised syntax using Natural Language [VK08]. This tool carries out a syntactic analysis with the help of a set of glossaries that identiﬁes syntactic components and ﬂags problematic phrases. An experimental version of RAT uses domain ontologies and structured content extracted from the requirements documents during the syntactic analysis in order to perform a semantic analysis [VK08]. As shown in 3.4, RAT’s approach starts with the syntactic analysis of the requirements document. Then, it extracts structured content from the document about each requirement which is used for phrasal and semantic analysis [VK08]. RAT supports a set of controlled syntaxes for writing requirements which include:

• Standard Requirements Syntax

• Conditional Requirements Syntax

• Business Rules Syntax

Three types of user glossaries are used to parse requirements documents: agent glossary, action glossary and modal word glossary [VK08]. It implements a deterministic ﬁnite automata based approach to parse the requirements, extract structured content and generate error messages.

23 State of the Art

Figure 3.5: QuARS High-level Architecture Scheme [Lam05]

RAT has been created as an extension to Microsoft Ofﬁce and can be installed as a plugin for Word and Excel. Ofﬁce libraries in .NET are used to access the textual requirements and analyse them [VKV14].

3.2.2 QuARS

The Quality Analyser for Requirements Specifications (QuARS) is a research tool that aids in the creation of quality requirements by allowing he user to perform an initial parsing of requirements by automatically detecting potential linguistic defects that can cause ambiguity problems at later stages of the software product development [Lam05]. QuARS also provides support for the consistency and completeness analysis of the requirements [Lam05]. The high-level architectural design of the QuARS tool can be seen in 3.5. The input is a requirements document in plain text. This file is passed on to the syntax parser which then produces a new file containing the parsed version of its sentences. This tool relies on a set of indicator-related dictionaries which must also be in plain text [QuAa]. The outputs of the tool include log files with the indications of the sentences containing defects and the calculation of metrics about the defect rates of the analysed document [Lam05].

3.2.3 Automated Requirements Measurement

NASA’s Software Assurance Technology Centre (SATC) developed a tool around the late 90s that automatically analyses a requirements document and produces a detailed quality report [CL14]. This report is based on a statistical analysis of word frequencies at many structural levels of the document.

24 State of the Art

Figure 3.6: Overall Smell Detection Process of Qualicen Scout [FFJ+14]

The tool is called Automated Requirements Measurement (ARM). It browses a requirements document searching for exact key words and phrases that could impact the quality of the requirements [McC01]. The SATC later developed eSMART, which is an update of ARM, that included additional features, such as a better user interface, the possibility to read the input speciﬁcations from a Microsoft Word formatted document and the ability to specify custom word lists for the quality indicators used in the analysis process [CL14]. Unfortunately, the work on the ARM tool stopped in 2011, although some versions of the tool are still used by various organisations.

3.2.4 Qualicen Scout

Qualicen Scout is a real-time quality analysis and visualisation tool for requirements and tests written in natural language [quab]. This tool features several different analysis to identify quality issues in the requirements also named requirement smells. The difference between a requirement smell and a defect is that a smell is only an indication for a possible quality defect [Are16]. Scout detects long and complicated sentences, usage of passive voice, multiple negations, vague phrases and pronouns, comparatives and superlatives, usage of a slash, duplicate requirements and, if required, Qualicen Scout can also perform structural analysis [quab]. The tool provides users with warning messages which a short description whenever a smell is detected. Some smells use NLP techniques such as morphological analysis and PoS tagging. The process of detecting requirements smells consists of four steps (Fig. 3.6): Parsing, Annotation, Smell Identiﬁcation and Presentation. Qualicen Scout integrates with some current tools like Qualicen PTC Integrity LM, Microsoft TFS, Visual Studio Team Services, GIT and SVN [quab].

3.2.5 QVscribe

QVscribe is a requirements analysis tool created by QRA Corp [QRA]. It combines natural language processing (NLP) with an expert system. QVscribe analyses requirements and alerts the author of ambiguous, overly complex and essentially malformed engineering requirements [QVs].

25 State of the Art

Figure 3.7: Example of QVscribe in Microsoft Word [QRA]

QVscribe performs syntactic and semantic analysis of requirements for quality and consistency. Once QVscribe is installed, it appears as a toolbar icon. It is used as a add-in tool for Microsoft Word, Visure Requirements and Marinvent’s Synthesis [QRA]. In order to start using QVscribe, first it is necessary to specify the location of the requirements in a document, then specify the acceptable terminology to be used in the requirements and identify specific parts of the requirements that should not be analysed [QRA]. After these steps, QVscribe can auto-find the requirements present in the document. The analysis provided by QVscribe is based on eight quality measures: Imperatives, Negative Imperatives, Options, Weaknesses, Vagueness, Subjectiveness, Continuances and Directives. The analysis can also be customised to match a company’s best practices for requirements documentation. The results of the analysis are displayed in a simple, interactive scorecard which indicates the quality of each requirement by number of stars (one to five) - an example of QVscribe in Microsoft Word can be seen on Fig. 3.7. Clicking on a requirement listed in the scorecard takes the engineer to the requirement in the document. There, the engineer can see highlights showing the quality indicators that triggered the given score for the requirement [QRA].

26 State of the Art

3.3 Conclusion

A tool that performs static analysis on requirements is a huge help when it comes to assuring the quality of the requirements documents. Libraries and toolkits such as the Stanford CoreNLP, the Natural Language Tool Kit (NLTK), the Apache OpenNLP and spaCy are paramount when developing a tool that analyses requirements documents. Most of these perform standard NLP techniques such as tokenization, PoS tagging, sentence splitting which is of great help when it comes to detecting possible defects in requirements. With that in mind, several companies have come up with solutions that are able to provide that needed assistance. Although the offers on the market for requirements analysis tools are scarce, there are some which can be of help to this research. Some example of such tools are the Requirements Analysis Tool (RAT), the Quality Analyser for Requirements Specifications (QuARS), Automated Requirements Measurement (ARM), the Qualicen Scout and QVscribe. Despite being useful for this investigation, they are not a viable solution for this particular case. Adaptability is an issue in the case of RAT, Qualicen Scout and QVscribe because they are available as extensions to several platforms such as Microsoft Word and Marinvent’s Synthesis and can not be integrated in the intended environment - CLM. Accessibility is also an obstacle, specifically in the case of ARM and QuARS, because these tools are not available for use, either because they are not available anymore (ARM) or because they were created as academic research tools (QuARS). Some of these tools demanded that the requirements were written in a determined structure and not just plain Natural Language text as expected in this investigation. However, the analysis provided by these tools is very detailed and advanced, providing a deeper evaluation of the requirements documents. That leaves the reasons mentioned before as the only setbacks of using them in this specific case. Were they to create an extension for IBM’s CLM and doing some adjustments to the input type, they could be viable solutions.

27 State of the Art

28 Chapter 4

Proposed Solution - RANT

Analysing requirements is a complex task that requires a considerable amount of time and resources. For that reason, it was critical to create a solution that not only helps the reviewers during that process but is also integrated in the requirements management tool already in use. In this chapter, the challenges faced during this investigation are carefully outlined, alongside the approach taken in order to work on those challenges. The tool developed for this project is an extension of IBM’s Collaborative Lifecycle Manage- ment (CLM) tool that is used in the Requirements Management (RM) environment. A compre- hensive description of this tool regarding its architecture and features can also be found in this chapter.

4.1 Challenges and Approach

The quality of requirements in textual specification documents has a significant impact on the final product [VAD09]. As a matter of fact, various studies have shown that fixing engineering errors in software projects can become exponentially costly over the project life cycle [HDS+04]. Furthermore, it is estimated that more than half of those errors are originated in the requirements [RR]. Hence, it is imperative that errors are found and corrected during the requirements stage, exactly when they occur and not later in the project’s life cycle. Defects in the requirements such as ambiguity, lack of readability or inconsistency can lead to delays when launching a product, when that could be avoided by simply finding and correcting issues in the requirements in due time. The task of finding errors in the requirements has been handed to software engineers and analysts who manually go through the requirements one by one trying to detect possible mistakes. Though, as human beings, sometimes errors can slip by because of how tedious and time- consuming this task is.

29 Proposed Solution - RANT

Considering that the market is as challenging as ever, it is paramount that time is not lost, especially analysing and correcting requirements when software could have been developed and a product launched. As nearly 90% of the requirements documents are written in Natural Language, the existent static analysis tools are not fit for that purpose [SARW12]. Most of the tools available in the market are not for requirements analysis but rather code syntax checkers, debuggers or static analysis tools designed to find errors in the software and not in the requirements. Nevertheless, there are some tools, as mentioned in Chapter3, that are suitable for requirements analysis but either require them to be written in a specific form, are not integrated in the tool that is being used, are not freely available or are not ready for use in a non-academic environment. Consequently, the need for a partial automation of the requirements review process, together with the lack of applicable solutions, led to the urgency to create a tool that checks the intended boxes. What makes this work challenging is the Natural Language Processing (NLP), since most requirements are written in Natural Language as it is easy to understand by all the people involved in the project. On the other hand, it can be ambiguous, inaccurate and inconsistent. Another challenge regarding this project is related to the fact that it needs to be integrated in the current requirements management tool being used by Bosch Car Multimedia: IBM’s Collaborative Lifecycle Management (CLM). The aim of this investigation is to develop a tool that comes to aid during requirements reviews and is integrated in the software being used. The intention is to partially automate the process of analysing requirements in order to decrease the time spent as well as increasing the number of defects correctly identified in those reviews. IBM’s CLM tool offers a section dedicated to the requirements elaboration, analysis and review processes called Requirements Management (RM). In order to be integrated in this environment and being able to access the requirements, an extension was the most reasonable solution. Being a task that demands the fullest attention to detail, the requirements review shall be made with the help of not only the tool but also a checklist that was elaborated during this research as well. Even though this tool together with the checklist provide insight on whether or not the requirements have issues, they are not supposed to be used on their own, but rather as a complement to the analysis of the developer. Towards partially automating this process, first it was essential to comprehend which international quality criteria should be applied to the requirements. Then, a checklist was created to serve as a guideline for the tool itself. Only at that point did the development of the Requirements ANalysis Tool (RANT) begin as an aid tool that makes use of NLP techniques and is integrated in the CLM application.

30 Proposed Solution - RANT

RANT aims at helping the reviewers achieve a defect free requirements document by analysing selected requirements and detecting if they comply with their expected structure as well as possible ambiguous words and expressions present in the sentence and alerts the user to that fact.

4.2 Checklist

There are various guidelines that help when writing good requirements which can either be provided globally, such as international standards, or internally, as an agreement between a company and its clients [SAAA+]. A checklist was created with the main goal of being used together with RANT, in the interest of complementing the review process performed by the analyst. It was elaborated at the beginning of this investigation so it could serve as a guideline to the development of RANT. In order to gather the needed items for this checklist, a research was done regarding Bosch’s internal quality criteria as well as international quality standards. The structure and the explanations given in the checklist were found in [KP15]. Good requirements should all follow the same set of criteria [Cor]. With that in mind, the checklist that was elaborated not only contains a list of criteria but also a group of simple guidelines, accompanied with practical examples. As it can be seen in Fig. 4.1, there is a list of criteria that should be fulﬁlled:

• Completeness

• Correctness

• Clearness

• Consistency

• Feasibility

• Veriﬁability

• Traceability

Each one of these criteria shall be applied when writing and reviewing a requirement. To have a good requirement, it is essential to check if each requirement is valid against each item of this list. Furthermore, there is a list of items that shall be checked when reviewing the requirements, which helps achieving those quality criteria. An explanation of each quality criteria can be found at the end of the checklist as depicted in Fig. 4.2.

31 Proposed Solution - RANT

Figure 4.1: Checklist for Requirements Review

32 Proposed Solution - RANT

Figure 4.2: Description of Quality Criteria (Continuation of the checklist)

33 Proposed Solution - RANT

4.3 Tool Architecture

RANT has a relatively straightforward architecture as it is shown in the architecture diagram in Fig. 4.3. Provided that one of the main goals of this project was to create a tool that is integrated with CLM, RANT has a structure that is based on a standard template for extensions for CLM supported by IBM. The extensions that are developed for CLM must follow a speciﬁc structure, particularly regarding RANT’s front-end environment - RANT_FE:

• An XML ﬁle

• A JavaScript ﬁle

• A CSS ﬁle

The XML file is used when adding RANT to the CLM dashboard. It contains all the necessary information to run the tool, namely the JavaScript and CSS files used to build RANT. The access to the data stored in the CLM tool, specifically in the RM environment, is only possible due to the RM API. The RM API is a service-centric mechanism for accessing and modifying data in the system and for interacting with mechanisms in the local client, such as showing user interface elements [RM]. Generally, the services are accessed via the RM object and its child objects, all of which are provided as part of the RM feature that is loaded into each extension. The JavaScript file contains all the functions necessary to perform calls to the RM API. These functions allow access to the requirements and their respective information. Some examples are the RM.Event.subscribe function, which subscribes RANT to an event and until the extension is closed or unsubscribes from the event, the given callback is invoked whenever the event occurs; another example is the RM.ArtifactAttributes function that associates a set of attribute values with an artefact; other examples along with their description are available in [RM]. Besides that, the requests done to RANT’s back-end structure are also contained on that single JavaScript file. In order to know when a requirement or a group of requirements is selected, the JavaScript file subscribes the notifications of the RM API for when that action occurs. When a requirement is selected, it is possible to gather data about the requirement such as the ID, the Primary Text, who created it, who changed it and when, its priority, its status, among other details. With all that information, only the relevant data concerning the analysis of the requirement is selected, namely the ID and the Primary Text, which will then be sent to RANT’s back-end structure - RANT_BE.

34 Proposed Solution - RANT

Figure 4.3: RANT Architecture Diagram

What occurs is the RANT_FE takes that relevant data and sends it to the RANT_BE. This is done via an HTTP request to the Node JS Server that is responsible for the back-end structure as well as the connection to the NLP component. Then, it stands by awaiting the result from the analysis performed.

Using Node’s module Python-Shell, which allows the communication between JavaScript and Python, RANT_BE is able to pass the data on to a Python script and then just waits a response.

From there, the script runs the analysis, which will be thoroughly detailed in Section 4.5, using spaCy’s NLP functionalities and returns a result. This component responsible for the NLP is called RANT_NLP.

When RANT_NLP has a result from all the analysis performed, RANT_BE is notiﬁed and it is sent back to RANT_FE also via HTTP.

The analysis from RANT is then showed on CLM’s dashboard on the left along with the respective ID and primary text.

35 Proposed Solution - RANT

Figure 4.4: Example of RANT integrated in the CLM RM environment

4.4 RANT User Interface

To be able to use RANT, ﬁrst it is necessary to add a link of the XML ﬁle, that should be stored and running on the same server as CLM, to CLM’s Widget Catalogue.

It is a simple process that is achieved by clicking Add Widget on the dashboard on the left side of the RM environment and RANT will then appear there.

A server has to be running the CLM software, where the catalogue is found, along with RANT_FE.

As seen in the example ﬁgure 4.4, RANT is seamlessly integrated in RM and can be used alongside other functionalities of the framework.

When requirements are selected, RANT performs the analysis and the results are returned in the form of a table, where details from the requirement are shown: the ID, the text and the result from RANT’s analysis.

Every time a requirement is selected, a new table with the respective result appears below the others. In the same way, when a requirement is deselected, the table regarding that requirement disappears.

As RM allows the user to select all requirements present in the page, RANT is also prepared to present the results of its analysis, independently of the quantity of requirements selected.

36 Proposed Solution - RANT

4.5 Requirements Analysis Features - RANT_NLP

Requirements specified using Natural Language, unlike those specified using formal models, are inherently subject to the possibility of being ambiguous, inconsistent and, in some cases, incomplete [Fat13]. This work is composed of two main components: a checklist and RANT. Both these components are used to help the developers and reviewers analyse the requirements, in a way that they complement each other. Since the checklist urges the analyst to look for contradictions in the requirements, traceability attributes, feasibility conditions, among others; RANT analyses the requirements in a more structural manner. As the requirements that are to be analysed are all written in Natural Language, this element of RANT - RANT_NLP, is the one responsible for all the Natural Language Processing techniques that lead to the final result presented by RANT. Regarding the several analysis performed by RANT, it is possible to observe an analogy to the checklist, in other words, RANT checks some of the items listed on the checklist. When analysing a requirement written in Natural Language, the first verification RANT_NLP runs is concerning if the requirement is in passive voice, just like on the checklist. It is possible to detect if a requirement is written using passive voice instead of the correct form which is active voice, using spaCy’s dependency parsing functionality. By making use of the dependency parsing, it is possible to detect if the subject in the requirement, instead of being nsubj, is nsubjpass, indicating that it is written in passive voice. Then, the requirement is analysed regarding the completeness of the structure of the sentence, which is the third item of the checklist. This means that, to be a good requirement, it must be a complete and correct sentence, just like in (4.1). The use of a subject is paramount because it implies a user, owner, system or entity to which the requirement is associated with. Likewise, the use of a verb and object are essential and should be an action phrase or expression of something that is to be done by, for, with or to the subject [Cor]. With the help of Python’s open-source library for advanced NLP called spaCy, it is possible to split the sentence into tokens and check if the requirement is structured as expected, by means of spaCy’s Part-of-speech (PoS) tagging functionality.

Subject (nsubj) + Predicate [Verb (verb) + Direct Object (dobj)] (4.1)

If a requirement is not following the complete intended structure, RANT alerts the user to the speciﬁc case, i.e., if the subject is missing, RANT alerts that the subject is missing and the same occurs for when the verb or the direct object are missing. Besides, RANT uses dictionaries with a selection of words and expressions that might lead to a requirement being vague and ambiguous and runs an analysis that detects if such word or expression is found in the requirement. Some examples of these words can be found in Fig. 4.5.

37 Proposed Solution - RANT

The second item on the checklist aims to make sure there is only one requirement specified per sentence. In RANT, this situation can be checked by detecting the use of conjunctions such as and, or, with, etc. Just like when analysing the requirements structure, RANT alerts the user for specific cases, varying with the different types of words and expressions used. If conjunctions are used, RANT warns the user and reminds that conjunctions should be avoided because they can add unnecessary complexity to a requirement. In the case of using let-out clauses, RANT alerts that they can compromise consistency and should be averted. Using words or expressions that impose suggestions or possibilities will also cause RANT to notify the user because the use of these can jeopardise clarity. Terms that are not well defined, such as appropriate, normal or user-friendly, will also cause RANT to tip off the user about their use due to that fact that these terms induce ambiguity. Lastly, RANT detects if words or expressions that are speculative or lead to wishful thinking are used and notifies the user that these should be avoided to ensure feasibility. The last three items present on the checklist are not yet able to be performed by RANT, meaning that the reviewer must interpret the requirement and extract the necessary information to make sure those items are checked.

38 Proposed Solution - RANT

Figure 4.5: Ambiguous words and expressions

39 Proposed Solution - RANT

40 Chapter 5

Experimentation

5.1 Experiment Design

Testing a software is a crucial step in every project. In this case, the validation of RANT was done using binary classiﬁcation [bin]. In a typical binary diagnostic test, a positive or negative diagnosis is made for each subject. When the diagnosis is compared to the true (known) condition, there are four possible outcomes: true positive, true negative, false positive, false negative. Considering a positive result as a defect found in a requirement and a negative result when none was, there are four possible outcomes to this classiﬁcation:

• a True Positive (TP) is when there is a defect in a requirement and it was detected.

• a False Positive (FP) is when it indicates the requirement has a defect but that is not true.

• a True Negative (TN) is when there is no defect and no defect was detected.

• a False Negative (FN) is when there is a defect in a requirement but it was not identiﬁed.

Using these values, it is possible to calculate the Precision, Recall and Accuracy values of the tool through the formulas (5.1) and (5.2), respectively.

TP Precision = (5.1) TP + FP

TP Recall = (5.2) TP + FN

TP + TN Accuracy = (5.3) TP + TN + FP + FN

41 Experimentation

Precision as defined in [Tin10] is a ratio of true positives and the total number of positives predicted by a model which translates to out of all the requirements the tool identified as having defects, how many actually had them. Recall as defined in [Tin10] is the ratio of true positives and the total number of actual positives which means that, out of all the requirements that actually have defects, how many were detected. Besides those values, it is also possible to calculate the accuracy of the tool, using formula (5.3). The accuracy value measures how well a binary classification test correctly identifies a condition. In this case, the accuracy is the proportion of true results among the total number of cases tested. In order to obtain all of these values, it was decided to conduct tests which consist in providing the tool a set of 65 dummy requirements provided by Bosch, which are representative of real requirements, and then getting the analysis from the tool. After that, this analysis was given to a group of 10 expert reviewers that belong to different teams inside Bosch. They were asked to individually provide their opinion on the analysis of the tool regarding that set of requirements: if they agreed or disagreed with the analysis done by the tool. If there was a situation where the experts did not agree with the tool, it was asked for them to specify why in a comment section. The results and its conclusions can be found in 5.2.

5.2 Results and Discussion

The results in detail can be found in Table 5.1. There, the verdict of each reviewer about each requirement can be found, (A) for Agree, (D) for Disagree as well as the analysis from RANT, (Yes) when a defect was found and (No) when no defect was found. In order to obtain a final verdict for each requirement, the majority of a certain verdict was considered and the final result: True Positive (TP), False Positive (FP), True Negative (TN) and False Negative (FN) is the translation of that value. In most cases, there was a consensual verdict of either Agreement or Disagreement. However, there were 29 cases were a consensus was not reached. 16 is the number of cases where only one reviewer disagreed with the rest, 11 cases where 2 reviewers disagreed and only 2 cases of 3 reviewers disagreeing. As such, all of the final results have a certainty percentage of at least 70% (7 out of 10 reviewers). From a total of 65 requirements analysed, 36 were True Positives and 19 were True Negatives. Regarding the False Positive results, which were 8, the majority of them was due to the use of conjunctions that could induce complexity but it was not considered a problem by the experts. In terms of False Negatives, there were only 2 cases and their reason is purely semantic, i.e., the problem was with the meaning of the sentence, which is out of scope for RANT.

42 Experimentation

Table 5.1: RANT Validation Test Results

Reviewers Assessment (Agree or Disagree) Tool Defect Req. ID R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Summary Analysis 1 Yes A A A A D A A A A A TP 2 Yes A A A A A A A A A A TP 3 Yes A A A A A A A A A A TP 4 Yes A A A A A A A A A A TP 5 Yes A A A A D A A D A A TP 6 Yes A D D A A A A A D A TP 7 Yes D D D D D D D A D A FP 8 No A A D A A D A A A A TN 9 No D D D D D A D D A A FN 10 No A A A A A A A A A A TN 11 Yes A D A A A A A A A A TP 12 No A A A A D A A A A A TN 13 No D D D D A D A D D D FN 14 No A A A A A A A A A A TN 15 Yes A A A A A A A A A A TP 16 Yes D D D D D D D D D D FP 17 No A A A A A A A A A A TN 18 Yes A A A A A A A A A A TP 19 Yes A A A A A A A A A A TP 20 Yes A A A A A A A A A A TP 21 Yes A A A A A A A A A A TP 22 Yes A A A A A A A A A A TP 23 Yes D D D D D D D D D D FP 24 No A A A A A A A A A A TN 25 No A A A A A A A A A A TN 26 Yes A A A A A D A A A D TP 27 Yes A A A A A D A A A A TP 28 Yes A A A A A A A A A A TP 29 Yes A A A A A A A A A A TP 30 Yes A A A A A A A A A A TP 31 No A A A A A A A A A A TN 32 Yes A A A A A A A A D A TP 33 Yes D D D D D D D A A D FP 34 Yes A A A A A A A A A D TP 35 Yes A A A A A A A A A D TP 36 Yes A A A A A A A A A A TP 37 Yes D D D D D A D D D A FP 38 No A A A A A A A A A A TN 39 No A A A A A A A A A D TN 40 Yes A A A A A A A A A A TP 41 Yes A A A A A A A A A A TP 42 Yes A D D D D D D D A D FP 43 Yes A A A A A A A A A D TP 44 No A A A A A A A A A A TN 45 No A A A A A A A A A A TN 46 Yes A A A A A A A D A A TP 47 Yes A A A A A A A A A A TP 48 Yes A A A A A A A A A D TP 49 Yes A A A A A A A D A D TP 50 Yes D D D D D D D D D D FP 51 Yes A A A A A A A A A D TP 52 No A A A A A A A A A A TN 53 Yes A A A A A A A A D D TP 54 Yes D D D D A A D D D D FP 55 No A A A A A A A A A A TN 56 No A A A A A A A A A A TN 57 Yes A A A A A A A A A A TP 58 Yes D A A A A A A A A A TP 59 No A A A A A A A A A D TN 60 Yes A A A A A A A A A A TP 61 No A A A A A A A A A A TN 62 No D A A A A A A A A A TN 63 Yes A A A A A A A A A A TP 64 Yes A A A A A A A A D A TP 65 No A A A A A A A A A A TN

43 Experimentation

Figure 5.1: RANT’s Validation Test Results

Concerning the False Positive results, RANT detects conjunctions and expressions that can lead to problems when interpreting the requirements. However, there are cases where the conjunctions are essential for the meaning of the sentence. In that case, RANT can’t distinguish between both cases and alerts the user which happened in all False Positive hits. The usage of conjunctions such as and, or and with were wrongly classiﬁed as defects, according to the experts in 7 out of 8 FP and the usage of the word when formed a let-out clause that could lead to problems with consistency but was disregarded by the experts. An example of such situation is the following requirement The VMPS shall receive a frame with CAN ID Value equal to 1003. RANT detected the use of the conjunction with and alerted the user for a possible complexity addition. However, the reviewers all agreed that it was necessary for this speciﬁc case and that there was no defect. Another example occurred with the requirement The VMPS shall respond only when a message is received. The result from RANT detected the use of a let-out clause expression when that could compromise consistency but was considered paramount in this case. A third example detected both and and or conjunctions in the requirement The VMPS shall encode the received alphanumeric or space characters, according to the International Morse Code encoding and return the encoding on signal Coded_Tx_Payload, alerting the user about adding unnecessary complexity to the requirement but was also deemed important to the meaning of the sentence. The results are shown in Fig. 5.1 regarding True Positive (TP), False Positive (FP), True Negative (TN) and False Negative (FN) values. By using these, it is possible to calculate RANT’s precision, recall and accuracy values which are, respectively, 0.818, 0.947 and 0.846.

44 Experimentation

In general, RANT’s results appear to be promising though there are no previous studies regarding other similar requirements analysis tools to use for comparison. However, RANT has a percentage of approximately 12% of False Positive results while static analysis tools aim at a 30% maximum so users don’t give up on using the tool and 20% as the goal to achieve, thus these results are very encouraging [BEB+10]. In general, the opinion of the experts is positive regarding RANT’s usability in the review process. They claimed that, even alerting the user about a few cases that were disregarded as issues, RANT could facilitate their work by saving some time and ﬁnding defects that could otherwise pass by.

45 Experimentation

46 Chapter 6

Conclusions and Future Work

An appreciation of the achievements that were accomplished during this work along with the plans for the continuity of this project are made in this chapter. Additionally, directives towards possible future work regarding improvements to the developed tool are also stated.

6.1 Achievements

Requirements engineering is a paramount step in every software project as, if done correctly, can lead to quality software and prevent future rework and respective costs. In order for that to happen, it is necessary to assure that the requirements are well-written, complete, clear and consistent by means of reviews. Reviews consist of a group of analysts going through each requirement manually, verifying if they are written according to the international standards criteria and internal rules as well. It is a very time and resource consuming task that beneﬁts from being partially automated so that it takes less time while ﬁnding all possible defects in the requirements. Being written in Natural Language, the requirements pose a challenge during its analysis as it can be vague and easily misunderstood by the different people that read it, from clients to developers to analysts. With that aim, RANT (Requirements ANalysis Tool) was developed as an extra help for the analysts to be used together with a checklist. RANT is a tool that employs a series of Natural Language Processing (NLP) techniques to perform analysis on requirements and alerts the user to possible issues. Additionally, RANT is integrated in Bosch’s Requirements Management tool CLM, which was a core goal of this work.

47 Conclusions and Future Work

Regarding the approach to the semi-automation of the review process, the results obtained by RANT were very promising with an accuracy value of 84.6%, a precision value of 81.8% and a recall value of 94.7%. Both main goals were accomplished resulting in a useful tool that is capable of providing encouraging results when looking for defects in requirements while being fully integrated in the currently used Requirements Management tool. The feedback from the experts that validated RANT’s results was also positive. They claimed that even with that percentage of False Positive and False Negative results, there is potential in the tool to be used by them during the review process, alongside the checklist. In conclusion, RANT has obtained promising results that, together with the reviewers assessment, lead to the belief that RANT has the capability to be used by requirements experts to aid them during the review process and provide useful feedback.

6.2 Future Work

The potential of performing analysis on requirements is tremendous, specially being written in Natural Language. Having the tendency to be ambiguous, Natural Language comes as a challenge to tackle when aiming for quality requirements. As the following steps towards an excellent tool, more functionalities would have to be developed in RANT, such as detecting similarity between requirements and the use of domain specific glossaries. These could bring benefits to RANT, possibly allowing it to analyse a requirement semantically. By doing that, the percentage of False Negatives could be even smaller. If RANT is to be used in the reviewers daily basis, some adjustments could be made towards improving RANT’s performance in terms of speed. Plus, there are other functionalities that could be added to RANT but are not related to NLP techniques, such as analysing the status of the requirement in terms of acceptance and determining if the requirement has dependencies with other requirements. A useful enhancement would be the classification of the defects in terms of how prejudicial that defect could be in the future, using categories or a scale. Having applied a few Natural Language Processing (NLP) techniques when developing RANT, the next step forward regarding this experiment, would be to implement a Machine Learning technique, more specifically a Convolutional Neural Network (CNN). CNNs are very similar to ordinary Neural Networks. They are also made up of neurons that have learnable weights and biases [LK17]. Although CNNs are most commonly known for its performance regarding Computer Vision, there are some studies that prove CNNs also provide promising results regarding NLP [Kim14].

48 Conclusions and Future Work

As future work, a CNN would be applied to the existing RANT tool so that it would learn how a quality requirement should be by means of training and testing with a large amount of data, making it able to better detect defects in the requirements in a shorter period of time. By using a CNN it would be possible to develop self improving algorithms that would analyse the network in order to identify which are the patterns that indicate the requirement is of quality and which are the common pitfalls.

49 Conclusions and Future Work

50 References

[10112] IEEE Std 1012-2004. IEEE Standard for System and Software Veriﬁcation and Valida- tion IEEE, volume 2012. 2012.

[Agi] Agile Manifesto | Software is Everywhere | Jama Soft- ware. https://www.jamasoftware.com/blog/ rethink-agile-manifesto-software-everywhere/. Accessed: 2018- 01-27.

[Apa] Apache OpenNLP Developer Documentation. https://opennlp.apache.org/ docs/1.8.4/manual/opennlp.html. Accessed: 2018-06-18.

[Are16] Brian Arendse. A thorough comparison of NLP tools for requirements quality improvement. Master’s thesis, 2016.

[BBH+14] Peter Braun, Manfred Broy, Frank Houdek, Matthias Kirchmayr, Mark Müller, Birgit Penzenstadler, Klaus Pohl, and Thorsten Weyer. Guiding requirements engineering for software-intensive embedded systems in the automotive industry: The REMsES approach. Computer Science - Research and Development, 29(1):21–43, 2014.

[BEB+10] Al Bessey, Dawson Engler, Ken Block, Ben Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles Henri-Gros, Asya Kamsky, and Scott McPeak. A Few Billion Lines of Code Later. Communications of the ACM, 53(2):66–75, 2010.

[BF14] Pierre Bourque and Richard E. Fairley. Guide to the Software Engineering - Body of Knowledge. 2014.

[bin] The 3 pillars of binary classiﬁcation: Accuracy, precision & recall. https://medium.com/@yashwant140393/ the-3-pillars-of-binary-classification-accuracy-precision-\ recall-d2da3d09f664. Accessed: 2018-06-11.

[BL] Steven Bird and Edward Loper. NLTK: The Natural Language Toolkit.

[Bos] Bosch Car Multimedia. https://www.bosch.com/products-and-services/ mobility/. Accessed: 2018-06-11.

[BSPM16] Euiwhan Kim Gi-Name Wang (eds.) Bhabani Shankar Prasad Mishra, Satchi- dananda Dehuri. Techniques and Environments for Big Data Analysis: Parallel, Cloud, and Grid Computing. Studies in Big Data 17. Springer International Publishing, 1 edition, 2016.

51 REFERENCES

[CL14] Nathan Carlson and Phil Laplante. The NASA automated requirements measurement tool: A reconstruction. Innovations in Systems and Software Engineering, 10(2):77–91, 2014.

[CMM] CMMI Institute. https://cmmiinstitute.com/. Accessed: 2018-06-19.

[Cor] IBM Corporation. Get it right the ﬁrst time: Writing better requirements. http://download.boulder.ibm.com/ibmdl/pub/software/ rationalsdp/documentation/product_doc/DOORS/version_9-2/get_ it_right_the_first_time.pdf. Accessed: 2018-07-01.

[D.L01] Elizabeth D.Liddy. Natural Language Processing. Encyclopedia of Library and Infor- mation Science, 2001.

[ESW14] Tj Eggebraaten, Rj Stevens, and Ew Will. Natural language processing (’NLP- overview’). US Patent 8,639,495, pages 1–19, 2014.

[Fat13] A. Fatwanto. Natural language requirements speciﬁcation analysis using part-of-speech tagging. In Second International Conference on Future Generation Communication Technologies (FGCT 2013), pages 98–102, Nov 2013.

[FFJ+14] Henning Femmer, Daniel Méndez Fernández, Elmar Juergens, Michael Klose, Ilona Zimmer, and Jörg Zimmer. Rapid requirements checks with requirements smells: two case studies. Proceedings of the 1st International Workshop on Rapid Continuous Soft- ware Engineering - RCoSE 2014, pages 10–19, 2014.

[HDS+04] B.; Haskins, B.; Dick, J.; Stecklein, R.; Lovell, G.; Moroney, and J Dabney. Error Cost Escalation Through the Project Life Cycle. Incose -Annual Conference Symposium Proceedings- Cd Rom Edition; 2004, page 8.4.2, 2004.

[Hul09] Elizabeth Hull. Requirements Engineering, volume 41. 2009.

[IEE08] IEEE. IEEE STD 1028-2008, IEEE Standard for Software Reviews and Audits, volume 2008. 2008.

[ISO] ISO/IEC 25010. http://iso25000.com/index.php/en/ iso-25000-standards/iso-25010. Accessed: 2018-06-12.

[ISO02a] ISO 12207:2008. International Standard Iso. Iso, 8(4):27, 2002.

[ISO02b] ISO 12207:2008. International Standard Iso. Iso, 8(4):27, 2002.

[JM09] Daniel Jurafsky and James H Martin. Speech and Language Processing: An Introduc- tion to Natural Language Processing, Computational Linguistics, and Speech Recogni- tion. Speech and Language Processing An Introduction to Natural Language Process- ing Computational Linguistics and Speech Recognition, 21:0–934, 2009.

[Kim14] Yoon Kim. Convolutional Neural Networks for Sentence Classiﬁcation. pages 1746– 1751, 2014.

[KP15] Chris Rupp Klaus Pohl. Requirements Engineering Fundamentals. 2015.

[Lam05] Giuseppe Lami. QuARS: A Tool for Analyzing Requirement | SEI Digital Library. (September), 2005.

52 REFERENCES

[LK17] Marc Moreno Lopez and Jugal Kalita. Deep Learning applied to NLP. Arxiv, 2017.

[McC01] J. R. McCoy. NASA software tools for high-quality requirements engineering. 26th An- nual NASA Goddard Software Engineering Workshop, IEEE/NASA SEW 2001, page 69, 2001.

[MSB+14] Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. The Stanford CoreNLP Natural Language Processing Toolkit. Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55–60, 2014.

[NLT18] NLTK Project. Natural Language Toolkit — NLTK 3.3 documentation. https://www.nltk.org/{%}0Ahttp://www.nltk.org/{%}0Ahttps: //www.nltk.org/{%}0Ahttp://www.nltk.org/, 2018. Accessed: 2018-06- 20.

[PEM03] Frauke Paetsch, Armin Eberlein, and Frank Maurer. Requirements engineering and agile software development. WET ICE 2003. Proceedings. Twelfth IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises, 2003., pages 308–313, 2003.

[PES17] Gerald Penn, Jason Eisner, and Noah Smith. Statistical Parsing. 13, 2017.

[QRA] QVscribe by QRA Corp - Analyze Requirements Documents in Seconds. https: //qracorp.com/qvscribe/. Accessed: 2018-02-05.

[QuAa] An Automatic Tool for Improving the Quality of Software Requirements. https: //www.ercim.eu/publication/Ercim{_}News/enw58/lami.html. Ac- cessed: 2018-02-04.

[quab] Qualicen Scoute. https://www.qualicen.de/en/products/scout.html. Accessed: 2018-06-19.

[QVs] Making the Analysis of Requirements Documents Easier : Automo- tive Design & Production. https://www.adandp.media/articles/ making-the-analysis-of-requirements-documents-easier. Accessed: 2018-02-05.

[Ref10] Iso Reference. ISO/IEC/IEEE 24765-2010(E), Systems and software engineering — Vocabulary. 24765, 2010.

[RM] RM Extensions API. https://jazz.net/wiki/bin/view/Main/ RMExtensions405API. Accessed: 2018-07-03.

[RMP13] Alpa Reshamwala, Dhirendra Mishra, and Prajakta Pawar. Review on natural language processing. IRACST Engineering Science and Technology: An International Journal (ESTIJ), 3(1):113–116, 2013.

[RR] James Robertson and Suzanne Robertson. The user story considered harmful.

[SAAA+] Mathias Soeken, Nabila Abdessaied, Arman Allahyari-Abhari, Andi Buzo, Liana Musat, Georg Pelz, and Rolf Drechsler. Quality Assessment for Requirements based on Natural Language Processing.

53 REFERENCES

[SARW12] Bahar Sateli, Elian Angius, Srinivasan Sembakkam Rajivelu, and René Witte. Can Text Mining Assistants Help to Improve Requirements Speciﬁcations? Mining Un- structured Data (MUD 2012), 2012.

[Som10] Ian Sommerville. Software Engineering. 2010.

[spaa] A short introduction to NLP in Python with spaCy – To- wards Data Science. https://towardsdatascience.com/ a-short-introduction-to-nlp-in-python-with-spacy-d0aa819af3ad. Accessed: 2018-06-20.

[spab] spaCy Library. https://spacy.io/. Accessed: 2018-06-16.

[Sta11] International Standard. INTERNATIONAL STANDARD ISO / IEC / IEEE Systems and software engineering — engineering. 2011, 2011.

[Sta17] Stanford University. Stanford CoreNLP – Core natural language software | Stanford CoreNLP. https://stanfordnlp.github.io/CoreNLP/, 2017. Accessed: 2018-01-30.

[Tha17] Jalaj Thanaki. Python Natural Language Processing. 2017.

[Tin10] Kai Ming Ting. Precision and Recall, pages 781–781. Springer US, Boston, MA, 2010.

[VAD09] S Vinay, Shridhar Aithal, and Prashanth Desai. An Approach towards Automation of Requirements Analysis. Proceedings of the International MultiConference of Engineers and Computer Scientists, I:1–6, 2009.

[VDA15] VDA QMC Working Group 13 Automotive SIG. Automotive SPICE { R }. page 132, 2015.

[VK08] Kunal Verma and Alex Kass. Requirements analysis tool: A tool for automatically analyzing software requirements documents. Lecture Notes in Computer Science (including subseries Lecture Notes in Artiﬁcial Intelligence and Lecture Notes in Bioin- formatics), 5318 LNCS:751–763, 2008.

[VKV14] Kunal Verma, Alex Kass, and Reymonrod Vasquez. Using syntactic and semantic analyses to improve the quality of requirements documentation. Semantic Web, 5(5):405– 419, 2014.

[YT06] Deniz Yuret and Ferhan Türe. Handbook of Natural Language Processing. Proceed- ings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics -, pages 328–334, 2006.