Funcionalidad De Kiuwan

Date: 02/16/2015 Review: Feb 2015

Version Changes Feb 2015  It updates contents to Kiuwan February 2015’s release.

We thank Lourdes Ortiz Saqudo, from Universidad de Huelva, for reviewing the document and detecting some errors.

CHANGES LOG ...... II

CONTRIBUTIONS AND ACKNOWLEDGMENTS ...... II

CONTENT ...... III

INDEX OF FIGURES ...... VI

1 INTRODUCTION ...... 1

2 KIUWAN ...... 1 2.1 FUNCTIONALITIES ...... 2 2.2 TARGET USERS ...... 3 2.3 HIGHLIGHTS ...... 3 2.4 ADVANTAGES ...... 4 2.5 ROI ...... 4 2.6 WHY KIUWAN? ...... 5

3 WORKING WITH KIUWAN ...... 7 3.1 BASIC CONCEPTS ...... 7 3.2 STATIC ANALYSIS WITH KIUWAN ...... 8 3.3 QUALITY METRICS AND INDICATORS ...... 12 3.3.1 First level indicators ...... 13 3.3.1.1 Quality indicator ...... 13 3.3.1.2 Effort to target ...... 13 3.3.1.3 Risk index ...... 13 3.3.2 Second level indicators ...... 14 3.3.2.1 Number of defects ...... 14 3.3.2.2 Duplicated code ...... 14 3.3.2.3 Lines of code ...... 14 3.3.2.4 Function points ...... 14 3.3.2.5 Complexity by function ...... 14 3.4 GENERATING AUTOMATIC ACTION PLANS ...... 15 3.5 QUALITY MODELS MANAGEMENT ...... 17 3.6 HELPING MANAGEMENT: DECISION QUADRANTS ...... 21

4 KIUWAN MAIN FEATURES ...... 24 4.1 CODE MEASUREMENT AND ANALYSIS ...... 25 4.1.1 Measure and analyze the code ...... 26 4.1.2 Analyze the software securely in the cloud ...... 27 4.1.3 Multi-language/multi-technology applications support ...... 28 4.1.4 Applications partial analysis ...... 29 4.1.5 Analysis configuration of ABAP applications ...... 29 4.1.6 Analysis configuration in C/C++ applications ...... 30

III

4.1.7 Comparing analyses ...... 31 4.1.8 Code metrics ...... 32 4.1.9 Duplicated code detection ...... 33 4.1.10 List of defects ...... 33 4.1.11 Defects Mute ...... 34 4.1.12 Technical Debt or Effort to Target ...... 35 4.1.13 Kiuwan Rule Developer & Rules API ...... 36 4.2 SECURITY AND PRIVACY ...... 36 4.2.1 Analyze the code locally ...... 37 4.2.2 Multiuser environment ...... 38 4.2.3 Multi-tenant architecture ...... 39 4.2.4 Processing personal data ...... 39 4.2.5 Access to Kiuwan ...... 40 4.2.6 Backup ...... 40 4.2.7 Data persistence policy ...... 40 4.3 CUSTOM QUALITY MODELS ...... 42 4.3.1 Kiuwan software indicators ...... 43 4.3.2 Índice de Riesgo ...... ¡Error! Marcador no definido. 4.3.3 A default quality model ...... 45 4.3.4 Customizing our quality models ...... 46 4.3.5 Reuse your rulesets from PMD, Checkstyle and FindBugs ...... 48 4.3.6 Comparing quality models ...... 49 4.3.7 Quality Models reports ...... 49 4.4 CAPABILITIES FOR DECISION MAKING ...... 51 4.4.1 Notifications ...... 52 4.4.2 Portfolios management ...... 52 4.4.3 Global View ...... 54 4.4.4 Historic evolution ...... 57 4.4.5 Trends ...... 58 4.5 QUALITY AND SECURITY CONTINUOUS MANAGEMENT ...... 59 4.5.1 Integration in your continuous deployment process ...... 60 4.5.2 Jenkins plug-in ...... 60 4.5.3 RESTful API ...... 61 4.5.4 Integration with JIRA ...... 62 4.5.5 IBM Urban Code...... 63 4.6 ACTION PLANS AND ‘WHAT-IF’ ANALYSIS ...... 64 4.6.1 “What if” analysis ...... 64 4.6.2 Action plans ...... 65 4.6.3 “Remediation Progress”...... 66 4.6.4 Reporting ...... 67

5 USAGE SCENARIOS ...... 69 5.1 AGILE PROJECTS DEVELOPMENT TEAMS ...... 69 5.2 DEVOPS ...... 69 5.3 QUALITY OFFICE ...... 71

5.4 OUTSOURCED DEVELOPMENT TEAMS MANAGEMENT ...... 72 5.5 CERTIFICACIÓN DE AENOR EN ISO 25000...... 74

6 CONCLUSIONS ...... 76

7 REFERENCES ...... 77

APPENDIX I. CHECKING QUALITY MODEL (CQM) ...... 78 BASED ON ISO-25000 ...... 78 MODEL STRUCTURED INTO LAYERS ...... 78 UNDERSTANDING THE INDICATORS ...... 78 AGGREGATIONS ...... 79

FIGURE 1.- KIUWAN: SOFTWARE IN THE CLOUD QUALITY AND SECURITY ...... 1 FIGURE 2.- TWO WAYS TO PERFORM STATIC CODE ANALYSIS IN KIUWAN ...... 8 FIGURE 3.-. KIUWAN LOCAL ANALYZER DOWNLOAD ...... 9 FIGURE 4.-. STATIC CODE ANALYSIS START ...... 9 FIGURE 5.-. VIEWING THE PROGRESS OF THE ANALYSIS ENGINES ...... 10 FIGURE 6.-. VIEWING THE DEFECTS ON KIUWAN ...... 11 FIGURE 7.-. RECALCULATING THE QUALITY INDICATORS ...... 11 FIGURE 8.-. ANALYSIS RESULT OF DUPLICATED CODE ...... 12 FIGURE 9.-. VIEWING METRICS ON KIUWAN ...... 12 FIGURE 10.-. FIRST LEVEL INDICATORS ...... 13 FIGURE 11.-. VIEWING THE DEFECTS BY CATEGORY ...... 14 FIGURE 12.-. SPECIFYING THE DESIRED LEVEL FOR THE DIFFERENT CHARACTERISTICS ...... 15 FIGURE 13.-. ACTION PLAN PRIORITIZED FOR A SPECIFIC AMOUNT OF TIME ...... 16 FIGURE 14.-. SOFTWARE QUALITY EVOLUTION BASED ON AN ACTION PLAN ...... 16 FIGURE 15.-. DETAILED ACTION PLAN ...... 17 FIGURE 16.-. SOFTWARE QUALITY EVOLUTION BASED ON AN ACTION PLAN ...... 17 FIGURE 17.-. SETTING KIUWAN QUALITY MODELS ...... 18 FIGURE 18.-. SETTING THE GLOBAL INDICATOR LEVEL OF DEMAND ...... 19 FIGURE 19.-. SETTING THE MODEL CHARACTERISTICS WEIGHTING ...... 19 FIGURE 20.-. SETTING THE WEIGHT FOR EACH TECHNOLOGY ...... 20 FIGURE 21.-. SETTING PRIORITIES AND REPAIR TIME ...... 20 FIGURE 22.-. SETTING ONE OF THE RULES IN THE MODEL ...... 21 FIGURE 23.-. DECISION QUADRANTS ...... 21 FIGURE 24.-. DECISION QUADRANTS II ...... 23 FIGURE 25.-. DEFECTS FOUND AFTER AN ANALYSIS ...... 26 FIGURE 26.-. APPLICATIONS ANALYSIS IN THE CLOUD ...... 27 FIGURE 27.-. MULTI-TECHNOLOGY SUPPORT ...... 28 FIGURE 28.-. APPLICATIONS PARTIAL ANALYSIS ...... 29 FIGURE 29.-. ANALYSIS CONFIGURATION FOR C/C++ APPLICATIONS ...... 30 FIGURE 30.-. COMPARING ANALYSES ...... 31 FIGURE 31.-. CODE METRICS ...... 32 FIGURE 32.-. DUPLICATED CODE DETECTION ...... 33 FIGURA 33.-. LISTADO DE DEFECTOS ...... 33 FIGURE 34.-. DEFECTS MUTE ...... 34 FIGURE 35.-. TECHNICAL DEBT / EFFORT TO TARGET ...... 35 FIGURE 36.-. RULE DEVELOPER & RULES API ...... 36 FIGURE 37.-. APPLICATIONS LOCAL ANALYSIS ...... 37 FIGURE 38.-. KIUWAN MULTIUSER ENVIRONMENT ...... 38 FIGURE 39.-. KIUWAN PLATFORM ARCHITECTURE ...... 39 FIGURE 40.-. SOFTWARE INDICATORS ...... 43 FIGURA 41.-. INDICADOR GLOBAL DE CALIDAD ...... 44

FIGURE 42.-. RISK INDEX ...... 44 FIGURE 43.-. ISO 25000 QUALITY MODEL ...... 45 FIGURE 44.-. QUALITY MODEL CUSTOMIZATION ...... 46 FIGURE 45.-. QUALITY MODEL CUSTOMIZATION II ...... 47 FIGURE 46.-. IMPORTING THIRD-PARTY RULES ...... 48 FIGURE 47.-. QUALITY MODELS COMPARISON ...... 49 FIGURE 48.-. QUALITY MODELS REPORT ...... 50 FIGURE 49.-. KIUWAN NOTIFICATIONS ...... 52 FIGURE 50.-. KIUWAN PORTFOLIOS ...... 53 FIGURE 51.-. DECISION QUADRANTS AT PORTFOLIOS LEVEL ...... 54 FIGURE 52.-. GLOBAL VIEW ...... 54 FIGURE 53.-. DECISION QUADRANTS AT GLOBAL LEVEL ...... 56 FIGURE 54.-. GLOBAL VIEW APPLICATIONS RANKING...... 57 FIGURE 55.-. HISTORIC EVOLUTION ...... 57 FIGURE 56.-. TRENDS ...... 58 FIGURE 57.-. INTEGRATIONS WITH THIRD PARTIES AVAILABLE ON KIUWAN ...... 60 FIGURE 58.-. INTEGRATION WITH JENKINS ...... 60 FIGURA 59.-. RESTFULL API ...... 61 FIGURE 60.-. INTEGRATION WITH JIRA ...... 62 FIGURE 61.-. INTEGRATION WITH IBM URBAN CODE ...... 63 FIGURE 62.-. “WHAT-IF” ANALYSIS ...... 64 FIGURE 63.-. ACTION PLAN ...... 65 FIGURE 64.-. REMEDIATION PROGRESS ...... 66 FIGURE 65.-. KIUWAN REPORTS ...... 67 FIGURE 66.-. KIUWAN INTEGRATION WITH JENKINS ...... 70 FIGURA 67.-. INTEGRACIÓN DE KIUWAN CON JENKINS II ...... 71 FIGURE 68.-. DEFECTS COSTS DEPENDING ON SDLC PHASES ...... 71 FIGURE 69.-. QUALITY OFFICE REPORT SAMPLE ...... 72 FIGURE 70.-. ISO 25000 CERTIFICATION ECOSYSTEM ...... 74

VII

Companies require software develoment to sustain their businesses. Quality and security assurance of the developed code is essential to:  Proper software maintenance and development cost management, either carried out by themselves or other resources.  Reduce issues —bugs— and technical requirements vulnerabilities —non-functional or structural— in their applications: security, performance, efficiency…  Align software systems with business.  Increase projects development teams productivity.  A major software development projects control —govern— or maintenance outsourcings. In many cases, these companies do not have the human resources nor the required infrastructure to automate this quality measurement, or they cannot afford hiring a software quality certification service in situ. For this reason SaaS (Software as a Service) benefits organizations and Kiuwan is the ideal answer for assessing Software Systems’ quality and security in the cloud. It is attractive and easy to use, without needing infrastructure or personnel resources, and due to its reasonable cost, highly justifies the ROI associated to the adoption of this service in the software Development Process.

Figure 1.- Kiuwan: Software in the cloud quality and security Kiuwan (www.kiuwan.com) is Optimyth Software’s solution in the cloud, for code quality and security Measurement and Analysis of business critical applications for organizations. This document describes fundamentals, core competencies and some of its possible use scenarios.

Code analyses run by Kiuwan are designed to measure, analyze and verify code quality and security of corporate applications portfolio, immediately providing essential results and reports.

Kiuwan provides the following features, which are detailed in chapter 4:  Source code static analysis, based on a set of our own analyzers (static analysis, metrics, duplicated code detection), either locally through a downloadable agent or in the cloud, uploading the code to Kiuwan platform.  Multi-technology scanning engines, capable of scanning Objetive-C, Android, Java, JSP, Javascript, PHP, C, C++, ABAP, COBOL, JCL, PL/SQL, Transact-SQL, SQL, Visual Basic, C# or VB.NET code, with about 3,000 rules that implement the best programming and design practices, able to detect critical errors and security vulnerabilities in the code.  Use of third-party rule sets. Kiuwan can enrich its quality models with rules imported from other static analysis tools, like PMD, checkstyle or Findbugs, and use them with its local analyzer.  Defects mute, to avoid false positives or rules exceptions.  Rule Developer & Rule API. To develop custom rules.  Duplicated code detection.  Getting more than 60 software quality metrics: number of violations, complexity, design, volumetric…  Calculation of software quality indicators, necessary for software applications portfolio govern and management: risk —associated to code structural quality—, quality index, effort of repairing defects (effort to target), technical debt…  Visualization, through dashboards, of the obtained results.  Action plan, including proposals for modifications, to reduce the software maintenance effort and, therefore, its TCO, with the option to export this plan to an incidences tracking tool, such as Atlassian JIRA.  Verifying the execution of the action plans (Remediation Process). To verify the compliance of the saved plans.  History of analysis results.  Comparison of results with other applications (Rankings).  Generation of automatic reports at executive and detailed levels.  Aggregation of information (Application, portfolio y global views).  Decision quadrants, to quickly obtain the health of our developments, or an application ranking that allows to see all the applications from different perspectives.  Compleye utility for the quality models management and customization.

 Categorization of applied rules and metrics.  Support and management for industry-recognized or made by customers Quality Models, with versioning capability to facilitate the measurement of different kinds of applications.  Specially designed to implement continuous code inspection, with integration capabilities (RESTfull API) in its Development Process (ALM) and DevOps, allowing to automate and know at every moment the applications quality and security status. It offers integrations with JIRA, Jenkins, IBM UrbanCode, CloudBees…

Unlike other static analysis products in the market, Kiuwan is designed to meet the needs of various profiles implied in Software Development, Software Quality Assurance or Validation & Verification processes, such as:  Developers  QA Engineers  QA Consultants  IT Managers

 No need to install software on workstation.  Option to download an agent that analyzes the code locally, to avoid having to upload the code to the cloud, when needed confidentiality requirements, or integrate Kiuwan in integration or continuous deployment processes.  Information privacy. The code is deleted from the portal once it has been analyzed.  100% customizable quality models (rules and metrics used by analyzers) and categorization (by default) of code quality rules.  Large set of analyzers. Compliance with coding standards, duplicated code detection, software quality metrics calculation.  Support of the most significant technologies: Objetive-C, Android, Java, JSP, Javascript, PHP, C, C++, ABAP, COBOL, JCL, PL/SQL, Transact-SQL, SQL, Visual Basic, C# or VB.NET.  Supported standards. Kiuwan helps the applications quality and security to accomplish industry standards, such as OWASP, CWE, WASC, PCI, MISRA o CERTC, ISO/IEC 25000 or ISO/IEC 9126.  Multi-technology support within the application.  It intuitively provides applications quality information at different levels (development teams, outsourcer, QA manager, IT manager).

 It manages applications portfolio, to group metrics and make benchmarks and rankings.  History and labeling of the obtained metrics.  Intuitive navigation.  Dashboards designed for different profiles (developer, QA Manager, IT Manager).  Obtaining automatic results reports.  Export results to different formats (PDF, Excel, XML).  Performance. Kiuwan has been designed to provide the best performance in all application layers. From the database structure to the memory cache and the application itself, both the server and the client sides. Everything has been optimized to manage small and large accounts alike.  100% functional Trial edition.  Simple hiring model: it just requires the number of applications needed to be analyzed.

There are several advantages offered by a service, like Kiuwan, for software applications quality and security assurance and management.  Detects in advance software weaknesses that could cause failures in operating time.  Suggests modifications (action plans) that reduce the code maintenance effort and, therefore, its TCO, and helps to verify its implementation.  Allows IT customers organizations the applications source code quality certification, automatically and without needing a separate deparment for that.  It fully conforms to methodologies, rules and best practices already established in organizations, allowing own rules implementation.  Simplifies this management in distributed work environments.  Shortens the adoption curve of software quality certification.  It does not require hiring specialist staff or buying any software or hardware.  Simplicity. It does not require you to know in advance your applications data, such as lines of code, used technologies, number of times you need to analyze the code, number of developers. You simply have to register the applications you need.  It allows to tackle “peak loads” of this activity, adjusting to the customer business needs, without imposing an “extra” cost.

To obtain ROI, it must be remembered that the two variables affected by software quality from the economic point of view: a) The cost our software “no quality” can have (the money we lose by not having the necessary quality). b) The “quality” cost (what I have to spend to achieve the quality I need).

For factual information of this second variable, we have basically two options, each with its advantages and disadvantages: a) Provide this function in an organic way, within some functional unity, specific department. In this case, besides the associated personnel costs, you will have to consider the tools associated to the function you will need to performance. b) Hiring an on-site service that provides the data that function would obtain. That service will impact the use of the tools within it as an expense and not as a cost.

Kiuwan improves this second option even more, since it provides this service in SaaS modality, which yields savings above 60-90%. But there are many indicators (KPIs) that the organizations can begin to measure with Kiuwan to improve their development proccesses and their applications portfolio maintenance. Some of them are: • % applications systematically verified. • % approved applications. • Quality indicator improvement. • Software characteristics main indicators improvement: Maintainability, Reliability, Efficiency, Security, Portability. • Software Development Risk reduction. • Technical Debt reduction. • # of redeliveries reduction. • % maintenance effort reduction. • Development process cost reduction. • % Productivity. • % Defect Removal Elimination (DRE). • % duplicated code reduction. • % code complexity reduction. • % production/exploitation incidendes reduction. • …

 Because of its functionality: Static code analysis, duplicated code detection, software quality metrics (# of violations, complexity, design, volumetric…) and software quality indicators...

 Since it offers this functionality without time limits, technologies, lines of code — LoC—, size, # users, # analysis of the code, reports, etc.  In view of its powerful customization capabilities beyond static analysis or QA metrics.  Due to its internal quality model configurable through a complete utility (Quality Model Manager).  By reason of its 'What-if analysis' Technical debt indicator, Risk indicator of risk or the organization of the applications portfolios.  Because of the management reports of the latest analysis or the comparative analysis with historical data and defects found, in excel/xml format.  Because it doesn't require additional payment consultancy for obtaining "qualified reports".  On the grounds that is is reliable (secure). Analyses can be run locally (through an agent) or in the cloud (uploading the code to Kiuwan site).  As it can be quickly adopted and is attractive and comfortable (it is a Cloud solution).  Because, in comparison with other alternatives, it has a greater usability, it is more professional and delivers much more functionality and management capabilities.

This chapter aims to introduce the basic concepts Kiuwan uses, as well as a quick initial tour of the use we can give to it, without going into details about each one of its functionalities, which will be seen in Chapter 4.

Before starting to work with Kiuwan, it is useful to understand the following concepts:  Application. In Kiuwan, an application is a piece of source code wanted to be analyzed and monitored independently, as well as observing its evolution over time. It can be identified with a name (label) and it can contain code written in different programming languages.  View (or Perspective). Kiuwan provides 3 “views” to sort information: o Application. This view provides access to the lower level of the applications and the analyses run on them. o Portfolio. This view provides access to the aggregated results, based on the configured portfolios “logic”. o Global. Groups applications and portfolios giving useful information at executive level.  Portfolio. It is a logical grouping of multiple applications. For example, if a company has part of their developments outsourced, it would be interesting to see the applications that each one of their suppliers have developed or maintain (software factories or integrators). Each factory would be a portfolio and the set of all of them, a portfolios group.  Rule/Standard. A rule is a program responsible for detecting some kind of problem in the source code. Kiuwan provides an extensive rule set (more than 3,000) which Kiuwan uses to verify the compliance with quality standards for each supported technology.  Quality model. It is the set of rules that participate in the analyses. The values of each rule in a quality model can be modified: category, priority, messages and/or modify its behavior through properties.  CQM. It is the Quality Model that Kiuwan provides by default, to start analyzing the applications quality and security from the very start (see Annex I) .  Defect/Violation/Vulnerability. They are the non-compliances that Kiuwan has found analyzing the source code with the defined rule set. The concept vulnerability is commonly used for security flaws.  Metric. It is a measure that Kiuwan calculates from the source code. There are volumetric metrics, lines of code metrics, function points metrics, maintainability metrics, code duplicity metrics, percentage of comments metrics or complexity by function metrics.

 Function points. Unit used to measure the functional size of an application, that unlike lines of code, allows to compare module or applications sizes developed in different technologies.  Quality indicator. Grade from 0 to 100 with whom Kiuwan indicates the quality level or the “health” of the analyzed application. High values indicate a good technical quality of the analyzed application. Values near zero mean poor quality.  Effort to target. It is the effort, measured in hours, to be incurred to repair the defects Kiuwan has found and reach the set quality goals. Its concept is similar to technical debt, so popular lately.  Risk index. It is a risk indicator calculated by Kiuwan. It represents the number of potential problems you assume by not paying attention to your source code quality.  KLOC. It is the equivalent of 1,000 lines of effective code, excluding comments and blank lines.

Kiuwan provides two ways to perform static code analysis:

Figure 2.- Two ways to perform static code analysis in Kiuwan The first way is to download the “analyzer” to the workstation in a zip file, execute and perform the verification of the code locally, in the user’s own infrastructure. To do this, they will choose the application to analyze (previously registered in Kiuwan) and the path where the application code is.

Figure 3.-. Kiuwan Local Analyzer download Once clicked on ‘Continue’, it will confirm the technology of the files and will start the static code analysis.

Figure 4.-. Static code analysis start

The second option is to upload the code to the cloud through a .zip with all the application source files. Kiuwan detects the technologies in the compressed file and runs the analysis of each one of them, giving a quality global grade. When the static analysis starts, Kiuwan invokes three analysis engines that harness the AST tree transformation of each one of the analyzed source files. These engines are: defects engine, metrics engine, duplicated code engine. As it is making invocations to these engines and getting results, Kiuwan shows the progress.

Figure 5.-. Viewing the progress of the analysis engines Once the results are obtained, they will be displayed in detail in “Defects” option, where even the line of code that is incurring the error is shown.

Figure 6.-. Viewing the defects on Kiuwan “Mute Defects” option allows to mute certain violations we don’t want to be computed in the calculation of the grade, either because they are false positives, either because it does not make sense to repair the generated code or it is third-party code that is not going to be modified at this moment. After muting the whole rule, Kiuwan allows to recalculate the quality indicators with the new scenario.

Figure 7.-. Recalculating the quality indicators

The detail of the analysis result of the duplicated code is showed on the same defects screen. In this case, Kiuwan indicates the pieces of duplicated code and the lines where they begin.

Figure 8.-. Analysis result of duplicated code

Figure 9.-. Viewing metrics on Kiuwan

Kiuwan’s metrics engine has its own screen which shows the different measures of complexity, size, comments, quality, coupling, etc. for the analyzed files,. In this section the main metrics and indicators are explained.

They are Quality Indicator, Effort to Target and Risk Index.

Figure 10.-. First level indicators

The grade obtained is calculated through a complex algorithm that takes into account the severity of the defects, the weight of the category in which the defect is, the analyzed code volume and the criticality of the language for Kiuwan user. Kiuwan allows to “interact” with this algorithm to “play” with its level of demand, the weights of the category and the priority of the rules.

Objectives are defined at the beginning of the application analyses. By default, they have to reach 70 points out of 100 in each category that it is being measured. The objectives are configurable. The tool has a repair effort assigned for each one of the more than 2,000 rules it incorporates. The sum of the repair efforts of each defect indicates the time needed to meet the targets. Repair efforts are configurable.

This indicator concentrates all evidence found in the application source code. To calculate not only is the Quality Indicator used but also the necessary effort to solve all the problems to reach the quality goal set for this application and the application’s size. The risk will show as

very high as it takes into account the low quality and the effort needed to reach the set goal. It is interesting to pay attention to the evolution of risk indicator over time.

One of the main indicators is the number of defects. That is the number of nonconformities detected by Kiuwan for the defined quality model, and it is essential to view this figure by priorities of the rules, as well as by category.

Figure 11.-. Viewing the defects by category Very high priority defects are the first ones to focus on.

Abusing of copying and pasting code is detrimental to the code quality. A defect that “lives” in a replicated fragment will “live” in two places and, therefore, the developer will have to make an extra effort, at least two times to eradicate this defect.

It is not the same to have 100 defects in 100 lines of code than in 1,000,000. The quality in the second case is much better, since the defect density is very low.

Another unit to measure the functional size of an application is the function points that, unlike the lines of code, they allow to compare sized of modules or applications developed in different technologies.

To know the complexity of the methods and functions of an application gives an idea of how easy or hard it will be to maintain it in the future.

Kiuwan allows to generate action plans to meet these needs.

When it comes to achieving certain quality goals, Kiuwan allows you to specify the desired level for each CQM category (see the section on CQM).

Figure 12.-. Specifying the desired level for the different characteristics According to the business needs, you can focus to improve in a particular category or in a subset. For example, if an organization suffers from excessive staff or development team turnover, whether external or internal, it takes considerable time to make changes. Then, it will be interesting to improve the code maintainability (and readability). In this case, Kiuwan will generate an action plan in which it will give instructions similar to the following: where and how to reduce code complexity or where and how to reduce the duplicated code, how to properly document the programs, how to eliminate dead code, how to follow homogeneous naming patterns, etc. If the developed application is a real time system, then it must approach 0 as far as efficiency defects are concerned. If the software that an organization develops must be properly executed in different platforms, either operating system’s (Windows, Unix, Aix…), with various features of machines, networks, etc. Then, no portability significant defect should be allowed. Kiuwan would give instructions on where and how to correct the use of specific commands of an operating system, absolute paths, IP addresses, etc.

Another common case is to have a set of “free” hours of a programmer and see what he can invest them on. In this case, you could indicate to Kiuwan the number of available hours and it would create an action plan prioritized for that amount of time:

Figure 13.-. Action plan prioritized for a specific amount of time In this example, 56 hours of one person’s time is being dedicated to correcting the application’s defects. Kiuwan will show the current picture and how the software quality will evolve if such time is invested:

Figure 14.-. Software quality evolution based on an action plan The action plan, exportable to EXCEL, PDF or integrated with incidence tracking systems like JIRA, details step by step the defects to be repaired:

Figure 15.-. Detailed action plan In order to check the progress in implementing the action plan, Kiuwan has an interface in which, after running a new analysis, it displays the percentage and number of repaired defects, as well as those which are pending.

Figure 16.-. Software quality evolution based on an action plan

When analyzing the developed software quality, it is important to select meticulously the quality model to apply.

A standard quality model (see Appendix CQM Model) which considers reliability as important as maintainability or security as important as performance should be enough, but what if my software has specific characteristics or needs? This is an issue that should not be taken lightly, since it might be quite usual. For example, aerospace applications may need really high security levels. Another case are real- time applications, those for which every millisecond counts, so they will seek the maximum efficiency.

Figure 17.-. Setting Kiuwan quality models

If I had a development team with a high level of turnover, I would emphasize, specially, maintainability. By improving it, the next programmer who “inherits” some software fragment will find it easier to change its code properly without recalling the former developer or, more accurate, some of their closest relatives. In Kiuwan, it is possible to set and manage the quality models. The model level of demand can be set. This is useful to be stricter in new developments, where it is easier to modify what is just coded, and be more permissive in old developments, where changes should be carefully done.

Figure 18.-. Setting the Global Indicator level of demand Depending on the application type, as mentioned, a category may be weighted above others, as well as give more weight to one language than others, e.g. because all business logic “resides” in one of them:

Figure 19.-. Setting the model characteristics weighting

Figure 20.-. Setting the weight for each technology Each violated rule yields a penalty that will impact the quality indicator and a repair effort that will impact the effort to target indicator. These values are configurable depending on the type of rule.

Figure 21.-. Setting priorities and repair time Finally, reaching the lower level, it is possible to enable/disable rules, as well as modify their criticality and/or properties.

Figure 22.-. Setting one of the rules in the model

Decision quadrants are a graphic representation based on the results of two indicators obtained by Kiuwan, from the previously selected different applications portfolios or from the applications global.

Figure 23.-. Decision quadrants

Kiuwan provides several decision quadrants geared for the different IT departments executive levels (CIO, CISO, Development Manager, Systems Manager). In this example, the axes of the graph would be:  The vertical axis corresponds with quality indicator, with values from 0 to 100, being 0 the intersection of the axes and 100 the highest point in the graph.  The horizontal axis would be the effort to targed, measured in the hours that we will spend repairing the defects in the code. It also has values from 0 to 100, reaching the highest value of that scale in the intersection and being lower the further to the right in the horizontal axis.

The scale can be based on absolute or relative values. This is useful in the case where the represented elements are highly concentrated in one point and so you will have a better view of them. For instance, if all applications have values from 70 to 80, it will be much more comfortable to see them represented in a graph based on relative scale. If, conversely, there are some applications with quality values near to 0 and other ones near to 100, this gap wouldn’t be correctly perceived in a relative scale graph, so it would be useful to see them in absolute scale graph. The four sectors in which the decision quadrant is divided are:  Conserve (top left corner). Kiuwan places in this quarter those applications or portfolios with high quality values and very low effort to target values, so no decisions should be made on them.  Observe (top right corner). Those portfolios or applications that, having high quality values, have, at the same time, high effort to target values, so they should be closely examined to see how they evolve.  Remediate (bottom left corner). These ones have low effort to target values, but also low quality values, which means they can be improved and, therefore, they require intervention.  Replace (bottom right corner). In this part of the graph are placed the portfolios or applications with high effort to target values and very low quality values, so they would be candidates to integrate in another existing portfolio or to be replaced by a new one.

To the right of the graph the legend is displayed, with the colors of the represented states.

Figure 24.-. Decision quadrants II The diameter of each circle (representing each portfolio or application) gives an idea of its size, measured in function points or lines of code (LoC), as stated in one of the combos above. The other setting we can perform is to select if we want to see the values on absolute or relative scale. We can see the detail of each portfolio or application by hovering the mouse over the respective circle.

To provide a better understanding between all of Kiuwan features, this document groups them into 6 sets, as on Kiuwan web1:

1. Code measurement and analysis 2. Security and privacy 3. Customizable quality models 4. Capacity for decision making 5. Quality and security continuous management 6. Action plans and ‘What-If’ Analysis

Nearly all features presented in this chapter have a video tutorial on Kiuwan Web with examples that help to easily introduce each one of them, they offer help boxes that explains them or they also have the on-line documentation.

1 See https://www.kiuwan.com/key-features to see this information updated.

Code quality and security impacts the economy Code quality and security are important aspects with a significant economic impact. This implies they should be measured accurately, using effective metrics, proven measurement practices and consistent indicators through the different technologies and languages used inside companies. Kiuwan makes it easy Unlike other static analysis tools, Kiuwan makes the activity of automatically analyze the code very simple. What is unique with Kiuwan are its multi-technology scan engines (15), with their approximately 4,000 rules and 100 metrics that implement best programming and design practices, able to detect critical errors and security vulnerabilities in the code. Standards it supports Kiuwan allows to accomplish your applications quality and security with industry standards, such as OWASP, CWE, WASC, PCI, MISRA or CERTC, ISO/IEC 25000 or ISO/IEC 9126.

Figure 25.-. Defects found after an analysis Measure, analyze and verify code quality and security of the applications portfolios of an organization is the main goal when we use Kiuwan. It immediately provides the essential results on the analyzed code, reporting the following:  Defects detection. Based on the adequacy of quality rules and best programming and design practices defined by the industry.  Software quality metrics.  Duplicated code detection. Its analyzers are capable of scanning Java, JSP, Javascript, PHP, C, C++, ABAP, COBOL, JCL, PL/SQL, Transact-SQL, SQL, Visual Basic, C# and VB.NET code (and new technologies are being continuously added). Kiuwan also helps to test mobile applications developed in Android or Objective-C (iOS). The complete list of defects in this screen can be exported to CSV format.

Figure 26.-. Applications analysis in the cloud If you chose to upload the code to Kiuwan platform and run the analysis there, the operation is simple. You just have to pack the code in a zip file. There is no need to worry about the directory structure. Kiuwan is responsible for searching where the sources are and informs us of what is going to be analyzed. Once that is done, you indicate the label you want to use for that analysis, the quality model to be used, your files encoding and that’s it! Click on “Run” and wait for Kiuwan to notify you that the analysis has been completed, or see on the screen that it has already ended. Are you worried about security? Well, don’t be. Remember that your code is uploaded through a securized connection (https) and Kiuwan never stores the code it analyzes; it is deleted as soon as the analysis is complete.

Figure 27.-. Multi-technology support If your application contains code developed in various technologies, you don’t have to repeat the analysis for every technology that is in your application. Kiuwan engines do it for you, as you can see in the figure above. Kiuwan recognizes the extensions of the files and carries out the association. Before running the analysis, you can verify possible unrecognized extensions and tell Kiuwan what language it corresponds to, so it won’t be left out. The following list shows the correspondence between language and extensions by default:  Objetive-C: .m  JCL: .jcl, .prc  Transact-SQL: .tsql  Javascript: .js  PHP: .php .php3 .php4 .php5 .php6 .phps .phtml  C/C++: .c .h .hh .cpp .hpp .cc .pc  Java: .java  JSP: .jsp .xhtml .jspx  Cobol: .cbl .cpy .cob  Abap IV: .abap  C#: .cs  Oracle PL/SQL: .st .sp .sps .trg .plsql .spp .sf .pkb .pks .fnc .spb  SQL-92 Standard: .xml .sql  Visual Basic: .cls .bas .frm  VB.net: .vb  Hibernate: .java  Android: .java

Figure 28.-. Applications partial analysis Kiuwan always gets the results of the analysis even if the analyzed application exceeds the size limits of your account; although it will only give you the results within your limit. In the report you will be informed of the total LOC. Therefore, if a 50Kloc application is analyzed, Kiuwan gives the results for the first 25Kloc, if you have a free account. This will give you a clear idea of what Kiuwan can do for your application quality, but we highly recommend to use these results carefully, since they do not reflect the entire application. For these partial analyses, the icon to represent the analysis status will be an orange thumb sideways, comparing it to the green thumb up for a complete successful analysis or the red thumb down for a failed analysis. In addition, you will be able to see a flag on every screen, warning you that the results correspond to the lines of code of your current limit.

To analyze Abap code, you need to export the code in the SAP server of the local file system. SAP JCO connector allows to extract all the necessary information to analyze the SAP applications that have been customized. Kiuwan Local Analyzer is distributed with a script that automates the source code extraction, through multiple remote calls into the SAP system, and download it to the local machine where the static analysis is performed.

Static analysis for C and C++ appications is a little bit different from other programming languages.

Figure 29.-. Analysis configuration for C/C++ applications

In these languages, there is a preprocessor that complicates things a bit. The resolution of the header files and macros, which are used in preprocessing phase, is essential for a complete and correct static analysis on a C or C++ application. This can be done by configuring Kiuwan Local Analyzer properly, which includes the option to read the log files generated when compiling your applications, and thus sets your static analysis with the same information used during compilation.

Figure 30.-. Comparing analyses

Compare the results of two analyses is easy. You have to select both analyses you want to compare and then you will be able to see the following lists: removed defects, removed rules, new defects and new rules. Graphics help you to quickly understand if you have more deleted defects than new ones or vice versa. This allows to have a greater control over the evolution of your development projects from the quality point of view.

Figure 31.-. Code metrics Kiuwan measures several metrics, code inherent, in each analysis, and organize them into different categories according to their nature:  Complexity: metrics related to code complexity, including CCN (cyclomatic complexity) or complexity by function metrics.  Coupling: metrics showing the dependence between application components, such as fan-out.  Documentation: metrics that measure how good or bad is the code documented.  Duplicated code: such as duplicated code ratio, #duplications, duplicated lines ratio or files with duplication ratio.  Effort. Effort required to reach the ‘target’ or ‘total quality’ at different granularity levels (by characteristic or priority).  Govern: depending on how exposed to risk your development efforts are.  Quality: adaptation to rules and best practices defined in quality model metrics. Number of defects, rule violations…  Size: they include physical and functional size metrics.

You can access all the information you may need about your code metrics in your applications’ metrics dashboard, including the evolution of the most relevant indicators, so you can evaluate the improvement of your development teams. You can also obtain a Metrics report (in CSV format) with all the indicators calculated, both at application and at file levels.

Figure 32.-. Duplicated code detection Code duplication is one of the most common software development bad practices. It is a useful exercise if you don’t want to start a program from scratch, but dangerous if abused. And it is a nightmare for code maintenance. Large software systems typically contain 10-25% of the redundant code. Imagine you have an error in a fragment of code that you have doubled in several modules. The best practice to apply is to refactor the code to use the functionality properly and avoid copying and pasting. With Kiuwan, you get this functionality from Defects screen, searching for occurrences of rules containing “duplicated code” literal, which locate small, medium or large duplicated code blocks. Kiuwan helps to track the rate of duplications found in the code, and it even says where to look in the code to fix the problem. Kiuwan’s duplication analysis is based on ‘tokens’, so we can detect a fragment of duplicated code, even when it is not exactly the same.

Figure 33.-. List of defects

Kiuwan indicators are based on evidence. Some of these evidence –along with code inherent metrics- are the defects found in the source code analysis. What does Kiuwan consider a defect? A defect is a violation of one of the rules defined in the quality model for a specific language and a software characteristic. Kiuwan provides a complete list of all the defects it detects in the code. The first thing it shows is a summary with the total number of violated rules, the total number of defects found and the number of critical defects. Each row of the list shows the defect related with a violated rule, in which you can see the number of affected files, the number of defects of the rule, the rule itself, the affected software characteristic, the code programming language and the estimated total effort to fix all those defects. You can sort the list by any of the above columns. Software developers will find all the information they need to repair the code. Each row expands to show the specific files where defects were found. For each file, you can expand it to see the defect specific line and even an extract of the code with the defect. Of course, you can export the list with all the detailed information to an Excel spreadsheet.

Figure 34.-. Defects Mute Kiuwan can “mute” defects for the whole application, a specific file or simply for a specific line of code in a file by dragging and dropping the rule that triggers the defects, the file or the line of code. You can perform this action in muted defects tab on Defects screen.

All Kiuwan indicators are recalculated on the fly and muted defects are saved to be taken into account in subsequent analyses. This characteristic gives great flexibility and has a very easy to use interface. For each muted defect, you can specify a reason why it is muted, as well as a comment. For example, if it is a false positive in a file, a line of code that must not be taken into account, a file that has too many defects of the same type or a line of code that has been generated and does not have to be taken into account, or any other basis that is reasonable for the user.

Figure 35.-. Technical Debt / Effort to Target The term “technical debt” was coined by Ward Cunningham to describe the obligation a software organization incurs when they are choosing a design or building approach valid in the short term, but that increases the complexity and is more expensive in the long term. To exploit the financial metaphor, technical debt is often expressed in dollars, euros, rupees… Kiuwan provides functionality to discover the debt, either intentional or unintentional. Kiuwan’s algorithms to calculate Technical Debt are based on factual evidence: the code defects it finds in the source code. Rather than taking the Debt in terms of money, Kiuwan uses the Effort to Target and the Effort to 100% Quality Indicator as substitutes for Technical Debt. The user can translate it to money, multiplying such effort by the cost of one development hour.

Figure 36.-. Rule Developer & Rules API Rule Developer is a tool to help develop and prove your custom rules in your quality models. Totally integrated with the IDE, it allows to easily encode rules logic and prove it against the application code. It even helps to generate the XML file with the meta information needed to import the rule to your Kiuwan rules library. Along with the tool, the complete Kiuwan rules API documentation is published, so there is virtually no limit when it comes to customize and extend their quality models.

Multi-tenant architecture Kiuwan is not an on-premise code review tool hosted in some cloud provider facilities. It has been designed from the start as a SaaS application and, therefore, it is “multi-tenant”, which prevents that anyone other than the owner of the application and its data can access data that is not theirs (malicious users, hackers, backdoors, system errors, information in logs,etc.).

There is no need to upload your code You can analyze the code locally by downloading the analyzer and running the analysis on your infrastructure and uploading your analysis results (encrypted), processing them safely and viewing the final results on Kiuwan platform. Where do the errors occur? To view where in the code errors or security vulnerabilities occur, you can choose: a) To upload the lines of code where they are (Kiuwan shows them in the defects report), or b) Not to upload anyhing of your code (in this case, Kiuwan only shows the number of lines where the erros or vulnerabilities were found).

Figure 37.-. Applications local analysis If you do not want or cannot upload the code to the cloud, it doesn’t matter. Kiuwan provides the option of downloading to the client’s computer an executable file with the analyzers and perform the code analysis locally in the client’s computer. Having the local analyzer option provides an additional benefic, it gives the possibility of integrating analysis and code quality and security measurement tasks with your continuous integration or deployment process, or being part of your ‘nightly build’. Results are securely uploaded to Kiuwan platform, where you can see them in dashboards, generate reports, run ‘What-if’ analyses, see the defects or metrics lists… As if you have run the analysis on the platform.

Figure 38.-. Kiuwan multiuser environment From small shops, with only a couple of applications, to companies with hundreds of applications and several development teams, Kiuwan can suit the needs of any size of organization. The wide range of Kiuwan characteristics makes it the perfect tool to every development team, and even other departments. From developers to QA engineers and managers of all types, all of them are going to use Kiuwan in different ways for different purposes. You can create all the users you need for your Kiuwan account, assigning them their own username and password, so they don’t have to depend on shared credentials.

Kiuwan physical architecture is showed in the following figure:

Figure 39.-. Kiuwan platform architecture Kiuwan is hosted securely on a platform that is at the forefront of the art estate, managed by Amazon, a leading provider of managed hosting and advanced connectivity solutions. Optimyth has chosen Amazon because of its reputation for quality and support services, as well as its unmatched ability to reliably host many of the web systems with most internet traffic. The architecture maintains all the components redundant and ensures their scalability by booting (on demand) service analysis servers.

(From Kiuwan Privacy Policy, https://www.kiuwan.com/privacy-policy) In response to OPTIMYTH concerns to ensure your data security and confidentiality, it has adopted the personal data protection security levels and installed the technical means at its disposal to prevent the loss, misuse, unauthorized access and theft of personal data supplied through www.kiuwan.com.

None of the data will be assigned, sold or rented at any time. In accordance with provisions of Law 34/2002 11th of July, de Servicios de la Sociedad de la Información y del Comercio Electrónico, users have the right at any time to see their data, unsubscribe or change their preferences. To exercise this right, users can write to the following address: Optimyth Software Technologies, S.L.U. en C/. Norias 80 - 2º, 28221, Majadahonda (Madrid), or send an e-mail to the following e-mail address: [email protected]. Users and their applications data security is ensured by software and hardware. The database server is in a network inaccessible from internet and software (only external input) is prepared to protect each customer data using passwords coding techniques (for each password, a different encoding) and through security tests (internal and external) on software, verifying there are not vulnerabilities to be exploited.

Access to Kiuwan is made by https secure protocol. If the user chooses to upload the code to Kiuwan server for its analysis, this will be deleted once the analysis is complete.

Kiuwan is designed from the beginning as a SaaS applications and, therefore, it is “multitenant”, which prevents anyone other than the application and its data owner can access data that is not theirs (malicious users, hackers, backdoors, system errors, information in logs, etc.).

(From Kiuwan Privacy Policy, https://www.kiuwan.com/security-policy) Backups of applications analysis data are held daily and kept for a minimum of seven days.

Users data persistence Once a user or an account is de-registered, in other words the user chooses to delete the data, none of it will be preserved. Analysis data persistence

Kiuwan follows this applications analysis data persistence policy:

 Last two analyses - all data is stored, including global indicators, metrics and defects per file. Regardless of their age.

 Old analyses less than three months - for application with more than two analyses that are less than three months old (besides the last two), fine grained metrics, such as file or function metrics, are deleted. Kiuwan keeps defects, metrics and indicators at application and language level to assure no functionality is lost for these analyses.  Ancient analyses - for application with more than two analyses at are older than three months (besides the last two), defects and more technical metrics are removed. Indicators and the most important metrics at application level are kept. Defects and simulator views won’t be available for these analyses.  Maximum number of analyses – Kiuwan stores 100 analyses per application. These analyses rotate to always have the last 100. This means that before storing the 101st analysis, the 1st one is removed.  For free accounts, only the last three analyses are stored, and the above policies still apply.

Kiuwan provides a default quality model To measure and provide relevant code quality and security indicators, you need a Quality Model (set of rules, metrics and indicators about the code). Selecting and customizing these sets to analyze and measure your code may be tedious and delicate for your organization at the same time. You are ultimately responsible for deciding the quality and security levels your applications should have. This Quality Model has to be easily understood and applied by all the actors in the evaluation and code certification processes, so it must be based on industry standards, such as ISO/IEC 25000 or ISO/IEC 9126. Therefore, the Quality Model you choose is a cornerstone in the SQA process. Kiuwan provides a default multi-technology Quality Model (CQM), which allows you to start analyzing your code from day one. …but you can customize it Kiuwan provides a powerful functionality that allows you to customize even the smallest detail of CQM, such as: add new rules or metrics from Kiuwan “library”, modify any of its parameters (priority, classification, parameters, sample code…). Or create your own model You can also create your own model based on CQM or from scratch, selecting the technologies, rules, metrics you need. I already have a Quality Model If you already use an open source tool in which you have invested time setting up a quality model with it (Checkstyle, Findbugs or PMD), you will appreciate not having to repeat the work. It can also been designed and implemented on Kiuwan. You just have to provide a few details about where and how you have configured these tools. Kiuwan incorporates the models (rulesets) of those tools, which thereafter can be treated as any other Kiuwan Library rule.

Figure 40.-. Software indicators Kiuwan calculates the indicators based on evidence (defects and metrics) taken from source code analyses. You can see the trend of all the indicators with colored arrows. Based on the values of previous analyses, it gives an immediate representation of the effectiveness of your efforts to improve your software quality. What is a defect for Kiuwan? A defect is a violation of a rule defined in the quality model for a specific language. Each rule in the model is classified into one of the five software characteristics defined in CQM. If there are not rule violations in one category, it means there are no defects. Therefore, a 100 value in the corresponding software characteristic indicator is the best score you can get. When we find a rule violation in a category, the indicator of that software characteristic reduces. How much? It will depend on the number of violations calculated for that rule in all the source code and the importance that rule has in CQM. The higher the number of violations and the importance of the rule, the bigger the decrease of the software characteristic indicator will be. Kiuwan provides indicators for:  Software characteristics defined in CQM: Efficiency, maintainability, reliability, security and portability.  Quality global indicators. Calculated as a weighted average of the software characteristics above. The weight for each characteristic can be customized in CQM. For example, there will be organizationts that want to give more relevante to maintainability than to reliability.

 Figure 41.-.Global Quality Indicator  Risk index. Associated with the structural quality of the software.  Effort to Target. The amount of work required to achieve the quality objective.

Figure 42.-. Risk Index Size does matter, at least when talking about software quality management. Kiuwan Risk Index takes into account the relative size of your application, which gives a closer estimation of the risk it faces in your developments. There are two main variables that produce development risk: quality and effort of repairing the issues (technical debt, if you will). The effort is normalized to take into account the application size. Thus, small efforts in small applications may yield higher risk, and big efforts in big applications may involve lower risk.

Figure 43.-. ISO 25000 Quality Model Without a quality model, software cannot be measured nor analyzed. Thus, Kiuwan provides a default quality model. It is calles CQM and is based on standard guidelines to evaluate software products, such as ISO/IEC 25000 "Product quality software requirements". Why CQM? In Optimyh, we have wide experience in introducing quality plans for all kind of organizations. We realized that organizations have common problems and interests. There was a need to define a quality model and an associated methodology, with the support of tools, products and solutions. Therefore, the adoption of CQM will provide destination benefits and will solve the most common problems. Using CQM saves time and money in quality assurance process. Short-term benefits obtained with the use of CQM are:  It abstracts from technical layer. Its quality information will be independent of programming languages and platforms.  Compares different versions of the same software. This responds to the most important question: has my software improved?  Compares the quality of different applications. It doesn’t matter if they are different type of applications or if they are developed in different technologies.  Evaluates technical requirements in order to accept software from an external supplier.

But there is more:  Aggregation of data. You can add the quality information of the different applications in order to obtain an evaluation of the software produced by a supplier, a country or an IT area in comparison with others.  A process of continuous improvement. You can apply a quality control methodology in your software life cycle process.

Figure 44.-. Quality Model customization Kiuwan offers to users with CQM a valid quality model. But we understand that just one model does not fit all type of applications that can be analyzed. Therefore, with Kiuwan you can create as many quality models as you need. You can use CQM as a template or start from scratch. What can be configured?  Model flexibility. How strict are you when calculating the quality indicators? As when in school, where achieving an A on a test depends on how strict the teacher is when correcting. There were some teachers with whom getting an A was much easier than with others. With that parameter, we can choose how strict we want to be when correcting.  The importance of software characteristics indicators. You can establish the weight for each software characteristics and how you want it to contribute to the Global Indicator. For example, you may want to give more importance to applications maintainability than to portability.  The importance of technologies. When applications are multi-technology, you may want to establish the weight of each technology and how they contribute to quality indicators. For example, you may want to give more importance to Java code quality than to JSP’s.

 The importance of the priority of the rules. You can set how important the priorities of rule violations are, meaning that you can decide how bad it is to violate a high priority rule compared to a medium priority rule. All this is reflected in the quality indicators.  Repairing difficulty. Referring to the time a developer has to spend to repair a defect of different priorities. It takes more time to repair a high priority violation than a low priority one.  Metrics. Metrics are what Kiuwan calculates when analyzing the code with its quality model. They are classified by language, priority and characteristics and you can study their evolution.  Rules. You can enable or disable rules, change their priority and even change the software characteristic they measure. Kiuwan has an extensive libraty of around 4,000 rules for you to choose. All this enriched with a timeline (baseline) to track all changes you make in your quality model.

Figure 45.-. Quality Model customization II

Figure 46.-. Importing third-party rules In the case you use any rules through a static analysis open source tools, such as PMD, Checkstyle or FindBugs, can be imported to Kiuwan custom rules library. They can be added to your custom quality models, classify them by software characteristic affected (efficiency, maintainability, reliability, portability or security) and assign them a priority and a repairing difficulty, in order to know the effort needed to correct the defects found based on these rules. But that is not all. You can take full advantage of all of Kiuwan characteristics for the defects provided for violations of these rules, such as the action plan obtained from a ‘What-if’ analysis, the possibility of muting defects that come from those tools for, for example, managing false positives, generating complete reports… Just a little warning: to take advantage of this capability, you need to run the analysis using Kiuwan Local Analyzer, since it incorporates those third parties’ engines. You just have to add the jar files with those third parties’ rules implementations in Local Analyzer’s lib. custom directory, so the rules can be executed when needed, and the results incorporated to Kiuwan own analyzers. Third party tools versions supported and incorporated to the local analyzer are:

 PMD 4.2.5.  Checkstyle 4.4.  Findbugs: 1.3.9.

The functionality to compare Kiuwan quality models helps to find changes between different versions of a quality model or between different quality models. These changes can be related to the quality model general configuration or the configuration of rules and metrics. By default, the loaded quality model will be compared to its previous version if available, but you can choose any previous version or any version of the models available in the account.

Figure 47.-. Quality Models comparison

Kiuwan can obtain a PDF report of your quality model and another one with the differences between the models you want to compare.

…

Figure 48.-. Quality Models report

Kiuwan is the only cloud solution that allows to:  Make decisions about your applications portfolio.  Compare your applications, see their evolution.  Explore the best scenario to meet your needs and resources.  Help to decide if an application should be stored, optimized or discarded. Dashboards for each role Kiuwan analyzers have a high level of accuracy detecting defects in the code. Kiuwan as well stands out for its capabilities to present all information (in dashboards and reports) assisting in making decisions based on our applications quality and security. Kiuwan’s views, diagrams and reports show the relevant information for your applications management, such as: Knowing the risk, the technical debt, the repair effort or other indicators at executive level of your applications portfolio. Depending on the role you have in your organization, Kiuwan provides this information with different views - one simple application, group (Portfolio) or your whole portfolio (Global) levels. Decision quadrants Kiuwan provides executive information at decision quadrants, at portfolio, portfolio groups and global (all the applications in one account) levels. Each diagram shows the information in a chart with several quadrants depending on their exposure to risk for development and quality. This way, you can decide how to improve the health of your portfolios or applications. The aim should be to move, over time, all the applications and portfolios to the optimal quadrant, which means that their exposure to risk is low and their quality is high. This is a good way to give priority to your development efforts.

Rankings You can also get a table with the classification of the portfolios or all the applications (ranking) at different levels: portfolios or global. So you can see what applications or portfolios are the best in terms of any of Kiuwan indicators.

Figure 49.-. Kiuwan notifications As an application in the cloud user, you will want to know the latest news of the platform you use every day. With the red notifications bell on the top of the screen, you can learn more about:  The new capabilities added to the platform.  New and recommended tutorials and other useful resources.  New supported languages.  Interesting blog posts.  Windows reserved for platform maintenance.  Other announcements of interest.

This is another feature that distinguishes Kiuwan from the rest. To create applications portfolios, first you have to decide the criteria by which you want to group your applications. For example, the software development team or the factory that has developed the application, the application business value –this actually is a group of portfolios that Kiuwan has by default-; everything that is relevant for you or your organization. Once you have this decided, you are able to create a portfolios group with the possible values of the specific criteria it can take (portfolio). For example, for business value, the possible values could be: critical, high, medium, low and very low. So, when you create an application –or any other time-, you can assign it to an existing portfolio, establishing the value it will have for that application. For example your on-line banking application could belong to your business-critical applications portfolio.

Why is it so great to be able to group the applications into portfolios? You can manage the health of your applications at this level, which means that we calculate all Kiuwan indicators for the portfolios and the portfolios groups, based on the applications data. Kiuwan provides a portfolio view (perspective), where you can track the indicators for the portfolios groups and individual portfolios.

Figure 50.-. Kiuwan portfolios For the portfolios groups, you can see the contribution of each portfolio to each indicator and to each portfolio as well. And for each portfolio, you can see the contribution of the individual applications to the portfolio indicators.

Figure 51.-. Decision quadrants at portfolios level

Kiuwan provides a global perspective to display and manage the health of your organization development. In this view, you can see the global indicators for your entire organization, based on the results of your applications last analyses. At a glance, you have the overall score of the health of your organization, based on the quality and the exposure to risk of your development efforts.

Figure 52.-. Global View

Decision quadrants in this view aim to identify those applications in your portfolio that require immediate action based on their criticality to the business and their exposure to any of the risks they face, such as: Global Risk (risk index), the failure probability (Production Risk), Maintenance (Risk for development) or security risks.

Figure 53.-. Decision quadrants at global level For this view, Kiuwan also provides a classification of applications (Ranking). This new classification will allow you to see all your applications in a different way. It sorts your applications by any of your Kiuwan indicators: Risk Index, Quality Indicators, Effort to Target or other metrics like size. Just open the applications details in the table and compare them one by one.

Figure 54.-. Global view applications ranking

Figure 55.-. Historic evolution When it comes to manage software quality, it would be desirable to track the evolution of all software quality indicators over time. Kiuwan does not throw any data. In fact, each time you analyze your applications with Kiuwan, we store all your data and it is displayed within the summary dashboards, in a time chart. You can see at a glance the trend of your application quality, but not just for a single application: Kiuwan provides the same functionality for the applications portfolios and for the global view, which summarizes the quality of all your applications with the same set of indicators.

Figure 56.-. Trends The colored arrows to the right of each indicator and metric tells you immediately if these have improved or not since the last analysis. Hovering the mouse over the arrow, the previous value and the improvement percentage are displayed. It seems very simple, but this gives a lot of information for you to take action and continuously measure and improve your applications quality. Each time you see a red arrow, it is a good time to run a what-if analysis and obtain a detailed action plan to improve your quality.

Quality and security continuous management Kiuwan is designed to make Continuous Code Inspection. Therefore, there are integration capabilities available in your Development Process (ALM or SDLC), which allow you to know the status of your applications’ quality and security. Agile, DevOps Kiuwan (and code analysis practice) is “neutral” from a methodological point of view, but its architecture and integration facilities enable it to fit “down pat” with agile methodologies and devOps. Kiuwan is continuously analyzed and deployed Considering that we do believe it, Kiuwan develops following these practices and with help from Kiuwan itself and some of the technological partners it integrates with. We have improved (and keep improving) our productivity, efficiency, quality and security levels. Kiuwan integrates with…  JIRA  Cloudbees  Jenkins  Urban Code  RTC  …and more coming. Or make your own integration Kiuwan provides RESTfull API, so you can invoke its functionalities and use them from your own environment.

Figure 57.-. Integrations with third parties available on Kiuwan Once you have the capability of measuring and analyzing your code locally, why not integrate the quality control with your continuous integration process? We make it simple for you, follow the instruction in the documentation to make your orchestrator invoke the Kiuwan analyzers. The results are automatically uploaded to your kiuwan account, so you are one click away to see the evolution of your applications quality for every deployment you do.

Figure 58.-. Integration with Jenkins Continuous code analysis. That is our mantra: measure the quality of your software and measure it as often as you can. If you have a continuous integration and deployment process driven by Jenkins, we have very good news for you: now you can easily integrate continuous code analysis into your process with Kiuwan's new Jenkins plugin. The plugin is open source, so the community can share it and improve it. You can download it from our GitHub repository or from the Jenkins CI repository.

If you are one of those who care about software quality and use Jenkins to steer your development process, just register for a free Kiuwan account. Bear in mind that to use the plugin from your Jenkins, you will need the Kiuwan local analyzer installed on the same machine. If you want to learn how easy it is to install the Kiuwan local analyzer, check out the tutorial. Now, download the plugin, configure it in your Jenkins installation and you are ready to go! You will be able to run a Kiuwan static analysis after your build and have a link to the results in the Jenkins build execution page.

Figure 59.-. RESTFull API There is a whole world out there that may need the information Kiuwan can provide. Just connect... Kiuwan has an exciting new feature: a RESTfull API that will allow anyone to pull out data from Kiuwan to use whenever is needed, wherever is needed, for example:  List of applications  Last application analysis results, including Kiuwan indicators with details and main metrics.  Applications defects.  Application files with defects and metrics.  Defects deltas for any given application’s analysis.  Etc. And this list continues growing.

Figure 60.-. Integration with JIRA Picture this: you decide to have two developers working on improving the maintainability of your application for a couple of weeks. Where do they start? You know you have hundreds of maintainability defects, but which are the ones that affect your application the most? Which are the ones that the developers can handle in those two weeks of work? That is exactly what the 'What If' analysis feature can do for you! Just say you want to use 160 hours of development time to fix maintainability defects and click the action plan. Kiuwan will give you the list of defects to work on. Now, wouldn't it be cool to have those defects as open issues in JIRA, so you and the developers can track them? Here you have it! Just click on the Atlassian Jira export icon and off you go. We can connect to your own JIRA instance or to your JIRA On-Demand instance, it doesn't matter. Select your JIRA project, decide the issue type from the ones you have defined in JIRA, assign the priority and create the issues. You can create issues per rule or per defect.

Figure 61.-. Integration with IBM Urban Code For those who have IBM UrbanCode Deploy as their preferred deployment automation platform we have great news. Kiuwan now is fully integrated with our IBM UrbanCode Deploy Kiuwan plug-in. With this new plug-in, you can measure your application's quality and security continuously as part of your deployment process. Minimize risk, make sure your deployed application have minimum levels of quality and security. Kiuwan will even help you identify the weak points in your application to help you improve. It is as easy as downloading the plug-in, install it in your IBM UrbanCode Deploy server/agent machine and you are set! Include a new step in your deployment process that will automatically take care of analyzing your software and upload the results to the Kiuwan platform for your convenience.

My application has plenty of different defects:  Where should I start to fix them?  How much time does it take me to repair each one of them?  Which are the optimal path and action plan to reach my quality goals?  I only have 20 hours to fix errors before the next delivery. What should I fix to aim the best possible quality?  Can I automatically transfer the errors-to-repair list to Development team?  Do I have “out-of-the-box” reports at executive level? Let Kiuwan help you to answer these and other questions that may arise…

Figure 62.-. “What-If” analysis This is probably one of the most interesting features of Kiuwan.

Wouldn't it be nice to know the effort it will take to reach the quality level you want? Or the other way around, what would be the quality of your application if you decide to invest a given effort to improve it? This is exactly what this Kiuwan functionality offers. Just decide what you want to simulate:  Effort. You can estimate the resulting quality indicator and risk index, if you invest the specified number of hours in one or more software quality characteristics.  Quality. You can estimate the effort you need to invest to reach the quality indicators you specify for each software quality characteristics. In one single view, Kiuwan will show you the current situation, the simulation parameters and the estimated results That's not all! Kiuwan generates an action plan for you to implement the simulation.

If you know the quality level you want to achieve, Kiuwan helps you to know how much it will take to achieve it, the shortest path to do it, maintain those levels over time and improve them continuously! Or if you want to increase your software quality indicators or you want to invest a little effort to improve one or more software quality characteristics, you would want to know where to start, wouldn’t you?

Figure 63.-. Action plan Kiuwan makes it really easy. Out from the defects lists, Kiuwan creates an action plan with defects you have to fix to reach your quality or effort goals. We encourage you to follow the order suggested by Kiuwan, if you want to see the improvement faster.

Your developers will be happy to have a bone to chew on and your QA department will know what to expect in the next release of your application. And remember that with this action plan you can automatically generate tickets in JIRA to fix the defects.

Figure 64.-. Remediation progress Kuiwan takes the usefulness of its action plans one step further through this feature. You can save an action plan and control its remediation progress. With this feature you can compare the latest analysis with any of the saved action plans. Kiuwan will give you the percentage of progress in the action plan based on removed defects and will give you full detailed information of what you have left.

Los desarrolladores, responsables de QA, gerentes de desarrollo, los ingenieros de QA, e incluso los CIOs necesitan información sobre la salud de sus esfuerzos de desarrollo de su organización en distintos niveles. Vd. tiene toda la información que necesita para todos ellos en Kiuwan, pero a veces es bueno tener informes para poder mostrarla y distribuirla.

Figure 65.-. Kiuwan reports Kiuwan provides the following set of reports for every role in your organization:  Application overview. A high level application's health overview in PDF format. It includes an introduction to Kiuwan's measurement methodology, a summary with the application analysis information, the Kiuwan indicators —risk index, quality indicator and reparation effort—, main code metrics values and top 10 repair first defects.  Defects list. Developers need all the details of the defects found. This excel format report lists all the violations to the rules with the file they were found, the violated rule and even the affected line in the code.  Metrics values lists. More details for the developers. This excel format report shows all the metrics values for each analyzed file.  Action plan. This is the final —and most useful— result of the 'what if' analysis feature. With this PDF format report you will get a summary of the what if analysis with the final quality values and efforts needed to execute the plan, and the detailed list of the defects to repair to reach the goal.  Organization overview. This report includes the same information as the application overview report, but referred to the whole organization based in the latest analysis of all the applications. The Kiuwan indicators here are an aggregation of the individual application's indicators.

 Portfolios overview. This is a PDF format summary of all the portfolios defined in your account with details of the applications that each portfolio includes and the criteria used to create them. Of course, for each portfolio you will have the Kiuwan indicators and main metrics values. o Quality Model. Quality Model Manager brings the following reports: o A PDF executive overview of a specific quality model, with all the high level detail you need to understand its scope. o An Excel spreadsheet with the detail all the rules and metrics documentation of a specific quality model.

In classical software development there are three common variables: cost, duration and scope. As they are getting set and times are urging, the fourth variable suffers: quality. Nowadays, with agile development, it is intended, in a quick way, to offer a product that provides value to the customer without suffering deterioration in quality. In one of the Lean principles underlying agile methodologies, we read: “Constant search for perfection”. According to Capers Jones, software projects tend to entropy or, in other words, to increase their complexity if action is not taken. Now, it seems necessary to control the quality for each iteration; check that defects, complexity and technical debt do not increase after each sprint; ultimately, deliver a better product increasingly well built. Kiuwan provided the possibility of being executed in continuous integration systems, streamlining quality measurement, valuable information for the actors involved in the sprint.

In many development companies we have the following dialogue, on a daily basis:

Developer: “Test this version and let me know if you see any failure". Testing: “OK. Including version download, deployment, running tests and analyzing the results, it takes three days". Developer: “Three days? But Business is asking for this yesterday! So you speak with them... They always complain how slow we are".

This conversation shows reality: development and business processes need greater fluidity, better communication... in short: it needs to be more agile. And this, nothing but this, is what DevOps proposes. In a fast paces world, where the market demands requests continuously (oh my!, the famous and overwhelming Time to Market). In short, a world where whoever comes second has everything to lose, then DevOps becomes a fundamental concept. Any business idea, requirement demanded by the client or defect (which always appear; yes, always) have to implement it or correct it within an agile process. And how do we get to be agile? One of the key points is "automatism".

Cambridge Dictionary definition for IT is: the study and use of electronic systems and computers for storing, analyzing and utilizing information. So what can I do to automatically recover the version the development team tells me to, run quality testst, perform deployments, etc.? Look for tools or mechanisms that undertake this work and continuously adopt this kind of solutions to speed the development and production environments. In the specific case of quality analysis, it is important that developers have tools already in their workstation (their IDE) to analyze code quality so they know in advance the quality of what they deliver. No less important is to have a continuous integration system that enables to download the correct version of the application from the code repository, to analyze the code, perform deployments, report results, etc. Kiuwan connects with continuous integration systems like Jenkins, so these automatisms we are talking about are possible. From Jenkins, it is possible to invoke Kiuwan, indicating the quality levels you want to achieve:

Figure 66.-. Kiuwan integration with Jenkins

Quality analyses are run automatically and it is possible to inform the user as to whether the level of quality is what is expected or not.

Figure 67.-. Integration of Kiuwan with Jenkins II With processes like the one above we can now change the conversation to: Developer: “Test this version and let me know if you see any failures". Testing: “OK. Tell me what version it is and I'll run the analyses. You will receive the results by e-mail". Developer: “Thank you".

The purpose of a software quality office is, in essence, to verify that the processes are being performed properly and the products are being delivered with acceptable quality. All possible improvements will be identified so quality will continuously improve in subsequent phases or deliveries. One of the delivered products is, obviously, the code. Therefore, the goal of a quality office is to review the code through some type of automatism (how about Kiuwan?). The fact of identifying defects in early stages of development will yield significant savings, since it is much cheaper to repair defects in development than in production.

Figure 68.-. Defects costs depending on SDLC phases

Kiuwan allows to make these reviews automatically, and what is more essential is that developers receive the feedback so that they can proceed to correct deficiencies before the cost increases. If the development team is located in the same facilities, an interesting option is to notify, via an incidences management system, the identified gaps in the code. If not, a report in Excel or PDF with the main violations could serve as an entry point for improving code quality in external devices. The more frequent and more automated is this circuit, the better it is.

Figure 69.-. Quality Office Report sample

Quality office, through these reviews, will identify improvements that will help to reduce the number of failures and incidences in the analyzed application. Thus, the application will reach its maintenance in a “healthy” status, whereas if there has been no verification, the “pathological application” syndrome will be suffered.

“Honey, don’t wait for me for dinner tonight, we have an issue in production and I’m going to stick around until we find and implement a solution”. How many times have you written a similar message to your wife, husband or partner when your systems start giving preoccupying symptoms of something serious, and you don’t know what’s going to happen. Moreover, once the symptoms are isolated or even mitigated it is time to find out what went wrong, the root cause of the problem. Now, you find out that particular application was developed by an external software factory under contract and they delivered a new release recently. After a thorough investigation you find out that there was a bug in the latest release that caused the problem. Given this situation, your partner candidly asks you: didn’t they run quality controls on the delivered release? Who is responsible for software quality control: the contractor factory or you? Where do you draw the line for each other responsibilities? Who rises an alarm when the quality indicators are in red? Very clever your partner…not so clever the situation.

First of all, it is important to point out that any software provider should have the mechanisms to assure the quality of all the software modules it develops. It would, not only allow them to deliver quality software to just one picky customer, they can leverage this for all their customers. In fact, quality assurance could become a continuous horizontal task for all the projects developed by the factory. Up front, it does not look beneficial for the software factory, since it implies to include new tasks –and cost– to their production workflow. However, what if they have to maintain the delivered software? Wouldn’t it be better to deliver the software with accepted quality levels? Wouldn’t it be better for the maintenance engineers to face readable, maintainable, bug free code? All these add up to the impact on the corporate image of a software factory delivering buggy applications… Ok, sounds good. A software factory should deliver quality software, but what about the contracting organization? They have the responsibility to certify the quality of the software they receive. Why? They are going to service their customers with this software. They can offer internal services –like an employee intranet– or an external service –like a web Application to offer and purchase their products–. As you can see it should be mandatory for any organization to control and assure the quality of all the software they get or develop. You can apply these checkpoints vertically to all the software factories you have under contract. The more you can automate this process the bigger the cost savings will be. How could Kiuwan intervene in this process? First of all, the contractor could provide in the RFP the quality model which will verify the quality on deliveries, as well as the criteria they will use. For example, quality levels above 80%, no serious violations, duplicated code below 5%, etc. Kiuwan can provide documentation along with the ruleset that makes up the quality model. Thus, the supplier or the outsourced development team could know the “test questions”. The contractor will be able to “examine” each delivery and verify with Kiuwan if the required quality criteria are being met. The provider finally could continuously analyze the development quality with their customer’s rulset and be aware, a priori, of “how is” what they will deliver. We could say, the line separating the responsibility of software quality is horizontal in the software factories and vertical in the organizations receiving the software. If you allow me the simile, these two lines build the software quality Red Cross that will monitor and ensure the health of your software. “Honey, today I will arrive on time”.

In late 2013, AENOR, ISO representative to Spain, added software product quality to its list of certifications, in its maintainability characteristic, based on IEC/ISO 25000 standard. For the organization concerned, this certification provides assurance that their software has been evaluated independently by a recognized entity as AENOR, obtaining, at the end of the process, a rating (from 0 to 5) and a report of recommendations and possible improvements for the adequacy of your system to the indicators and metrics that allow to see if that system meets the different characteristics and subcharacteristics of the standard. Software product certification and evaluation involve the implementation of an ecosystem in which, for automating the collection of metrics that can achieve the indicators that make up the standard, tools are needed. Kiuwan has been the first one to adapt and align their metrics to AENOR assessment model, as driving schools prepare their students to undertake their driving license, companies that analyze their applications with Kiuwan, seeing these metrics, can anticipate the grade obtained in the evaluation by AENOR.

Figure 70.-. ISO 25000 certification ecosystem The benefits of this scenario for organizations go even further. Applications evolve and presumably these organizations want to know how to evolve their quality indicators, and they may not be every bit time re-certifying their applications. Having a solution like Kiuwan can give them peace of mind that metrics equivalent to AENOR’s are maintained or, what is

desirable, they improve over time and whenappropriate (e.g. see that the rating could have evolved from "3" to "4" or from "4" to "5"), repeating AENOR assessment process.

The future of cloud computing, the need to combine reduction of software delivery times –for reasons of business- and the quality of those deliveries, the increasing adoption of agile methodologies or better collaboration and development teams management, as seen in this chapter, makes that solutions like Kiuwan can provide new and interesting capabilities to development teams, QA managers, project managers or CIOs. Capers Jones, in his interesting book The Technical and Social History of Software Engineering [6], makes it clear how software engineering will evolve in the coming years, from his own point of view. He states that static analysis is one of the more efficient practices for DRE (defects removal efficiency) reduction and for him, one of the essential metrics for software quality improvement. Turn this practice into something easy to implement, usable and highly integrated with Saas platforms, as well as with development processes, tests or quality, it is something that tools like Kiuwan are helping to achieve.

[1] Software Quality in 2014: A survey of the state of the art. Capers Jones. [2] http://mashable.com/2014/08/31/how-safe-is-icloud. [3] Chidamber, S.R.; Kemerer, C.F. IEEE Transactions on Software Engineering Volume 20, Issue 6, Jun 1994 Page(s):476 – 493. [4] Inversion of Control Containers and the Dependency Injection Pattern, by Martin Fowler. [5] Fernández,C.M.; M.Piattini, Modelo para el gobierno de las TIC basado en las normas ISO, Madrid: ed. AENOR, 2012. [6] The Technical and Social History of Software Engineering, Capers Jones. Addison- Wesley.

Checking Quality Model for Software (CQM) is a model to evaluate the internal quality of a software product and it is designed by Optimyth.

CQM is based on ISO-25000 standard, which defines the scope and quality characteristics. ISO-25000 defines three validation aspects:  Internal quality: sef of characteristics from an internal point of view.  External quality: characteristics obtained by putting the software into production.  Quality in use: software perspective from the user point of view.

CQM methodology extracts data from the code and its configuration following a pyramidal structure:

Indicador CQM Indicador de las características del software. Indicador de las dependencias entre tecnologías.

Métricas del código fuente.

An indicator can be defined as something that helps us to understand where we are, where we are going and how far we are from our goal. Therefore, it may be a number, a chart, etc. Indicators are information of system characteristics represented in summary form. CQM indicators are normalized to be represented within three regions: 0-30, 30-70 and 70- 100.

This normalization allows us to make comparisons between different characteristics, such as the ones shown in the following graphs:

Indicators can be added in average. This allows us to generate indicators of applications groups. For example, we can compare the software produced in India to the one produced in USA, or compare the software quality of the software provided by different suppliers.