Course 2: «Open Source Software (OSS) Engineering Data»

Course 2: «Open Source Software (OSS) Engineering Data».

1st Day: Metrics and Tools for Software Engineering in Open Source Software

1. Open Software / Hardware Technologies: Introduction to Open Source Software and related technologies. 2. Software Engineering FLOSS: Free Libre Open Source Software in Software Engineering. 3. Metrics for Open Source Software: product and project metrics and related tooling.

2nd Day: Research based on Open Source Software Data

4. Facilitating Metric for White-Box Reuse: we will discuss a metric derived from the analysis of Open Source Project which facilitates the white-box reuse (reuse based on the source code analysis). 5. Extracting components from Open-Source: The Component Adaptation Environment Approach (COPE): we will discuss the results of the OPEN-SME EU Research Project and in particular the COPE tool for extracting reusable components from Open Source Software. 6. Software Engineering Research based on Open Source Software Data: Data, A recent example: we will discuss a recent study of open source repository mailing lists for deriving implications for the evolution of an open source project. 7. Improving Component Coupling Information with Dynamic Profiling: we will discuss how dynamic profiling of an open source program can contribute towards its comprehension.

In various points during the lectures students will be asked to carry out activities. Open Software / Hardware Technologies

Ioannis Stamelos, Professor Nikolaos Konofaos, Associate Professor School of Informatics Aristotle University of Thessaloniki

George Kakarontzas, Assistant Professor University of Thessaly

2018-2019 1 F/OSS - FLOSS Definition

● The traditional SW development model asks for a “closed member” team that develops proprietary source code. ● FLOSS is an alternative model for developing, distributing and using software ● A FLOSS software provides three basic “freedoms” to its user:  Use the software at his own will (as he wishes)  Copy it and distribute copies as many times as he wants  Change it and distribute the changes as he wants ● The above freedoms need the source code to be openly available 2018-2019 2 Free Software

● Free Software Foundation, under the leadership of Richard Stallman (http://www.fsf.org) ● Free software license cover the three freedoms mentioned above

● Free has to do with the freedom of using the source code ● “Free as in freedom, not as free beer”

2018-2019 3 Open Source Software

● Open Source Software is similar to Free Software (http://www.opensource.org) ● License addresses more issues:  Single programmer’s code integrity  Distribution of new developments  Obligation to avoid bans over specific populations  ... For more, read on the link above...

2018-2019 4 2018-2019 5 Development Process (1)

● Different than traditional software development approaches (e.g. Waterfall model)

● Considered as similar to a kind of “extreme spiral model”, extreme programming or other agile methods ● None of the above can accurately represent the F/OSS development process: they neglect the fact that the code is open to everyone

2018-2019 6 Development Process (2)

● In a typical scenario, an individual has a problem as a computer user that he wants to solve (has a “personal itch”): – Wants a brand new app – Wants a non-commercial app – Existing commercial or F/OSS apps are not satisfactory

● He produces an initial version and opens the code with an appropriate license

2018-2019 7 Development Process (3)

● In case there is interest around the project, people start using it, reporting issues and sending new pieces of code (commits)

● The coordinator (or the coordination team), typically the original programmer(s) integrate the add-ons and produce new releases, openly available on the Internet

● This cyclic procedure repeats as long as the project is alive, i.e. there are interested users and developers that support it

2018-2019 8 Development Process (4)

● The most significant advantage is the availability of a multitude of volunteers who develop and inspect continuously the source code (“peer review”)

● Because of the numbers of users/developers finding/fixing bugs and releasing the new code is fast, much faster than with closed code and closed teams. This fact is epitomized with “given enough eyeballs, all bugs are shallow” (Raymond)

● This type of collaborative and open development has produced dedicated software tools that have been adopted by traditional teams as well (CVS/SVN/GIT, Bugzilla etc)

2018-2019 9 F/OSS today

● Several modern F/OSS projects (e.g. ODOO, Alfresco, SugarCRM) are hybrid, meaning in general that: – The project is initiated by a company or the community has evolved into a company – There is an open sourced core (community) version, and a commercial (enterprise) edition – There are paid services over the code (installation, training, parameterization, adaptation, training, support, etc)

2018-2019 10 Known F/OSS projects (1)

● The flagship OS: The Linux Kernel, also Android (~80% open sourced). MacOS is open since some months ● Web servers: Apache Web Server, ~50% globally, also nginx ~20% ● PC Software: Thunderbird, Webmail – Firefox, Chromium – LibreOffice/OpenOffice - VLC ● RDBMS: MySQL, PostgreSQL. Surprisingly, also SAPDB

2018-2019 11 Known F/OSS projects (2)

● Programming: Perl, Python, PHP, Java (since some years) ● The known LAMP of software development: – Linux – Apache http Server – MySQL – Perl/Python/ PhP

● ...

2018-2019 12 Where F/OSS resides

● On self managed web sites (e.g. Apache.org)

● Collectively, in forges:  SourceForge  Github  RubyForge, Tigris.org, BountySource, Launchpad, BerliOS, JavaForge, GNU Savannah, Gitorious...

2018-2019 13 Some current statistics on GitHub

● https://octoverse.github.com/

2018-2019 14 People Location

2018-2019 15 Repositories

2018-2019 16 Advantages

● Interoperability

● Ease of extension

● Transparency

● Security (!?), Reliability

● Cost savings (zero acquisition cost, free updates, reduced vendor lock-in risk)

2018-2019 17 Drawbacks - Weaknesses

● Development continuum is not guaranteed (however the same issue appears in closed source as well)

● Possibility of forking

● Hard to assess the maturity of a specific FLOSS project

● Lack of usability (in recent years this issue has been addressed in many cases)

● Possibility of lack of support

2018-2019 18 Open Source Issue Survey: GitHub

Incomplete or confusing documentation: ~90% Unresponsiveness: ~80% Dismissive responses: ~55% Conflict: ~45% Unexplained rejection: ~30% Unwelcoming language or content: ~15%

2018-2019 19 Open Source criticism

● "Ubuntu Spyware: What to do?", από τον Richard Stallman:

https://www.fsf.org/blogs/rms/ubuntu-spyware-what-to-do

● Lennart Poettering statement about open source communities:

https://plus.google.com/+LennartPoetteringTheOneAndOnly/posts/J2TZrTvu7vd

● Criticism on "given enough eyeballs, all bugs are shallow", from Bob Glass:

http://books.google.gr/books?id=3Ntz-UJzZN0C&pg=PA174&redir_esc=y#v=onepage&q&f=fals e

2018-2019 20 Technical Infrastructure: web site

● Open source projects should have two different websites: – A user website – A developers’ website

● These two websites contain different types of information and serve different purposes.

● There should be a way to reach the developers’ website from the user oriented website.

● An example are the LibreOffice users’ website (top) and developers’ website (bottom).

2018-2019 21 Technical Infrastructure: “Canned” Hosting

● A “canned” hosting site offers most of the necessary collaboration tools for an open source project. For example it could offer some or all of the following: – Public version control repositories – Bug tracking – Wiki space – Mailing list hosting – Continuous integration testing and other services.

● For many projects, canned hosting is an adequate and perfectly acceptable solution.

● The most popular such service today is GitHub (https://github.com/)

2018-2019 22 Technical Infrastructure: Mailing Lists/Forums

● Mailing lists or forums refer to message-based communication platforms where posts are organized in threads (i.e. topics).

● Users can subscribe and create posts and get notified when answers become available.

● Usually there is web-based access to old messages and searching facilities.

● Some things to consider when choosing a mailing list/forum solution: – It should provide mail and web-based access – It should provide spam checking and moderation capabilities – It should provide archiving and searching

2018-2019 23 Technical infrastructure: forum software

● Discourse — https://discourse.org/ – You can install it yourself. Provides both mail and web interfaces. Commercial support is available but it is open source software.

● Google Groups — https://groups.google.com/ – Not an open source service. Provides searchable archives, moderation and spam-prevention as well as both mail and web access.

● And many others: – GroupServer — http://groupserver.org/ – Sympa — https://www.sympa.org/ – Mailman — http://www.list.org/

2018-2019 24 Technical infrastructure: version control

● Version control system is a system which allows of tracking and controlling changes on project’s files. Particularly the project’s source code but also other files (e.g. web pages).

● Currently the most used version control system is Git (usually hosted at GitHub): https://git-scm.com/

● Other options include the following: – Mercurial: https://www.mercurial-scm.org/ – Subversion: https://subversion.apache.org/

2018-2019 25 Technical infrastructure: bug tracker

● A more generic platform than the name implies, because it tracks: – Bugs – Feature requests – One time tasks – Anything with start time and end time in the project

● Classic lifecycle of a bug: – Filing (open issue) – Others read the issue and ask clarifications – Reproduction of the bug and confirmation – Assignment – Resolution

2018-2019 26 Answer the following questions

● Is freeware F/OSS? ● Is pirate software F/OSS? ● Is F/OSS always better? ● Does all software code need be F/OSS? ● Would you fly on a airplane running on F/OSS?

2018-2019 27 Suggested Reading

 Producing Open Source Software, K. Fogel – Iranian translation available (2017)  Open Source Software: A Survey from 10,000 Feet, S. Androutsellis-Theotokis, D. Spinellis, M. Kechagia, G. Gousios (2011)  Introduction in Free Libre Open Source, J. Barahona, J. Pascual, G. Robles (2010)  Books are not enough: the F/OSS phenomenon evolves rapidly, read the news!

2018-2019 28 More readings on the legacy of F/OSS

● http://www.catb.org/~esr/w E. Raymond ritings/cathedral-bazaar/

● http://www.oreilly.com/ope nbook/freedom/index.html R. Stallman

● http://www.oreilly.com/ope Voices from the open nbook/opensources/book/i source revolution ndex.html Various authors

2018-2019 29 Other Sources

 Internet  Journals (Journal of Systems & Software, Information & Software Technology)  Conferences (Open Source Systems)  https://ellak.gr/ – Open source – Open data – Open content – Open hardware

2018-2019 30 Software Engineering FLOSS

George Kakarontzas University of Thessaly

1 Agenda

 Introduction  Open Source Tools for Software Engineers  Use and distribution of Open Source Software from and for software engineers  Conclusions

2 Introduction

 Open Source software for software engineers has the following advantages:  It constitutes an economic alternative f0r software development (e.g. free IDEs)  Provides the possibility of reuse of libraries and frameworks which make possible the creation of large and useful applications from relatively small groups of developers  Provides access to source code for modification and reuse (under conditions)  Provides a source of knowledge (e.g. Code reading)  But also has some issues that require attention:  The license must be compatible with the distribution of the software product (this concerns the distribution of commercial software mainly).  It must provide assurances of quality and long-term support.

3 Tools of Open source software for software engineers

4 Types of tools  Software engineering is a process that, among others, has the following basic activities:  Analysis: specifies the requirements of the users  Design: designs a solution that satisfies the requirements  Implementation: the code is implemented in accordance to the requirements  Testing: quality assurance of the application and its quality characteristics.  The above mentioned activities are usually intermingled with different intensity at all the iterations of a software project which are of the same duration (e.g. one month).  In all the above-mentioned activities different FLOSS tools are used.

5 Requirements analysis

 Usually we do not use specialized tools for this. The requirements are noted in a document in accordance with the users and some templates are used.  Some tools with which to draw use case diagrams may also be used.  Many times simply the requirements are recorded in a Wiki (a Web application, which allows its users to edit its content in collaboration with others). Relative Open Source tools:  Libre Office (Recording requirements with text) - https://el.libreoffice.org/  Umbrello, Papyrus etc (Use case diagrams) - https://umbrello.kde.org/, https://eclipse.org/papyrus/  Wiki (Collaborative online editing) – Solutions provided by the software version control system (e.g. GitHub) or internal solutions to companies (e.g. MediaWiki - https://www.mediawiki.org/wiki/MediaWiki) 

6 Software Design

 The software is usually designed in a language called UML (Unified Modeling Language) and enables software engineers to capture the design of the software with diagrams (class, sequence, component diagrams, etc.). Usually they need many different diagrams because the software under development has different views that should be designed (static, dynamic, architecture, use cases, etc.) Main open source programs for UML design:  Umbrello:https://umbrello.kde.org/  Papyrus: https://eclipse.org/papyrus/  ArgoUML: http://argouml.tigris.org/  Open ModelSphere: http://www.modelsphere.com/org/ 

7 Software implementation

 Software implementation is usually done with Integrated Development Environments (IDEs). Several popular software development environments are open source software:  Eclipse IDE: http://www.eclipse.org/  NetBeans IDE: https://netbeans.org/  Android Studio: https://developer.android.com/studio/index.htm l , κ.α.  IDEs usually support all software development processes related to coding, testing and sharing of code through repositories (e.g. GitHub). They support the software engineer in many different ways (automatic code completion, code generation, graphical user interfaces creation and much more).

8 Software testing

 Software testing ensures the correctness and quality of the software:  Functional correctness: During software development, tests are carried out by the software engineers themselves, who have the objective of controlling individual units of the program being developed. These tests are aimed at finding errors and correcting them before the SOFTWARE PRODUCT is made available to end users.  Quality control: It concerns other qualitative characteristics that may be important for an application (e.g. speed-performance, safety, usability, etc.) and aims at finding problems related to these qualities and correcting them before The product of the software is made available to end users. Open source tools for software testing:  JUnit – http://junit.org  NUnit – http://nunit.org  Apache JMeter - http://jmeter.apache.org/

9 Use and distributions of open source software for software engineers

Ημερίδα με θέμα: «Ελεύθερο και Ανοικτό Λογισμικό για Μηχανικούς στο Δημόσιο και Ιδιωτικό Τομέα» - TEE Κεντρικής και Δυτικής Θεσσαλίας 29.03.2018 10 Open source software licenses

 Open source licenses do not limit software use but may restrict the distribution of open software and can be divided into two major categories:  Copyleft licenses (A term indicating the contrast with the term copyright): These licenses put restrictions for commercial use. Permissive licenses: They do not pose any substantive restrictions for commercial use.  Software licenses affect software engineers in two ways:  When using open software in their own products the FLOSS license should be compatible with the license of their own software.  When they wish to distribute their own programs as open source software they should choose the appropriate license according to their goals.

11 Some standard licenses License Meaning GNU GPLv3 Strong copyleft License. It requires full disposal of the source code and its modifications by others under the same license. The license notes should be retained. There is an express grant of patent rights.

Apache License 2.0 Permissive License. It mainly requires retention of the license notes. It also provides an explicit grant of patent rights. Larger programs that use software under this license can be made available with a different license and without an open source obligation.

MIT License Simpler permissible license with claims only in respect of the license notes. Larger programs that use software under this license can be made available with a different license and without an open source obligation. 12 Open software selection criteria by software engineers

 Reputation of open Source: is the Open software popular? Is it widely used? Support Community: Is there active development and support from a large community of software engineers?  Standards and interoperability: Is it compatible with relevant standards? Existence of commercial support: Is there a possibility of paying support for the software? Documentation: Is there good documentation available? (Books, websites etc.)  License: Is the license of the software compatible with the license of the software under development?

13 Open Software License selection criteria

 Who is the software for (businesses, students, academics, ordinary users)? Depending on the category, the license must be selected. For example, for most software companies a copyleft license would be prohibitive, so if we are aiming for a software that we produce to be used in commercial products of others we could choose a permissive license. How many and who are expected to contribute to open software and what are their interests? For example, if the project wants to attract many independent software engineers from all over the world, a copyleft license will work better, because it ensures that their work will not be exploited by a third party. What is the business model that we want to follow with the open software we produce? I.e. How will this software be funded? There are many business models: dual-licensing, selling services for the software, selling other merchandise, selling software as a service etc. ( https://en.wikipedia.org/wiki/Business_models_for_open-source_software )

14 Conclusions

 The open source software is now the basic toolbox of Software engineering.  The ability to reuse open source software by software engineers makes it possible to create large and useful applications from relatively small groups and in a relatively short period of time. Attention is required in the software license that is incorporated to produce new software, so that it is compatible with the intentions of the company that develops the new software, especially if the license is copyleft. Caution is required in the selected license of a new open source software in order to be able to finance it depending on the selected business model and the target audience of the new application.

15 Exercise

• In groups of 4, select a software project that all of you or some of you were involve in and according to this project’s characteristics, select either a copyleft (e.g. GNU GPL) or a permissive (e.g. MIT) license. You should consider the software engineers that you want to attract to your project. You should also take into account the possible business models ( https://en.wikipedia.org/wiki/Business_models_for_open-source_sof tware ) • Justify your selection. Any questions?

17 METRICS FOR OPEN SOURCE SOFTWARE Kakarontzas George Agenda

• Metrics definition and the distinction between project and design metrics • The GQM approach in defining appropriate metrics • Metrics’ thresholds and their statistical derivation with examples • Interpretation based on thresholds • Composite metrics and thresholds • Examples of metrics relevant to FLOSS What is a metric?

• The numerical value that corresponds to a particular characteristic of a measured entity. • The measured entity can be anything of interest. In software it is usually some entity of the source code or the design (e.g. the source code a software class or a class in a UML diagram). • The characteristic can be any characteristic of the measured entity that bears some significance (e.g. the number of methods in a class). • The numerical value is expressed in a unit. For example if we measure the height of a man and say that he is 180 cm high then the value is 180 and the unit is cm. This is important because if the unit changes so does the value (e.g. 180 cm = 1.8 m) What to measure?

• Metrics quantify aspects of the software product or its development process and they are used to reveal potential problems (e.g. badly structured code). • They are an important tool in evaluating the quality of FLOSS. • They can be classified in two broad categories: • Project metrics: they quantify the aspects of a software development project and can be used to predict the dynamics of a project. • Design metrics: they measure directly characteristics of the design of the software product. These measurements relate to the product’s quality, size and complexity. Why measure: The Goal-Question-Metric (GQM) model

• Basili and Rombach proposed the Goal-Question-Metric (GQM) model which is based in setting objectives or goals prior to measuring activities. • GQM has the following 3 steps: 1. List the major Goals of your measurement activity 2. For each Goal set the Questions that must be answered to determine if the Goal is met. 3. For each Question decide what Metrics need to be collected to answer it. • Why do we measure? • To answer a Question. • Why is answering the question important? • Because answering it helps us achieve a Goal. Metrics’ Thresholds

• Thresholds divide the value space of a metric into regions (e.g. low, normal, high, very high) • Thresholds are important because without them we cannot interpret the observed value of a metric? • E.g. Are 10,000 hairs in a man head too many or too few? • How do we derive these thresholds? • Using Statistical Information • E.g. An average male person has between 80,000 and 120,000 hairs in his head! • Using Generally Accepted Values • E.g. Three meals a day are considered a normal value for the average number of meals a person has daily. Statistically Derived Thresholds

• In order to derive thresholds it is important to know the mean (or average) value of a metric (휇) and the standard deviation of a metric (휎), where: σ푁 푎 • 휇 = 푖=1 푖 푁

σ푁 푥 −휇 2 • 휎 = 푖=1 푖 푁 • The importance of these two values lies in the fact that in a normal (bell- shaped) distribution nearly 70% (68.26%) of the observations lie within one standard deviation from the mean. Statistically Derived Thresholds (cont.)

• Lower margin: 휇 − 휎 • Higher margin: 휇 + 휎 • Very high: 휇 + 휎 × 1.5 • A value is considered to be very high if it is 50% higher than the threshold for a high value. Examples of Statistical Thresholds in Software

• Suppose we want to know the statistical thresholds of the following metrics: • Average Number of Methods (NOM) per class • Average Lines of Code (LOC) per method • Average Cyclomatic Number (CYCLO) per line of code (i.e., density of branching points) • These metrics are: • Elementary addressing fundamental issues of size and complexity • They vary independently to each other • They are independent of the project size Deriving the Statistical Thresholds for the Example Metrics • Based on 45 Java Projects and 37 C++ projects of various sizes (20K to 2M lines of code) and diverse domains both OSS and commercial we can derive the statistical thresholds given in Table 1.

Java C++ VERY VERY METRIC LOW AVG HIGH LOW AVG HIGH HIGH HIGH

퐶푌퐶퐿푂Τ퐿푂퐶 0.16 0.20 0.24 0.36 0.20 0.25 0.30 0.45 Table 1

퐿푂퐶Τ푀푒푡ℎ표푑 7 10 13 19.5 5 10 16 24

푁푂푀Τ푐푙푎푠푠 4 7 10 15 4 9 15 22.5 Using Thresholds to Interpret Metrics

• Based on the information from Table 1 we can state for example that a Java method is very long if it has at least 20 LOC, or that a C++ class has few methods if it has between 4 and 9 methods.

Java C++ VERY VERY METRIC LOW AVG HIGH LOW AVG HIGH HIGH HIGH 퐶푌퐶퐿푂Τ퐿푂퐶 0.16 0.20 0.24 0.36 0.20 0.25 0.30 0.45 Table 1

퐿푂퐶Τ푀푒푡ℎ표푑 7 10 13 19.5 5 10 16 24

푁푂푀Τ푐푙푎푠푠 4 7 10 15 4 9 15 22.5 Using Elementary Metrics’ Thresholds to Derive Composite Metrics’ Thresholds

• Thresholds of elementary metrics can be used to derive thresholds of metrics composed by them. • For example Weighted Methods per Class is the sum of the complexities of a class methods. For statistical purposes we can derive WMC thresholds by using the following formula:

퐶푌퐶퐿푂 퐿푂퐶 푁푂푀 푊푀퐶 = × × 퐿푂퐶 푀푒푡ℎ표푑 푐푙푎푠푠

• Similarly we can derive other composite metrics. For example:

퐿푂퐶 푁푂푀 • 퐿푂퐶Τ푐푙푎푠푠 = × 푀퐸푇퐻푂퐷 푐푙푎푠푠

퐶푌퐶퐿푂 퐿푂퐶 • 퐶푌퐶퐿푂Τ푀퐸푇퐻푂퐷 = × 퐿푂퐶 푀퐸푇퐻푂퐷 Using Elementary Metrics’ Thresholds to Derive Composite Metrics’ Thresholds (cont.)

• To compute a threshold for high WMC we select the high statistical values from Table 1 for the three elementary metrics and multiply them. Similarly we can compute low, average, high and very high thresholds for the two other size and complexity metrics mentioned earlier, i.e., LOC/Class and AMW (Average Method Weight) a.k.a. CYCLO/Method (see Table 2).

Java C++ VERY VERY METRIC LOW AVG HIGH LOW AVG HIGH HIGH HIGH

푊푀퐶 4.5 14 31.2 105.3 4 22.5 72 243 Table 2

퐴푀푊 1.1 2 3.1 7 1 2.5 4.8 10.8 = 퐶푌퐶퐿푂Τ푀푒푡ℎ표푑

퐿푂퐶Τ퐶푙푎푠푠 28 70 130 292.5 20 90 240 540 Summary

• We introduced the concept of metrics and distinguished between software project and design metrics. In this course we will mainly focus on software product metrics. • We briefly discussed the GQM approach that helps with the identification of appropriate metrics. • We used basic statistics such as the mean and the standard deviation to derive statistical thresholds for metrics. These thresholds help us interpret the observed metrics’ values. • Finally we discussed composite metrics which are metrics derived from other elementary metrics and discussed how to get their statistical thresholds from the thresholds of the elementary metrics from which they are derived. SOURCE CODE METRICS 16

Object-oriented metrics

• Non-object oriented metrics measure size, structure etc. • Lines of Code (LOC), Halstead, Cyclomatic Complexity etc.

• Object-oriented programming requires a different approach to design and implementation.

• Object-oriented metrics measure: • Coupling • Inheritance • Cohesion • Size • Structural Complexity • etc. 17

Chidamber και Kemerer Metrics

• Set of six metrics proposed by Chidamber and Kemerer in 1991 [1] specifically for object-oriented software (improved in 1994 [2]) and measure the complexity of object- oriented design: 1. Weighted Methods per Class (WMC) 2. Depth of Inheritance Tree (DIT) 3. Number of Children (NOC) 4. Coupling Between Objects (CBO) 5. Response set For a Class (RFC) 6. Lack of Cohesion of Methods (LCOM)

✓ [1] Shyam R. Chidamber and Chris F. Kemerer. Towards a metrics suite for object oriented design. In Proceedings of 6th ACM Conference on Object-Oriented Programming Systems Languages and Applications (OOPSLA), pages 197–211, 1991.

✓ [2] Shyam R. Chidamber and Chris F. Kemerer. A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering, 20(6):476-493, June 1994 18

Lack of Cohesion in Methods (LCOM)

• Cohesion: the degree to which the methods of a class are related. • High cohesion: Good division of classes, simplicity and high reuse. • Low cohesion: Increased complexity, increased probability of error in development. Classes with low cohesion probably have to be "broken" into 2 or more subclasses. •

• Assume class C1 with methods M1, M2, …, Mn, and {Ii} = The set of variables in the class used by the method Mi. • There are n such sets I1, …, In. • LCOM (1991) = «The number of foreign sets formed by the intersection of the N sets» or «The number of pairs of methods of a class that do not share variables».

• LCOM (1994) = The number of pairs of methods in a class that do not share variables minus the number of pairs of methods that share variables. 19

Lack of Cohesion in Methods (LCOM) class C1 class C2 class C3 class C4

a1 a4 a1 a4 a1 a4 a1 a4

m1 m4 m1 m4 m1 m4 m1 m4 LCOM Ορισμός / Τύπος

Lack of Cohesion of Methods (LCOM1) LCOM1 = Number of pairs of methods that do not share variables (attributes) (1) (Chidamber & Kemerer, 1991)

P = Number of pairs of methods that do not share variables P – Q, αν P – Q  0 LCOM2 LCOM2 = (2) (Chidamber & Kemerer, 1994) Q = Number of pairs of methods that share variables 0, διαφορετικά

LCOM1(C1) = P = NP – Q = 6 – 1 = 5 LCOM2(C1) = P – Q = 5 – 1 = 4 LCOM1(C2) = 6 – 2 = 4 LCOM2(C2) = 4 – 2 = 2 M M! LCOM1: # Method Pairs = NP = = LCOM1(C3) = 6 – 2 = 4 LCOM2: LCOM2(C3) = 4 – 2 = 2 2 2!  (M – 2)! LCOM1(C4) = 6 – 1 = 5 LCOM2(C4) = 5 – 1 = 4

4 NP(Ci) = = 6 2 20

LCOM – Example

public class User { private String firstname; private String surname; private Vehicle vehicle; private Card card;

I1 = { firstname, surname } public void setName(String f, String s) { firstname = f; surname = s; }

I2 = { vehicle } public void setVehicle(Vehicle v) { vehicle = v; }

I3 = { card } public void setCard(LocalDate issueDate, int duration) { Card card = new Card(issueDate, duration); this.card = card; }

I4 = { firstname, surname } public void printName() { System.out.println( firstname + " " + surname); } } 21

LCOM – Example

Pair (mi, mj) Ii ∩ Ij setName, setVehicle  setName, setCard  setName, printName {firstname, surname} setVehicle, setCard  setVehicle, printName  setCard, printName 

LCOM1 5 LCOM2 4 22 LCOM – EXAMPLE public class User { LCOM1,2 = 0 private String firstname; public class Driver extends Person { private String surname; private Vehicle vehicle;

I1 = { firstname, surname } I1 = { vehicle } public void setName(String f, String s) { public setVehicle(Vehicle v) { firstname = f; surname = s; vehicle = v; } } LCOM1,2 = 0 } I2 = { firstname, surname } public void printName() { System.out.println( firstname + " " + surname); } }

public class Customer extends Person { private Card card; LCOM1,2 = 0 I1 = { card } public void setCard(LocalDate issueDate, int duration) { Card card = new Card(issueDate, duration); this.card = card; } } 23

LCOM – Qualitative effects

• LCOM evaluates the coherence of a class based on the similarity of the methods. If two methods use common variables in the class, they are considered similar. • • The cohesion of a class methods is desirable, and it improves data encapsulation.

• The lack of cohesion may mean that one class should "break" into two or more classes.

• Lack of cohesion is indicative of poor design.

• Increases complexity and therefore increases the likelihood of errors.

• Thresholds are determined empirically through research: • 0 (good cohesion), • 1—20 (average cohesion), and • > 20 (bad cohesion). 24

Weighted Methods per Class (WMC) • Measures the structural complexity of a class • The sum of the complexity of the methods of a class.

• Assume class C1, with methods of M1,…, Mn and c1, …, cn the complexity of the methods. Then: n • WMC = ci i=1

• Determination of complexity ci : • Unweighted WMC: The number of methods if the complexity of each method is considered equal to 1. In this case, the WMC=n • McCabe: The metric of the cyclomatic complexity of McCabe. • Lines of code 25

McCabe cyclomatic complexity

• Estimates the complexity of the algorithm in a method • The number of test paths • Methods with low complexity are generally better • 26

WMC - Example Class Foo { int local_var; Assuming all int pInt; methods have public Foo(){ complexity equal to local var = 1; pInt = new int; 1” } public int f1() { return local_var; WMC(foo) = 4 } public int f2() { return ++local_var; } public int f3() { return 1/f1();} } 27

WMC - Qualitative effects • The WMC gives a good prediction of the time/effort needed to implement/maintain a class • • High values are indicative of problems in class maintenance.

• High values make reuse more difficult. • The more the methods in a class, the greater the potential impact on its children. • Children of a class inherit paternal-class methods. • Classes with more methods are more likely to be application specific, thus limiting the possibility of reuse • 28

Number of Children (NOC) • The number of direct descendants of a class in a hierarchy of classes • • The theoretical basis of the metric NOC is related to the concept of the scope of the properties of a class. It's an indication of how many subclasses will inherit the methods of the paternal class..

• Example: NOC(A) = 2 NOC(B) = 2 29

NOC - Qualitative effects

• It may mean an error in the use of the inheritance.

• If one class has many children it will require more number of tests and more careful testing of the methods since these are inherited by a large number of children classes.

• A positive factor is that high values for NOC may indicate greater chances of reusing a class. 30

Metrics of Robert Martin • In 1994 Robert Martin proposed a set of object-oriented metrics that are popular until today.

• These metrics, focus on the relationships between the packages in a project.

• The level of detail of these metrics is lower than those of the CK.

• These are 5 metrics: 1. AC (Afferent Coupling) 2. EC (Efferent Coupling) 3. I (Instability) 4. A (Abstractness) and 5. D (Normalized Distance from Main Sequence) 31

EFFERENT COUPLING (CE ή EC) • CE  Counts interrelationships between classes. It is calculated as the number of classes in a package that depend on classes in other packages.

• Allows you to measure the vulnerability of the package to changes occurring in packages that it depends upon. • • EXAMPLE Qualitative effects ➢ High value of metric > 20 indicates instability of the package as changes in one of the many external classes can cause the need for changes to the package.

➢ Preferred values are between 0 and 20, while higher values can cause problems with Package Α has outgoing dependencies with maintenance and code 3 other packages (Υ, Ζ, W) development. 32

AFFERENT COUPLING (CA ή AC) • CA : It is an addition to the CE metric and is used to measure incoming dependencies between packages.

• Allows you to measure the sensitivity of other packages to changes in the package under analysis. Qualitative effects • Example ➢ High values of the metric usually constitute high stability. This is due to the fact that a class depends on many other classes. Therefore it cannot be significantly altered as in such a case the likelihood of spread of changes is increased.

Package A has only 1 incoming ➢ Preferred values are between 0 and 500. dependency (Χ) 33

INSTABILITY (I)

• I : Measures the relative susceptibility of a class to changes. • The ratio of outgoing dependencies to all package dependencies (values 0..1).

• Where Ce: Outgoing dependencies, Ca: Incoming dependencies • Qualitative effects • Example Two types of packages: ➢ With many outgoing dependencies and a few incoming (I close to 1), that are somewhat unstable due to the possibility of easy changes to these packages ➢ With many incoming dependencies and a few outgoing (I close to 0), more difficult to change because of the greater responsibility they possess.

Package Α has 3 outgoing (Y, Z, W) Preferred values of metric I must belong to intervals 0 through 0.3 or 0.7 to 1. Packages and 1 incoming (Χ) dependency must be either very stable or unstable, which means that intermediate stability packages should be avoided. This means that packages should have clear roles in the system. 34

ABSTRACTNESS (A)

• Α : Measures the degree of abstraction of the package and is similar to instability. • It is defined as the number of abstract classes in the package to the number of all classes. • • Qualitative effects: • Preferred values are either close to 0 or 1. • Packages that are stable (I close to 0), meaning that they depend to a very small extent from other packages, should also be abstract (A close to 1). Similarly, very unstable packets (I close to 1) should consist of concrete classes (A close to 0). • Combining abstraction and stability, Martin defined the main sequence In the optimum case, the instability of a class is compensated by the abstractness, namely: I + A = 1. Well-designed classes must be grouped around the points of the graph along the main sequence. 35 NORMALIZED DISTANCE FROM MAIN SEQUENCE (D) • D : Measures the balance between stability and abstractness. • Defined as follows: •

Qualitative effects: The value of the metric must be as low as possible so that the components are close to the main sequence. EXAMPLE: COLLECTING METRICS FOR AN OPEN SOURCE COMPONENT USING ECLIPSE ADD-INS 37

Eclipse Metrics Plugin

• Some of the metrics of Chidamber and Kemerer and Martin as well as many other useful metrics are provided by the Eclipse Metrics Plugin that is freely available and can be installed in Eclipse by the • http://metrics2.sourceforge.net/ • The installation process and the basic use of this add-on are provided in the following slides. 38 Install the Add-in

• From the Help menu of Eclipse we select “Install new Software…” And then in the dialog box “Install” We give in the text box “Work with:” the update site http://metrics2.sourceforge.net/updat e/ and press enter. • It will appear in the list of available add- ons in “Metrics plugin for Eclipse” which you should choose. • Press the “Next” and follow the next steps of the wizard (you accept the license, etc.). • Once you finish installing the add-on you restart Eclipse to complete the process. 39 Testing the Add-on

• We will test the add-on in an existing software component that can be used in Java programs to send e-mail. • This is the component Apache Commons Email from the Open Source components Apache Commons that contains many useful software components for Java programs. • The code as well as all usage information of the component is available from its page at http://commons.apache.org/email/ • In our exercise we do not need to understand how we can use this component but only to analyze its source code. 40 Enable Metric Collection • Having entered the project in Eclipse we need to Enable metrics. • We open the project and make sure we are in Java Perspective (Window → Open Perspective → Java). • To activate the Metrics collection we right-click on the project from the “Package Explorer”, We go to “Properties”, Choose “Metrics” and check the checkbox “Enable Metrics”, as the picture shows. • Now each time the project is compiled the collection of its metrics will be refreshed by the add-in Eclipse metrics. 41

Metrics display • To see the Metrics tab, select Window → Show View → Other And from the dialog box “Show View” Go to Folder Metrics and select the “Metrics View”. • This will display the Metrics tab which will initially be empty because we have not yet calculated the metrics of the project (If we haven't yet Build it) 42 Building the project to calculate the metrics

• To calculate the metrics of the project we go to the menu “Project” and select “Build All”. • Note that this option may not be available if the selection “Build Automatically” is checked. If the selection “Build Automatically” is checked τThe project is always "built" automatically every time something changes (e.g. we make a change to a Java file and we store the new file). When the project changes, the metrics are automatically recalculated as well. 43 Metric limits

• The Metrics window is specialized for everything we choose from “Project Explorer”. • If we click on the project the Metrics for the project are displayed. • If we click on a class, the metrics for that class are displayed, κ.ο.κ. • Some metrics appear in red if they are out of bounds. • To see the metric limits you select Window → Preferences And from the tab “Metrics Preferences” choose “Safe Ranges”. • Limits are provided by default for only three metrics. The rest should be put by the programmers according to the desired limits (e.g. depending on what the technical Director of the project considers safe). • What is Gitential

• Gitential is a service that analyzes source code repositories to help in the identification of problems with the process of software development. • It can be used freely for Open Source projects. • Gitential doesn’t evaluate code quality with code metrics that we discussed earlier. Thus it can be considered complementary to these metrics. Creating a project in Gitential

• We can provide our GitHub credentials and Gitential will display our projects in GitHub to select the one we want to analyze. • The service is free for Open Source projects. • Then we can select a period (start date – end date) for the analysis. • Some, but not all, of the metrics provided by Gitential are discussed in the following. Basic statistics comparing to previous period Detailed progress Developer Utilization Productive vs. unproductive work Languages Used Developer productivity Activities

• If you have some project in Github you give Gitential a try to analyze your open source software project and provide project analytics. • You can also insert an open source project in Eclipse and collect metrics with the Eclipse metrics plugin. What does it tell you for the code quality? References

• Michele Lanza and Radu Marinescu. 2010. Object-Oriented Metrics in Practice: Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object- Oriented Systems (1st ed.). Springer Publishing Company, Incorporated. • Shyam R. Chidamber and Chris F. Kemerer. Towards a metrics suite for object oriented design. In Proceedings of 6th ACM Conference on Object-Oriented Programming Systems Languages and Applications (OOPSLA), pages 197–211, 1991. • Shyam R. Chidamber and Chris F. Kemerer. A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering, 20(6):476-493, June 1994 • Robert C. Martin. 2008. Clean Code: A Handbook of Agile Software Craftsmanship (1 ed.). Prentice Hall PTR, Upper Saddle River, NJ, USA. • The Code Project: “Object-oriented metrics by Robert Martin”, at https://www.codeproject.com/Articles/1007524/Object-oriented-metrics-by-Robert- Martin (available online May 2019) Thank you for your attention. Any Questions? Facilitating Metric for White-Box Reuse ( FWBR) George Kakarontzas University of Thessaly

1 Introduction

• In this presentation we will discuss a metric that has been proposed for the reusability assessment of object-oriented classes based on their source code metrics. • The proposed Facilitating Metric for White-Box Reuse (FWBR) was derived using analysis results from 29 open source Java projects, and constitutes an example of how Open Source Software data can be used in software engineering research. • The projects are of various sizes and fields of application. • For each project, metrics were collected and the FWBR metric was derived using statistical methods. No. Project No. of classes Application domain Framework and environment for the development and execution of mobile agents 1 Aglets 447 2 Ant 493 Java Library and program builder from the command line 3 Argo UML 1.578 UML Modeling Tool 4 Atunes 656 Audio Player and music organizer 5 Borg Calendar 147 Calendars and Activity Tracking system 6 Cocoon 85 Spring-based development framework 7 Columba 1.100 Email client and Email management tool 8 Compiere 2.221 ERP 9 Contelligent 836 Content Management System (CMS) 10 DrJava 2.207 Development environment for writing Java programs 11 EuroBudget 155 `Personal Finance Management tool 12 FreeCS 141 WebChat Server 13 FreeMind 406 Software for mind maps 14 GFP 312 Personal Finance Management Program 15 JabRef 1.987 Bibliographic references Manager 16 JRdesktop 58 Remote Desktop management, Remote Assistance, and desktop Sharing tool 17 jBPM 520 Business Process Management (BPM) 18 JEdit 508 Editor for programming 19 jeeObserver 172 J2EE Application Server Performance monitoring too 20 JMoney 54 Personal Accounting Manager 21 Magnolia 955 Content Management System (CMS) 22 OpenEJB 1.759 Enterprise Java Beans (EJB) Container and server 23 Projectivity 452 Project Management Platform 24 RapidMiner 2.745 System and data mining engine 25 Rhino 335 Implementation of Javascript written in Java 26 SportsTracker 56 Application for recording sports activities 27 StoryTestIQ 377 Framework for automatic construction of acceptance tests 28 SweetHome3D 167 Application of interior design spaces 29 OpenProj 846 Project Management Application Total number of 21.775 classes

3 Metrics Used:(1) Metrics by Chidamber-Kemerer

Metric Description Alleged correlation of the metric with the reuse of a class Weighted Methods per Class The sum of the complexity of the methods of classes. It is the Negative (WMC) number of classes if the complexity of each method is considered equal to 1. Depth of Inheritance Tree The depth of inheritance of each class in the inheritance hierarchy. Unspecified: The DIT can be seen from different points of view. If we think (DIT) In the case of Java, DIT is at least 1 since all classes inherit from of a framework (framework) that is designed for reuse one class can have java. lang. Object. This applies to other languages such as C#. high DIT and yet be very reusable. But there are also cases in which large DIT means that a class is very application-specific and therefore not a reusable class. Number of Children (NOC) The number of direct descendants of a class. Positive Coupling Between Objects The number of other classes that a class is associated with. Negative (CBO) Response set For a Class The number of methods that may be called in response to receiving Negative: Although no explicit relationship is reported, the increased effort (RFC) a message from one class. This includes only the methods of class to test and correct errors can be important for reusing a class using a white- and the methods they call directly due to practical difficulties in box reuse approach in which developers need to adapt the class to Reuse in calculating the metric, approach also followed by the CKJM tool we a new slightly different environment. Therefore we assume a negative used. relationship of the RFC to reuse. Lack of Cohesion of Methods The number of method pairs of a class with a similarity of 0 minus Unspecified: LCOM could be considered a negative feature for reuse (e.g. (LCOM) the number of method pairs of a class with a similarity other than controllers controllers that control multiple classes and are not reusable are 0. The similarity of methods is decided by their access to common expected to have high LCOM). However, reuse is not explicitly stated in the variables. original paper of Chidamber and Kemerer.

4 Metrics used: (2) D-Layer metric

5 Metrics used: (3) CDS metric

• CDS is the number of the dependencies of a class. This set of classes is extracted by following the dependencies of a class recursively until we reach classes with D-Layer 0 (I.e. Classes that have no internal dependencies). • High CDS values are indicative of problems in white box reuse.

6 COPE Tool

• The aforementioned metrics are combined in a relational database and are analyzed by a tool called the Component Adaptation Environment (COPE). • COPE also integrates various tools, such as the CKJM tool (Spinellis 2005) and Classycle (Elmer 2011), to provide a more user friendly and productive environment Statistical regression

• We applied regression with progressive addition of variables (Forward regression) In the logarithmic-transformed values of the CK metrics used as forecast values of the D-Layer dependent variable. • Regression with progressive addition of variables was made from the beginning of the axes. The analysis resulted in the introduction of all six predictive variables with a final R2 = .714

8 Regression results

9 Comparative tests

• We compared the metric FWBR with three other metrics proposed in the literature: • Bansiya J., Davis C.: “A hierarchical model for object-oriented design quality assessment”, IEEE Transactions on Software Engineering, vol. 28, no. 1, pp. 4- 17, January 2002 • Inoue K., Yokomori R., Yamamoto T., Matsushita M., Kusumoto S.: “Ranking signiﬁcance of software components based on use relations”, IEEE Transactions on Software Engineering, vol. 31, no. 3, pp. 213 – 225, March 2005 • Nair T. G., Selvarani R.: “Estimation of software reusability: An engineering approach”, SIGSOFT Softw. Eng. Notes, vol. 35, pp. 1–6, January 2010

10 Comparative test #1

• The first comparison examines if the proposed metric outperforms other proposed metrics, in relation to the Cluster Dependencies Size (CDS) of a class. As can be seen in Table the proposed metric outperforms the three other proposed metrics in relation to CDS. In this analysis all 21,775 classes were considered and FWBR is significantly negatively correlated ( ρ =−0.657, p < 0.01, 1-tailed) to CDS. Bansiya’s metric Bansiya and Davis (2002) is also negatively correlated but with a weaker correlation ( ρ = −0.226, p < 0.01, 1-tailed) and the same holds for Inoue (Inoue et al., 2005) ( ρ = −0.180, p < 0.01, 1-tailed). On the other hand the reuse metric proposed in Nair and Selvarani (2010) is positively correlated to CDS according to our analysis (ρ = 0.359, p < 0.01, 1- tailed). • This analysis verifies that using the proposed metric will indicate classes that are more probable to attract fewer classes in total when reused in another context.

11 Comparative test #2 • As a second comparison of the metrics, a correlation analysis was performed with the D-layer of each class and different partitions of the D-layer that maintain the relative order of the dependencies. These partitions are an attempt to simulate the architectural layers of an application which are significantly fewer than the D-layers. The D-layer structure was partitioned to different sizes (i.e. 5, 6, 7 and 8 parts) to simulate different candidate layered architectures. It is important to note that none of these partitions actually corresponds exactly to a true layered architecture since they do not represent the actual architectural partitions of the systems’ architects, but they partition the classes to larger groups of classes than the D-layers which maintain the essential property of the layered architecture (i.e. classes at each layer refer or use classes only at lower layers and never on layers above them). • The results are again very favorable for the proposed FWBR metric which shows a strong negative correlation to all layered partitions and the original D-layer. Again the Bansiya and Inoue metrics are outperformed by FWBR but they are also significantly negatively correlated to the different partitions albeit with a weaker correlation compared to FWBR. Nair metric exhibits a positive correlation to the layered partitions. The details of this analysis can be seen in Table. 12 Comparative test #3

• In the third test the data was split to D-layers and for each individual D-layer a correlation analysis was performed to CDS per project. This comparison attempts to capture the more reusable classes per layer. Since classes at the same D-layer of a project are classes that have the same role in the same system they are more similar (e.g. GUI classes, controllers, entities, etc.) and a successful reuse metric should highlight the more reusable among them favoring those with fewer dependencies. Since current work is interested in white-box reuse as a starting point for componentization, re-architecting and framework extraction, we want each class to require fewer dependencies to be reused. Notice that for at least two reasons the correlation of a reuse metric to CDS may not be easy to establish separately for each D-layer: • 1. First the classes of a D-layer may be very few (e.g. 8 classes) in which case a significant correlation cannot be established. • 2. Second, as discussed earlier, in higher D-layers the dependencies of each class belonging to this D-layer may be the same or very close to the dependencies of any other class belonging to the same D-layer. This may be the result of a design that overlooked dependencies control and is something that we noticed in the majority of the analyzed projects. Again the correlation cannot be established since CDS does not differ significantly for the classes of the analyzed D-layer.

13 Percentages of negative correlations of CDS with metrics by level and project

14 Percentage of positive correlations of CDS with metrics by level and project

15 Regression by project category

16 Calibrated metrics by project category

• CBO is the most negative factor in white box reuse regardless of project size and LCOM is the least important factor • Medium projects: NOC is exempt from regression coefficients • Large projects: The RFC replaces DIT as the second most aggravating factor for white box reuse

17 Possible calibration Benefit

• Small projects: The maximum improvement we can have in our sample with the use of calibrated metrics is 9%. • Medium projects: The maximum possible improvement in our sample is 6%. • Major projects: the maximum possible improvement in our sample is 5%.

Related publication: George Kakarontzas, Eleni Constantinou, Apostolos Ampatzoglou, Ioannis Stamelos: “Layer assessment of object-oriented software: A metric facilitating white-box reuse”, Journal of Systems and Software, Available online 29 August 2012 (http://dx.doi.org/10.1016/j.jss.2012.08.041)

18 Extracting Components from Open Source: The Component Adaptation Environment (COPE) Approach

George Kakarontzas, Ioannis Stamelos, Stefanos Skalistis and Athanasios Naskos Agenda

• OPEN-SME Overview and Contributions • COPE Component Adaptation Environment • Conclusions Agenda

•OPEN-SME Overview and Contributions • COPE Component Adaptation Environment • Conclusions What is OPEN-SME

 Open Source Software (OSS) reuse has the potential to improve software quality, shorten time-to-market and bring competitive advantages to Software Development SMEs.  Currently OSS reuse is restricted to: Whole OSS projects (e.g. Apache web server, MySQL Database) Opportunistic reuse of isolated classes (i.e. copy-paste-adapt reuse). Well-known selected infrastructure components (e.g. Apache Commons)  OPEN-SME proposal was to extend the landscape of OSS reuse to domain-specific components extracted by arbitrary OSS projects. Challenges

 Valuable OSS components exist in every OSS project. However it is difficult to recognize them, extract them, test them, document them etc.  During software development, usually there is no time for the aforementioned activities. Developers often prefer to develop new code from scratch although this code has been written before many times by many others.  Even when developers recognize the opportunity to reuse OSS code there are several uncertainties related to the provided functionality and quality. What the component does exactly? How well it does it? OPEN-SME main idea  Introduce a reuse service that will be operated by SME Association Groups (AGs) (e.g. Greek Association of Computer Engineers, Vasteras Science Park in Sweden) on behalf of their SME software development members.  This service will be operated by software experts of the SME AGs who will produce components from OSS projects, test them, generate documentation, resolve licensing etc. asynchronously to application development by SMEs and independently from the SMEs. The components will be related to domains that are relevant to the SMEs.  When the SMEs will want to reuse them, the components will already be there. OPEN-SME Overview

D E P

n r

C S E T

o n

f g

O s (

A E P

p n r

p g

l i

i n

)

D E P

n r

C r s (

C a E (

)

Result 6: Business Models and Cost Models Agenda • OPEN-SME Overview and Contributions •COPE Component Adaptation Environment • Conclusions COPE Architecture COPE OSS Facts Database Schema The concept of a ‘Reuse Project’

• SME AGs experts, called reuse engineers create reuse projects, analyze the source code of these projects, extract components, test these components and so on. • A “Reuse Project” serves as an extended version of a regular F/LOSS project. It combines the source code related information (of the original F/LOSS project) with information resulted from the analysis process carried out by the reuse engineer. Reuse Project Lifecycle  A Reuse Project’s lifecycle consists of the following phases: Analysis phase: the source code of the target F/LOSS project is being analyzed and the results of this analysis are being stored in the database. Component Recommendation: in this phase COPE automatically suggests class clusters that could serve as reusable components. Component Making: this set of functionalities allows the user to extract components from the reuse project by either using class clusters recommended in the Cluster Recommendation phase or by selecting a single class that along with its dependencies will form a reusable component. Knowledge Management: in this phase the user provides information for the generated components. Using the “Semantic Application” feature, the user can describe the functionality of each component. Moreover s/he can classify the resulting component to a specific domain and concept and finally upload the component to the COMPARE component repository. Creating a Reuse Project Performing Static Analysis with COPE Static Analysis Results Source File Indexing

 Source File Indexing is another analyzer that creates an index from the source files of a Reuse Project. This index enables the feature of the internal search engine and provides five different fields for targeted search: Full Text: performs search for the specified keywords to the whole class file Name: performs search for the specified keywords to the name of the class file Attribute: performs search for the specified keywords to the attributes of the class file Method: performs search for the specified keywords to the methods of the class file Comment: performs search for the specified keywords to the comments of the class file Searching text with COPE Documentation Generation  Documentation Generation creates the Javadoc documentation for the source code of the Reuse Project augmented with UML diagrams for each class and package. Internally we reuse the APIVIZ project (http://code.google.com/p/apiviz/) Pattern Analysis • Pattern analysis is scanning the source code of the Reuse Project for design patterns. Not only it identifies the type of the pattern but it is also able to determine the classes that participate on this specific pattern. • Design patterns’ participants for certain patterns may be good starting points for components’ extraction. Class cluster recommendation

• Using the Cluster Recommendation options, the Reuse Engineer can easily come up with some recommendations of class clusters that could form possible components. For the time being COPE provides two methods for recommending such class clusters: • Dependencies Recommender: uses a genetic algorithm in order to form class clusters using the source code of the Reuse Project. • Pattern Recommender: forms a cluster for each pattern detected in the source code of the Reuse Project. Class-based component extraction

 For component extraction another very useful approach is to select a class and extract a component based on this class. The resulting component will have the interface of the public methods of the class and will include all the required classes for the reuse of this class. The reuse engineer can select this class based on the metrics that are presented in the main window, and especially the Cluster Size, Layer and R (our own reusability index) metrics. Classes which are lower in the layered digraph of the project (have small layer value), have few dependencies (have small Cluster Size) and have larger R value (are more reusable) are good candidates for reusable components. The reuse engineer can extract components by right- clicking any class from the main window that seems promising based on the aforementioned metrics and extract a component for this class. Recommenders Dialog • All recommenders present a similar dialog to the reuse engineer who can examine the recommendations Component Making  Based on the analyses and recommendations carried out earlier the Reuse Engineer can now produce independent software components and then place those components in the repository using the ‘Knowledge Manager’ feature.  Four different kinds of component makers:  Interface Maker: This component maker uses as input the clusters produced by the “Dependencies Recommender”  Dependency Maker: This component maker presents all the classes of the project along with their reusability assessment. The reuse engineer can select a class and extract a component providing the functionality of the selected class.  Adapter Pattern Maker: This component maker presents the clusters produced by the 'Pattern Recommender' and displays clusters involved in Adapter pattern instances.  Proxy Pattern Maker: This component maker presents again the clusters produced by the “Pattern Recommender” but this time it displays only clusters involved in Proxy pattern instances. User Interface for Component Makers

• UI for component makers is similar to all makers such as the one for the Dependency Maker depicted below. Generated Components • Extracted components will be opened for further processing using an IDE (e.g. Eclipse or NetBeans). The reuse engineer will use the IDE to comprehend the component, create test cases for it or execution scenarios, discover further dependencies that are required which are not recoverable through static analysis (e.g. data dependencies). The component can then be tested dynamically using the test cases or execution scenarios that were developed by the reuse engineer. Performing Dynamic Analysis  Dynamic Analysis, gives the opportunity to the Reuse Engineer to test and validate the components he extracts from COPE’s Reuse Projects. More specifically, Dynamic Analysis can provide the following information: Statement Coverage of the Component Statement Coverage per Method of the Component Linear Code Sequence and Jump (LCSAJ) coverage of the Component LCSAJ Coverage per Method of the Component Control Flow Graph per Method of the Component  In order for the Dynamic Analysis to be performed at least one component should have been generated using one of the available Component Makers Performing Dynamic Analysis Step 1

• First we select an available component Execution Scenarios o The reuse engineer needs to define the execution scenario for the component to be tested. An execution scenario consists of a fully functional class that uses the main functionality of a specific component. o Essentially the reuse engineer after extracting the components’ source files it performs some testing on the component using either JUnit tests or simple programs. These programs which test the component’s functionality are called in COPE terminology ‘execution scenarios’. In principle unit tests of the OSS project, if available, can be reused as execution scenarios. Selecting an Execution Scenario Coverage

• The reuse engineer can be informed about the coverage that the execution scenario achieves Method paths followed

 Moreover by selecting a specific method from the table in the centre of the dialog and clicking to the “Details” button, the Reuse Engineer is prompted with a new dialog which presents new coverage information, specifically for the selected method.  COPE provides not only the LCSAJ (Linear code sequence and jump) and statement coverage metrics for the selected method but also it visualizes the paths that were covered by the execution scenario provided (text highlighted in green). Method Control Flow Graph

• COPE provides the option of displaying the Control Flow Graph for a specific method. By clicking the “Display Control Flow Graph” button, the control flow graph opens in a new dialog Component Validation  Component Validation is a five step process: 1. Class Selection: The candidate classes include those that implement a provided interface of the component or the class from which the component was extracted. 2. Select Included / Exclude Interfaces: The provided interfaces of the extracted component should be added to the included interfaces list whereas the required interfaces of the component should be added to the excluded list. 3. Select Include Exclude Scope: This defines the scope of the validation process. The reuse engineer may select from the most general package (as it is recommended), up to a specific class. 4. Select Test Generation Properties: The reuse engineer must define the input and output interfaces. Input interfaces are those that are provided by the component and are going to be tested based on the execution scenario. 5. AspectJ: Aspect Oriented Programming is used to trace component's behavior as it is executed. An AspectJ file is generated according to the fields of 1, 2 and 3 steps. Reuse Engineer may make any change in the displayed code and save the changes. Performing Validation • Once the AspectJ file has been generated the Reuse Engineer can proceed in performing validation process by clicking to “Perform Validation Process” button. If no errors occur a success message is prompted after a while. Additional validation features •In addition the reuse engineer can: • Visualize the FSM of the component validation process • Generate automatically test cases using the ModelJUnit library (https://sourceforge.net/projects/modeljunit/) • All products of the testing and validation process are included in an automatically generated HTML website which documents the testing coverage and validation results in detail. • This test documentation package is included in the component’s package for the benefit of reusers. Generated Test Documentation The Knowledge Manager

• The Knowledge Manager provides the Reuse Engineer with a way of providing meta-information for the components s/he generates. Moreover s/he can classify those components to a specific domain and concept. • Moreover, the Knowledge Manager serves as an intermediate between the COPE platform and the component repository of COMPARE. Characterizing a component with the Knowledge Manager Classifying the Component in its Domain Adding Languages & Technologies Synchronizing with COMPARE repository Component Searching COMPARE provides a page for each Component Component Wiki and Forum Users can download the components from their file repositories Reuse engineers can update components’ details Agenda • OPEN-SME Overview and Contributions • OPEN-SME Tools: OCEAN Search Engine • OPEN-SME Tools: COPE Component Adaptation Environment •Conclusions Concluding remarks

 OPEN-SME tried to bridge the gab between reusing whole OSS projects and the opportunistic reuse of small granules (e.g. classes).  The inspiration from this comes from the success of OSS component libraries such as the Apache Commons (http://commons.apache.org/). OPEN-SME was an attempt to extend this by providing components from existing OSS projects. More components means more attention from the developers More domain specific components means more value from reuse  OPEN-SME provided two processes and three tools that attempted to enable OSS component reuse for software development SMEs. Related Publication: George Kakarontzas, Ioannis Stamelos, Stefanos Skalistis, Athanasios Naskos: Extracting Components from Open Source: The Component Adaptation Environment (COPE) Approach. EUROMICRO-SEAA 2012: 192-199 https://doi.org/10.1109/SEAA.2012.55

COPE Resources: Source code: https://bitbucket.org/gkakaron/copeswingapp/src/default/ Wiki: https://bitbucket.org/gkakaron/copeswingapp/wiki/Home

COPE is a NetBeans project and is provided under the terms of the GNU GENERAL PUBLIC LICENSE

Any questions? Software Engineering Research based on Open Source Software Data: A recent example George Kakarontzas Introduction

• FLOSS open data gave the opportunity to software engineering researches worldwide to study more systematically various other aspects of software besides the source code itself. • In this Section we will discuss a studys about software engineering research based on data derived from Open Source Software repositories, such as mailing lists and issue tracking systems. • More specifically we will examine a published research article about decision-making during the evolution of a programming language based on the analysis of mailing lists. Investigating developers’ email discussions during decision-making in Python language evolution • Related publication: • Pankajeshwara Sharma, Bastin Tony Roy Savarimuthu, Nigel Stanger, Sherlock A. Licorish, and Austen Rainer. 2017. Investigating developers' email discussions during decision-making in Python language evolution. In Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering (EASE'17). ACM, New York, NY, USA, 286- 291. DOI: https://doi.org/10.1145/3084226.3084271 • This publication is an example of using data from FLOSS mailing lists to derive conclusions about software engineering related topics (in this case decision-making in the evolution of a major programming language). Research issue addressed

• Due to the distributed nature of OSSD, email has traditionally been used as the main mode of communication between developers. • Other research has signified that decision-making is the most important factor for the success of organizations and in particular for OSSD communities. • This work addresses the gap of existing research in relation to decision-making for the evolution of a programming language. • Python community has established a decision-making process for proposals concerning the language’s evolution, called Python Enhancement Proposals (PEPs). Python Enhancement Proposals (PEPs) process • A new idea is recommended and discussed in an appropriate forum (using one of the mailing lists, e.g., python-ideas or python-dev). • If the idea gains momentum, a draft PEP document is created with a unique PEP ID number. • Draft is circulated within the Python community for discussion. • The PEP will eventually be accepted or rejected either by vote of community or by Python’s creator, Guido von Rossum, the Benevolent Dictator For Life (BDFL). • Accepted PEPs proposing changes to Python are marked as final when implementation of the changes is complete. Then these PEPs are frozen and no further changes take place. • Accepted PEPs that describing processes and conventions (such as voting), are “living documents” that can be changed as needed. These are marked as active instead of final. Types of PEPs

• Three types: Process, Informational, and Standard Track. • Process PEPs concern processes that should be used during development (e.g. https://www.python.org/dev/peps/pep-0010/ describes the voting guidelines). • Informational PEPs provide information to the development community (e.g. the philosophical underpinnings of Python development (e.g., PEP 203)) and some provide release information for versions of Python (e.g., PEP 3984). • Standard Track PEPs are about changes to the Python language and concern the evolution of the language itself. Dataset

• The dataset included 375 PEPs between July 2000 and July 2015. • 34 (9.1%) Process PEPs • 48 (12.8%) Informational PEPs and • 293 (78.1%) Standard Track PEPs • Recent PEPs include only eight of states (draft, accepted, rejected, withdrawn, final, replaced, deferred, and active). • A PEP starts at the draft state. When formally accepted it is assigned the accepted state. When the implementation is provided it is assigned the final state. • Python community has published a simplified version of this process (https://www.python.org/dev/PEPs/PEP-000) • However, prior research indicates a more complex process. • The authors of this work used the methodology proposed in prior work, to identify the state progression of PEP and the Figure in next slide shows the states and their frequencies. Edges represent transitions between states, while thickness of edges stands for the frequency of each transitions State progression of PEPs in the dataset Research questions

• The progression of PEPs through states is therefore known (see figure in previous slides). • Unknown: Developers’ involvement in decision-making process, based on email discussions, for different PEP types and the and PEP document states. • Idea: research developer involvement patterns to improve management of an OSS team, using email discussions about PEPs from six mailing lists, • The authors investigated two Research Questions regarding developer involvement in decision-making using email: • RQ1: What level of interest (in terms of discussion) has each of the three types of PEP (Process, Informational, and Standard Track) generated among developers? • RQ2: Do certain states of the decision-making process generate more discussion among developers? Method • Dataset of 42,672 email messages in six mailing lists: • python-dev (25,406) [considered the main forum] • python-lists (11,929) • python-checkins (5,445) • python-ideas (3,823) • python-distutils-sig (3,777) • python-announce-list (425) and • python-committers (279). • Focus on emails with explicit references to PEP ID numbers either in the title or in the body text. • Emails were classified by PEP type and ID to answer RQ1 (level of interest of developers by type) • Messages that were exchanged between two pairwise states (e.g., between draft and accepted) based on timestamps were grouped. Then the number of messages between transition of a PEP from one state to another were counted for specific PEPs to answer RQ2 (Do certain states of the decision-making process generate more discussion among developers?) Basic statistics of email messages for each PEP type Outliers in Informational and Process PEPs

• Informational PEPS (2 outliers): • PEP 440 (294 emails): new approach for versioning and dependency specification • PEP 356 (229 emails): release info of Python ver. 2.5 • Process PEPs (2 outliers): • PEP 8 (2,504 emails): the Python style guide • PEP 1 (358 emails): describes the purpose of PEPs and provides guidelines. Outliers in Standard track PEPs (1)

• Standard track PEPs: 10 outliers (595-2,055 emails): • PEP 308 (2,055 emails): new syntax for conditional expressions. 17 options were proposed. The decision was deferred to the BDFL. • PEP 318 (1,000 emails): 27 different syntaxes for function decorators. Community converged but BDFL rejected it. Final decision contained in PEP 318. • PEP 285 (902 emails): new built-in type bool. The BDFL decided after rejecting three other proposals. Outliers in Standard track PEPs (2) RQ2. Do certain states of the decision-making process generate more discussion among developers?

• The top ten pairwise states (of the 78 possible pairs (combinations of 13 states) account for more than 95% of the email, with the top 5 accounting for 84%. • Since the top ten pairwise states account for (almost) all discussions, we discuss the main trends for the top ten pairwise states. • Figure shows the number of messages for all PEPs between pairwise states as bars and average number of messages per PEP as square points. • Pairs draft-accepted, draft-final, and draft-rejected generated the most messages. It is interesting that draft proposals that succeed (represented by draft-accepted and draft-final) and those that are rejected (represented by draft-rejected), generated a similar amount of discussion suggesting the community’s healthy balance between both successful and rejected PEP drafts discussions. Pairwise message counts and average number of messages per PEP Other pairs of PEPs

• Dip in the number of messages per PEP for the accepted-final and draft-deferred pairs (values of 37 and 41 respectively). Once a PEP is accepted, there is not a lot to discuss. • A draft is deferred due to important issues that prohibit, no need for much discussion other than identifying reason(s) to defer. • No evidences of PEP that was accepted and then rejected (i.e., the pair accepted-rejected is missing). • Average number of messages for PEPs that are withdrawn (i.e., the draft-withdrawn pair) is relatively high (74). There is significant discussion before a PEP is withdrawn. Pairwise pairs of informational states

• The next three pairwise states (active-final, draft-active and active- draft) concern Informational and Process PEPs as they include the state active. • Average no. of messages discussing PEPs involving the active state is arpox. 60. • The last pair state (incomplete-final) is similar to draft-final since some PEPs were assigned the incomplete state rather than the draft state. Draft and accepted states

• Certain states generated substantial amount of emails. • Four out of the top five pairwise states involve the draft state. • There is variation in the average number of messages depending upon the end state to which the PEP transitions (e.g., draft-deferred generated fewer messages per PEP than draft-final). • The other starting states that attract a lot of discussion are the accepted and the active states (4th and 7th ranked pairs). Contributions and Implications regarding RQ1

• Standard Track PEPs generated the most developer discussion because they concern the evolution of the language itself. Other PEPs are of more social nature. • There is a lot of passion involved in the discussion of the evolution of the languages (e.g. PEPs 238 and 285) and this poses challenges for the project managers. • PEPs with multiple alternatives generate a lot of emails and require BDFL intervention for a final decision. • For project management with too many alternatives pose problems it may be desirable to explore alternatives before voting (with involvement of BDFL early on). Balance between enabling an open community vs. reducing choices early on to enable easier decision-making, must be maintained. Large number of alternatives increase the potential for polarized opinions. • Proposals for new syntax generate greater no. of emails than modifications to existing features. Python and other OSSD communities embarking on similar topics can plan for such discussions (e.g., by exploring alternatives with experts early). Contributions and Implications regarding RQ2

• Certain pairs of states generated more discussion (the top three: draft-accepted, draft- final, and draft-rejected). • Both accepting and rejecting PEPs attract similar amount of discussion, something deemed as healty. • Less discussion for deferred PEPs, compared to accepted, rejected, or withdrawn. T • Transitioning from some states is not taken lightly • Rejection needs strong justification • This has implications for community members willing to participate in the decision-making process, and the likely challenge of managing such a large amount of knowledge available in emails (up to thousands of messages per PEP). • Early phases of PEPs generated a large amount of discussion, particularly when PEPs were in the draft state. The other starting states that generated relatively large amounts of discussion include the accepted and active states. These results indicate that when PEPs are in certain states (e.g., the draft state), the PEP authors and the BDFL should be ready to handle the volume of ensuing discussion (in terms of allocating time and resources for managing the discussion). Exercise

• In groups of four visit the website https://flosshub.org/biblio and find a publication that used Open Source Software data for software engineering research. • After reading the paper discuss and report the key elements: • Which data was used? • What were the research questions answered? • What were the answers to these research questions? • What were the implications for the communities of open source software? Improving Component Coupling Information with Dynamic Profiling

George Kakarontzas and Christina Pardalidou University of Thessaly Agenda

❑Introduction ❑Proposed analysis and tooling ❑Experiments ❑Small program scenario ❑Larger program scenario ❑Conclusions and ideas for extensions Introduction Coupling and program comprehension

 In FLOSS the source code is available which makes it possible to read it in order to comprehend. However it is preferable to first study the design of the program before delving into the details of the source code.  Even the design, however, can contain too many details (e.g. large number of classes, large number of associations between these components etc.)  In this research we propose a solution to this problem, which – when automated – will help towards Open Source Program comprehension, specifically for projects lacking such design documentation. What is coupling? B

class A extends B { private C c; … public void someMethod(D d) { A E e = new E(); } … } C D E  Excessive coupling is not desirable  And it usually signifies low cohesion Coupling metrics

 Static metrics measure coupling without executing the program.  Coupling Between Objects (CBO): S. R. Chidamber and C. F. Kemerer  But there are also dynamic metrics which measure coupling when the program runs.  Suite of dynamic metrics: E. Arisholm, L. C. Briand and A. Føyen,  Run-time Coupling Between Objects (RCBO): RC Mitchell and Power  Estrangement Between Classes: G. Kakarontzas, V. C. Gerogiannis, S. Bibi and I. Stamelos  The benefit of dynamic metrics is improved accuracy. The problem with reverse engineering

 Reverse engineering in the context of this work is generating information, such as diagrams, from an existing system with the purpose of system comprehension.  Can be challenging:  Diagrams can be large and tangled making comprehension of even moderately large systems very difficult.  Some information will not be reverse engineered (e.g. missing dependencies or associations via interfaces).  All lines and classes look the same (i.e. associations and classes are indistinguishable from one another). Diagrams can be large and tangled Missing information

IB A

C All lines and classes look the same class psnbtech

«enumeratio... JPanel TileType -nextType -side SidePanel

-tiles -currentType JFrame Tetris -tetris

-tetris

Clock -board JPanel BoardPanel

-logicTimer Proposed analysis and tooling Proposed Analysis

 During application profiling, users capture snapshots, which contain the call tree of methods executed as well as their execution times.  Many such snapshots are collected and consolidated in a single CSV file which is the output of our open source tool, COMPOSITOR (COMPonent cOupling diScrimInaTOr using pRofiling - https://github.com/chripard/COMPOSITOR)  The exported CSV file contains the names of all system classes as rows and columns. In the cells the time that the row class has spent inside the column class is exported. Algorithm implemented by COMPOSITOR

open directory merge call traces into one .csv file create Map1 (keys are unique method names) from the merged file for each unique method name in Map1 add inner method calls and time spent as values if method call already exists time spent = time spent/2 end if end for create List with unique class names that is ordered create Map2 (keys are unique class names) for each unique class name in Map2 find method calls to other classes & add them to total time spent end for write Map2 to a new .csv file (output file) Example of the output of the analysis

Average time spent from each Row class inside each Column class in Milliseconds

BoardPanel Clock SidePanel Tetris TileType

BoardPanel 0 0 10,048 119,134 Clock 0 0 0 0 SidePanel 0 0 8,012 29,507 Tetris 4,259 136 0 331 TileType 0 0 0 0 Experiments Description of the experiments

 We have used our technique in two programs to understand the analysis and the behavior of the tool.  Both programs were Open Source Tetris games since it was easy to profile both applications without requiring domain expertise.  Initially we analyzed a small Tetris program and then a larger one, to understand how to use the proposed analysis and the tool in these two distinct scenarios.  Small Tetris game: https://github.com/PSNB92/Tetris/tree/master/src/org/psnbtech  Larger Tetris game: https://github.com/seanws/Tetris Small Tetris game class psnbtech

«enumeratio... JPanel TileType -nextType -side SidePanel

-tiles -currentType JFrame Tetris -tetris

-tetris

Clock -board JPanel BoardPanel

-logicTimer Conclusions from the analysis of the small Tetris game  Some classes do not appear to be related at all in the UML diagram; however, they are related in code.  Although some classes are related in the class diagram, in reality they are not dependent on each other.  Some elements that seem unimportant by examining the class diagram are very important when examining the profile- deduced intensity.  All classes’ associations are now augmented with their strength according to their usage by other classes and therefore are differentiated. Characterizing associations

 We can create a range of values with the different values that we have obtained through our analysis and separate the values in 5 areas (i.e. a Likert scale): very strong, strong, medium, weak and very weak associations.  For this purpose, we obtain the median of the whole range, which splits the range in two (strong and weak associations) and then for each subrange we obtain again the median of the respective subrange to differentiate between very weak (very strong) and weak (strong) associations as depicted in the following table: Assigning numerical values

 Now for each association type we assign a value (1= VW, 2=W, 3=N, 4=S, 5=VS) and we create an NxN table (where N is the number of classes) and at each cell we place the association strength (1-5) obtained from our profiling analysis.  For example, the association of Tetris to BoardPanel is 4,259 which places it to the Weak (W) category. Therefore, in the respective cell intersection of Tetris with BoardPanel we assign the value 2 (for W). Coupling importers and exporters Tetris Board Clock Side Tile Imported Panel Panel Type Coupling

Tetris 2 1 1 1 5 BoardPanel 4 1 1 5 11 Clock 1 1 1 1 4 SidePanel 4 1 1 4 10 TileType 1 1 1 1 4 Exported Coupling 10 5 4 4 11

• Classes TileType and Tetris should be changed carefully since their changes may cause major changes to other classes. • Classes BoardPanel and SidePanel may need to change frequently due to their dependencies to other classes. Larger program analysis

 A useful addition for larger programs is that the analysis allows us to focus in the most strong associations and respective classes first. Tangled Diagram including ALL associations Class diagram of the VS class associations Class diagram of the VS and S class associations Conclusions for the large program scenario

 Besides providing the benefits of the small program case, the larger program also provides a method to study the system gradually by first focusing on the VS associations and respective classes and then gradually adding more detail Conclusions & Ideas for extensions Conclusions  We used profiling of object-oriented programs to determine the intensity of the associations among program components during runtime based on CPU time as criterion.  We applied our approach and tool in two programs with different sizes and discussed the significant findings.  Our approach provides the means to:  differentiate among similar associations and highlight the most important ones.  identify missing associations from generated reverse engineered class diagrams.  pinpoint situations in which seemingly existent relationships are really non-existent  identify the biggest coupling importers and exporters in a system  provide a stepwise approach in the study of large reverse engineered UML diagrams. Ideas for extensions

 In the future, we plan to apply our approach in larger programs and try to associate the profile-induced coupling intensity with change proneness.  Furthermore, we plan to use different ways of establishing the coupling intensity besides CPU time such as the memory usage and compare these different approaches.  An important practical extension would be to create a plugin for an IDE (e.g. an eclipse plugin) that will provide assistance on the comprehension of software using our approach. Thank you for your attention. Questions? Improving Component Coupling Information with Dynamic Profiling George Kakarontzas [email protected] and Christina Pardalidou [email protected] Related publication: George Kakarontzas and Christina Pardalidou. 2018. Improving component coupling information with dynamic profiling. In Proceedings of the 22nd Pan-Hellenic Conference on Informatics (PCI '18), Karanikolas Nikitas and Mamalis Basilis (Eds.). ACM, New York, NY, USA, 156-161. DOI: https://doi.org/10.1145/3291533.3291576