IMPLEMENTING OPEN SERVICES FOR LIFECYCLE COLLABORATION API FOR THE FREENEST PLATFORM
Joni Katajamäki
Bachelor's Thesis December 2012
Software Engineering School of Technology OPINNÄYTETYÖN KUVAILULEHTI Tekijä(t) Julkaisun laji Päivämäärä KATAJAMÄKI, Joni Opinnäytetyö 3.12.2012
Sivumäärä Julkaisun kieli 28 Englanti
Verkkojulkaisulupa myönnetty ( X ) Työn nimi IMPLEMENTING OPEN SERVICES FOR LIFECYCLE COLLABORATION API FOR THE FREENEST PLATFORM Koulutusohjelma Ohjelmistotekniikka
Työn ohjaaja(t) PELTOMÄKI, Juha
Toimeksiantaja(t) HUOTARI, Jouni
Tiivistelmä
Opinnäytetyö kuvaa FreeNEST ohjelmistokehitysalustan erilaisia työkaluja ja niiden välisiä integrointeja, sekä näiden integrointien ongelmia. Vaihtoehtoisena integraatiotapana tutkittiin Open Services for Lifecycle Collaboration (OSLC) yhteisön vastaavia alustoja varten luomia standardeja.
Työssä kuvataan OSLC rajapinnan toteuttamista FreeNESTin Bugzilla-työkaluun. Kuvauksessa käydään läpi integroinnin teknisiä yksityiskohtia, liittyen muun muassa RDF-datamalliin. Opinnäytetyössä kuvataan myös projektissa käytettyä Go-ohjelmointikieltä ja perustellaan sen valintaa toteutustyökaluna.
Avainsanat (asiasanat) FreeNEST, OSLC, RDF, Golang, REST
Muut tiedot DESCRIPTION Author(s) Type of publication Date KATAJAMÄKI, Joni Bachelor´s Thesis 3.12.2012
Pages Language 28 English
Permission for web publication ( X ) Title IMPLEMENTING OPEN SERVICES FOR LIFECYCLE COLLABORATION API FOR THE FREENEST PLATFORM Degree Programme Software Engineering
Tutor(s) PELTOMÄKI, Juha
Assigned by HUOTARI, Jouni
Abstract
This thesis describes the different tools of FreeNEST project platform, the existing integrations between them and the problems of those integrations. A specification created by Open Services for Lifecycle Collaboration (OSLC) is researched as an alternative to the current way of integrating tools with various plugins and other non-scalable solutions.
The thesis describes the technical details of the OSCL specifications, a metadata model known as RDF and it also walks through the basics of the Go programming language used in the FreeNEST OSLC implementation. The thesis also includes a description of the technical details and challenges encountered when implementing an OSLC interface to FreeNESTs bug tracking component called Bugzilla.
Keywords FreeNEST, OSLC, RDF, Golang, REST
Miscellaneous CONTENTS
1 INTRODUCTION ...... 1 1.1 FreeNEST platform ...... 1 1.2 Thesis objectives ...... 1 2 THEORETICAL BACKGROUND ...... 1 2.1 Resource Description Framework ...... 1 2.2 Representational Data Transfer ...... 3 2.3 Open Services for Lifecycle Collaboration ...... 5 2.3.1 OSLC Core ...... 5 2.3.2 OSLC domains ...... 5 2.3.3 Service Provider architecture ...... 6 2.3.4 Query Capability service ...... 8 2.3.5 Delegated Dialog services ...... 8 2.3.6 Creation Factory service ...... 9 2.3.7 User Interface Preview ...... 9 2.4 Google Go ...... 10 2.4.1 Google’s reason for Go ...... 10 2.4.2 Improving build times with packages ...... 10 2.4.3 Making teamwork easier ...... 12 2.4.4 Reducing syntax complexity and improving safety . . . 12 2.4.5 Lighter object orientation ...... 13 2.4.6 Familiar language, modern features ...... 15 2.4.7 Easy concurrency ...... 15 2.4.8 Why use Go for the OSLC API ...... 16 3 CURRENT FREENEST INTEGRATION METHODS ...... 17 3.1 Top bar ...... 17 3.2 Foswiki as an integration platform ...... 17 3.3 Problems with the current integration methods ...... 17 4 PROPOSED INTEGRATION METHODOLOGY ...... 18 4.1 Service providers in FreeNEST ...... 18 4.2 Bugzilla as stepping stone to OSLC ...... 19 4.2.1 Bugzilla data model ...... 19 4.2.2 Bugzilla OSLC queries ...... 20
4 5 RESULTS AND CONCLUSIONS ...... 21 5.1 SkyNEST project implementation ...... 21 5.2 Possible problems with the new integration model ...... 21 5.3 Conclusions ...... 22 6 REFERENCES ...... 22
LIST OF TABLES
1 HTTP methods used in REST ...... 4 2 OSLC workgroups ...... 6
LIST OF FIGURES
1 Service Provider architecture ...... 7 2 Selection dialog sketch ...... 9 3 UI Preview scetch ...... 10 4 Bugzilla database to OSLC change request conversion ...... 20
5 TERMINOLOGY
ALM - Application Lifecycle Management means continuously managing the whole life of an application from the development to support through governance, development and maintenance.(Application Lifecycle Management, 2012)
API - An application programming interface (API) is a specification intended to be used as an interface by software components to communicate with each other.
Bugzilla - An open source defect tracking system used by the Mozilla project. It contains features that even large commercial defect trackers don’t have. Bugzilla has an open- source license and is completely free to use.
Changezilla - A name given to a custom installation of Bugzilla inside FreeNEST. Unlike Bugzilla which tracks defects, it is used for tracking change requests.
Foswiki - An enterprise grade wiki software that contains an easy to use text editor and has support for various plugins. FreeNEST has many custom plugins for FosWiki, such as linking bugs from Bugzilla and test cases from Testlink.
FreeNEST - Software development platform developed by the SkyNEST project.
SkyNEST - Project maintaining and developing the FreeNEST platform. The project is running in JAMK University of Applied Sciences.
Testlink - A test management tool used by testing teams to write test cases and test plans and report their results in an orderly way.(Testlink website, 2012)
Uniform Resource Identifier - A sequence of characters that can be used to identify a name or a resource. Abbreviated as URI.
Uniform Resource Locator - A URI referencing an Internet resource. Abbreviated as URL.
Wiki - A wiki is a collaboratively edited website, usually containing knowledge from the project or community it was created for. 1
1 INTRODUCTION
1.1 FreeNEST platform
FreeNEST is a project management platform developed by the SkyNEST project at JAMK University of Applied Sciences. The project was started by a testing engineer Marko Rintam¨akiwhile working for Ixonos Oyj. While other open source project man- agement tools exist, FreeNEST aims to provide a full ALM solution for managing a project’s life cycle.
A FreeNEST installation is meant to be used by a single team, working on a single project. Setting up a complete environment for a project should be as easy and as fast as possible. While FreeNEST currently contains tools that are most useful in software projects, the project will expand to cover other kinds of projects. The platform is open source web based software. It is built by integrating existing open source project management tools into a single platform.
1.2 Thesis objectives
This thesis aims to find a better, more unified way of integrating the various tools in FreeNEST. Since the components of FreeNEST are open source tools without any centralized development, a unified vocabulary, interface and data model needs to be created. These can be implemented using a specification from the Open Services for Lifecycle Collaboration, or OSLC. Since implementing the whole specification is a very large task, implementing a small subset for a single tool first is a useful experience.
2 THEORETICAL BACKGROUND
2.1 Resource Description Framework
All data in OSLC follows the RDF model. RDF is meant to be used for web resources by supporting linked data and providing features that help merging together data with differing schemas.(RDF website, 2012)
In practice, RDF is a metadata language that makes statements about web resources. While these resources originally meant just web documents, they can now be also thought as any kind of resources that appear in the web. For example, a person can have their personal data on the web, and that data can be presented by RDF. 2
RDF data is formed from statements often called ”triples”. These triples have three parts, subject, predicate and object. Subject is the entity the statement describes, object is a entity the subject has a relationship to and predicate describes the kind of relationship there is between subject and object. The following sentence is an example of a triple-like structure:
John is a friend of Jill
In that statement, the subject is John, the predicate is ”is a friend of” and the object is Jill. While the sentence does not follow any known RDF format, the structure of RDF data contains similar components. A collection of RDF triples forms a directed graph. RDF does not have a single official textual representation, but the most common ones are are RDF/XML, N-Triples and Turtle. RDF is also a part of movement called Semantic Web, where from RDF borrows the idea to use web links as resources. These links can of course be opened and navigated to, and they can contain more RDF data. Because a URL describes the location of a single web page, it can be used as a globally unique identifier. The uniqueness of identifiers makes it possible to combine different RDF documents from various sources to form a larger data graph. Here is an N-Triples formatted version of the previous example sentence:
The people URLs are only examples, but the middle one (the predicate) is a real RDF predicate defined by the Friend Of A Friend (FOAF) RDF ontology. Ontologies are a way of defining relationships between objects. There is no central authority that defines the relationships and hierarchies that RDF documents can represent and because of that, there are many organizations who have created their own public RDF ontologies. These can be used by anyone, making it easier for the users of RDF data to use the same vocabularies when describing the same kind of data.
The FOAF defines an RDF ontology that allows RDF data to describe relationships between people. In the example, a FOAF predicate called ”knows” is used to form a relationship between the John and Jill URLs. The ”knows” predicate is defined by the FOAF specification as the following: ”A person known by this person (indicating some level of reciprocated interaction between the parties)”(FOAF Vocabulary Specification, 2012). 3
Since full URLs like those in the N-Triples example are quite difficult for humans to read, most RDF document formats allow replacing the common parts of URLs with previously specified prefixes, and organizing the triples to a hierarchy. Below is an example of an Turtle formatted version of the previous document:
@prefix example:
The prefixes are defined on the top of the document, giving a shorter notation for the URLs. Turtle is an extension of N-Triples, adding more advanced features to the notation, instead of just simply listing triples.
RDF/XML is a popular format in the RDF world, because it can be read using existing XML libraries and tools, but its can be more difficult for humans to decipher. The following is a RDF/XML version of the previous examples:
2.2 Representational Data Transfer
Representational Data Transfer (REST) is a kind of software architecture that is used in distributed systems like the World Wide Web. The term was coined by Roy Fielding in his doctoral dissertation in 2000.(Representational Data Transfer, 2012)
REST is a client-server architecture, clients send requests to the server, server processes them and returns responses to them. This means that servers do not have to concern themselves with the client state and user interface, and clients do not have to care about the data storage. The architecture makes it possible to develop the client and server separately, as long as the interface between them stays the same. 4
REST conformant interfaces are often called RESTful.
Originally REST was designed to be used with the HTTP protocol, but it does support others that can represent the same kind of functionality. Individual resources in REST can be identified with URIs and the resources can be retrieved and changed with HTTP methods. For example, a web store with a REST API could have the following URL representing a collection of the products sold in the store: http://www.fakestore.com/products/
Then, all of the products could be referred to using the previous URL with the products identifier, which in the next example is ”123”: http://www.fakestore.com/products/123
This product’s information could then be retrieved by sending a HTTP request with the method GET to its URL. RESTful interfaces often use methods described in Table 1.
HTTP method Description GET Retrieves a resource from the given URL. The request can also specify the content type that the client can process, the server should then respond with the spec- ified content type (such as XML) if it is supported. POST or PUT These methods are used to update existing resources or to create new ones. The request body contains the document data that is then inserted into the docu- ment at the specified URL. Usually there is need to support either POST or PUT, not both. DELETE Removes the specified resource from the server.
TABLE 1. HTTP methods used in REST
Methods in table 1 are often more than enough for most interfaces, since they allow retrieving, updating, creating and deletion of resources. 5
2.3 Open Services for Lifecycle Collaboration
Open Services for Lifecycle Collaboration community, or OSLC, was started in 2009 to define specifications to address these integration problems. The goal of the community is to make it easier for tool developers and vendors to create integrations between different products by creating unified technical and conceptual specifications. OSLC commu- nity has several independent workgroups that focus on different domains of application lifecycle.(Open Services for Lifecycle Collaboration, 2012)
2.3.1 OSLC Core
OSLC Core workgroup defines the specific technical details used by the other specialized domain workgroups.
The Core group has chosen REST and Linked Data as the main technical implementation details. OSLC identifies resources through URLs, which can be then modified using HTTP methods. Resources in OSLC are often data about common concepts in ALM software, such as bugs (change requests in OSLC vocabulary).
The data in OSLC is represented with RDF, which allows software vendors to easily add custom extensions to the data provided by their OSLC API implementation. OSLC only defines a limited set of data that must be provided about a specific kind of resource.
On the surface the various OSLC features may seem to be an arbitrary combination of features, relating to the data model, vocabulary and even user interfaces. But these features are needed to provide a general integration interface between any tools that have an OSLC implementation.
2.3.2 OSLC domains
Because there are dozens of different tools in the ALM world, OSLC contains many workgroups whose mission it is to identify common data in ALM tools that are a part of the same problem domain. Table 2 provides a short description of all of the working groups in the OSLC community.
All of the groups work independently, but the specifications produced by them form the whole of OSLC. A full implementation of all the domains specifications is a massive undertaking, but this thesis will focus on the implementation details of a small subset, consisting of Core and Change Management. 6
Workgroup name Description Configuration Management Tries to find common definitions for managing soft- ware versions and configurations. Reporting Defines ways to form reports about the data provided by ALM tools. Change Management Focuses on tools that manage product change infor- mation, this often means bug reports. Quality Management Specifies definitions for test cases, test plans and test results. Requirements Management Creates data definitions for system requirements, such as what a system must do to fulfill its users needs and how processes surrounding it must be or- ganized to make that system functional. Asset Management Focuses on defining asset data management. Assets in the software world are often binary files, such as dynamic linking libraries (DLLs), images or some kind of schematics. Architecture Management Defines data about project planning, analysis, design, construction and governance. Automation Specifies the resources for automation plans, requests and results in the software development cycle. Estimation and Measurement Provides specifications for estimating project’s differ- ent aspects, such as its scope, cost and schedule. It also has definitions for monitoring, controlling and calibrating project estimates. ALM-PLM Interoperability Will define data for different interoperability scenarios with different ALM tools. Work in progress. Performance Monitoring Gives specifications about different performance data aspects of applications. Reconciliation Specifications that will give different applications a way to know that they are using the same resource
TABLE 2. OSLC workgroups
2.3.3 Service Provider architecture
OSLC API clients can use Service Provider Catalog and Service Provider RDF documents to discover what services an OSLC server implementations offers. Service Provider Catalog offers links to all of the Service Providers of the implementation and individual 7
Service Providers list links and metadata to different services offered by the provider. Figure 1 shows the structure of Service Provider data.
FIGURE 1. Service Provider architecture
These documents describe all the information needed by clients to use the supported services. OSLC services provide different types of functionality defined by the OSLC Core specification. Here is an example of a Service Provider document using the Turtle notation:
@prefix oslc:
oslc:creation
The document describes a Service Provider located in ”http://example.com/bugz/comp1” which has a QueryCapability URL that can be used for queries and a CreationFactory URL that can be used for creating new resources. It could contain other useful metadata, such as a human readable description of each component or technical details needed by the clients.
2.3.4 Query Capability service
Query capability is a way of querying OSLC resources. A resource is a network data object or a service identified by a URI. For example, in the Change Management domain, a resource could be a bug report for a product. Querying these resources enables searching needed data and creating dynamic reports from that data. OSLC does not mandate a specific query syntax, but it does provide a definition for a query language that is relatively simple to implement. The OSLC query syntax is meant for querying RDF data, so tools that have a different data model, such as a relational database, need a way to convert the OSLC queries to their own formats.
Queries can be sent with a GET HTTP request, making it possible to link to queries. The following is an example query which retrieves a resource with an identifier of ”123” from a fictional ALM product: http://alm-ex.com/oslc/cr/product/1?oslc.where=dc:identifier=123
The part after the question mark in the query are the query terms. Queries support searching for specific values or terms, limiting returned data to specific values and or- dering the result set.
2.3.5 Delegated Dialog services
Delegated dialogs are a way to provide easy integration between different OSLC compli- ant tools. These dialogs can be embedded to tools with minimal code changes and they can be used to search and create resources of a tool inside an other tool.
A selection dialog allows users to enter search terms to find a list of resources. These resource links can be then inserted to the parent document that the dialog is attached to. For example, this could be used to easily link bugs to test cases, without the need to 9 switch between the test and bug management tools. A basic selection dialog design can be seen in Figure 2. Selection dialogs counterpart is the creation dialog, which allows users to create new resources inside separate tools in a uniform way, making it easy to create bug reports and other small resources quickly.
FIGURE 2. Selection dialog sketch
2.3.6 Creation Factory service
Creation factories are a way to create resources by sending RDF documents to a address. These can be used to automate resource creation, for example creating a bug report if a unit test fails. The factory may provide a Resource Shape. The shape is a document that describes the information needed by the factory to create a resource.
2.3.7 User Interface Preview
UI preview is a small but important feature in OSLC that make it possible for users to see useful information about a resource with a glance. When a user hovers over a resource link, a small pop-up shows up at the mouse cursor, showing a small preview of the resource, such as in Figure 3. 10
FIGURE 3. UI Preview scetch
2.4 Google Go
Go is an open source programming language originally created by Google engineers Rob Pike, Ken Thompson and Robert Griesemer. Work on the language began in late 2007, and the implementation was released as open source in 2009. The language tries to solve today’s software engineering problems in large scale programs, such as those encountered by Google.
Go has some modern features, like automatic memory management with a garbage collector, but unlike many new languages, it has strong typing and is compiled, instead of interpreted.
2.4.1 Google’s reason for Go
Google server software is written mostly using C++ and some Java and Python. Google has millions of lines of code written by thousands of engineers. C++ is a language known for its long build time with a large code base and even with Google’s massive distributed build system, compilation of a large binary can last as long as 27 minutes.(Go at Google, 2012)
Also, when working with the same code base with dozens, maybe hundreds of people, having strict programming conventions becomes necessary. Most large software devel- opment companies programming with C++ elect to use only a subset of the language, banning the use of features deemed too complex, such as templates and operator over- loading.
2.4.2 Improving build times with packages
Because of the prevalence of C++ and C, it is often thought that long build times are inevitable with compiled languages. This is not actually true. The use of syntactically 11 defined packages and dependency declarations in Go makes it possible to read exponen- tially less data while compiling, making build times much shorter than in the C family of languages.
Package names are usually short and descriptive, but not unique. The package paths differentiate between packages with the same name. Each Go source file in the package path must begin with the package name declaration, exemplified below:
package http
Declaring dependencies in code is handled by the ”import” statement, as shown below. The ”net/http” is a unique identifier for the package, a common convention is that the last part of the identifier is the name of the package.
import"net/http"
If a package with the same name exists, the following syntax can be used to locally rename the package.
import myhttp"net/http"
When a package is imported, its exported contents can be used with the packages name, as shown in below. It is possible to export package contents directly to the local scope, but this is considered to be an extremely bad practice in normal code.
// Run net/http packages inbuiltHTTP server http.ListenAndServe(":8080", nil)
Because the package system describes all the dependencies in the program, binaries can be built using a single command, no build files are necessary. The binaries created by the Go compiler are built statically, so distribution is extremely simple, since the user just has to download the program binary for the right architecture, there is no need to install any code dependencies. 12
2.4.3 Making teamwork easier
Large software projects often have their own common programming conventions to make it easier for programmers to understand their colleague’s code. These conventions usually define the code’s stylistic guidelines, such as the placement of braces and naming of variables. They can also ban the use of some programming language features because of the added complexity is deemed to outweigh the benefits of those features.
The philosophy of Go is ”Less Is More”. In practice this means that new features are only added to the language through careful discussion with the language’s original creators and community, who try to weigh the price of the added complexity versus usefulness. This has the benefit that Go avoids the feature bloat and that there is no need to select a small subset of the language to use in a project.
Go’s syntax has been designed to be easy to parse programmatically and as such it is fairly simple to create tools for it. One such tool that comes with the standard Go package is gofmt. The gofmt tool can automatically format Go source code to a standard format, so that there is no need to agree any other code format conventions besides ”use gofmt on your code”. Since the Go community highly endorses the use of gofmt, most of open source code written in Go follows a common format, making it easier to read code written by others.
Another small, but useful feature in Go, is the fact that the visibility of functions, variables and types (”names”) inside a package is decided by the capitalization. If a name begins with a capital letter, it is visible from outside the package, if its lower case, it can only be used inside the package. This makes it easier to see the visibility of a name at a glance and also gives yet another language enforced naming convention to the programmers.
2.4.4 Reducing syntax complexity and improving safety
Even though Go’s syntax is similar to the C-family of languages, it tries to remove some of the syntactical complexity and ambiguity of those languages.
While Go supports pointers in much the same way as C and C++, it does not support pointer arithmetic and it is perfectly legal to take the address of a stack variable. These features remove opportunities to practice ”pointer magic” where arbitrary memory can be read and there is also no need to think about the situations in which you can or can not take the address of a variable.
Go also does not support using some common statements as expressions. Below are some examples of expressions that are perfectly valid in C and C++, but not in Go. 13
// Assigning2 to the indexi and then incrementingi by1 arr [i ++] = 2
// First incrementsi by1 and then assigns value2 to the incremented indexi arr [++ i] = 2
// The return statement returnsa which is2 returna=2
These expressions can result in code that is harder to understand and as such Go does not allow these expressions.
2.4.5 Lighter object orientation
Go eschews the type of object oriented programming practiced in Java and C++. While the language supports data with methods like Java and C++, Go does not support inheritance. The language designers feel that inheritance causes programmers to over engineer their early code in the fear that changing a base class later can cause massive changes throughout the code later. Inheritance can also create large hierarchies in problem domains that do not really have natural hierarchies. Go does not have classes, but since any data type, even custom integer types, can have methods structs work as replacements for classes.
While inheritance is sometimes said to be necessary for code reuse, Go’s way of reusing code is done with interfaces and a feature called embedding.
Interfaces in Go can be combined to create larger interfaces and unlike in C++ and Java, there is no need for types to declare that they satisfy an interface. All a type needs to do, is to implement the functions defined by the interface, there is no ”implements” syntax. This allows interface designers to create minimal interfaces that can be later expanded if necessary. Often Go interfaces only contain a single function. The example below shows how to implement an interface called Reader.
// This interface contains only1 function type Reader interface{ Read (b []byte) (n int, e error) }
// Implements Reader implicitly type MyType struct{ 14
contents string }
//Method declaration for MyType func (mt *MyType) Read(bytes []byte) (readn int, error err) { ...// method contents omitted }
While Python and many other dynamic languages have similar typing system called ”duck typing” it does not exactly match the one Go uses, because dynamic languages do all of their type checks at run-time. Go uses both compile-time and run-time checks when doing interface typing. The way Go handles interface typing is called structural typing.
While Go does not have true inheritance, it does support a similar feature called embed- ding. In practice it means that a structure type can contain other types, and the inner types contents become directly accessible through the outer type. The following code example shows a way to embed an Inner type to an Outer type, notice that while the Outer structs declaration contains the Inner structs name, it has no variable name for it, meaning that Inner is embedded.
//Inner type type Inner struct{ ... }
func (in Inner) PrintFoo() { ... }
type Outer struct{ Inner// Notice no variable name is set when embedding }
// Instances of Outer can use the Inners values and methods directly: outer := new(Outer) outer.PrintFoo() 15
The main difference between true inheritance and embedding, is that when a method of the inner type is called, the receiver is still the inner type. Embedding is really just an easy to use syntax obviating the need to write extra methods to access the functionality of the inner types.
2.4.6 Familiar language, modern features
While Go is reminiscent of the C-language family, it has features that are often considered to be part of new languages. Go supports closures, meaning anonymous functions that can refer values defined in the surrounding function. An example of this can be seen below:
function main() { a := 3 f := func(){ fmt.Println(a) } f()// Prints "3" }
Functions are also first class values in Go, they can be used as parameters to functions and stored in variables.
2.4.7 Easy concurrency
Modern computers often contain more than a single CPU core. Programming multi- core programs is difficult in many languages. Which is why it has features that support concurrent programming. It is possible to run functions in a lightweight process called a ”goroutine”. These goroutines can communicate with each other safely and the pro- grammer does not have to worry about setting up memory barriers and synchronization. Using goroutines can help separating parts of the program to independently executing components. Below is an example of a goroutine writing the result of a computation to a channel.
package main
import( "fmt" "math/rand" 16
)
func main() { // Createa channel to read the responses from. c := make(chan int)
// Launchesa new goroutine and // runs the function"foo" in it. go foo(c)
// Reada value from the channel, blocks // while waiting for an answer. res := <-c
// Print the result fmt.Println(res) }
// Returna random integer toa channel func foo(result chan int){ result <- rand.Int() }
Channels use the arrow operator (<-) to move data to and from them. Data flows in the direction of the arrow. If a channel has no internal buffer, like in previous code example, channel reads and writes block the goroutines. This means that channel operations can function as communication and synchronization points.
2.4.8 Why use Go for the OSLC API
FreeNEST is developed by a large group of people, most of whom are students with varying level of programming knowledge. Some FreeNEST projects use more common dynamic languages such as PHP and Python, but the nature of those languages seem to cause problems when the programs start to grow in scale. Without unit testing it is hard to be certain that a program created with a dynamic language does not contain any serious faults, since there are no compiler errors when giving wrong kind of data to a function. PHP and Python with their various frameworks and libraries can also be overwhelmingly complex, making it harder for new project members to grasp.
While Java has static typing and it is a fairly simple language, its runtime can not be distributed with FreeNEST virtual machines due to its license. There is an open imple- 17 mentation of the Java runtime called OpenJDK; however, based on previous experience, it has problems working with several libraries.
Go has static typing, it is simple, the binaries are easy to build and distribute, it has good tools and a fine standard library. The license is also a permissive BSD style license. These features make it a good language to use in a project that will have a lot of different people working on it over the years.
3 CURRENT FREENEST INTEGRATION METHODS
3.1 Top bar
The main integration method of the FreeNEST platform is a component called simply the ”top bar”. It aims to make it easier for users to navigate around the various FreeNEST tools. In practice the top bar is a navigation box located at the top of each web page in FreeNEST. It contains links to each of the installed tools, a list of users currently online, a mood meter for reporting the users current mood and the logout button. The top bar uses JavaScript injection methods to create itself in all the different tools inside the FreeNEST installation.The top bar also contains a content plugin that searches for words written in camel case (such as AdminUser), it then searches for a topic named like that in Foswiki and if it finds one, it converts the word to a link to that topic. This plugin makes it easy to link wiki topics between FreeNEST tools.
3.2 Foswiki as an integration platform
Foswiki is an open source wiki software forked from TWiki as a response to its com- mercialization. Foswiki contains a simple topic editor and it supports various plugins created by people all around the world. The FreeNEST Foswiki has several plugins for querying the various tools of FreeNEST for data, and inserting human readable format- ted lists from that data to a wiki topic. Currently there are integrations for Bugzilla, Changezilla, TestLink and Trac. The plugins combined with the ability to create ready- made templates for wiki topics make it easy to add reports and planning documents for a project.
3.3 Problems with the current integration methods
Many companies often use some of the tools contained in FreeNEST, creating their own plugins and integrations. Most often these integrations never leave the company’s 18 private servers, creating a duplication of effort. FreeNEST tries to solve that problem by having a ready made integrated package of popular project management tools and publishing it to the world.
Even though these plugins work, and they reduce the need to reinvent the wheel for companies, they are not perfect. Each tool has its own vocabulary for their respective project management domain, making it somewhat hard to provide a unified language used by all of the tools. All of the tools also store their data in different formats, some use text files, some SQL databases. There is a need for a way to unify the vocabularies and data formats, and to have access to the data inside those tools in a unified way.
4 PROPOSED INTEGRATION METHODOLOGY
FreeNEST contains several commonly used open source ALM tools and with an OSLC API implementation it could easily connect to other vendors tools improving FreeNEST’s ease of adoption. While a full implementation for all of the FreeNEST tools is a very large project, a great amount of value can be created with a FreeNEST compatible implementation for a single tool.
4.1 Service providers in FreeNEST
On the surface, implementing the service provider architecture of FreeNEST is fairly simple. There is a service provider catalog in each FreeNEST instance, which works as a ”roadmap” for all of the services provided by the instance by linking to service provider documents, each corresponding to a single tool in the FreeNEST instance.
But the modularity of FreeNEST makes the problem more difficult. Each tool component of a FreeNEST installation can be easily removed or installed, making it impossible to have hard coded definitions of supported services. That is why the OSLC implementation must be able to handle situations where only a subset of FreeNEST components are available. Luckily the tools are installed using Debian packages making it possible to query the operating system for the existence of specific components.
Since the service provider architecture makes it possible to have service provider catalogs that link to other catalogs, it is possible to have a central server that provides an API access point to all of the FreeNEST instances in a network. Using the catalog, a client could locate any FreeNEST in the network and navigate to it, making it possible to create a corporate project web that has all of the project data accessible in a common format through a single location. 19
4.2 Bugzilla as stepping stone to OSLC
Bugzilla is a bug reporting engine used by dozens of large software projects. It is also one of the most important development components in a FreeNEST installation. Most of the tools in FreeNEST use a MySQL database to store their data, and Bugzilla is no exception. That is why creating an implementation for Bugzilla could produce code that can be reused when making OSLC implementations for the other FreeNEST components. Bugzilla belongs to the Change Management domain in OSLC.
4.2.1 Bugzilla data model
Bugzilla stores its data in a MySQL relational database and it has its own vocabulary that differs from the OSLC Change Management specification. This means that there needs to be a data mapping converting queries and resources to and from the Bugzilla database. It might be easier to create an OSLC interface that handles communication with Bugzilla using its existing XML/RPC API, but that implementation could not be then used with any other FreeNEST tools, because other tools do not have that API. Most FreeNEST tools use a MySQL database and if a good implementation is made for Bugzilla, it could be used for those other tools as well.
Figure ?? shows the Bugzilla database fields that are needed to form a change manage- ment resource.
The left side of Figure ?? contains Bugzilla database fields that are needed to form the OSLC change request on the right side. The change request is presented in RDF Turtle format. Turtle documents begin with a list of prefix declarations, giving definitions for all of the ontologies used in the document.
The change request resource in Figure ?? describes a Bugzilla bug. As the figure shows, OSLC and Bugzilla use very different vocabularies for corresponding values. Care must be taken by the service implementors to make sure that meaning is not distorted in the conversion.
Since all resources must be identified by a URI, the service must provide a way to have each bug resource accessible through a URL. In this integration, the URL is formed using the identifier of a bug contained in the bug id column of the bugs table in the Bugzilla database. This is just one of the ways the pure database values have to be modified to provide a working change management service. Similar URLs have to be provided for the users whose data is shown as a part of the RDF document. Section 7 of Figure ?? shows that a single field in the database can correspond to multiple values in the document, 20
FIGURE 4. Bugzilla database to OSLC change request conversion
so bug status must be converted to and from the various status values contained in the RDF document.
4.2.2 Bugzilla OSLC queries
FreeNEST OSLC implementation should support simple queries through the OSLC query syntax, but while OSLC queries look somewhat similar to SQL queries, their underlying functionality is very different. SQL queries correspond to a specific database schema, while OSLC queries are meant for retrieving data from an RDF graph. This difference makes it fairly difficult to create an OSLC to SQL query converter. Below is an example 21 of an OSLC query and the MySQL query it has to be converted to.
The following OSLC query retrieves bug resources from a tool with the specified severity and creator: nestdemo.com/bugs?oslc.where=cm:severity="high" and dcterms:creator{foaf:accountName="[email protected]"}
It must then be converted to its corresponding MySQL query that looks something like the following:
SELECT * FROM bugs INNER JOIN profiles AS reporter ON bugs.reporter = reporter.userid WHERE bugs.bug_severity = "high" AND reporter.login_name = "[email protected]"
While the queries do look a somewhat similar, there are some differences. The most visible one between the queries is that the OSLC query does not have the concept of tables, because it is querying an RDF graph. As such, the query conversion does not only have to worry about converting the terminology from OSLC to MySQL, it also has to be able to understand the differences between the data models. After a query is converted and executed on the database, the result set must then be transformed into an RDF document.
5 RESULTS AND CONCLUSIONS
5.1 SkyNEST project implementation
The SkyNEST project at JAMK produces the Bugzilla to OSLC implementation. Even though it is not (as of 28.11.2012) complete, the results are promising. The data of each bug can be accessed through a link, queries can be made to the service and it is possible to generate a UI preview from a link. 22
5.2 Possible problems with the new integration model
While OSLC does provide some much needed consistency to the data and user experience in FreeNEST, implementing it is no simple task. Creating OSLC integration even for a single tool such as Bugzilla is a major undertaking, let alone for all of the FreeNEST components. A great deal of resources are needed to have a full implementation for the whole platform.
Since many FreeNEST tools use MySQL databases, there is a real need for extremely robust OSLC to SQL conversion techniques. Most of the solutions for that area are either proprietary or insufficient for the needs of the project. SkyNEST project has produced a converter implementation for OSLC queries that works with Bugzilla, but only time will tell if that is a sufficient solution.
Even though Google Go is a language that is well suited to this project, it is possible that some technical issues will arise due to its relative newness. Go’s garbage collector has some known memory leak problems on 32-bit systems, meaning that it can fill the memory of a computer in certain cases. This bug is being worked on by the developers, however it is not a high priority. The worst case scenario is that FreeNEST instances would have to run only on 64-bit machines if they wish to have the OSLC features. Java does not have this problem and most of the OSLC community seems to use Java as their language of choice. It is possible that Java would have been a better alternative to the project, but most of Go’s problems seem to be caused by the youth of its implementation rather than the language itself.
5.3 Conclusions
OSLC specifications make it possible to link FreeNEST to the larger, more expensive sys- tems like IBM Jazz, removing some of the potential customer’s vendor lock-in problems. While its implementation is no easy task, it is certainly a worthwhile project. It brings FreeNEST a major step closer to being a well integrated open platform for software projects.
While OSLC is meant to be used with software projects, its extensibility could make it possible to use its features and philosophy in other kinds of projects as well.
6 REFERENCES
Open Services for Lifecycle Collaboration. referenced 8.11.2012. http://en.wikipedia. org/wiki/Open Services for Lifecycle Collaboration 23
Application Lifecycle Management. referenced 23.10.2012. http://en.wikipedia.org/ wiki/Application lifecycle management
Bugzilla website. referenced 23.10.2012. http://www.bugzilla.org/
Testlink website. referenced 23.10.2012. http://www.teamst.org/index.php/news-mainmenu-2/ 1-latest/124-what-is-testlink
Foswiki website. referenced 8.11.2012. http://foswiki.org/
RDF website. referenced 30.10.2012. http://www.w3.org/RDF/
FOAF Vocabulary Specification. referenced 8.11.2012. http://xmlns.com/foaf/spec/
Go (programming language). referenced 8.11.2012. http://en.wikipedia.org/wiki/Go % 28programming language%29
Go at Google. referenced 8.11.2012. http://talks.golang.org/2012/splash.slide#3
Representational Data Transfer. referenced 20.11.2012. http://en.wikipedia.org/wiki/ Representational state transfer
Application programming interface. referenced 20.11.2012. http://en.wikipedia.org/ wiki/Application programming interface