Open Government Data in Kosovo and Crowdsourcing

Home , Open data, Open government

International Journal of Civil Engineering and Technology (IJCIET) Volume 9, Issue 4, April 2018, pp. 1333–1343, Article ID: IJCIET_09_04_150 Available online at http://iaeme.com/Home/issue/IJCIET?Volume=9&Issue=4 ISSN Print: 0976-6308 and ISSN Online: 0976-6316

OPEN GOVERNMENT DATA IN KOSOVO AND CROWDSOURCING

Isak Shabani and Qazim Tairi Faculty of Electrical and Computer Engineering, University of Prishtina “Hasan Prishtina”, Prishtina, Kosovo

ABSTRACT With the development of information technologies, open data released by central and local governments and crowdsourcing means are emerging in Kosovo. Technological equipment, mobile devices are inseparable from the daily use, and their capabilities continue to advance rapidly. Different Application, internet access almost everywhere has impact the way of storing information and communicating but also the fundamental nature of social interactions. The period of innovation and creativity has made many traditional business models both in production and in distribution have to transformed, or if they do not respond the transformation, their business may have failed in a short time. Large institutions, serving society for generations, are challenged to keep pace as we evolve as a digital society. Education, health care, defense, and public safety are at the forefront, but all governments are challenged to balance the traditions and values they were designed to uphold while adapting to lead in a digital environment. In this paper, we will explain the importance of open government and crowdsourcing, and examines the role that e-government plays in driving the open data phenomenon. Online and open government's presence makes services and standards for supporting helps governments to understand better the needs, wishes and beliefs of citizens, and create better and more socially rooted policies. Keywords: e-Government, e-Participation, Open government, Open information, Open dialog, Crowdsourcing, Open data Cite this Article: Isak Shabani and Qazim Tairi, Open Government Data in Kosovo and Crowdsourcing, International Journal of Civil Engineering and Technology, 9(4), 2018, pp. 1333–1343. http://iaeme.com/Home/issue/IJCIET?Volume=9&Issue=4

1. INTRODUCTION Nowadays the society has changed a lot. The epoch of information and technology has made the life easier. For long time it has been widely recognized that representative democracy, in which citizens representatives identify needs and problems of society and design public policies and services for addressing them, should be combined with continuous citizens involvement. This is based on an understanding of the growing complexity and multi-

http://iaeme.com/Home/journal/IJCIET 1333 [email protected] Open Government Data in Kosovo and Crowdsourcing dimensionality of social problems and needs, which necessitates the exploitation of citizens extensive knowledge on them. This gave rise to a new model of democracy, which is termed “participatory democracy”, and combines decision making by citizen’s representatives with citizen’s participation, with the latter not replacing but supporting and enhancing the former [3]. Many governments all over the world have shown consistent interest in and made considerable efforts for enabling the participation of citizens in the formulation of public policies through various mechanisms. Three main motivations of government for this: • To produce better quality policies (proportionality and fairness), • Build trust and gain acceptance of policy-making processes (awareness and consensus-building) and • To share responsibility for policy-making (inclusion and participation). The development and increasing penetration of information and communication technologies (ICT), and the Internet, provides an efficient and inclusive channel for the extensive application of the above principles. This gave rise to the development of electronic participation (e-participation). The first generation of e-participation was based on the development and operation of official government e-participation spaces, in which electronic discussions of various levels of structure were, conducted [3]. Government agencies are increasingly interested in exploiting the capabilities provided by the information and communication technologies (ICT), and especially the Internet, for engaging the citizens in their public policymaking processes, and this has lead to the gradual development of the e-participation domain. This offers to governments highly important capabilities to collect valuable information, knowledge, ideas and opinions from large number of citizens on the complex problems and needs of modern societies, which can be quite useful for designing better quality policies, and, at the same time, to build trust and gain more acceptances of their policies [5]. It was highly government-controlled, as the owner government agencies defined and controlled the topics and rules of these electronic discussions taking place there. For the reason that the use of these official e-participation spaces by the citizens has been, in general, limited, the emergence of the new Web 2.0 enables the gradual emergence of a second generation of e-participation, taking place in popular social media where citizens choose to discuss and generate content, which is characterized by less government control and more citizens control [3, 5]. In this new paradigm of e-participation government agencies post content (e.g. short or longer text, images, video) to various social media on their policies under formulation or implementation, and then collect and analyze citizens interactions with it (e.g. views, comments, likes/dislikes, retransmissions, etc.). Therefore, in this second generation of e- participation governments continue defining and controlling the discussion topics to some extent (through making postings and initiating discussions on the topics they choose), but do not control the rules of the discussion (which are the ones defined by the social media) [3, 5, 7]. Governments produce huge amounts of information that are only now being recognized as national assets. Public Sector Information (PSI) plays a fundamental role in the provision of public services. Over the past decade, governments have launched policy initiatives to promote the reuse of PSI—from developing open license models to establishing regulatory frameworks and making “open data” available on government websites. Companies that tap into this information can apply newfound knowledge to better understand customers, create products and services to meet demand, and discover new markets. Governments have the

http://iaeme.com/Home/journal/IJCIET 1334 [email protected] Isak Shabani and Qazim Tairi opportunity to use this information to improve transparency, efficiency, and service delivery to citizens [1].

Figure 1 Open Government: Empowerment through information access In this paper we will describe the convergence of PSI and technology, and examines the role that e-Government plays in driving the open data phenomenon.

2. OPEN GOVERNMENT Open government is a principle that gives citizens the right to access the documents, data, and proceedings of government to allow for greater transparency. It reflects the necessary balance between freedom of information and protection of privacy legislation in many jurisdictions [1]. In an information economy, data is the raw material for new products and services. Governments around the world have articulated their commitment to facilitating a more open approach to governing. The Open Government Partnership (OGP) was launched in September 2011, when the eight founding governments (Brazil, Indonesia, Mexico, Norway, Philippines, South Africa, United Kingdom, and the United States) endorsed an Open Government Declaration. The declaration outlines a voluntary, multilateral mandate to “promote transparency, empower citizens, fight corruption, and harness new technologies to strengthen governance”. The OGP is a dynamic global movement that promotes open government, free dialogue, and sharing among governments, civil society, and private sector organizations. President Obama and the White House were also willing and supportive about the Open Government. President Obama about this issue said “My Administration is committed to creating an unprecedented level of openness in Government. We will work together to ensure the public trust and establish a system of transparency, public participation, and collaboration. Openness will strengthen our democracy and promote efficiency and effectiveness in Government.” One of the countries most committed to open government is Canada. In this point, we will take as a model country, with the goal to effort to foster greater openness and accountability, to provide Canadians with more opportunities to learn about and participate in government, to drive innovation and economic opportunities for all Canadians and, at the same time, to create a more cost-effective, efficient, and responsive government. Canada’s approach to open government is described as a “three-legged stool” based on Open Data, Open Information, and Open Dialogue.

2.1. Open Data Open data is information that is accessible, available in digital machine-readable format, and reusable under open license terms. The full Open Definition breaks this down into the following characteristics [9]:

http://iaeme.com/Home/journal/IJCIET 1335 [email protected] Open Government Data in Kosovo and Crowdsourcing

• Availability and access: Data should be available in readable format, preferably by download over the Internet. • Reuse and redistribution: Data should be available under terms that permit reuse and redistribution. • Universal participation: Everyone should be able to use, reuse, and redistribute data without discrimination. The importance of Open Data stands on the economy. In 2013, The European Commission predicts that the open data revolution could bring as much as $55 billion in economic benefit a year to the continent alone [10]. There are many kinds of open data that have potential uses and applications [12]: Culture: Data about cultural works and artefacts for example titles and authors and generally collected and held by galleries, libraries, archives and museums. Science: Data that is produced as part of scientific research from astronomy to zoology. Finance: Data such as government accounts (expenditure and revenue) and information on financial markets (stocks, shares, bonds etc.). Statistics: Data produced by statistical offices such as the census and key socioeconomic indicators. Weather: The many types of information used to understand and predict the weather and climate. Environment: Information related to the natural environment such presence and level of pollutants, the quality and rivers and seas.

2.1.1. Open Data in Kosovo Kosovo is the youngest country in Europe, and has start give many attentions the open data, with the aim of facilitating the work, and make the life easier for its citizens. Data Kosovo is a nonprofit organization that believes in using civic-tech and digital humanitarianism to open government. This initiative promotes the idea that governance data should be made freely available for everyone to use and republished as. They wish, without restrictions from copyright, patents or other mechanisms of control. Contributes to the development of a local and vibrant tech scene. Kosovo has made a big leap into e-government and some of the platforms are: • Open Government Data Portal: The Open Government Data Portal is developed by Open Data Kosovo to serve as a tool for government officials to enable transparency through the publication of government datasets [11]. • Amnesty Education Platforms: An online micro-learning platform providing information and case studies on human rights defenders and a complimentary targeted proposal for an engaging and informative social media micro-learning campaign [11]. • Open Businesses: Open Businesses is a business registration search engine with data from over 170,000 businesses in Kosovo. The data was scraped from the Kosovo Business Registration Agency (ARBK). Opening business data allows economic analysis of sectors as well as various economic activities. It creates space for analyzing economic trends in different periods and regions. It creates space for market analysis by local and international investors. Opening these business data enables data linking with other sectors such as public procurement or property declaration, which would strengthen anti-corruption mechanisms [11].

http://iaeme.com/Home/journal/IJCIET 1336 [email protected] Isak Shabani and Qazim Tairi

• e-Procurement: A platform that visualizes procurement data from local municipalities and automatically detects irregularities through a contextual and custom built red flagging algorithm [11]. • Election monitoring visualizer: A prototype data visualization platform that presents election monitoring data collected by election observers from the Democracy in Action (DiA) initiative. The platform displays election ballots and their observed irregularities, offering disaggregation by hour, gender, municipality, and polling station [11]. • e-Municipality: The portal is dedicated to natural and legal persons in the territory of the Municipality of Prishtina. Through this portal, you can be informed which documents you need to prepare beforehand before applying for any documents that you can get at the counters of the Municipality of Prishtina. Figure 2 shows Kosovo ranked in Global Open Data Index [12]:

Figure 2 Kosovo in Global Open Data Index [12]

2.1.2. Open Information and Open Data Portal in Kosovo Open Information makes access to online information requests faster and easier. Open information consists: • Modernizing the Administration of ATI , • Virtual Library, • International Aid Transparency Initiative, • Advancing Recordkeeping and • User-Centric Web Services. Currently, the Open Data portal in Kosovo contains 62 sets of data, 17 organizations and 5 groups [13].

2.2. Open Dialogue Open Dialogue advantages tools like social media to engage citizens in policy development. Open dialogue consists: • Consulting citizens and • Open regulation.

http://iaeme.com/Home/journal/IJCIET 1337 [email protected] Open Government Data in Kosovo and Crowdsourcing

3. CROWDSOURCING Crowdsourcing is an online, distributed problem-solving and production model that has grown in use in the past decade. It has been gaining traction as a public participation tool for governance and planning, as well as a method for building common resources or processing large batches of data to streamline government functions [2]. Crowdsourcing is a great example of democracy in action, as enabled by e-government technologies. The use of crowdsourcing has created new opportunities for citizens to engage with governments at all levels [1]. The capability of a large network of people, termed as ‘crowd’, networked through web technologies, to perform difficult design and problem-solving activities is gradually recognized by management researchers and practitioners. This is referred to as ‘collective intelligence’, which is defined as a ‘form of universally distributed intelligence, constantly enhanced, coordinated in real time, and resulting in the effective mobilization of skills’ [3]. Once open data is available, the network effect takes hold and data becomes increasingly more valuable as more people contribute and participate. Some crowdsourcing examples are: Oxford English Dictionary OED or Wikipedia. A common and widely used definition stands as follows: • Crowdsourcing is a type of participative online activity in which an individual, an institution, a non-profit organization, or company proposes to a group of individuals of varying knowledge, heterogeneity, and number, via a flexible open call, the voluntary undertaking of a task [2]. The crowd should participate bringing their work, money, knowledge and/or experience, always entails mutual benefit. The user will receive the satisfaction of a given type of need, be it economic, social recognition, self-esteem, or the development of individual skills, while the crowdsourcer will obtain and utilize to their advantage what the user has brought to the venture, whose form will depend on the type of activity undertaken [2]. • When an organization embarks on a crowdsourcing venture, it is important to consider first the kind of problem it wants to solve and the kinds of solutions it wants to receive. Only then should the organization consider the tactical means for executing that kind of arrangement [2]. Crowdsourcing is a strategic approach to problem-solving, while specific tools (e.g., websites, social media, and wikis) are tactical implementations of strategies. Crowdsourcing can be accomplished through any number of new media tools, including wikis, blogs, websites, social networking sites (e.g., Facebook, Twitter), mobile apps, mapping software, and so on. Many tools enable communication, and so many tools can make crowdsourcing possible [2].

3.1. Four Approaches to Crowdsourcing A typical example in this direction is the study by Brabham at [2], which, based on the analysis of several case studies, identifies four dominant crowdsourcing approaches: • The knowledge discovery and management approach (i.e. an organization tasks crowd with finding and reporting information and knowledge on a particular topic), • The broadcast search approach (i.e. an organization tries to find somebody who has experience with solving a rather narrow and rare empirical problem), • The peer-vetted creative production approach (i.e. an organization tasks crowd with creating and selecting creative ideas)

http://iaeme.com/Home/journal/IJCIET 1338 [email protected] Isak Shabani and Qazim Tairi

The distributed human intelligence tasking (i.e. an organization tasks crowd with analyzing large amounts of information)

3.2. Best Practices for Executing Crowdsourcing Applications As crowdsourcing, cases have proliferated in business and the public sector and as empirical research on these cases comes to light, scholars and practitioners of this area have identified several common principles or best practices [2]. Brabham presented 10 best practices by categorizing them in 3 steps: • The Planning Phase  Clearly define the problem and solution parameters  Determine the level of commitment to the outcomes  Know the online community and their motivations • The Implementation Phase  Invest in usable, stimulating, well-designed tools  Craft policies that consider the legal needs of the organization and the online community  Lauh a promotional plan and a plan to grow and sustain the community  Be honest, transparent, and responsive  Be involved, but share control • The Post-Implementation Phase  Acknowledge users and follow through on obligations  Assess the project from many angles. 4. METHODOLOGY Governments around the world are establishing more consistent information management structures, standardizing the way this data is stored and presented, and endorsing the publication of federal data in machine-readable formats. The methodology purposed by Charalabidis, Triantafillou, Karkaletsis, Loukis includes four stages (Figure 3), and which enable the policy-makers users of it:

Figure 3 Four stage methodology for e-government The first stage Listen, includes listening and monitoring what people say, what their needs, their opinions, positions and proposals are on a topic (e.g. a public policy under formulation) defined by the policy-maker. For this purpose, a focused crawler is necessary, that is, a program which browses the Web in an automated, organized fashion, and attempts to visit solely content that is relevant to a previously known topic (or set of topics) [3].

http://iaeme.com/Home/journal/IJCIET 1339 [email protected] Open Government Data in Kosovo and Crowdsourcing

The second stage Analyze, includes advanced processing and analysis of the content, from which are identified relevant citizens’ needs, opinions, concerns, proposals, sentiments and other information hidden within the citizens conversations. As most of the available information is in textual form, the processes related to this stage will mainly make use of linguistic techniques. Each web page found by the crawler in the previous stage will go through a series of automated analysis processes: Language Detection, which will recognize the language used in the page. Opinion and Argument Extraction, with the use of the appropriate semantic similarity measures and inference mechanisms that will allow the identification of analyzed content that is pertinent to the policy Sentiment Analysis, using smart sentiment classifiers analyzes the mentions, which recognize the polarity (positive, neutral, negative) of the elements identified above. Argument Summarization, with the use of the appropriate algorithms for generating qualitative information about opposing arguments, in the form of anonymity-preserving and automatically-generated summaries The Receive stage aims to present to the end-user (policy maker) the knowledge acquired from the previous phases in a complete, coherent and usable manner. The platform will provide an aggregated view of the discovered opinions, their polarity, their association with policy concepts and statements, and statistical indications of their significance and impact [3]. The three stages above perform a ‘passive crowdsourcing’, where the policy maker collects and analyses content which has been created without his/her intervention or moderation. In the fourth stage, ACT, the policy maker uses the collected cluster of problems, needs and proposals to formulate a draft-policy agenda, which can be tested out against social opinion. This is going to be achieved by ‘active crowdsourcing’ where policy text is posted to various social media (e.g. blogs, Twitter, Facebook, YouTube, etc.), soliciting citizens remarks, opinions, positions and proposals on it [3]. 5. NEXT STEPS E-GOVERNMENT AND CROWDSOURCING Government workers must be convinced that crowdsourcing is an outlet to help them get their jobs done. They need to work with the crowd, not consider interacting with the crowd as an additional task. This does not mean applying crowdsourcing to every task. Tasks that can benefit from crowdsourcing require diverse expertise, creativity, and varying experiences [6]. However, participants should be discouraged from putting in information that is out of context or not integrated with the rest of the material in a module, section or other component of the final product. A key requirement to keep participation high is to ensure that user input is considered. A moderator is needed to provide feedback on the status of an idea, why an idea was rejected, and the results of accepted ideas. Awards for accepted ideas might motivate continued participation. Although monetary awards might be enticing, even more effective might be inclusion and collaboration in the idea implementation. Overall, crowdsourcing could be used at any stage of a project or move through them all. Specifically, it can be used to generate and prioritize ideas, plan for implementation of selected ideas, support implementation, design and development, and support promotion of the final product through increased access to information, and new products and services that can be co-created with citizens [6].

http://iaeme.com/Home/journal/IJCIET 1340 [email protected] Isak Shabani and Qazim Tairi

5.1. Some successful characteristics of crowdsourcing Many are skeptical of the use of crowdsourcing in general pointing to the lack of verity in the produced work [6]. To counteract these criticisms, the most successful e-government crowdsourcing projects have the following characteristics: • Moderated, there is a moderator and feedback. Someone responds to posts and provides feedback including results of the input. • Assessed, there are features that help the best ideas (as identified by the participants) be easily seen. Typically, this is done through voting. Voting can be on many levels – usefulness, relevance, accuracy, etc. Combined scores incorporating the various criteria can also be calculated using scoring or the analytical hierarchy. • Organized, there are linkages between related posts. This is often enabled through participants annotating the posts with tags. Tags should be at the individual post level even in a discussion so that they are specific and relevant. • Promoted, the crowdsourcing project is well known by those who could provide significant input. This means it must be advertised and those experts should be encouraged to participate. • Rewarded, there is some reward or motivational factor for participating. This might be recognition, nonmonetary award or monetary reward. • Managed, in some cases, participation should be limited. Although no one knows where the next best idea or contribution will come from, participation by the uninformed or by those without expertise might slow the process down. Coordination costs should not exceed results value and efficiency benefits. Depending upon the goals of the crowdsourcing project, different characteristics might be emphasized [6]. 6. POLITICAL INFLUENCE How to build an efficient, transparent, and democratic government is an enduring topic? However, the level of public trust in the government is falling, in both democratic and undemocratic countries. The government must find a right way to improve the efficiency and transparency. In this context, E-government came into being [8]. According to the definition proposed by the UN and American Society for Public Administration (ASPA), E-government is utilizing the Internet for delivering government information and services to citizens [8]. Organization for Economic Cooperation and Development (OECD) gave another illustration, which is using ICTs and particularly the Internet as a tool to achieve better government [8]. Based on these definitions, we find that the essential purpose of E-government is improving the efficiency and transparency of government through offering official information and online service [8]. In the past, people almost had no access to official data, including financial budget and the process of policy-making. This situation means that public employees could escape from supervision to some extent. The development of E-government, however, creates a new way for citizens to get close to relevant materials including financial budget and policy- making process. As a result, the situation of corruption will fall [8]. In the beginning, the primary purpose of E-government was offering online data so that ordinary people could become more knowledgeable about government and public affairs [8]. Then, many online services were supplied by the government, which aim to facilitate people’s daily life. With the help of E-government system, people can accomplish many things online, such as paying tax, submitting documents and so on, which had to be done offline in the past

http://iaeme.com/Home/journal/IJCIET 1341 [email protected] Open Government Data in Kosovo and Crowdsourcing

[8]. These two functions, open data, and online service, have much common ground to some extents [8]. Political participation was the new functional development of E-governance. Online participation or E-participation means citizens can express their opinions, take part in the public issues discussion and even monitor government and officers through online platforms [8]. E-governance, without a doubt, offers citizens an appropriate and workable approach to achieve their political rights. As a result, this function is also called E-democracy. Overall, the functions of E-governance can be divided into three aspects, which are: open data, online service, and E-participation [8]. In the past, political participation was hard to achieve because of the lack of suitable approaches. The E-government system offers a new approach for citizens to take part in public issues [8]. In the online platform, citizens can express their opinions about public affairs and even monitor the process of policy-making [8]. The strategy of the new mayor recognized that increased transparency and increased citizen input, both through structured surveys and open-ended comments, could leverage several positive outcomes [8]. These advantages included access to useful information on citizen demands, tools for managing relationships with citizens and the legislature, and increased information about each other [8]. These benefits, plus the obvious incentives these systems provided, helped reduce resistance to change and made the bureaucracy a bit more willing to respond to the mayor’s needs [8].

7. CONCLUSION This study contributes on Open data in Kosovo e-Government and Crowdsourcing by helping us identify aspects of new technology that are likely to affect citizen participation and democratic institutions. Around the world, governments have been invested in crowdsourcing. To investigate the effectiveness of applying crowdsourcing to the data understanding phase, we conducted experiments using public datasets provided by the government of Kosovo. This paper provides an overview of crowdsourcing and how it works on government context also hopes to inspire future crowdsourcing in government, offering four approaches and best practices based on problem type. It can be an extremely useful resource for government. It supports transparency, promotes participation and has the potential to bring together wide areas of expertise to allow for real collaboration. The first generation of e- participation has been based on the construction of official websites of government agencies, which provided to the citizens’ information on their activities and policies, and capabilities for participation in consultations; therefore, it was highly government-controlled and had a top- down orientation. This second generation of e-participation is less government-controlled and more citizens-controlled (as governments continue controlling the discussion topics to some extent - through making postings and initiating discussions on the topics they choose - but do not control the rules of the discussion - which are the ones defined by the social media). For government, it is time to move onto the next steps and encourage participation, show results and develop the next stages of even more effective crowdsourcing projects. Finally, based on analysis of outcomes, theoretical and practical arguments, stage the address quality issues with aim of improvement Open Government Data. The benefits of this paper are multiple, starting with promotion of Open Government Data and pushing the governments of respective countries to publish more public data with the aim of increasing of accountability and transparency. In addition, the evaluation process may have multiple impact pointing out the problems of public administration in their processes, finding the main factors of causing poor Open Government Data and crowdsourcing whether in production or in publication process.

http://iaeme.com/Home/journal/IJCIET 1342 [email protected] Isak Shabani and Qazim Tairi

REFERENCES

[1] M. J. Barrenechea, T. Jenkins, “E-Government or Out of Government,” Open Text Corporation, Canada, 1st ed, chp 12, April 2014. [2] D. C. Brabhan, L. V. Feller, “Using Crowdsourcing in Government”, Collaboration across Boundaries Series, IBM, 2013. [3] Y. Charalabidis, A. Triantafillou, V. Karkaletsis, E. Loukis, “ Public Polisy Formulaton through Non Moderated Crowdsourcing in Social Media” International Conference on Eletronic, Part of LNCS book series, vol. 7444, Springer, Berlin, Heidelberg, 2012, pp. 156-169. [4] C. Lampe, P. Zube, J. Lee, Ch. H. Park, E. Johnston, “Crowdsourcing civility: A natural experiment examining the effects od distributed moderation in online forums”, Government Information Quarterly, vol. 31, ScienceDirect, 2014, pp. 317-326. [5] Y. Charlabidis, E. N. Loukis, A. Androutsopoulou, V. Karkaletsis, A. Triantafillou, “Passive crowdsurcing in government using social media”, Transforming Government: People, Process, and Policy, vol. 8 Issue 2, pp. 283-308, 2014. [6] J. Warner, “Next steps in e-government crowdsourcing”, Proceedings of the 12th Annual International Digital Government Research Conference: Digital Government Innovation in Challenging Times, pp. 177- 181, ACM, USA, 2011. [7] L. Spiliotopoulou, Y. Charalabidis, E. N. Loukis, V. Diamantopoulou. “A framework for advanced social media exploitation in government for crowdsourcing”, Transforming Government: People, Process, and Policy, vol. 8 Issue 4, pp. 545-568, 2014. [8] M. J. Ahn, S. Bretschneider, “Politics of E-Government: E-Government and the Political Control of Bureaucrracy”, The American Society for Public Administration, vol 71 Issue 3, June, 2011, pp. 414-424, Wiley. [9] This link was accessible on 12/10/2017, http://opendefinition.org/od/2.1/en/ [10] McKinsey Global Institute, "Open data: Unlocking innovation and performance with liquid information", MGI, October 2013. [11] This link was accessible on 12/10/2017 http://opendatakosovo.org [12] Global Open Data, https://okfn.org/opendata, this link was accessible on 15/10/2017 [13] Open Data Portal in Kosovo, http://opendata.rks-gov.net, this link was accessible on 20/10/2017 [14] Himanshu Pandey, Dr. Santosh Kumar, Waseem Ahmad and Dr. V.K Singh, Proposed Methodology for Crowdsourcing and Agile Development. International Journal of Advanced Research in Engineering and Technology, 9(2), 2018, pp 68–76.

http://iaeme.com/Home/journal/IJCIET 1343 [email protected]