Free and Open Source Software for Geospatial (FOSS4G) Conference Proceedings Volume 14 Portland, Oregon, USA Article 5 2014 Evaluation of Web Processing Service Frameworks M. Ebrahim Poorazizi University of Calgary (Canada) Andrew J.S. Hunter Follow this and additional works at: https://scholarworks.umass.edu/foss4g Part of the Geography Commons Recommended Citation Poorazizi, M. Ebrahim and Hunter, Andrew J.S. (2014) "Evaluation of Web Processing Service Frameworks," Free and Open Source Software for Geospatial (FOSS4G) Conference Proceedings: Vol. 14 , Article 5. DOI: https://doi.org/10.7275/R5PG1PXB Available at: https://scholarworks.umass.edu/foss4g/vol14/iss1/5 This Paper is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Free and Open Source Software for Geospatial (FOSS4G) Conference Proceedings by an authorized editor of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. Evaluation of Web Processing Service Frameworks Evaluation of Web Processing Service Frameworks by M. Ebrahim Poorazizi and Andrew J.S. Hunter publishing vector spatial data, generally encoded us- ing Geography Markup Language (GML) (Vretanos, University of Calgary (Canada). [email protected] 2002). A WCS defines a standard interface and oper- ations that enable interoperable access to spatial cov- Abstract erage (Spatial information representing space/time- As geoprocessing on the web has matured in recent varying phenomena) datasets (Evans, 2003). A WMS years, an increasing number of geoprocessing ser- delivers visualizations of data and, unlike WFS and vices and functionality are becoming available in the WCS, does not deliver the data directly (de La Beau- form of online Web Processing Services (WPS). Con- jardiere, 2004). sequently, the quality of such geoprocessing services In the context of processing services, the Open is of importance to ensure that WPS instances ful- Geospatial Consortium (OGC) has standardized the fill users’ expectations. In this paper, we illustrate, WPS interface for publishing of spatial processes, and discuss initial results from a quantitative analy- the discovery of, and binding to, those processes by sis of the performance of WPS servers. To do so, we users (Schut, 2007). A spatial process may include used two test scenarios to measure response time, re- algorithms, calculations, or various kinds of models, sponse size, throughput, and failure rate of five WPS which are exposed as a service instance, and oper- servers including 52◦ North, Deegree, GeoServer, Py- ating on spatial data. A WPS, thus, can be used to WPS, and Zoo. We also assess each WPS server in design and develop a wide variety of GIS function- terms of qualitative metrics such as software archi- alities, and be made available to users across a net- tecture, perceived ease of use, flexibility of deploy- work, as well as provide access to previously defined ment, and quality of documentation. A case study functions, calculations, or computational models. addressing accessibility assessment is used to eval- With the emergence of geoprocessing on the web, uate the relative advantages and disadvantages of the WPS specification and its (application) profiles each implementation, and point to challenges expe- have been applied to a wide array of use cases, rienced while working with these WPS servers. from accessibility assessment (Steiniger, Poorazizi, Keywords: OGC WPS; Geoprocessing; Perfor- & Hunter, 2013) to ecological modeling (Dubois, mance Evaluation; Benchmark. Schulz, Skøien, Bastin, & Peedell, 2013). The increas- ing use of WPS instances has also raised pertinent quality concerns — users/developers are likely to 1 Introduction be concerned about the Quality of Service (QoS) at- tributes such as performance, reliability, and security. With the development of geospatial services, web- The performance of a particular WPS is often of based GIS (Geographic Information Systems) have importance to users, arguably the most important, progressed towards a service-oriented paradigm when evaluating the QoS of a specific service. More- (Mayer, Stollberg, & Zipf, 2009). Today, spatial ser- over, performance has a direct effect on other QoS vices can be used to effectively support common attributes; for example, poor performance will affect tasks undertaken by spatial information users, for reliability, scalability, capacity, accuracy, accessibility, example, discovery and access to, process of, or vi- and availability (Cibulka, 2013). sualization of spatial data. Catalogue Services for A developer’s concerns, during designing and the Web (CSW), Web Feature Services (WFS), Web development of a WPS, are often twofold. As noted, Coverage Services (WCS), Web Mapping Services from a quantitative perspective, performance is one (WMS), and WPS are common services defined by of the key principles that can ensure both user and the OWS (Open Geospatial Consortium Web Ser- application developer satisfaction. From a qualita- vice) initiative. A CSW provides the ability to pub- tive point of view, quality concerns such as software lish and search collections of descriptive informa- architecture, perceived ease of use, flexibility of de- tion (metadata) (Solntseff & Yezerski, 1974) for spa- ployment, quality of documentation, and support ac- tial data and services (Nebert, Whiteside, & Vretanos, cessibility are important factors that can guide devel- 2007). A WFS is the main geospatial service for opers during selection of a WPS framework that fits OSGEO Journal Volume 14 Page 29 of 48 Evaluation of Web Processing Service Frameworks a particular application domain best. WPS, and Zoo – using two test plans in an accessibil- Several reviews have been reported in the litera- ity assessment scenario. To do so, the WalkYourPlace ture that evaluates spatial services from both a quan- Transit Model (Steiniger et al., 2013) was used to de- titative and qualitative perspectives. MapServer’s sign the geoprocessing workflow. The workflow was WMS has been assessed and optimized by Kalbere then developed using Python and wrapped and ex- (2010). COSMC (Czech Office for Surveying, Map- posed as a standard WPS using the candidate WPS ping and Cadastre) and CENIA (Czech Environmen- servers. The sample locations were selected using tal Information Agency) WMSs have been tested a stratified random sampling approach within the for availability and performance (Horák, Ardielli, bounds of the City of Calgary, Alberta, Canada. Dur- & Horáková, 2009). Bermudez et al. (2009) com- ing experiments we controlled the number of concur- pared the ability of WFS and SOS (Sensor Observa- rent requests, and the WPS input parameters to as- tion Service) to publish time series data. Tamayo sess the performance and load capacity of the WPS et al. (2011) presented an empirical study of in- servers. The remainder of the paper is structured stances of servers implementing SOS in terms of as follows. Section two introduces the WPS speci- compliance with OGC’s SWE (Sensor Web Enable- fication. The specification of candidate WPS servers ment) and interoperability, and in our previous work is described in section three. Section four explains we evaluated performance of three SOS servers – 52◦ the methodology used to evaluate the WPS servers, North, MapServer, and Deegree – based on differ- along with a description of the case study, technical ent test scenarios (Poorazizi, Liang, & Hunter, 2012). architecture, test scenarios, and hardware configura- Moreover, a WMS performance shootout has been tion of the servers used. Section five presents the re- presented annually since 2007 at the FOSS4G (Free sult. In section six, the WPS servers are assessed in and Open Source Software for Geospatial) confer- terms of qualitative metrics. Section seven summa- ence, which provides a standardized procedure for rizes our findings. measuring and comparing the performance of WMS server installations (http://wiki.osgeo.org/wiki/ FOSS4G_Benchmark). 2 Web Processing Service Within the geoprocessing domain, there have been few attempts to evaluate WPS servers. Scholten The OGC released version 1.0.0 of the WPS specifi- et al. (2006) investigated efficiency of web ser- cation in June 2007 (Schut, 2007). The specification, vices for geoprocessing in a Spatial Data Infras- along with the OGC Web Processing Service Best tructure (SDI), but focused on caching, network Practice discussion paper, describe a web service in- adaptation, data granularity, and communication terface that defines how a client and server should modes. Michaelis and Ames (2009) evaluated the cooperate during the execution of a spatial analysis, WPS 0.4.0 specification, identified challenges, and and how results of the process should be presented proposed potential enhancements from an imple- (Schäffer, 2012). Clients can send requests via three mentation perspective. In addition, a WPS shootout core operations using three methods: Key Value was presented at the FOSS4G conference 2011, which Pairs (KVP) encoding via HTTP’s (HyperText Trans- evaluated five WPS servers, 52◦ North, Deegree, fer Protocol) GET, XML (eXtensible Markup Lan- GeoServer, PyWPS, and Zoo, in terms of compli- guage) via HTTP’s POST, or a SOAP/WSDL (Sim- ance with OGC’s WPS, and interoperability (http: ple Object Access Protocol/Web Service Description //wiki.osgeo.org/wiki/WPS_Shootout). The main Language) approach. The WPS specification defines achievement of the aforementioned works is that three mandatory