A Restful Middleware Application for Storing JSON Data
Total Page:16
File Type:pdf, Size:1020Kb
School of Computer Science Third Year Project Report A RESTful Middleware Application for Storing JSON Data Author: Alexandru Aanei Supervisor: Prof. Norman Paton A report submitted in partial fulfillment of the requirements for the degree of BSc (Hons) Computer Science with Industrial Experience May 2016 Abstract Over the last decade, Representational State Transfer, commonly known as REST, has become a popular style for designing the APIs of distributed applications which integrate with the World Wide Web. REST has also been adopted by data storage solutions, such as Document-oriented databases, to expose interfaces which benefit from the advantages of HTTP. Such solutions, however, do not fully respect the REST architectural style. This report presents two main issues identified in the HTTP APIs of data storage so- lutions, which prevent them from being entirely RESTful. Consequently, a design for a RESTful interface which aims to solve those issues is proposed. The report details how the interface was implemented in an application which uses an embedded database to allow clients to store and retrieve data. The functional evaluation of the interface reveals that the design successfully supports common data storage operations, but has limited querying functionality. Furthermore, the performance evaluation indicates that the application performs satisfactorily in most scenarios. Based on the limitations identified, directions for future work are provided in the conclusion. Acknowledgements First of all, I would like to thank my supervisor, Prof. Norman Paton, for all his support throughout the project, especially for helping me organise my ideas and for showing me how to develop and sustain an argument. I would also like to thank my family and friends for the help, support and words of encouragement offered so far. 1 Contents 1 Context 7 1.1 JSON . 7 1.2 The REST Architectural Style . 8 1.3 REST Specifications . 10 1.4 REST Interfaces for Data Storage . 11 1.5 Report Outline . 11 2 Motivations 12 2.1 RPC-style Endpoints . 12 2.2 No Use of Hypermedia . 15 2.3 Conclusion . 20 3 Design of the Interface 21 3.1 Choice of Specification . 21 3.2 JSON:API Extensions For Data Storage . 22 3.2.1 Root Endpoint . 22 3.2.2 ID Generation Strategy . 22 3.2.3 Pagination Strategy . 23 3.2.4 Filtering Strategy . 23 3.2.5 Entity Tags . 23 3.3 Design Solutions to the Existing Problems . 24 3.3.1 Hypermedia Support . 25 3.3.2 Relationship Modelling . 28 3.3.3 RPC-like Endpoints . 30 3.4 Conclusion . 31 4 Implementation 32 4.1 Implementation Overview . 32 4.2 Choice of Programming Language . 33 4.3 Database Choice . 33 4.4 Architectural Overview . 34 4.5 Testing . 37 4.6 Conclusion . 38 5 Evaluation 40 5.1 Functional Evaluation . 40 5.1.1 Purpose . 40 5.1.2 Method . 40 2 5.1.3 Findings and Results . 41 5.2 Performance Evaluation . 42 5.2.1 Purpose . 42 5.2.2 Method . 43 5.2.3 Findings and Results . 43 5.3 Conclusion . 46 6 Conclusions 47 6.1 Future Work . 47 A Interface Documentation 56 A.1 URI Schemes . 56 A.2 Document Structure . 57 A.2.1 Top - level . 57 A.2.2 Resource Objects . 57 A.2.3 Attributes . 58 A.2.4 Relationships . 59 A.2.5 Member Names . 59 A.3 Errors . 59 A.4 Fetching Data . 60 A.4.1 Fetching Available Collections . 60 A.4.2 Fetching Collections . 61 A.4.3 Fetching Individual Resources . 63 A.4.4 Fetching Relationships . 64 A.5 Querying Supported . 65 A.5.1 Inclusion of related resources . 66 A.5.2 Sparse Fieldsets . 66 A.5.3 Sorting . 66 A.5.4 Paging . 66 A.5.5 Filtering . 67 A.6 Creating Resources . 67 A.7 Updating Resources . 69 A.8 Updating Relationships . 71 A.9 Deleting Resources . 74 A.10 HEAD and OPTIONS . 75 A.11 A Full JSON:API Document . 76 3 List of Figures 1.1 High-level mode of operation of the application . 7 4.1 The application running in a terminal . 33 4.2 System context view of the interface . 34 4.3 The main components of the implementation . 35 4.4 Complete architectural view of the interface . 36 4.5 Screenshot of an end-to-end test running in Postman . 39 5.1 Screenshot of the blog web application used for functional evaluation . 41 5.2 Plot showing the average response time for resource fetching, in milliseconds 44 5.3 Plot showing the average response time for resource creation, in seconds 44 5.4 Plot showing the average response time for collection fetching, in seconds 45 4 List of Listings 1 Example of JSON document . 8 2 ElasticSearch search HTTP request using query parameters . 13 3 ElasticSearch search HTTP request using the Query DSL . 13 4 ElasticSearch update request for a stored document . 14 5 CouchDB /db/_missing_revs endpoint usage example . 15 6 Fetching a list of collections in Kinto . 16 7 Application-side joins in ElasticSearch . 17 8 Nested objects in ElasticSearch . 18 9 Inline objects in CouchDB . 18 10 Joins in CouchDB . 19 11 Fetching all available collections . 25 12 Fetching the articles collection . 26 13 Fetching a collection of tags related to an article . 27 14 Fetching the tags relationship . 28 15 Adding a tag to an article . 29 16 Removing tags from an article . 30 17 SQL query for fetching the latest posts of the blog . 42 18 Request used for profiling . 45 19 Example of a resource object . 58 20 Example of an error response . 60 21 Root endpoint GET request . 60 22 Fetching a Collection . 62 23 Fetching an individual resource . 63 24 A conditional GET request . 64 25 Fetching a relationship . 65 26 Creating a new resource . 68 27 Creating a new collection . 69 28 Updating an existing resource . 70 29 Conflicted update request . 71 30 Complete update request for a relationship . 72 31 Adding members to a relationship . 73 32 Removing members from a relationship . 74 33 Deleting an individual resource . 75 34 Example of a HEAD request . 75 35 Example of an OPTIONS request . 76 5 List of Tables 1.1 Example usage of HTTP methods and URI schemes in a REST API . 10 A.1 URI schemes of the interface . 56 6 Chapter 1 Context The scope of this project is to develop an application which exposes a RESTful interface for data storage. The application itself acts as a middleware between a client and a database, therefore it.