Building Highly-Interactive, Data-Intensive, REST Applications: The Invenio Experience Michelle Annett, Eleni Stroulia {mkannett,stroulia}@cs.ualberta.ca Department of Computing Science University of Alberta these changes in Internet user’s usage preferences Abstract and practices, it should come as no surprise that the number of applications utilizing Web 2.0 With the explosion of Web 2.0 ideas and technol- technologies and ideas such as REST, “mashups” ogies such as XML, REST, and RIAs (Rich Inter- and Rich Internet Application (RIA) frameworks net Applications), developers are now creating has dramatically increased. “mashup” applications that aggregate numerous Of these ideas, mashups and RIAs are proba- sources of information and promote rich user inte- bly foremost in the minds of users. Mashups are raction. Although many innovative mashups are web applications that aggregate multiple data being created, there has been little research sys- sources into one application. Although different in tematically examining which technologies to use many ways, most mashup applications contain the and how to design and implement such applica- following three elements: (a) one or more data tions. We describe the features and complexity sources, (b) a website to display an interpretation inherent within a data-intensive, REST-based, of the data sources, and (c) a client web browser, RIA entitled, Invenio . Invenio combines a variety which acts as the user interface for the website. of different technologies (Yahoo! Maps, Amazon On the other hand, RIAs are web services that use Associates Web Service, REST, and the Flex the web client to process a user interface and use framework) to geographically visualize aggre- an application server to store program states and gated music chart information. We report on our user data that is generated. The proliferation of experiences in designing and authoring Invenio, accessible REST APIs and rich user-interface use Invenio’s requirements as a case study to ex- frameworks that can run with zero installation amine relevant technologies and recommend a set requirements on browser sandbox environments of possible “best practices” for developing other greatly supports the development of highly inter- RIAs. active, web-based, data-exploration tools. Given the fact that every web service or ap- 1 Introduction plications stores, retrieves and modifies data to some capacity, it is surprising to find that there As the size of the Internet dramatically increases are few recommendations or instructions in exis- each year, so do its user’s demands. An Internet tence that explain which frameworks or APIs user is no longer content interacting with static work best. Without such information, deciding HTML web pages or simple searching portals; upon the types of data visualizations to use, the instead, users want applications and services that data sources (and their organizational structures) promote collaboration and provide them with rich and the affordances (or constraints) of user inte- content that can easily be manipulated. Given raction can lead to mass confusion and frustration for developers and users alike. Many developers end up utilizing a bricolage approach to design Copyright 2008 Michelle Annett and Eleni Stroulia. and implementation, which ultimately wastes val- Permission to copy is hereby granted provided the orig- uable time and resources. Not only does the inal copyright notice is reproduced in copies made. amount of time spent on a project increase, but the 1 number of “workarounds” or “hacks” that are iTunes, and cross-continental success. With re- needed also increases (thereby making the spect to retail and media sharing, the ever- project’s code unnecessarily complex). growing popularity of online digital media me- Sharing one’s experiences, either by provid- chanisms, such as peer-to-peer networks, torrents, ing documentation, recommendations, guidelines and online music stores have shifted the impor- or instructions, can potentially save everyone time tance of the Internet in music consumer’s pur- and effort. Providing detailed examples and justi- chasing habits and the borderless selling of fications for one’s work – as we do in this paper – products. From 2005 to 2007, there was a steady can not only help novel developers, but it can also decline in physical CD sales (-7.8%, -4.9%, and - increase the quality of an expert’s work and in- 9.5%), and an astronomical increase in digital spire other developers to increase creativity within album and single sales (+150%, +20.8%, and their own applications. As well, we can use these +45%) [2,3,4]. Given these sales figures and the guidelines and recommendations to help push the compelling evidence for the impact of on-line evolutionarily pace and optimality of technologies word-of-mouth, it should come as no surprise that forward, by constantly demanding more im- consumers are now, more than ever, looking at the provements and innovations in Web 2.0 technolo- Internet as an influential source of music informa- gies and illustrating the importance of cohesion tion and recommendations. and interoperability between resources, frame- Coupling the importance of the Internet for works and APIs. consumers within the music domain with the pop- The remainder of this paper discusses the de- ularity of RIAs and mashups, the authors have sign rationales of a REST-based RIA service, In- chosen to create Invenio, a highly interactive, venio, and the experiences encountered during its data-intensive mashup service that uses geovisua- development. We discuss our motivation in lization techniques to facilitate the discovery of choosing the application domain and review re- new music by its users. lated visualization and mashup research (Section In looking at the geovisualization and ma- 2), we then discuss the Invenio software architec- shup work occurring in academia and industry, ture (Section 3) and workflows (Section 4). Sec- there are a number of innovative projects that tion 5 concludes with our reflection on the lessons have been pursued. Wood et al. have utilized a we learned from the experience and our advice on geographic map with data dial overlays to demon- “good practices” for building REST-based RIA strate the power of geovisualization mashups [5]. applications using complex compositions of exist- They have taken cell phone text message queries ing resources. sent to a popular cell phone provider and com- bined them with Google Earth to assess trends in 2 Motivation text message queries. A data dial displays all of the times of day that a query was made from a Under the Web 2.0 umbrella, we now recognize a specific location (Figure 1A). The angle of a line collection of technologies that flexibly and crea- leaving a specific point indicates the time that a tively support information sharing on the web. query was made and the length of the line indi- On-line wikis, blogs, and RSS feeds have had an cates the number of queries made at that time. In immeasurable influence in various domains, as contrast to Wood et al. ’s “typical” geovisualiza- they provide an easily accessible platform for tions, Kwan and Lee used two non-traditional word-of-mouth advertising, online retail, product visualization techniques (linear and topography research, and media sharing. inspired) to visualize a geo-coded (geographical On the marketing/advertising front, it is inter- coordinates) data set that was composed of esting to consider the celebrity gossip blog Pe- 129,000 human activities [6]. Using horizontal rezHilton.com [1] whose founder, Mario and vertical lines (Figure 1B), as well as peaks Lavandeira, started posting audio and video links and valleys (Figure 1C), Kwan and Lee have ex- of his favorite unsigned or overseas artists such as tended the boundaries of geovisualization re- Leona Lewis, Amy Winehouse, and Mika on his search in a very creative direction. Although both blog. Before his postings, each of these artists had sets of researchers use different ways to express local “cult” followings, but after they were temporal data (one in 3D and the other using 2D blogged about, they obtained number one hits, top dials), neither contain capabilities which might album and singles downloads on Amazon and allow one to discover trend patterns that occurred 2 Service Purpose Visualization Interactive Zooming / Types Visualizations Filtering Deploys and tracks user created videos across Line chart, bar chart, TubeMogul [7] multiple websites (AOL, Google, MySpace, Ya- pie chart, Google None N/A hoo!,YouTube, Viddler) Map Line chart, bar chart, QLikView Musiq Visualizes online and radio station airplay from Charts – yes; On some visualiza- pie chart, geographic Tracker [8] Mediaguide service Geographic map – no tions, yes map Users click on map Visualizes disease outbreak bulletins (Google Filter by disease markers to view HealthMap [9] News, curated personal accounts, World Health Google Map type , resource additional informa- Organization official alerts) geographically provider tion Table 1 - Commercial Mashup Services. over long periods of time, such as days or weeks. A) B) This temporal dimension is something that we have included within Invenio. As illustrated in Table 1, there are a variety of successful, online mashups that have been created to visualize aggregate data. TubeMogul [7] pro- vides users with a Google Map on which their demographics are shown. Unfortunately, Tube- Mogul does not utilize the full power and capabil- ities that the map can provide: instead of enabling C) interactivity with this map, the TubeMogul de- signers have chosen to leave it static and to not provide any way to determine the regional popu- larity of one’s videos over time. The QLikView Musiq Tracker [8] service does provide useful information, but most often there is too much in- formation presented in awkward or confusing ways. The data is largely static and the interactive functionality is very slow, leading to a very inef- Figure 1 - Geo-Spatial Visualizations from [5,6].