Yarely – a Software Player for Open Pervasive Display Networks

Yarely – A Software Player for Open Pervasive Display Networks

Sarah Clinch, Nigel Davies, Adrian Friday, Graham Clinch School of Computing and Communications, InfoLab21 Lancaster University, Lancaster, LA1 4WA. United Kingdom. [s.clinch | nigel | adrian] @comp.lancs.ac.uk, [email protected]

ABSTRACT signage system that has operated at Lancaster University This paper describes Yarely, a software player designed to since 2005. The system consists of a wide range of displays support the next generation of pervasive display networks. including indoor and outdoor installations and both pro- We identify five design goals for future digital signage play- jected and LCD technology. Content creators in e-Campus ers: the ability to provide basic signage functionality (media use the system to create “channels” of signage content that scheduling and playback) and support for openness, extensi- they subsequently publish within the network. Display own- bility, resilience and appropriation. The paper describes the ers subscribe their displays to these channels and the system design and implementation of Yarely in light of these design automatically schedules content to each display based on the goals. We present an evaluation of the system in the form combination of subscriptions and channels relevant to that of deployment and usage data alongside a reflection on the display. Of particular note is the fact that e-Campus offers degree to which Yarely successfully addresses the needs of an API that enables developers to create new applications future pervasive display systems. to control the signage network [14]. The e-Campus system was based on an experimental software infrastructure that relied on a complex series of mes- Categories and Subject Descriptors sage events to coordinate playout amongst screens. Whilst H.5 [Information Interfaces and Presentation]: Mis- our initial predictions for the software suggested that the un- cellaneous derlying eventing system would be able to handle large num- bers of displays (estimated at between 7,000–15,000 events General Terms per second with 250 displays) we found that during a typ- ical day’s operation on campus with 25 screens the system Design, Reliability quickly began to lag. This slowdown was despite a relatively modest event channel throughput measured at ap- Keywords proximately 40 events per second. We found that this was public displays, system architecture, experiences partly due to the communications overhead of the e-Campus API over the event channel, but also due to the 250,000 or 1. INTRODUCTION so database queries needed to calculate the schedules for the displays given the 66 content channels and 3,000 or so In this paper we describe Yarely – a software player de- content items in the system. Despite providing the main sig- signed to support the next generation of digital signage and nage infrastructure on campus for seven years, this reliance pervasive display networks. Our work is motivated by a on a common eventing infrastructure and centralised high vision of large-scale networks of pervasive displays forming level scheduler led to a number of concerns with the existing a new communications medium that is open to content and system: applications from a wide range of sources [4]. We believe this openness will encourage innovation and reduce reliance on excessive fragility our system was extremely sensitive to traditional models of funding through advertising – enabling interruptions in network connectivity with these typi- a broad range of signage scenarios such as emergency coordi- cally leading to blank screens or “stuck” displays. nation, personalised information, situated applications and community information sharing. scalability concerns our eventing infrastructure struggled We have previously reported on the design and implemen- to scale as the number of displays grew above 20, and tation of the e-Campus system [14] – an experimental digital became less reliable as each new screen was added.

no support for off-line nodes our system could not support ad-hoc deployment of screens (e.g. at conferences) Permission to make digital or hard copies of all or part of this work for without connection back to our infrastructure. personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies problems with firewalls our system did not have support bear this notice and the full citation on the first page. To copy otherwise, to for traversing through firewalls which limited deploy- republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ment options. PerDis 2013 June 04 - 05 2013, Mountain View, CA, USA Copyright 2013 ACM 978-1-4503-2096-2/13/06 ...$15.00. difficulty in pre-fetching content because our displays

25 received instructions on a per content item basis they We also recognised that in order to have our player widely could only pre-fetch a single item of content at a time. adopted it was crucial to develop a multi-platform solution and one that would scale to very large deployments. Together these limitations acted as a barrier to scaling the system beyond the University campus, and even prevented 2.4 Resilience and Disconnected Operation us from increasing the size of the on-campus display network. Core to our design goals is the need to provide a resilient The large-scale, open nature of the future display networks player. Our experiences with e-Campus taught us that hav- we envisaged could not be met by our existing software. ing a tightly-coupled system that relied on network connec- In 2011 we decided to completely re-engineer our software tivity led to significant periods of time when screens failed. infrastructure from the ground up in order to support our vi- We were also aware that while a campus may have generally sion of large-scale open signage systems. In so doing we drew good network connections, many displays are installed at lo- on our extensive experience with the e-Campus system. We cations with little or no network connectivity. As a result begun the re-engineering by creating a new software player we set ourselves the goal of developing a system that was for our signage network. In this paper we describe the design fully functional without network connectivity. In particular, and implementation of this component. we wished to create a system that could make autonomous We observe that in contrast to many papers on digital scheduling decisions and could source content from multiple signage that focus on user experience or new display tech- local sources without network connectivity. nologies, we explicitly aim to provide the community with engineering details of a single component in order to help fa- 2.5 Appropriation cilitate other researchers to create robust open signage sys- Our vision of an open pervasive display network relies on tems. display networks providing users with the ability to appro- priate displays for a wide range of applications. Such ap- 2. DESIGN GOALS propriation might range from simply tailoring content such as news or weather broadcasts through to users executing Our experiences with the e-Campus system enabled us their own applications on a public display and interacting to arrive at a series of concrete design goals for the Yarely with the display for significant periods of time. As a result, player. we set ourselves the design goal of supporting a wide range 2.1 Addressing General Functional Require- of sensing and appropriation methods. In contrast to many ments systems, we specifically designed our player to enable users to execute native applications locally on the display. Clearly, at a minimum, the player needed to support general signage functionality such as displaying content of a range of media types and on a range of displays. Since we 3. DESIGN AND IMPLEMENTATION are only concerned with the requirements for the player in this paper, we do not consider the more general requirements 3.1 Overall Architecture relating to issues such as content creation and scheduling. For the purposes of this paper we consider digital signage systems to comprise of two main architectural segments – 2.2 Openness Display Nodes (upon which our player software executes) In developing our player we considered it critical that the and sources of content located within the broader network display was able to serve as a component of a future open (e.g. as a Web or cloud service). We recognise of course that display network. Most signage players assume that they display networks are significantly more complex and include are controlled by a single authoritative source that supplies components such as ad-brokers, content creation tools and them with both content and a play out schedule. Our goal scheduling, but this simple model serves to highlight the fact was to design a player that could easily be connected to that Display Nodes are typically separate to content sources. multiple content sources simultaneously. This represents a An important architectural decision for any signage sys- fundamental shift in thinking of signage systems – instead of tem is the distribution of functionality between display clients a player being treated as a simple “slave” that plays the con- and cloud based services such as content sources. For exam- tent that is sent to it, we aimed to create a more intelligent ple, it is possible to construct signage systems in which dis- player that is able to schedule content from multiple sources play clients simply run standard Web browsers and all of the that are not necessarily aware of each other and hence may functionality is provided through cloud services. In contrast, provide the player with conflicting play out requirements. many existing commercial signage systems have relatively To achieve such a degree of openness we specifically set our- feature-rich clients in order to enable them to survive peri- selves the goal of providing a mechanisms that would enable ods of disconnection. Our experiences with e-Campus have components developed by third parties to interact with our led us to believe that this latter approach has significant player. value in the absence of guaranteed high-bandwidth network connectivity and Yarely is explicitly designed to support dis- 2.3 Extensibility connected operation. In addition to being open to content from many sources we set ourselves the explicit goal of creating a player that 3.2 Communicating with Yarely was extensible. In particular, we wished to develop the Within our system Display Nodes obtain instructions re- player such that new components that provided functions garding which content to play in terms of Content Descriptor such as content rendering, scheduling and environmental Sets. The purpose of the Content Descriptor Set is to pro- sensing could be developed and included with minimal changes vide a description of a set of content items to be played by to the core player. the node, the circumstances in which they should be played,

26 Figure 1: Overview of our network architecture. and the location of any required media. Our player can thus be used in conjunction with any signage system that is able to export (or translate) its playout requirements in this format. We term any system component that creates Content Descriptor Sets a Descriptor Factory. Sample De- scriptor Factories include the e-Campus Channel system and a future application store for public display networks [3]. Communication between the Display Node and Descriptor Factory (DF) occurs through the transmission of a Content Descriptor Set from the DF to the Display Node [Figure 1]. A Content Descriptor Set is encoded using XML [Figure 2] and, from a design perspective, transmission of the Content Descriptor Set is protocol-agnostic. We consider the relationship between a Display Node and Descriptor Factory to be a many-to-many relationship: each Display Node may use Content Descriptor Sets from multiple DFs and each Figure 2: A sample Content Descriptor Set. DF may serve many nodes. A Content Descriptor Set is comprised of a combination of two main types of element: content-sets and content- Sensor Management items. A content-set is essentially a container element for The sensor management module reads data from sen- content-items and other content-sets, whilst a content-item sors and environmental sources (e.g. context, user pres- describes a single item of media that may be rendered at a ence and interaction) which may trigger injection of display. interactive content onto the node. Our Content Descriptor Set format [Figure 2] is designed to be as open as possible. We use cross-platform, human- Playlist Generation and Scheduling readable XML with a structure that is designed to extend This module processes the available subscriptions to as requirements for new features present themselves. We derive a playlist of content items to be played in the use a recursive structure comprised of nested content sets to immediate future and is responsible for the scheduling support merging of content from numerous content sources. of these content items onto the screen in response to changing circumstances (e.g. time, sensing events) and 3.3 Node Architecture in line with the constraints speciﬁed in the Content Yarely features a component-oriented design. We divide Descriptor Set. our node into ﬁve key modules [Figure 3]: Content Rendering and Lifecycle Management Content rendering refers to the physical representation Subscription Management of content items onto output hardware (e.g. display, The subscription management module is responsible speakers). The lifecycle of such content items includes for maintaining connections with the Descriptor Fac- the caching of media at the display node, the prepara- tory for the purpose of requesting and receiving the tion of content in advance of playback, playback, and Content Descriptor Sets that describe the subscrip- cleanup on completed playback. tions. It polls periodically for changes in its subscriptions and passes any changes on to the playlist gener- Analytics ator and scheduler. The analytics module is responsible for returning data

27 to external servers (e.g. the Descriptor Factory or ap- currently integrated into our scheduler – each content ren- plication provider). These may take the form of sum- derer is handled as a separate Python application. mary statistics or notifications on a per-event basis. Our current implementation is focused on the Mac OS X operating system (versions 10.6+) but the bulk of the Each module is composed of a managing element plus code is contained within cross-platform Python modules [Ta- a set of further plug-ins which handle specific software or ble 1]. Renderers are developed using the PyObjC bind- hardware requirements. The module for Content Rendering ings for AppKit, QTKit and WebKit (for the default Image, and Lifecycle Management for example, relies on a set of Video and Web renderers respectively) and the Python libvlc renderer plug-ins needed to handle content types including bindings (for the default media stream renderer which also Web pages, images, pre-recorded and live streamed video. acts as an alternative video renderer). The renderers are platform specific and are the only element of the system that needs to be adjusted from system to Python Lines system. To support lifecycle management caching plug-ins Files of Code provide file fetching over HTTP and other mechanisms (e.g. bit torrent or other content distribution networks). Similar Cross-Platform 58 7587 granularity is required for other modules (handling many Subscription Management 9 1344 different sensors, communicating subscriptions over numer- Sensor Management 4 413 ous transport mechanisms etc.). Scheduling 6 1802 Configuration 3 501 Unit Tests 8 388 Other (helper modules, 28 3139 base classes etc.) Mac OS X 24 1415 Rendering 8 411 Facade 3 306 Other (helper modules, 13 698 base classes etc.)

Table 1: Proportion of cross-platform versus platform-speciﬁc code within the Yarely packages

4. EVALUATION 4.1 Usage Our first demonstration of Yarely was in November 2011, with a first deployment in February of the following year. Our initial deployment consisted of 4 display nodes, one at each of four European universities. The deployment was extended in March 2012 to include approximately 30 additional display nodes at our institution, and has been continuous use on campus since March 2012 as the dominant digital signage platform. The system is integrated with our previous user Figure 3: Node architecture. facing frontend, e-Channels, and supporting the same usage model and content patterns we’ve previously reported [2]. 3.4 Implementation and Deployment Our client software is implemented in Python and we use the ØMQ transport layer [6] to transmit events between por- CDS requests tions of the software (fulfilling the role of the internal event- content requests ing system shown in Figure 3). Data exchanged over ØMQ 800 is encapsulated using XML. Requests The Subscription Management, Sensor Management and 400 Scheduling modules are each mapped to separate Python applications. In our current implementation we have config- 0 ured our players to use HTTP to fetch Content Descriptor 0 5 10 15 20 25 Sets from a backend server that supports our existing chan- Number of displays nel system interface [2]. Individual content items are also fetched over HTTP via an Apache web server. We check for modifications to the content items and Content Descrip- Figure 4: Number of Content Descriptor Set (CDS) tor Sets every 2 minutes (configured in the root Content requests is approximately linear with number of ac- Descriptor Set XML on each node). We have not yet imple- tive displays, whilst number of content items fetched mented the analytics module, and lifecycle management is is lower due to effective caching strategy.

28 As we can see from plotting Content Descriptor Set (CDS) of new components. requests and content requests against number of displays (snapshot shown 1st April 2012–11th February 2013) in Fig- 4.2.4 Resilience and Disconnected Operation ure 4, the new architecture exhibits good scalability charac- Our system is able to survive extended periods of network teristics. The number of CDS requests has a linear upper disconnection and is no longer dependant on a single cen- bound with the number of active displays. The median num- tral event server. Indeed, at a recent conference demonstra- ber of displays online is 18/hour, corresponding to a median tion of the system, an extended network problem prevented CDS requests of 511/hour. The median number of content demonstrations of other signage platforms but our player items fetched is 0 (maximum 545/hour), which demonstrates continued to cycle through the content it had been able to that content is effectively cached by the system. cache before the outage. As a result of this we have anec- dotal evidence that the system is perceived as significantly 4.2 Reflection on Design Goals more reliable than our previous implementation. One inter- esting side-effect of this is that, in the absence of detailed 4.2.1 Addressing General Functional Requirements system monitoring, it is harder for us to detect system fail- Our system supports many general signage requirements: ures as in most cases the displays continue to show content date- and time-based scheduling, ordering of content items, even when not receiving updates. play-out of multiple content types, control of display hard- 4.2.5 Appropriation ware and simple monitoring. We note however that, in contrast to the e-Campus system, our current player does not Our system has been used to support two distinct forms of support synchronisation of content between displays. We appropriation. We enable users to personalise content using hope to add this, together with the concept of visual trans- the Tacita system [8]. In addition, we have used the system actions [14] in due course. to support instantiations of user virtual machines as part Our initial playlist generator and scheduler cycled through of our research into cloudlet systems. Both appropriation each content channel using a round-robin algorithm. We mechanisms are triggered using our sensing module. found we needed to extend this in three important respects. Firstly, we introduced a set of priority levels to allow campus- 5. RELATED WORK wide emergency announcements to interrupt running con- The term digital signage is typically used to refer to an tent (we’ve since been able to reuse the priority system for electronic display used to show a range of media items in- triggering personalised content). Secondly, we needed to cluding information, advertising and television broadcasts. support live broadcast streams of unknown duration—this Such displays are located in a range of shared, public and broke our initial design assumption that content length was semi-public, spaces. always known a priori. Finally, we found that long-running There are many feature-rich commercial signage platforms, content (specifically movies from an art festival) were domi- often providing tool chains supporting content creation, ac- nating the airtime of the displays. Even lowering the priority quisition, scheduling and playback over a range of hardware failed to address the imbalance between the duration of most and media types. In our experience, these systems typi- of the content (typically 15 seconds) and these movies, lead- cally follow a broadcast model where content timelines and ing to priority inversion; we therefore introduced rate pacing content items are pushed to player nodes for play-out (anal- of each channel to enforce airtime ratios using a credit-token- ogous to our former e-Campus system [14]), rather than the based rate pacing scheme. open model of many content sources we advocate. As commercial closed-source software, details of the internals and 4.2.2 Openness performance are not available for comparison. One exem- In terms of openness we have been able to construct a plar worthy of note due to its potential for extensibility, player that can accept content and scheduling instructions is Signagelive [13]. Signagelive provides cloud-based media from multiple sources. We believe that the format we have delivery, playback on Android, Windows, SMIL and Bright- used for these instructions can be readily used by new and Sign devices, supports HTML 5, and it’s OpenSplash player existing signage systems and content creation tools. These is open source. Content Descriptor Sets have replaced the APIs we provided Within the research domain, the focus is not typically in earlier testbed implementations [14]—providing equiva- on system architecture, but rather broader aspects such as lent functionality without the need for application writers how the display supports interaction, community or work- to maintain persistent connections to a display in order to place awareness, or influences its deployment context. The issue commands. earliest examples, e.g. Plasma Poster Network [1], took the form of digital noticeboards that cycled through a number of 4.2.3 Extensibility information articles, photographs, local intranet pages and Our component-oriented approach, in which modules com- user contributed items. Other similar scale digital signage municate through ØMQ provides significant potential for ex- deployments have been situated in public spaces in New- tensibility. Whilst our implementation is written in Python, castle, UK (a cafe [7]), Brisbane, Australia [12] and Wray, ØMQ bindings exist for over thirty programming languages UK [16]. allowing new plug-ins to be written in the language that best Although not a focus of this paper, content and applica- suits the functionality to be provided (for example, allowing tions for displays have been extensively researched. Muller¨ easy integration with hardware components with restricted developed prototypes including iDisplays [10] and Reflec- APIs). Since its initial deployment our system has been ex- tiveSigns [9] in which face recognition triggers content adap- tended with new renderers to support streaming video and tation to passers by (somewhat analogous to FLUMP [5]). VNC connections – demonstrating the post-hoc integration Strohbach and Martin [15] focussed on context-awareness,

29 developing a middleware for access to sensor information. for the 21st century. Computer, IEEE, 45(5):58–64, Researchers in Finland deployed UBI-Hotspots [11], an May 2012. urban deployment of 12 interactive displays in a mixture of [5] J. Finney, S. Wade, N. Davies, and A. Friday. Flump: indoor and outdoor locations. Their modular web browser- The FLexible Ubiquitous Monitor Project. In based architecture is flexible, and has allowed the creation Cabernet Radicals Workshop, May 1996. of many applications, trialled ‘in the wild’ with citizens of [6] iMatix Corporation and Contributors. The intelligent Oulu. Virtual machines simplify deployment, and a cen- transport layer - zeromq. http://www.zeromq.org/ tralised event-channel (FUEGO) is used to coordinate com- [Last accessed: February 2013], 2007–2013. ponents. Deployment of content and applications is under [7] C. Kray, A. Galani, and M. Rohs. Facilitating tight centralised control. opportunistic interaction with ambient displays. In We have designed our system to be open to content from Workshop on Designing and Evaluating Mobile numerous sources with no centralised control and our com- Phone-Based Interaction with Public Displays at CHI, ponent division is motivated by the desire to ensure exten- 2008. sibility and permit execution of the common code across [8] T. Kubitza, S. Clinch, N. Davies, and different operating systems and hardware. M. Langheinrich. Using mobile devices to personalize pervasive displays. In Demo. at HotMobile ’12, 2012. 6. CONCLUSION [9] J. Muller,¨ J. Exeler, M. Buzeck, and A. Kruger.¨ Despite extensive research in the field of digital signage Reflectivesigns: Digital signs that adapt to audience relatively little has been published on the engineering of attention. In Proceedings of the 7th International such systems. In this paper we have described the design Conference on Pervasive Computing, Pervasive ’09, and implementation of Yarely – a software player designed pages 17–24, Berlin, Heidelberg, May 2009. to support the next generation of digital signage networks. Springer-Verlag. Our system explicitly supports our design goals of open- [10] J. Muller,¨ O. Paczkowski, and A. Kruger.¨ Situated ness and extensibility by allowing multiple content sources public news and reminder displays. In Proceeedings of and scheduling systems to publish Content Descriptor Sets the European Conference on Ambient Intelligence, that can be passed to displays. Our system has now been AmI ’07, pages 248–265, Berlin, Heidelberg, 2007. in operation at a number of sites for over a year and has Springer-Verlag. demonstrated significantly improved levels of resilience as [11] T. Ojala, H. Kukka, T. Lindén, T. Heikkinen, compared to our previous testbed. We believe that the de- M. Jurmu, S. Hosio, and F. Kruger. Ubi-hotspot 1.0: sign goals, system architecture and experiences we have re- Large-scale long-term deployment of interactive public ported will benefit other researchers considering the creation displays in a city center. In Proceedings of the 2010 of a dedicated signage system. In particular, we hope that Fifth International Conference on Internet and Web other researchers will see fit to adopt the format we use for Applications and Services, ICIW ’10, pages 285–294, Content Descriptor Sets such that interoperability between Washington, DC, USA, 2010. IEEE Computer Society. signage systems becomes commonplace. [12] F. Redhead and M. Brereton. Designing interaction for local communications: An urban screen study. In 7. ACKNOWLEDGEMENTS Proceedings of the 12th IFIP TC 13 International The authors acknowledge the financial support of the Fu- Conference on Human-Computer Interaction: Part II, ture and Emerging Technologies (FET) programme within INTERACT ’09, pages 457–460, Berlin, Heidelberg, the 7th Framework Programme for Research of the Euro- 2009. Springer-Verlag. pean Commission, under FET-Open grant number: 244011. [13] Remote Media Limited. signagelive — cloud-based digital signage software. http://www.signagelive.com/ [Last accessed: 8. REFERENCES February 2013], 2013. [1] E. F. Churchill, L. Nelson, L. Denoue, and [14] O. Storz, A. Friday, and N. Davies. Supporting A. Girgensohn. The plasma poster network: Posting content scheduling on situated public displays. multimedia content in public places. In Proceedings of Computers & Graphics, 30(5):681–691, 2006. the 9th IFIP TC13 International Conference on [15] M. Strohbach and M. Martin. Toward a platform for Human-Computer Interaction, INTERACT ’03. IOS pervasive display applications in retail environments. Press, September 2003. Pervasive Computing, IEEE, 10(2):19–27, 2011. [2] S. Clinch, N. Davies, A. Friday, and C. Efstratiou. [16] N. Taylor and K. Cheverst. Creating a rural Reflections on the long-term use of an experimental community display with local engagement. In digital signage system. In J. A. Landay, Y. Shi, D. J. Proceedings of the 8th ACM Conference on Designing Patterson, Y. Rogers, and X. Xie, editors, Ubicomp, Interactive Systems, DIS ’10, pages 218–227, New pages 133–142. ACM, 2011. York, NY, USA, 2010. ACM. [3] S. Clinch, N. Davies, T. Kubitza, and A. Schmidt. Designing application stores for public display networks. In Proceedings of the 2012 International Symposium on Pervasive Displays, PerDis ’12, New York, NY, USA, 2012. ACM. [4] N. Davies, M. Langheinrich, R. José,and A. Schmidt. Open display networks: A communications medium