Data Streams Permeate Almost Every Aspect of Computing and Data Communication

Data streams permeate almost every aspect of computing and data communication. From medical imaging to YouTube videos, from stock market data to sports results or from seismic sensor data to baby monitors, data streams play an increasing role in almost every aspect of our lives. They can be found in every application area imaginable, including medicine, finance, education, engineering, physical sciences, media, entertainment, and the arts. Although increasing processing capacity has made data stream processing more accessible, increases in storage capacities and network bandwidths are making it possible to store and access ever increasing volumes of data. This in turn leads to an increase in demand for processing the available data streams, e.g. in the case of multimedia archives there is an acute need to index, search and summarise the data if it is to remain useful. Furthermore, new applications with increasing resource requirements will continue to emerge and many of these applications will require processing resources beyond the capabilities of much of the existing infrastructure of today. Large organisations, such as media corporations and telecommunications providers, are building significant standing infrastructures for specialized data stream processing, but these are not suited to meeting the diverse and transient requirements of a broad range of users, from home musicians to scientific collaborators. Requirements of economy, flexibility, scalability, mobility and location mit- igate against this approach. These users need techniques that allow them to dynamically build and control transient infrastructures for processing their data streams on demand. Questions arising from our previous work are: • Where can the necessary compute power be found to meet these needs? • Can cycle-harvesting (opportunistic) services handle these streams? • Can the use of on-demand virtual machines assist? • What are the real issues in frequency/volume? • What kind of stream datatypes yield most benefit? • What kind of opportunistic resources are best? • What categories of stream execution are best handled this way? • Can idle cores and GPUs be effectively exploited? We will explore architectures that would allow elements from existing infrastructures to be accessed and composed on demand, enabling new possibilities for communities for whom it is infeasible, un- economical or unnecessary to build a dedicated, standing infrastructure. For example, a school might use on-demand infrastructure for indexing, summarizing and searching an archive of audio and video streams from practical science experiments. Or scientists might construct an on-demand infrastructure to process the output data stream from a radio telescope or particle accelerator, with the possibility of performing further stream processing to visualise the output. Alternatively, an on-demand infrastructure might be created to transform an archived video stream for playback on a low-power mobile device.

1Image of An Bradan´ Feasa courtesy of Ois´ın Mac Suibhne and Alanna Avant, http://www.suibhne.com/suibhne.html 1 Figure 1: On-demand stream processing stack

We propose a comprehensive programme of research encompassing the end-to-end challenges presented by stream processing using infrastructure on demand. Figure 1 illustrates the relationship between three work packages:

WP1 – Applications We will examine the requirements of a diverse range of applications involving the acquisition, synthesis and processing of data streams. A number of specific applications will be investigated as a means of identifying challenges and evaluating solutions. WP2 – Infrastructure on demand Supporting the requirements of a diverse range of applications is a challenging problem, especially as they may be executed on computing resources distributed both geographically and across administrative boundaries. We propose to address this challenge by investigating techniques for dynamically building infrastructure on demand. We will incorporate the use of existing infrastructure resources but will also investigate the addition of hooks to existing infrastructures that enable the dynamic creation of virtual machines (VMs). Such VMs could be configured to meet the requirements of specific stream processing applications. Although such facilities would be useful across a range of processing models, special consideration will be given to the needs of data stream processing applications. WP3 – Stream processing extensions An infrastructure on demand facility, by itself, will not provide for the execution of complex stream processing applications. We will develop stream processing extensions that will provide (i) a framework within which applications can describe infrastructural, quality-of-service, fault-tolerance and other requirements; (ii) the ability to execute complex stream processing workflows using infrastructure on demand; and (iii) interfaces that will give both existing and new applications access to the stream processing facility.

1 WP1: Applications This work package will investigate stream processing applications of various types, and will be led by Dr. Jonathan Dukes. Stream processing applications consist of stream acquisition, stream synthesis and stream processing. Figure 2 illustrates these stream flows. Stream Acquisition Streams of data come into existence either by the acquisition of external information or by synthesis. External information comes either from interfaces to the natural world or from other computing systems. Such information may already be in the form of a time series of discrete values, in which case it can be directly used as a data stream, or it may be a continuous variable that needs to be sampled to create a data stream suitable for further processing. Many scientific grand challenges produce extremely large data streams, of the order of petabytes per year. The prime global example is the CERN 2 Figure 2: Stream flows

LHC [1]. Examples of interest to Irish scientists (from the recently submitted HEA PRTLI4 e-PSI proposal) are the LHCb experiment [2], HESS [3], CTA [4] and the Solar Dynamics Observatory (SDO) [5]. While these are on a grand scale, smaller-scale examples such as streaming data from weather stations or from seismic sensors are also of interest. There are already several information and monitoring systems that can be used to acquire data streams, including the very widely deployed gridICE [6] and R-GMA [7, 8, 9] (the latter is specifically targetted at streams). For the purposes of this research we propose to acquire H.323[10] and AccessGrid[11] audio and video streams. For non audio/video data streams we intend to acquire streams using R-GMA, and particularly to avail of the HESS, CTA and SDO sources if possible. Stream Synthesis A great majority of the streams that are synthesised are audio or video streams. Some common applications are audio synthesis (voice and music), avatar video synthesis, navigable world synthesis, and scientific visualisation. Audio is generally synthesised directly and fed to the sound subsystems of desktop PCs. Video stream synthesis is more demanding, and raises interesting questions. Quite complex video streams may be synthesised directly, e.g. using DirectX [12] or OpenGL [13] and dis- played using PCs’ graphics accelerators. Rendering engines can be used for more complex streams: for example OpenGL can be rendered on clusters running Chromium [14]. For immersive visual environments a “cave” can be used. The various scales on which rendering can be performed are illustrated in Figure 3. “Caves” and desktop PCs only serve one user at a time per system whereas rendering engines can be constructed to serve multiple users while scaling performance per user. Stream processing Data streams, whether acquired or synthesised, may be processed to produce one or more new or transformed streams. Processing may be performed on general processing architectures or using specialized stream processing hardware (e.g. GPUs or CPUs with specialized stream processing capabilities). Processed streams may be processed further, rendered (e.g. in the case of video streams) or stored for future use. 1.1 Motivating Application Areas A number of specific application areas will provide focal points for our investigation of on-demand infrastructure for stream processing. In each application area, we will identify specific requirements and challenges and develop proof-of-concept applications. Work in each of the identified application areas will build on past and ongoing research and expertise in the Department of Computer Science at Trinity College Dublin.

3 Figure 3: Rendering engine performance scale spectrum

1.1.1 Scientiﬁc Visualisation Early work by the proposers has concentrated on dedicated multiuser rendering engines, e.g. recently a multi-user multi-modal multi-scale grid-enabled visualisation engine [15] has been constructed, including a 9-node 2-d SCI torus running Chromium for rendering [16]. We will explore the possibility of integrating on-demand resources into rendering engines, which raises interesting questions: • What are the limitations in creating on-demand engines to user-speciﬁed scales? • Is it feasible to reliably run a complex engine such as Chromium on transient resources? • Does this model permit the steering of visualisations? • How do such engines scale in performance relative to the number of nodes? • What are the key contributions to time-variance in setup, execution and teardown?

1.1.2 Multimedia Use of multimedia data streams in entertainment, communication, education, scientific, medical and other applications is growing rapidly. IPTV services, which deliver familiar television services to clients over an IP network, are being rolled out by many service providers (e.g. BT, AT&T, France Telecom and Deutsche Telekom). New services, such as Joost! [17], claim to change how we watch television. Digital music downloads are challenging physical media sales for market share. Ser- vices such as YouTube and Google Video provide channels for the distribution of this content. More recently, research has emerged in an area referred to as CARPE: capture (or continuous) archival, and retrieval of personal experience [18]. MyLifeBits [19] is one example of ongoing work in this area. More everyday examples are the audio and video acquisition for voice over IP (VoIP) using SIP protocols [20] or Skype [21], and videoconferencing using H.323 services like Netmeeting [22] and Polycom [23] or other systems such as VRVS [24], AccessGrid [11] or EVO [25]. Past successes in capturing, creating, storing and transporting multimedia data are driving an ex- plosion in the volume of multimedia data that can be accessed. This in turn leads to in a new “grand challenge” for multimedia research: to “make capturing, storing, finding, and using digital media an everyday occurrence in our computing environment” [26]. Although significant progress is being made in the area of multimedia digital asset management and multimedia information retrieval, substantial challenges remain. Open challenges include querying video content-based on temporal concepts (e.g. a sequence of shots in a dialogue between two people) and automated summarization of video content (although progress has been made in areas such as sports) [27]. A related challenge 4 is the development of high-performance algorithms and infrastructures that can scale in tandem with the volume of multimedia content available [28]. We will explore on-demand solutions. • Is the scalability of on-demand resources and associated algorithms sufficient to meet the growing multimedia and information management challenges? • Is is feasible to perform multimedia processing on transient resources? Despite developments in the “nuts & bolts infrastructure” that underpins today’s multimedia streaming applications [26], distributing multimedia content is still demanding on network and server resources, particularly as new applications and services continue to increase the volume of data available. Content Distribution Networks (CDNs) (e.g. Akamai [29]), play an important role in allowing distributed multimedia streaming services to scale in the number of clients. By strategically replicat- ing content at the “edges” of the network, close to clients, resource bottlenecks around central servers can be avoided. Peer-to-peer content distribution solutions, such as CoopNet [30] and SplitStream [31], provide an alternative approach, as do hybrid approaches that use peer-to-peer networks in conjunction with content distribution networks. Asymmetry and contention are important factors. We will explore the relationship between content distribution and content processing using on-demand resources, which poses further questions about locality: • Will the availability of on-demand infrastructure for stream processing influence content distribution decisions? • Conversely, will content distribution influence where we perform stream processing?

1.1.3 Applications Targetting Specialized Stream Processing Hardware Two disparate example applications which can harness bitstream processing are cryptography and biotechnology. Both display independent bulk data processing needs, though quite different data processing needs, e.g. floating point versus integer processing, sparse versus compact coherent memory access patterns. These application spaces expose diverse demands on highly parallel architectures and yet there is relatively little research relating them to the emerging hardware paradigm (GPUs, etc). Early work by the proposers has started with a successful implementation of the AES private block cipher encryption algorithm on pre DirectX10 graphics processors [32]. We will extend this to other private block ciphers using post DirectX10 compatible cards. On top of these cryptographic algorithms various modes of operations will be applied. Public cryptographic algorithms such as RSA and Diffie-Hellman will also be developed. Indications are that suitable algorithms within this field will display significant increases in performance. As well as exploring offloading work to on-demand GPUs, etc, we will investigate extracting further parallelism from harnessing multiples of these on demand, and their application to encryption for tunnelled networking. In biotechnology an interesting case is molecular docking. We will extend the AutoDock [33] implementation for emerging highly parallel hardware. This complements our existing research with RCSI (FightMalaria@Home [34]) to harness eHiTS [35], SGA [36] and BOINC [37]. Autodock predicts the interactions of ligands with biomacromolecular targets for use in fields such as computer- aided drug design. The focus will be on the acceleration of affinity grid generation algorithms used to represent energy interactions of probe atoms and macromolecules. We will also analyse the core docking algorithms which use pre-calculated affinity grids in combination with ligands to determine energy interactions, the ultimate goal of docking. Currently the docking problem is one which trades accuracy in docking procedure for processing requirements. AutoDock has been used in distributed computing applications such as FightAids@Home [38]. We hope to see improvements in work unit throughput over current client approaches which solely run on the CPU in a similar manner to a related project, Folding@Home [39]. The ultimate (somewhat distant) goal is near-interactive visualisation and steering of docking computations, all exploiting on-demand stream processing resources. • How effectively can GPUs, etc, be instanced with on-demand virtualised contexts?

5 2 WP2: Infrastructure on demand This work package deals with enabling technologies for creating infrastructure on demand, and will be led by Dr. Stephen Childs. These technologies allow for the dynamic creation of stream processing systems by facilitating the creation and management of customised processing nodes; providing support for the description, discovery and allocation of resources; and allowing secure usage and accounting of resources across institutional boundaries. There are a number of trends that make infrastructure on demand an interesting problem. Firstly, applications are becoming more and more specialised. Even for a well-defined application (e.g. SIP-compliant Internet telephony), there are many competing implementations differentiated by factors such as functionality, implementation approach, or even just their appearance. Users typically have their own favourite application, software development framework, or operating system to which they typically remain loyal. Generic execution environments with a “lowest common denominator” software base force all users to adapt to an externally-imposed method of working. Users ideally need to be able to comprehensively customise the environment in which their programs are run. In order to deploy customised environments, users must be able to describe their requirements in detail. This is currently an open problem. We will investigate suitable description mechanisms both for manipulation by users and for automatic processing by software agents. Another trend that cannot be ignored is the emergence of substantial standing infrastructures with well-established management and security systems. These may be as simple as a small office network, or as complex as a worldwide grid. Such infrastructures often have spare capacity, which could be made accessible (either for altruistic reasons or for financial reward) if suitable mechanisms were available. Such access should be provided securely, with minimal disruption to existing configurations. We will explore methods of “injecting” external execution environments into existing infrastructures in a manner that respects the operational policies of those infrastructures. 2.1 Virtual Machines On Demand With the widespread availability of high-quality open-source virtual machine implementations such as Xen [40], the research frontier is now the coordination and management of large numbers of virtual machines. The Globus project is currently working on Virtual Workspaces [41], a generalised architecture for the management of virtual execution environments (VEEs), i.e. any dynamic, isolated, customisable space in which to perform processing. VEEs can be implemented in various ways: the simplest is a Unix user account, and the most powerful is a fully-fledged virtual machine. The Virtual Workspaces software builds on the Globus Toolkit 4 (GT4) [42] web-services-based Grid software. Data stream processing applications may require a specialised environment incorporating application- specific libraries and tools. Ideally, users would like to specify their environment requirements in detail, leaving the system to configure compute nodes accordingly. This may be possible by simple software installation on existing nodes, or it may require the creation of new compute nodes from filesystem images supplied by the user. The general availability of virtual machine (VM) technology makes the latter approach feasible. Isolated/tunnelled and customised networking may be required. We will explore the integration of VM technology with existing remote job execution systems (such as Condor or gLite). Our aim is to provide dynamic creation and configuration of compute nodes/networks in a manner compatible with existing middleware to enable the on-demand creation of stream processing infrastructure composed of multiple custom compute nodes and virtual networks. A range of processing environments could be made available, varying in customisability, stability, and longevity. The problem of efficiently deploying arbitrarily customisable execution environments is currently an active research area. There is little agreement on how to describe an execution environment. We envisage that resource providers and clients will need a range of options: automated installation of new execution environments from scratch using standard package repositories; deployment of node instances based on “template” filesystems; and deployment of fully-configured, certified, signed filesystems (virtual appliances) that can be rapidly deployed with minimal localisation. A potential starting point for machine description is the XML format used for configuration profiles in the Quattor [43] fabric management system. This schema incorporates hardware information, service configuration, network parameters and software package lists, providing a comprehensive pic- ture of a machine’s configuration. Profiles could be shipped to hosting servers, who can either install the machines from scratch, or use copy-on-write techniques to rapidly reconfigure an existing stored 6 filesystem image. References to software packages could be adapted to allow standard packages to be sourced from a local mirror rather than from the original location. User requirements will drive the need for virtual resources with a variety of properties, from long- running Xen VMs running on specialised hosting nodes to transient, lightweight Cooperative Linux [44] nodes using idle capacity on desktop PCs. We will need to explore the use of general-purpose tools that can create and manage VMs hosted on a variety of technologies. We will investigate techniques for securely managing and deploying virtual machine images in a manner compatible with existing system administration procedures. This work will build on an existing deployment of Condor on Cooperative Linux virtual machines hosted on PCs in undergraduate computer labs. Solutions will first be tested in a testbed environment, and will be migrated to a production environment as soon as they are proved stable. This will allow for analysis of user experience, and for realistic interactions with system administrators, ensuring the approach taken will be deployable and suitable for widespread adoption. Figure 4 shows a possible architecture for a Sruth resource provider site. Software managers access an entry point directly to install template images into the local image database and to control deployment of virtual services. Sruth publishes into supported information systems: the entry point advertises the location of the services it supports and the image database advertises the capabilities of installed system images.

Figure 4: Possible Site Architecture

Remote users access the site using standard methods: the site appears as having the superset of capabilities available on installed images. When a job is targeted to the site, the entry point negotiates with the local resource management system (LRMS) to instantiate new VM(s) on compute containers if needed, and to target the job to the appropriate virtual node. The VM containers are physical machines conﬁgured with a VMM and a network-accessible management interface. VMs for long- running services are hosted on service containers; VMs for compute nodes are on compute containers. Entry points on multiple sites would allow rapid provision of resources based on time-varying demand. Building on standard protocols (e.g. WS-Agreement) would enable uniform negotiation of short- and long-term virtual leases across multiple sites. • How can we manage the scale and variety of resources made available through on-demand techniques? • Can on-demand services be made sufﬁciently responsive to meet future stream processing and protocol demands?

7 2.2 Discovery and Allocation of On-Demand Virtual Resources The use of dynamic virtual machines and the exploitation of idle capacity introduces extremely distributed, transient resources covering a wide spectrum of processing power and configurations. There- fore, providers need methods for describing the resources they offer so that users can locate the specific resources they require. Conversely, users need to be able to describe the execution environment that they need to run their code. Cooperating parties need to be able to agree on a common language so that expectations can be met. This implies a need for resource description schemas for use in resource description, discovery, and customisation. The Condor system [45] exploits idle resources to process high-throughput batch jobs. It uses Clas- sAds to describe both resources and user requirements. A matchmaker compares job requirements to available resources, and selects the best place to run the job. In the Grid world, resource brokers play a similar role to the Condor matchmaker, but on a wider scale; they search global information systems to match job requirements with capabilities published by sites. The GLUE schema is used to describe resource capabilities; the Job Description Language (e.g. JDL, a modified version of ClassAds) is used to describe job requirements. We aim to exploit resources located on a variety of infrastructures and owned by a wide range of providers. Our approach will need to take into account the following realities: the existence of multiple resource management middlewares (which requires interoperability services); the complexity of application workflows (which requires workflow engines); and the diversity of policies (which requires support for federation and translation). We will investigate an agent-based approach, where resources will be represented by production agents; these will interact with social agents who act on behalf of users. Social agents negotiate to acquire resources, acting to fulfil the requirements of users within the framework of policies specified by themselves and by resource providers. Therefore, much of the complexity of resource allocation should be provided by autonomous agents that negotiate with target infrastructures on behalf of clients, taking into account any constraints provided by the user. This approach builds on existing work [46, 47, 36, 48] that explores such issues in a Grid environment, where production agents typically model entry points to Grid infrastructures. We aim to extend this work to incorporate hierarchies of production agents reaching down to basic infrastructural elements such as virtual machines. The higher levels are further discussed in Section 3. These agents will largely operate on the control plane, and so will not be directly involved in the transport of streamed data. However, it will be necessary to supply information important for streamed data into the pool of properties navigated by agents. In addition to machine properties such as compute power, memory size, and installed software, network-related properties such as round-trip time (RTT) will need to be published to allow locality to be taken into account when matchmaking. We will also investigate methods for bridging between differing schemas used by various target infrastructures. The resources published at a particular location will fluctuate due to the presence of transient compute resources and virtual machines. We will also explore the enforcement of “truth in advertising” within a resource description framework. In many current information systems, the resource provider is wholly responsible for ensuring that the descriptions they publish are accurate: there is no independent verification of their claims. As networks of cooperation expand and the links of trust between resource providers and clients stretch, it will be increasingly important to validate that the resources being offered actually provide the capacity advertised. We will investigate methods of automati- cally submitting tamper-proof benchmarks to independently validate the performance of advertised resources. • What level of detail about execution environments is needed by stream processing applications? • Can we flexibly and securely manage allocations or reservations of time to users from a wide range of communities?

2.3 Generic Execution Services for Stream Processing The VM-on-demand services will directly support existing stream data processing applications on standard CPUs by providing the customised execution environments needed. For example, a multimedia streaming client will specify its processing, memory and networking requirements, as well as 8 all the software it requires (e.g. specific versions of OS and libraries). We will explore the integration of virtual machine invocation with existing distributed execution frameworks. Automatic creation of VMs based on user requirements will provide transparency for application developers. We will also investigate policies for satisfying jobs with requirements for low latency. For example, it may be desirable to pre-allocate a number of VMs within a site to imme- diately service such jobs without the overhead of VM creation (this is analagous to the thread pool maintained by a web server for handling download requests). As well as these generic execution services, extensions will be needed to support specific requirements of stream processing applications (e.g., timeliness, composition of services and workflows). These extended services are the subject of Section 3. • How can automatic invocation of custom VMs be integrated into dedicated and opportunistic distributed execution systems? • What are the best policies for enabling low-latency access to custom execution environments?

2.4 Specialised Execution Services for Stream Processing The demanding requirements of multimedia applications have driven the development of specialised stream processing hardware, to the extent that such hardware is now almost ubiquitous. For example, modern CPUs have built-in instruction set support for stream processing, many commodity computers now include a high-performance graphics processing unit (GPU) and specialised devices for encod- ing and decoding media streams are commonplace. We will explore methods for making this largely untapped stream processing capacity available on-demand alongside general-purpose processing resources. The primary bitstream processing architectures of interest are commodity GPUs and the Cell Broadband Engine. Both provide a highly parallel hardware solution, though in differing ways. The GPU applies a more rigid parallel processing model, imposing various restrictions on memory access and flow control flexibility. These restrictions allow a greater amount of transistor logic to be expended on processing power and thus the theoretical performance of the GPU is higher than both the Cell and CPU architectures. Intel recently announced a prototype 80 core processor which is similar to the GPU design (currently 128 processors), consisting of a large array of identical processing cores. This approach leads to better performance, however poses the greatest software development difficulty in attempting to leverage the potential throughput. The Cell delivers a compromise by com- bining a single general purpose processor with 8 simpler processors on a single die, which eases its adoption by existing and new applications. In practice the future architectures will most likely lie somewhere between the current multi-core CPUs and the higher simpler multi-core approach of the GPU. Sruth will be targetted to both the GPU and Cell. As part of our infrastructure-on-demand framework, we will explore techniques for making specialised processors available for stream processing. This will include description of such processors’ capabilities to enable them to be advertised. • How can GPUs and Cell processors be made accessible within a distributed computing environment? • How can GPUs and Cell processors be securely shared between VMs and hosting machines?

2.5 Security for On-Demand Virtual Resources Authorisation, authentication and accounting (AAA) are fundamental to any distributed computing infrastructure. It is essential to ensure that resource providers can reliably identify prospective clients, can control access in a fine-grained manner, and can track resource usage and enforce quotas. The introduction of transient resources and inter-institutional usage patterns adds new dimensions to the already-complex problem of distributed AAA. National research and education networks (NRENs) such as HEAnet are pioneering federated iden- tity schemes (e.g. Eduroam). The Virtual Organisation Membership Service (VOMS) is an authorisation framework developed in the Grid community that supports the creation of roles using extensions 9 to standard X.509 grid proxies. Globus Toolkit 4 provides good support for specifying and evaluating complex policies. Maintaining end-to-end trust relationships between client and resource provider multiple target infrastructures is a challenge. There are issues of trust to do with sending potentially sensitive data to be processed on transient nodes. Encapsulation and isolation of execution environments can help. Authorisation is a problem with many dimensions. At its most basic, authorisation consists of access control decisions: is a particular user allowed access to the system based on credentials he is presenting? However, this is actually a special case of a more interesting question: how much access should a particular user be granted? Traditionally, operating systems have supported a rather static set of permissions (e.g. read, write, and execute) to resources; in a streaming environment, permissions must include concepts of timeliness, allowing resource shares and reponse times to be specified. We will investigate specification of such parameters, and enforcement on varied resources. It will also be important to investigate translation to and from schemes already commonly used by resource providers. Such policies will provide a foundation for higher-level resource management systems such as agent-based schemes. In addition to basic access control and resource allocation issues, resource providers need the ability to control the kind of applications that run on their systems. A site that is unwilling to allow arbitrary code to be run on its resources may be willing to run specific trusted applications (e.g. scientific code exploring an area of research in line with the goals of the site). In this case, the site will need to be able to identify incoming applications or virtual appliances securely, and to restrict access to recognised code signed by a trusted party. • Can on-demand approaches co-exist with existing security mechanisms and policies? • Can on-demand environments provide sufficient isolation from hosting computers and networks to ensure security is not compromised?

3 WP3: Stream Processing Extensions WP3 will investigate extension services for supporting stream processing applications on top of infrastructure on demand. Gabriele Pierantoni, the principal developer of Social Grid Agents, will lead this work package. Infrastructure on demand provides for the execution of jobs within cus- tomized processing environments, which can be configured to meet specified requirements. An infrastructure on demand facility by itself, however, will not provide for the execution of stream processing workflows, which may have requirements relating to, for example, inter-node dependencies, end-to-end quality-of-service and fault-tolerance. An example of an analogous problem can be found in the Condor high-thoughput batch processing system. While Condor provides for the batch execution of jobs on idle computing resources, it does not take into account data dependencies between two or more jobs. Condor’s DAGMan meta-scheduler extends the Condor batch processing system to provide such functionality. Similar provisions will be needed to facilitate the execution of stream processing workflows using infrastructure on demand. These provisions, which we refer to as stream processing extensions, must provide a means of executing both existing and new stream processing applications using infrastructure on demand. The extensions, however, will need to go much further than simple job execution. Applications will need a rich framework to describe end-to-end quality-of-service, fault-tolerance and other parameters and requirements. Mechanisms will also be required for composing complex workflows from available infrastructural resources, according to specific high- and low-level criteria and across multiple infrastructural providers. To the greatest extent possible, applications that use existing frameworks and communication protocols should be supported, alongside applications that are specifically developed to use the proposed infrastructure-on-demand facility. 3.1 Stream Processing Workflow Description The workflow topology for a stream processing application can be represented as a directed graph, where nodes represent the processing elements required by the application and edges represent the flow of data streams between nodes. A stream processing workflow may be as trivial as a single node with one or more input data streams and zero or more output data streams. More complex workflows may describe schemes for parallel stream processing on distributed resources using, for example, 10 pipelining, multiplexing or hybrid approaches. For example, multimedia workflow nodes may represent multimedia stream sources (e.g. a media file, a network-attached video camera or a remote multimedia archive), stream processing tasks (e.g. video colour adjustment or overlay mixing) and stream sinks (e.g. a device that will render the stream or a remote multimedia archive). Indeed, this approach is used by existing multimedia processing frameworks, such as Microsoft’s DirectShow SDK [49]. Although the stream processing jobs described in this manner are typically executed on a single, stand-alone computer, distributing multimedia processing tasks over networked computers has been proposed in the past (e.g. [50]). A workflow description for an application will be used throughout the lifetime of its execution, from the initial provision of infrastructure on demand, through the scheduling of individual workflow jobs, to monitoring and managing the execution of the workflow and accounting for the resources consumed. We will build on schemes for describing both batch processing job descriptions (e.g. Condor ClassAds) and workflow descriptions (e.g. Chromium) to provide a framework for describing stream processing workflows. The proposed framework must go beyond simply describing the workflow topology. Application developers, administrators and users will need to be able to specify detailed infrastructural requirements and scheduling parameters for each processing node, as well as quality-of-service, locality, fault-tolerance and other criteria. • How does one describe the diverse requirements of individual applications?

3.2 Stream Processing Scheduler The underlying processing facilities exposed as infrastructure on demand will each have their own mechanisms for scheduling jobs for execution. While the infrastructure on demand facility will hide the differences between disparate resource providers, the underlying systems are still oriented to- wards batch processing. With few exceptions (such as Condor’s DAGMan and WebCom-G), these underlying systems do not support the execution of workflows. Scheduling complex stream processing workflows will require a stream processing scheduler that operates at a higher level, above the scheduling of individual processing jobs. Since our proposed approach will be to dynamically configure infrastructure on demand to meet specific applications requirements, the stream processing scheduler will need to fulfil an expanded role, requesting resources from the infrastructure-on-demand facility to meet application requirements. Although there are many efficient constraint-driven algorithms, e.g. Dijkstra’s Algorithm, Dynamic Programming or Hill Climbing, the extent to which resources can be selected independently by a scheduler, using only a static workflow description provided by an application, is an interesting open question. For example, it seems likely that the resource requirements for a simple scienfitic visualisation application, servicing a single user, could be specified sufficiently using a workflow description. A multimedia stream processing application, however, that interacts with a content distribution network to service multiple users, might benefit by playing an active role in the selection of resources, particularly with regard to locality. Ultimately, the stream processing scheduler will have to instantiate each node in the workflow description by scheduling a job for execution on the underlying processing resources. Furthermore, on-demand infrastructures created to service a particular user request may be long-running and may be required to execute independently of an interactive user session. Stream processing workflows will need to be monitored to ensure they execute reliably and within the performance or quality-of-service bounds specified in the application’s workflow description. Support for redundancy and failover must be provided to enable processing to continue even if nodes fail. This will be particularly important if transient nodes (e.g. compute nodes provided by a Condor pool) are used in the computation. How can node failure be hidden from applications and users? What measures are needed to provide sufficient fault-tolerance? What impact will these measures have on application performance? Monitoring execution of stream processing workflows will also provide accounting information for processing, communication, storage and other resources. This might conceivably be required, for example, to pay for underlying resources used by the application. A high-level approach that performs

11 accounting across entire workflows, rather than individual nodes, will be required. • How can the large space of potential execution environments be efficiently searched to find the best place for a particular process to run? • How can we compose the located resources to construct a stream processing workflow that satisfies individual application requirements? • How can we exploit spatial locality when selecting the best place for a particular process to run? • What happens when a node’s execution is pre-empted to return control to its rightful user?

3.3 The Social Grid Agent Model The initial prototype of Social Grid Agents (Pierantoni et al, 2007 [46, 47, 36, 48]), addresses the problem of the growing complexity of grid computing (multiple grids, complex jobs, multiple actors and different allocation scenarios) subject to various constraints (different relationships among resource owners, institutions, and users; and the need of a ﬂexible and scalable solution able to adapt to future middleware with unknown characteristics). The architecture described in Figure 5, divides the agents in two broad categories with different behaviours and relationships, Production Grid Agents and Social Grid Agents.

Figure 5: Social and Production Grid Agents model.

Production Grid Agents are wrappers of one or more existing grid services such as job submission, security and file storage and are controlled by one or more Social Grid Agents. Social Grid Agents evaluate the requests that they receive, compute the required factors and exchange in social and economic transactions to obtain the needed factors. Once the factors are obtained, the Social Grid Agents instruct the Production Grid Agents they are in control of to define a production topology able to encompass all the needed services. Social Grid Agents are not only concerned with the exchange of factors and the definition of production topologies, but they are also used to define social topologies capable of reflecting the diverse and possibly complex relationships among resource owners, users and institutions. On close inspection it appears that this model is very well suited to the requirements of the Stream Processing Scheduler. A Production Agent provides implicit interoperability, as arbitrary low-level services can be wrapped in their specific agent. The stream processing layer is isolated from underlying details. Social Agents support complex high-level market-oriented resource allocation. They can take the global view or, if needed, a more in-depth local view. They support workflow across differing stream processing infrastructures. They may react to changing conditions by reallocations or redundant allocations, particularly when degradation occurs. Thus we propose to explore this model in the context of stream processing, not just for scheduling but also for coarse-grain workflow and other decision making. • Can an agent-based approach meet the complex matchmaking requirements presented by on-demand stream processing resources?

12 3.4 Access Mechanisms and Application Interfaces Many of the applications that we will explore are already supported by existing software frameworks and protocols. For example, a multimedia streaming client might use the RTSP protocol to request and control the playback of a multimedia stream. Similarly, a visualisation application might use the Chromium parallel rendering system. If a stream processing facility is to be widely adopted, it should support existing applications with little or no modification, in addition to new applications, developed to specifically target the use of infrastructure on demand. The provision of access mechanisms to provide existing applications with access to the infrastructure on demand facility will be an important aspect of our proposed research. For example, a media player client on a mobile device, which uses the RTSP protocol to request the playback of a video stream, requires the video stream to be transcoded before it can be rendered. Limited processing, memory and network capacity, however, prohibits transcoding the stream on the device itself. Instead, the RTSP request to setup the stream could trigger the execution of a stored workflow description. The workflow retrieves the stream identified by the mobile client, transcodes it according to the parameters supplied in the RTSP request, and delivers the transcoded stream to the client. An alternative example is provided by the Chromium system [45] for interactive rendering on clusters of workstations, which uses graph-based representations of stream processing workflows. A Chromium configuration usually launches a mothership instance and one or more crappfaker and crserver instances. Chromium provides a facility that particlly automates this procedure. By ex- tending the facility, existing Chromium configurations would be able to make use of our proposed infrastructure-on-demand facility. • What interfaces will be required to make the service available to application developers and end users? • Can the programming overhead to use this framework, and gain its advantages in full, be maintained to an acceptable level?

3.5 Stream Query Services Streams can be queried, for example as in the Borealis [51] and TelegraphCQ [52] stream query processing systems. If a data stream has a flat structure composed of known simple fields then it may be viewed as a serialised two-dimensional table of rows and columns. Queries could be posed that select only rows of interest. Once posed, a query could remain in force until explicitly terminated. This query processing could be considered to be a three-port filter component inserted in the data stream, with input, query and output ports. Multiple filters could be composed, i.e. input and output ports could be chained, into parallel/serial data flow graphs, each filter subject to its own query. Intuitively a filter is a black box with these ports. Naively one might construct production agents that instantiate the components, and then one might feed the data between agents as needed. But then the data stream flows via the stream services layer rather than the infrastructure layers, thereby mixing what should be orthogonal layers. A better approach might be to construct agents that instantiate the components and simply direct the data flows, but a more interesting approach would be to construct production agents that only deal with the definition of the ports, not their instantiation or operation, i.e. construct Stream Schema Agents. The actual instantiation of components and creation of flows would be devolved to associated stream processing and transport technologies. The schema could be generalised to definition of an input table, operation and output table. Query operations could be executed by query engines. Other operations could be executed by appropriate on-demand resources (i.e. execution units). The question of how to dynamically compose the three ports then maps to a resource matching and allocation problem that is amenable to solution using market-oriented social agents. Our initial unpublished research has explored the use of R-GMA [7, 8, 9], a stream-oriented relational grid information system that we have contributed to since its inception. Input streams are inserted into R-GMA source tables using SQL INSERT statements, and output streams to destination tables result from queries using SQL SELECT statements. We will explore the instantiation of such components, execution of data stream flows, dynamic queries, query stream flows, the use of a LRMS (such as Condor) with query stream flows, with/without on-demand virtual resources, with/without 13 specialised stream processors. Query streams are a very interesting concept. • How can one dynamically compose query stream ports? • How can one synchronise data and query stream flows?

4 Infrastructure The proposers have specialist expertise in e-infrastructures due to their involvement in running the national computational Grid infrastructure (Grid-Ireland), where Dr. Coghlan is Director of the Op- erations Centre, and Dr. Childs is the Deputy Grid Manager. The Grid-Ireland infrastructure provides an existing set of static resources, to which we propose to add on-demand virtual machines. The methodology for test and deployment will follow Grid-Ireland’s standard process. In this model, development and experimental work is performed within the isolated TestGrid environment [53]; the certiﬁed services that result are then rolled out to the production grid using an automated deployment architecture [54]. In addition, the proposers currently operate an expanding set of opportunistic on- demand resources hosted on VMs in undergraduate teaching laboratories. These resources, which run the Condor middleware, have been successfully used for executing jobs, both independently and in conjunction with the production EGEE middleware. All of the hosting machines include graphics accelerators (GPUs); we intend to add a Cell processor and to explore techniques for making all of these available as the genus of a national on-demand stream processing capacity. • Can on-demand stream processing resources be harnessed by a national Grid?

5 Project Management The project will be organised into work packages (WPs) as described above. Each WP will have a WP leader, who is reponsible for managing the WP actions and team members, and for ensuring the successful execution of project tasks. In addition, the WP leaders will be responsible for managing the research output of their WP; this will include publication in appropriate conferences and journals. All project members will make use of the existing infrastructure for collaborative work available within the CAG research group: this includes a wiki, web server, source control (CVS, subversion), and issue tracking systems (e.g. Request Tracker or Trac). The provisional assignment of work package leaders is as follows: WP Leader Expertise 1 Jonathan Dukes Real-time streaming, multimedia, opportunistic computing 2 Stephen Childs Virtual machines, Grid, system management software 3 Gabriele Pierantoni Software agents, Web services, Globus Toolkit The programme of research will be conducted within the Computer Architecture and Grid Research Group of Trinity College Dublin (https://www.cs.tcd.ie/research groups/cag/), and so the governance and management is very self-contained. The project coordinator (Dr. Brian Coghlan, research group leader) has overall responsibility for the project schedule and budget, as well as for reporting to SFI. He will track project expenditure and provide the interface to administrative officers at SFI and TCD. He will also have overall responsibility for research outputs, in particular liaison with TCD’s Research Office to ensure that the maximum benefit is obtained from IP generated by the project. The technical progress of the project will be monitored by the Project Executive, composed of the proposers. They are responsible for taking a project-wide view of progress, monitoring expenditure, and coordinating the protection of IP. The regular operational meetings are as follows: • Weekly WP Progress Updates: short per-WP review meetings for members of each WP focussing on technical progress. Collaborators from other WP will participate when required. • Monthly Project Management Meetings: detailed technical assessments of project progress relative to milestones with a focus on WP interdependencies and associated risks. • Half-Yearly All Hands Meetings: presentation of technical results achieved by WPs to the whole project team to promote deeper collaboration and feedback within the project.

14 Major milestones and deliverables are (M for milestone / D for deliverable): Year Item Detail 1 M1.1 Completion of requirements analysis D1.1 Analysis of motivating applications requirements D1.2 Analysis of infrastructural requirements of stream processing applications D1.3 Analysis of stream processing extension (SPE) requirements 2 M2.1 Completion of basic stream processing prototypes D2.1 Prototype of motivating streaming applications D2.2 Prototype of basic infrastructure-on-demand (IOD) services D2.3 Prototype of basic stream processing extensions 3 M3.1 Advanced stream processing prototypes D3.1 Advanced IOD services, e.g. agent-based discovery, specialized processing architectures D3.2 Advanced SPEs, e.g. workflow agents, access mechanisms, stream query processing 4 M4.1 Performance analysis, software tuning, documentation and final demonstration system D4.1 Documentation, final services and final demonstration system D4.2 Final motivating applications, integrated with on-demand services 6 Gantt Chart

Figure 6: Project Gantt Chart

15 References [1] Large hadron collider (LHC). URL http://lhc.web.cern.ch [2] The large hadron collider beauty experiment (LHCb). URL http://lhcb.web.cern.ch [3] HESS Project. URL http://www.mpi-hd.mpg.de/hfm/HESS/HESS.html [4] Cherenkov telescope array (CTA). URL http://www.mpi-hd.mpg.de/hfm/CTA/ [5] Solar dynamics observatory (SDO). URL http://sdo.gsfc.nasa.gov/ [6] GridICE. URL http://gridice.forge.cnaf.infn.it/ [7] S. Fisher, Relational model for information and monitoring (2001). [8] B. Coghlan, A. Djaoui, S. Fisher, J. Magowan, M. Oevers, Time, information services and the grid, in: Proc.BNCOD 2001 - Advances in Database Systems, edited by O’Neill, K.D., and Read, B.J., RAL-CONF-2001-003, Oxford, 2001. [9] The relational grid monitoring architecture (R-GMA). URL http://www.r-gma.org/ [10] H.323. URL http://www.itu.int/rec/T-REC-H.323/e [11] Access Grid. URL http://www.accessgrid.org/ [12] Microsoft DirectX. URL http://msdn.microsoft.com/directx [13] OpenGL. URL http://www.opengl.org [14] G. Humphreys, M. Houston, Y. Ng, R. Frank, S. Ahern, P. Kirchner, J. Klosowski, Chromium: A stream processing framework for interactive graphics on clusters, in: Proceedings of the 29th International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH), 2002. [15] R. Watson, S. Maad, B. Coghlan, Multiscale multimodal visualization on a grid, in: Proc. Cracow Grid Workshop 2006 (CGW06), Academic Computer Centre CYFRONET AGH, Cracow, Poland, 2006. [16] R. Watson, S. Maad, B. Coghlan, Integrating a common visualization service into a metagrid, in: Proc. Workshop on State-of-the-Art in Scientific and Parallel Computing (PARA06), Umea, Sweden, 2006. [17] Joost. URL http://www.joost.com [18] J. Gemmell, H. Sundaram, Capture, archival, and retrieval of personal experience, IEEE Multimedia (2006) 18–19. [19] J. Gemmell, G. Bell, , R. Lueder, MyLifeBits: a personal database for everything, Communications of the ACM 49 (1) (2006) 88–95. [20] J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R. Sparks, M. Handley, E. Schooler, SIP: Session initiation protocol, IETF RFC 3261 (June 2002). [21] Skype. URL http://www.skype.com/ [22] Microsoft Netmeeting. URL http://www.microsoft.com/windows/NetMeeting/default.asp [23] Polycom. URL http://www.polycom.com [24] VRVS. URL http://www.vrvs.org [25] EVO. URL http://evo.vrvs.org [26] L. Rowe, R. Jain, ACM SIGMM retreat report on future directions in multimedia research, ACM Transactions on Multimedia Computing, Communications and Applications 1 (1) (2005) 3–13. [27] A. Smeaton, Techniques used and open challenges to the analysis, indexing and retrieval of digital video, Information Systems Journal (in print). 1 [28] M. Lew, N. Sebe, C. Djeraba, R. Jain, Content-based multimedia information retrieval: state of the art and challenges, ACM Transactions on Multimedia Computing, Communications and Applications (2006) 1–19. [29] G. Palis, A. Vakali, Insight and perspectives for content delivery networks, Communications of the ACM 49 (1) (2006) 101–106. [30] V. Padmanabhan, H. Wang, P. Chou, K. Sripanidkulchai, Distributing streaming media content using cooperative networking, in: 14th International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV), 2004. [31] M. Castro, P. Druschel, A. M. Kermarrec, A. Nandi, A. Rowstron, A. Singh, SplitStream: High-bandwith multicast in cooperative environments, in: 19th ACM Symposium on Operating Systems Principles (SOSP), 2003. [32] O. Harrison, J. Waldron, Aes encryption implementation and analysis on commodity graphics processing units, in: Proc. Workshop on Cryptographic Hardware and Embedded Systems, Vienna, Austria, 2007, (To appear). [33] AutoDock. URL http://autodock.scripps.edu [34] FightMalariaAtHome. URL http://bioinformatics.rcsi.ie:8080/mmg/faces/Projects.jsp?form1: relatedLinkMalaria_submittedField=form1:relatedLinkMalaria [35] eHiTS. URL http://www.simbiosys.ca/ehits/ [36] G. Pierantoni, E. Kenny, B. Coghlan, A prototype of a social and economic based resource allocation system in grid computing, in: Proc. 6th International Symposium on Parallel and Distributed Computing (ISPDC 2007), IEEE Computer Society Press, Hagenberg, Austria, 2007. [37] BOINC. URL http://boinc.berkeley.edu/ [38] FightAids@Home. URL http://fightaidsathome.scripps.edu [39] Folding@Home. URL http://folding.stanford.edu [40] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, A. Warfield, Xen and the Art of Virtualization, in: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, ACM, 2003. [41] K. Keahey, I. Foster, T. Freeman, X. Zhang, Virtual Workspaces: Achieving Quality of Service and Quality of Life in the Grid, Scientific Programming Journal. [42] I. Foster, Globus toolkit version 4: Software for service-oriented systems, in: IFIP International Conference on Network and Parallel Computing, no. 3779 in LNCS, Springer-Verlag, 2005, pp. 2–13. [43] R. G. Leiva, M. B. Lopez, G. C. Melia, B. C. Marco, L. Cons, P. Poznanski, A. Washbrook, E. Ferro, A. Holt, Quattor: Tools and Techniques for the Configuration, Installation and Management of Large-Scale Grid Computing Fabrics, Journal of Grid Computing 2 (4). [44] Cooperative Linux. URL http://www.colinux.org/ [45] M. Litzkow, M. Livny, M. Mutka, Condor - A hunter of idle workstations, in: Proceedings of the 8th International Conference of Distributed Computing Systems, 1988. [46] G. Pierantoni, E. Kenny, B. Coghlan, An agent-based architecture for grid societies, in: PARA 06, Umea, Sweden, 2006. [47] G. Pierantoni, B. Coghlan, O. Lyttleton, D. O’Callaghan, E. Kenny, S. Maad, G. Quigley, Social grid agents as a metagrid technology: An approach for flexible resource allocation in heterogeneous grid middlewares, in: Proc. Cracow Grid Workshop 2006 (CGW06), Academic Computer Centre CYFRONET AGH, Cracow, Poland, 2006. [48] G. Pierantoni, E. Kenny, B. Coghlan, An architecture based on a social-economic approach for flexible grid resource allocation, in: Cracow Grid Workshop (CGW05), Cracow, Poland, 2005. [49] Microsoft DirectShow. URL http://msdn2.microsoft.com/en-us/library/ms783323.aspx [50] M. Lohse, M. Repplinger, P. Slusallek, An open middleware architecture for network-integrated multimedia, in: Joint International Workshop on Interactive Distributed Multimedia Systems / Protocols for Multimedia Systems, 2002. 2 [51] D. J. Abadi, Y. Ahmad, M. Balazinska, U. Cetintemel, M. Cherniack, J.-H. Hwang, W. Lindner, A. S. Maskey, A. Rasin, E. Ryvkina, N. Tatbul, Y. Xing, S. Zdonik, The design of the borealis stream processing engine, in: In Proceedings of the 2nd Biennial Conference on Innovative Data Systems Research (CIDR’05), 2005. [52] S. Chandrasekaran, O. Cooper, A. Deshpande, M. J. Franklin, J. M. Hellerstein, W. Hong, S. Krishnamurthy, S. Madden, V. Raman, F. Reiss, M. Shah, TelegraphCQ: Continuous dataflow processing for an uncertan worl, in: In Proceedings of the 1st Biennial Conference on Innovative Data Systems Research (CIDR’03), 2003. [53] S. Childs, B. Coghlan, D. O’Callaghan, G. Quigley, J. Walsh, E. Kenny, A virtual testgrid or how to replicate a national grid, in: Proceedings of the EXPGRID workshop on Experimental Grid testbeds for the assessment of large-scale distributed applications and tools, Paris, 2006. [54] B. Coghlan, J. Walsh, D. O’Callaghan, Grid-ireland deployment architecture, in: P. M. Sloot, A. G. Hoekstra, T. Priol, A. Reinefeld, M. Bubak (Eds.), Advances in Grid Computing - EGC 2005, LNCS3470, Springer, Amsterdam, The Netherlands, 2005.