Towards Web-Scale Workflows for Film Production Chun Ouyang Marcello La Rosa Arthur H.M. ter Hofstede Marlon Dumas Queensland University of Technology, Australia University of Tartu, Estonia Katherine Shortland Australian Film, Television, and Radio School, Australia Abstract quent. For example, the production manager – who makes sure all departments operate within the budget and time The screen business encompasses all creative and man- constraints – often has to wait until the day after to fin- agement aspects related to film, television, and new me- ish the previous day’s shooting progress report, due to de- dia content, from concept to production and distribution. lays in the completion of the on-set documents by other Companies in this industry face increasing competition stakeholders. There is an opportunity to optimize and due to market globalisation. To stay competitive, they automate film production processes to reduce production are turning to contemporary technology-enabled business costs. Moreover, by saving time otherwise spent in costly improvement methods, such as business process manage- and tedious activities, the production team can invest more ment. Processes in the screen business, particularly film on creative activities, like the shooting, thus increasing the production, generally consist of highly interdependent quality of the final product. steps that manipulate heterogeneous data and involve a va- Despite its potential benefits, the use of workflow man- riety of stakeholders in a distributed and mobile work en- agement for film production is a direction yet to be ex- vironment. Despite its potential benefits, the use of work- plored. Major challenges hindering the application of flow systems for automating film production processes is workflow systems in this domain include: largely unexplored. This paper presents a case study that • The variety of independent entities involved. highlights some of the key challenges that lie ahead on the road to Web-scale workflows for film production. • The distribution and mobility of stakeholders. • The degree of data heterogeneity. Film Production Automation: Requirements • The need for high degree of flexibility. The screen business is characterized by business pro- These requirements closely match those that web-scale cesses with high demands for creativity and flexibility. workflow management is meant to fulfill [3]. Indeed, web- These processes span a value chain consisting of four ma- scale workflows promote the encapsulation of capabili- jor phases: development, pre-production, production, and ties as Web services with self-described and openly ac- post-production. The production phase involves many cessible interfaces, in line with the principles of Service- stakeholders and it is usually the most expensive phase. Oriented Architectures (SOA). These independent ser- For example, most cast and crew are contracted during vices are composed and orchestrated by means of a work- production. Furthermore, additional costs are associated flow system that is itself structured according to the prin- to the rental of the shooting equipment, such as cameras, ciples of SOA. The resulting web-scale workflow archi- cranes, and action vehicles. tecture naturally supports the coordination of independent The production process includes daily shooting activ- and distributed entities in a flexible manner, while the use ities over a period of weeks or months. Shooting activ- of the eXtensible Markup Language (XML) across the ar- ities include acting, camera and sound recording. These chitecture addresses the data heterogeneity requirement. activities are interdependent and involve heterogeneous Below, we articulate the results of a hands-on investi- data, e.g. logs and technical notes, time-sheets for cast and gation into the automation of film production processes crew, daily shooting progress report, next-day’s shooting using an open-source workflow management system, schedule, and revisions of cast, crew and locations. namely YAWL (Yet Another Workflow Language) [1]. At present, shooting is a highly manual activity. It This ongoing work has led to the development of an ap- involves processing rather large amounts of data on a plication platform, namely YAWL4Film, that exploits the daily basis and coordinating many geographically dis- principles of web-scale workflow in order to coordinate tributed stakeholders, which is time-consuming and error- work distribution within production teams, to automate prone. Not surprisingly, delays in the schedule are fre- the collection of documents and data on a daily basis, 1 Figure 1. The YAWL system architecture: the YAWL services in the coordination layer access the data layer to support one or more functions in the business logic layer and/or to interact with users via the presentation layer. to generate reports, to ensure data synchronization across which the set of active work items can be queried, work different (disconnected) nodes, and to document experi- items can be checked-out (indicating the start of the work) ences gained in a production project (especially with re- and checked-in (indicating completion of the work). Since spect to exception resolution) for reuse in future projects. communication with the worklist handler (as well as other services in the YAWL system) is via XML, it is possible to The YAWL System build customized Web applications on top of the worklist handler to expose work-lists and work-items to end users. The YAWL system is structured according to the prin- The system is shipped with a default renderer that gen- ciples of SOA. It consists of a number of independent erates Web forms with a basic layout. Alternatively, the services that expose endpoints accessible through stan- forms connector service can be combined with the work- dard technology – XML over Hypertext Transfer Protocol list handler to enable connections to custom-made Web (HTTP) – and described by means of publicly accessible forms. The organization and storage of data entries may interfaces. The architecture, shown in Figure 1, follows a be delegated to a document management service. multi-tier model where services composing the workflow The routing of manual work items is governed by a system form a coordination layer blended between the tra- role-based access control mechanism handled via the re- ditional data and business logic layers. source service and based on the task-role associations The core service is the workflow engine. This service is specified in the process model. Roles and their capabilities responsible for creating and routing work items according are defined in an organization model and can be loaded to to a YAWL process model and managing the coordination the resource service via an administration interface. data (e.g. active tasks and execution traces). A work item The data entered by the user through a Web form is is an instantiation of a task in a process model, together validated by the data handler. This service also provides with its associated data. The workflow engine routes work data manipulation and aggregation capabilities. For exam- items either to a user (manual task) or to software appli- ple, the data handler may be used to generate reports by cations exposed as services in the coordination layer (au- aggregating data from multiple work items. Aggregation tomatic task). functions are defined as XQuery expressions. The worklist handler is responsible for offering and al- The worklet service allows users to dynamically locating manual work items to users and transferring the change the process model at runtime, by plugging self- associated data. This service provides an interface through contained sub-processes (called worklets) drawn from a 2 repository. This capability, offered via the worklet inter- list”, “location notes”, and “shooting schedule”) available face, is used to handle both expected and unexpected ex- from the pre-production phase. Next, the shooting pro- ceptions and to store information allowing users to better cess starts and is carried out on a daily basis. Each day, deal with such exceptions in future occasions. tasks are performed along two main parallel streams. One stream focuses on the production of a “call sheet”, as cap- Why YAWL? tured by the flow of tasks starting from Begin Call Sheet In the screen business, information needs to be available to Finish Call Sheet. A “call sheet” is a daily shooting to the production team at the right time and with a pro- schedule for a specific day. It is usually maintained by the fessional look & feel. The YAWL language, based on in- Production Office and is sent out to all cast and crew one sights gained from the workflow patterns research [2] and day in advance. A draft call sheet can be created from the on concepts from Petri nets, can capture sophisticated or- shooting schedule. It may go through any number of revi- der dependencies among tasks. sions before it is finalized, and most of the revisions result The production process involves many stakeholders from the changes to the “shooting schedule”. and involves complex data that need to be validated, an- The other stream is specified by the flow of tasks start- alyzed and aggregated for decision-making and report ing from Kick Off Onset to Distribute DPR. At first, tasks generation. The YAWL system offers such capabilities are executed on-set to record the logs and technical notes and provides a resource service that supports complex re- about individual shooting activities into a number of docu- source allocation policies.