Orc-X: Combining Orchestrations and XQuery (work in progress) Kristi Morton David Kitchin William Cook University of Texas at Austin fkmorton, dkitchin, [email protected] Abstract combinators aggregate invocations of services into orchestrated, In designing a language for distributed computing, the handling structured concurrent programs. Structured concurrency [14] disal- of data and distribution can be viewed as largely orthogonal con- lows arbitrary message passing and constrains communication and cerns, as long as the data model supports the communication re- synchronization, in the same way that structured programming [8] quirements of the distribution model. This view contrasts strongly disallows arbitrary jumps and constrains program control flow. Be- with approaches based on distributed objects, which typically en- cause of these restrictions, Orc’s combinators obey strong algebraic force a tight coupling of state and behavior. We have previously laws, and Orc programs are easier to reason about locally. presented Orc, a language that provides simple but powerful con- The original design of the Orc language focused on concur- structs to orchestrate distributed computations. Previous versions of rency constructs and included only basic scalar data types. The data Orc included only simple data types, since these were sufficient to model was considered orthogonal to the main intent of Orc, which demonstrate the concurrency primitives. However, Orc’s commu- is concurrency and communication. However, without a data model nication model is based on web services, which support complex it is difficult to write large applications or interface with complex XML documents in addition to simple data types. Thus it is natural services. We have chosen to extend Orc by embedding support for to consider XML as an appropriate data model for Orc. We present the XML [5] data model. XML has proven to be a robust data model Orc-X, an extension of the Orc language with an XML data model for communication in many wide-area systems by supporting com- and XML-specific data management capabilities from XQuery. We plex structured data models that can be extended as the system demonstrate that Orc-X is well-suited for the application domain evolves. Since Orc is a communication-centric language, the in- of distributed resource management protocols such as the Narada ternal data model should fit well with the communication model. mesh overlay protocol. Furthermore, Orc’s intended use is in orchestrating web services, which rely on XML for messaging and representing data content. Categories and Subject Descriptors D.1.3 [Concurrent Pro- Finally, XML is a natural fit for Orc because it is easy to serialize gramming]: Distributed programming and ship XML data across a network. General Terms Languages, Design We present Orc-X, an extension of the Orc programming lan- guage which enhances structured concurrency compositions from Keywords XQuery, Orc, orchestration, Narada Orc [18] with XML data and XQuery processing. XQuery [3] is a declarative query language for XML-based applications. XQuery 1. Introduction leverages XML for representing semi-structured data as well as ef- Modern applications increasingly require support for concurrent ficiently querying XML data. computation and management of complex, distributed data. For We demonstrate Orc-X by showing an example implementa- example, the emerging area of grid computing has led to increasing tion of the Narada overlay protocol. This distributed protocol en- focus on applications that manage distributed resources to create tails sending periodic, asynchronous messages around an overlay overlay networks or other network services. These applications network for managing its structure. The structure of the overlay must be resilient to failure of nodes and communication. Such network is encoded in the node-local state, which includes neigh- applications operate in a dynamic environment, in which nodes bor and group memberships as well as routing tables. This infor- join and leave the network. They should also support evolution, mation is naturally represented in XML and we leverage XQuery by allowing a single node to be upgraded without bringing down to query and manage the semi-structured XML data at each node. the complete network. Current programming languages and models Furthermore, since the Narada protocol sends concurrent messages do not facilitate building these kinds of distributed data-oriented that probe for changes to the overlay network, Orc-X is well-suited applications. to orchestrate the concurrent invocation of these messages, while The Orc language is a domain-specific language that targets managing time-outs and communication failures that inevitably applications with distributed and concurrent computations. Orc’s arise in distributed environments. Finally, the Narada implemen- tation demonstrates the potential of Orc’s combinators, which con- cisely and elegantly express distributed computations. Permission to make digital or hard copies of all or part of this work for personal or Organization The rest of this proposal is organized as follows. classroom use is granted without fee provided that copies are not made or distributed Section 2 presents an introduction to Orc, XQuery and Narada. for profit or commercial advantage and that copies bear this notice and the full citation Section 3 presents an introduction to the language concepts of Orc- on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. X and section 4 discusses our Narada implementation. Section 5 Technical Report TR-07-63 presents related work and section 6 discusses the challenges in Copyright c The University of Texas at Austin, Department of Computer Sciences, combining Orc with XQuery as well as the interesting future chal- 2007. Research Scientists: Kristi Morton, David Kitchin and William Cook. lenges of Orc-X in other application domains. 2. Background def Metronome = Rtimer(1) >> Signal | Metronome 2.1 Orc Orc is an executable process calculus that expresses structured con- def WeatherUpdates(ac) = currency. There are three main operators for composing expres- Metronome >> Weather(ac) >m> Email(a,m) sites sions: parallelizing, sequencing and selective pruning. In Orc, The Orc operators are arranged in increasing order of prece- are responsible for performing data operations such as computa- dence as: def, where, |, >x>. The operator >x> is right- tion and communication. A site call can represent a function call or associative, the operator where is left-associative, and the operator web service invocation, for example, and may return at most one | is fully associative and commutative. result. If it never returns a result, we say it is silent. In the follow- ing examples we use the site call Weather(ac) as a web service 2.2 Examples that provides weather forecasts for the given airport code ac, and the site call Email(a,m) as a function that sends a message m to Fork-join Parallelism address a. In concurrent programming, one often needs to spawn two indepen- The sequential composition defines an ordering between two dent threads at a point in the computation, and resume the compu- expressions, written as: f >x> g or simply f >> g. The x in the tation after both threads complete. Such an execution style is called first sequential composition is bound to the value published by f, fork-join parallelism. There is no special construct for fork-join in while the second sequential composition has no variable binding. Orc, but it is easy to code such computations. The following code The following example invokes the Weather web service, binds fragment calls sites M and N in parallel and publishes their values as the result to a message m and e-mails the message to address a. a tuple after they both complete their executions. Weather("AUS") >m> Email(a,m) (let(u,v) where u in M) where v in N The parallel composition allows two expressions f and g to be Time-out composed as: f | g. Here f and g are treated as independent com- Distributed resource management protocols interact in environ- putations and executed asynchronously in parallel. Upon termina- ments that exhibit dynamic behavior. For example, a ping request tion of both expressions there may be up to two results published. to a remote peer may not return a result if the peer’s communica- The results are published as soon as a computation terminates, and tion link is down. We show in section 4 how time-outs are used to no result is published if the computation never terminates. In the mitigate this problem. Because Orc has time built into its seman- following example there could be zero, one or two weather reports tics [20], it is easy to express time-outs. The expression published depending on the responsiveness of the Weather services for Austin and Barcelona. Upon each publication Orc will create a let(z) where z in (f | Rtimer(t) >> let(3)) new thread for invoking Email with m bound to the published web either publishes the first publication of f, or times out after t report within the scope of the new thread. units and publishes 3. A typical programming paradigm is to call (Weather("AUS") | Weather("BCN")) >m> Email(a,m) site M and publish a pair (x,b) as the value, where b is true if M publishes x before the time-out, and false if there is a time-out. In The asymmetric parallel composition is used for selective prun- the latter case, x is irrelevant. Below, z is the pair (x,b). ing, which is written as f where x in g. This construct starts ex- ecuting f and g in parallel. Any subexpression within f that is de- let(z) where z in pendent upon x will suspend executing until x has a value bound. ( M >x> let(x,true) | Rtimer(t) >x> let(x,false) ) When g has published a value, x is bound to the result and any Non-strict Evaluation; Parallel-or subexpressions within g that are still executing are terminated.