A Formal Framework for Component Deployment
Y. David Liu Scott F. Smith Johns Hopkins University
OOPSLA'06, Portland, Oregon A Menagerie of Deployment Systems
CLI Assemblies JSR 277 InstallShield
RPM OSGi
Dpkg EJB Manifests
Portage Bazaar
CORBA D&C RubyGems
CTAN CPAN Foundations?
CLI Assemblies JSR 277 InstallShield
RPM OSGi Dpkg ? EJB Manifests
Portage Bazaar
CORBA D&C RubyGems
CTAN CPAN An Analogy: Programming Languages
Fortran Smalltalk Pascal
Lisp Java
Perl C++
Scala C#
Scheme C
Haskel ML An Analogy: Foundations of Languages
Fortran Smalltalk Pascal
Lisp Java λ Calculus
Perl Object Calculi C++ etc. Scala C#
Scheme C
Haskel ML This Work
CLI Assemblies JSR 277 InstallShield
RPM OSGi
Dpkg Application Buildbox EJB Manifests
Portage Bazaar
CORBA D&C RubyGems
CTAN CPAN This Work An abstract, platform-independent, vendor- independent study of component deployment
● Designing components as deployment units
● Formalizing the entire deployment lifecycle
● Proving deployment invariants Design objectives: simple (capturing recurring themes) and expressive This Work An abstract, platform-independent, vendor- independent study of component deployment
● Designing components as deployment units
● Formalizing the entire deployment lifecycle
● Proving deployment invariants Design objectives: simple (capturing recurring themes) and expressive This Work An abstract, platform-independent, vendor- independent study of component deployment
● Designing components as deployment units
● Formalizing the entire deployment lifecycle
● Proving deployment invariants Design objectives: simple (capturing recurring themes) and expressive 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 5429 5233
NetLib Browser
build ship
execute 1690 (testing) NetLib 5233
Browser
Browser 5429 5233
NetLib Browser This Work An abstract, platform-independent, vendor- independent study of component deployment
● Designing components as deployment units
● Formalizing the entire deployment lifecycle
● Proving deployment invariants
● Deployment ''never goes wrong''
● Version compatibility Design objectives: simple (capturing recurring themes) and expressive This Work An abstract, platform-independent, vendor- independent study of component deployment
● Designing components as deployment units
● Formalizing the entire deployment lifecycle
● Proving deployment invariants Design objectives: simple (capturing recurring themes) and expressive Why Foundations?
● Fosters next-generation deployment systems – Elucidates subtle issues – More features proposed from academic research community – Deployment systems with provably correct properties
● Complements modularity research – when and where of linking Why Foundations?
● Fosters next-generation deployment systems – Elucidates subtle issues – More features proposed from academic research community – Deployment systems with provably correct properties
● Complements modularity research – when and where of linking Basics Application Buildbox
1690 5233
NetLib Browser
5429
NetLib
An imaginary box where an application ''hatches'' throughout the deployment lifecycle Deployment Unit: Assemblage
Net 5233 Plugins
send readfile Browser
timeout start
● Real-world analogues: JAR, C .so library, DLL, CLI Assembly ● Assemblages were first developed in [Liu and Smith, ECOOP'04], but without deployment Version Identifiers
Net 5233 Plugins
send readfile Browser
timeout start
● Globally Unique ● Real-world analogues: COM+ GUID, CLI Assembly strong names Side-by-Side Deployment
1690 5233
NetLib Browser
5429
NetLib
Two versions of the NetLib are deployed in the same buildbox Basic Construct: Assemblage Interfaces
Net 5233 Plugins
send readfile Browser
timeout start
Real-world analogues: Manifest files, Deployment Descriptors Two Kinds of Assemblage Interfaces
Net 5233 Plugins
send readfile Browser
timeout start
Mixers: regular dependency Pluggers: hot deployment dependency Interfaces are Bi-directional: Imports, Exports
Net 5233 Plugins
send readfile Browser timeout start Multiple Interfaces
Net 5233 Plugins
send readfile Browser
timeout start
GUI
initGraphics draw
● Name management is crucial for deployment.
● Avoid global name clashes Interface: Unit of Versioning Dependencies
5233 Net Plugins
Browser
GUI initGraphics draw
initGraphics draw
GUILib
0872 What is NOT Possible...
5233 Net Plugins
Browser
GUI initGraphics draw
initGraphics draw initGraphics draw
GUILib GUILib
0872 5422 Assemblages in Shipped Form
Net 5233 Plugins
send readfile Browser
timeout start
Net -> NetLib.1690.Socket version constraint Component Wiring: Mixing
1690 5233 Socket Net Plugins
send NetLib Browser timeout
Net -> NetLib.1690.Socket
● Between a pair of mixers ● Matching of functionalities ● Matching of version constraints Component Wiring: Plugging
5233 3265 Plugins Main
readFile Browser Flash start
Main -> Browser.5233.Plugins
● Wiring at hot deployment time ● Between a plugger and a mixer ● Matching of functionalities ● Matching of version constraints Compatibility Set
1690 5233 Socket Net Plugins
NetLib Browser
3370 < : 1690 Net -> NetLib.1690.Socket
● Subversioning: a partial order ● We do not hardcode the strategy on how two versions are semantically compatible Act 2: Component Deployment Lifecycle Deployment 1690 5233 1690
Site NetLib Browser NetLib Transitions 5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
5429 5233 3265 Flash NetLib Browser Flash hot deploy 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy Development Site Transitions
5429 5233
NetLib Browser
build ship
execute 1690 (testing) NetLib 5233
Browser
Browser 5429 5233
NetLib Browser 5429 5233
NetLib Browser
build ship
execute 1690 (testing) NetLib 5233
Browser
Browser 5429 5233
NetLib Browser 5429 5233
NetLib Browser
build ship
execute 1690 (testing) NetLib 5233
Browser
Browser 5429 5233
NetLib Browser 5429 5233
NetLib Browser
build ship
execute 1690 (testing) NetLib 5233
Browser
Browser 5429 5233
NetLib Browser Formalism Choice
● Labelled Transition System (LTS) for deployment operations – Each transition step is an application buildbox evolution step – Labels are ''commands'' which deployment system users can trigger
● Run-time behaviors captured via a minimalistic programming language Shipping a Component
1690 Socket Net 5233 Plugins
NetLib Browser
ship (Browser, 5233, {Net}) Shipping a Component
Shipped Assemblage
Net 5233 Plugins 1690 Socket Net 5233 Plugins Browser NetLib Browser
Net -> NetLib.1690.Socket
ship (Browser, 5233, {Net}) Why Not Always Ship the Entire Closure?
1690 Socket Net 5233 Plugins
NetLib Browser Why Not Always Ship the Entire Closure?
1690 Socket Net 5233 Plugins
NetLib Browser
● Components are independently deployable units!
● Off-the-shelf commercial components, libraries
● Updates, patches
● Sometimes not realistic, such as native code Installing a Component
shippedbrowser
3370 Socket Net 5233 Plugins
NetLib Browser
3370 < : 1690 Net -> NetLib.1690.Socket
install (shippedbrowser) Installing a Component
3370 Socket Net 5233 Plugins
NetLib Browser
Net -> NetLib.1690.Socket
3370 < : 1690 install (shippedbrowser) Cyclic Dependencies
shippedA shippedB
P 7421 Q 0088
A B
P -> B.0088.Q Q -> A.7421.P
Example: System.dll and System.xml.dll in .NET Cyclic Dependencies
shippedA shippedB
7421 P Q 0088
A B
P -> B.0088.Q Q -> A.7421.P
install (shippedA) Cyclic Dependencies
shippedB
7421 P Q 0088
A B
P -> B.0088.Q Q -> A.7421.P
install (shippedA) Cyclic Dependencies
shippedB
7421 P Q 0088
A B
P -> B.0088.Q Q -> A.7421.P
install (shippedB) Cyclic Dependencies
7421 P Q 0088
A B
P -> B.0088.Q Q -> A.7421.P
install (shippedB) Updating a Component
7622 Socket Net 5233 Plugins
NetLib Browser
Net -> NetLib.1690.Socket 9985 Socket
NetLib
9985 <: 1690, 7622 <: 1690 update (NetLib, 7622, 9985) Updating a Component
7622 Socket Net 5233 Plugins
NetLib Browser
Net -> NetLib.1690.Socket 9985 Socket
NetLib
9985 <: 1690, 7622 <: 1690 update (NetLib, 7622, 9985) Updating a Component
7622 Socket Net 5233 Plugins
NetLib Browser
Net -> NetLib.1690.Socket 9985 Socket
NetLib
9985 <: 1690, 7622 <: 1690 an update is not necessarily an upgrade Hot Deployment
flash Running application
3265 Main 7622 Socket Net 5233 Plugins
send readfile Flash NetLib Browser start timeout Main -> Browser.5233.Plugins
h = plugin flash with Plugins >> Main; Hot Deployment
Running application
7622 Socket Net 5233 Plugins 3265 Main send readfile NetLib Browser Flash start timeout
h = plugin flash with Plugins >> Main; Hot Deployment
Running application
7622 Socket Net 5233 Plugins 3265 Main send readfile NetLib Browser Flash start timeout
h = plugin flash with Plugins >> Main; h..start(); Multiple Plugins: Hot Update
3265 Main Flash 7622 Socket Net 5233 Plugins
send readfile NetLib Browser start 3211 timeout Main Flash
h1 = plugin flash1 with Plugins >> Main; ... h2 = plugin flash2 with Plugins >> Main; Act 3:
Invariants, Invariants! Theorems: Buildbox Well-formedness
● Theorem: no deployment operations can turn a well-formed buildbox into a non-well- formed one.
● Theorem: no reductions at run time can turn a well-formed buildbox into a non-well- formed one. 1690 5233 1690
NetLib Browser NetLib
5429 5429 5233
NetLib NetLib Browser update
install remove
1690 5429 5233
NetLib 3265 NetLib Browser
Flash 5429 5429 5233
NetLib NetLib Browser 4423 execute
Flash 5233 5233 5429 Browser Browser NetLib hot update 3265
Flash 5429 5233 3265 NetLib Browser Flash hot deploy 5429 5233
NetLib Browser
build ship
execute 1690 (testing) NetLib 5233
Browser
Browser 5429 5233
NetLib Browser Specifying Version Compatibility
How do a deployment-site run and a pre- shipping test-run correspond? Suppose we have a component X...
2700 P X m method n ... n int z = P::m(3); ... locating method m imported/exported from P On The Development Site
2700 m P X On The Development Site
2700 m P X
execute (testing)
2700 m P X On The Development Site
2700 m P X
execute (testing)
2700 m P X at run time P::m is bound to assemblage Y version v On The Development Site
2700 m P X
ship (X, 2700, {P})
2700 m P execute X (testing)
2700 m P X at run time P::m is bound to assemblage Y version v On Any Deployment Site
2700 install m P X On Any Deployment Site
2700 install m P X m P
X 2700 On Any Deployment Site
2700 install m P X m P
X 2700
. anyLTS steps
m P
X 2700 On Any Deployment Site
2700 install m P X m P
X 2700
. any LTS steps
execute m m P P
X 2700 X 2700 On Any Deployment Site
2700 install m P X m P
X 2700 at run time . any LTS steps P::m is bound to . assemblage Y' version v'
execute m m P P
X 2700 X 2700 Theorem on Version Compatibility
● Y = Y'
● v = v' or v' is a subversion of v Future Work
● Keep the platform-independent spirit, with more expressiveness gains – security in deployment – distributed deployment (e.g. sensor network applications)
● A closer look at Java deployment – an effort to map back to the real world Related Work
● Many real-world systems
● Formal treatment is rare – [Buckley, CD'05]: formalized name-binding of CLI Assemblies
● platform-specific ● no modeling of deployment lifecycle ● no invariant properties proved Related Work: Real-world Systems
CLI Assemblies JSR 277 InstallShield
RPM OSGi
Dpkg Application Buildbox EJB Manifests
Portage Bazaar
CORBA D&C RubyGems
CTAN CPAN Related Work
● Many real-world systems
● Formal treatment is rare – [Buckley, CD'05]: formalized name-binding of CLI Assemblies
● platform-specific ● no modeling of deployment lifecycle ● no invariant properties proved A Retrospective
● For deployment systems designers: – platform-independent communication – foster next-generation deployment systems
● For deployment system users: – tools with well-defined user interfaces – tools with provably correct properties
● For module system researchers: – a foundational study of when and where of linking