Running Pharo on the GemStone VM James Foster VP of Finance & Operations, GemTalk Systems LLC ESUG 2017 – Maribor, Slovenia 4 September 2017 Agenda • GemStone/S Introduction • Replacing Base Class Libraries • Questions 2 Limitations of traditional Smalltalks • Object space (image) must fit in (virtual) RAM • Object space visible to one (single-threaded) VM • Sharing objects between VMs is difficult – Convert to non-object format (binary, XML, SQL) – No built-in object identity for multiple exports • Object state is lost when VM exits 3 Welcome to the magical world of GemStone • Object space limited by disk, not RAM • Object space shared across multiple VMs on multiple hosts • Transactional persistence • Image from http://www.flickr.com/photos/laffy4k/182219003/ • Creative Commons License http://creativecommons.org/licenses/by-sa/2.0/ 4 GemStone/S Architecture 5 Gem Types • Linked Gem OS Process Application & – Gem in application process space GCI Library Gem • Remote Procedure Call (RPC) Gem – GCI library in application space – Gem has separate process OS Process 1 OS Process 2 Application & GCI Library TCP/IP Gem 6 One-Machine Process Locations (Linked Gem) Stone SPC Application & GCI Library Gem Repository 7 One-Machine Process Locations (RPC Gem) NetLDI Stone SPC Application & GCI Gem Repository Library 8 Two-Machine Process Locations (Gem on Stone Host) Client Host Stone Host NetLDI N Stone E T W SPC O R K Application & GCI Gem Repository Library 9 Two-Machine Process Locations (Gem Remote from Stone) Gem Host Stone Host NetLDI N NetLDI E Stone T SPC Page W Server O R K Application Gem Repository & GCI Page Library Server Remote SPC 10 Three-Machine Process Locations Client Host Gem Host Stone Host NetLDI N NetLDI N Stone E E T T SPC W Page W O Server O R R K K Application Gem Repository & GCI Page Library Server Remote SPC 11 Shared Page Cache 12 Agenda • GemStone/S Introduction • Replacing Base Class Libraries • Questions 13 Replacing Base Class Libraries • Smalltalk was intended to allow exploration • Portability – ANSI – Grease • Can we go to a lower level? • One approach to learn more 14 Option 1: Complete Replacement • Not possible in most Smalltalks – Creating methods requires code • GemStone schema editing with Topaz – Build process starts with a few base classes – But no methods for any class (initially) • Challenges – Difficult to debug (most tools are internal) – VM has knowledge of schema 15 Option 2: Parallel Class Hierarchy • GemStone’s namespace model allows separate hierarchy 16 Pharo Global Lookup Image a CompiledMethod Smalltalk (a SmalltalkImage) globals a GlobalVariable a SystemDictionary (Association) key a Symbol value an Object 17 GemStone/S Global Lookup Repository a CompiledMethod AllUsers a UserProfile (a UserProfileSet) symbolList a SymbolList a SymbolDictionary a SymbolAssociation (Array) key a Symbol value an Object 18 Unique or Shared SymbolDictionary Instances Repository AllUsers Bob Carol 1. UserGlobals 1. UserGlobals a SymbolList a SymbolList 2. Globals 3. Published SymbolDictionary instances 19 Unique or Shared SymbolAssociation Instances Bob’s SymbolList Carol’s SymbolList 1. UserGlobals 1. UserGlobals 2. Globals assoc assoc key key #Array value value Array 20 Unique Keys Bob’s SymbolList Carol’s SymbolList 1. UserGlobals 1. UserGlobals 2. Globals assoc assoc key #Array #List key value value Array 21 Unique Values Bob’s SymbolList Carol’s SymbolList 1. UserGlobals 1. UserGlobals 2. Globals assoc assoc key key #OrderedCollection value OrderedCollection value OrderedCollection 22 Problems with Parallel Class Hierarchy • GemStone’s namespace model allows separate hierarchy • But complications exist: – Literals – Classes known to the VM 23 Complication: Literals (and their superclasses) • Array: #() • BlockClosure: [] • Boolean: true, false • ByteArray: #[1 2 3] • Character: $a • Float: 1.23 • SmallInteger: 42 • String: ‘Smalltalk’ • Symbol: #Array • UndefinedObject: nil 24 Complication: Classes Known to the VM • Behavior, Class, Metaclass • Exception, MessageNotUnderstood, ZeroDivide, … • Pragma • Process • ProcessorScheduler • So, we need to use (some) base classes 25 Problem: Conflicting Implementations • Array>>printOn: – Pharo • #(1 2 3) printString '#(1 2 3)' – GemStone • #(1 2 3) printString 'anArray( 1, 2, 3)' 26 Option 3: Parallel Methods in GemStone Classes • Each GemStone class has a collection of MethodDictionary instances – Methods are compiled into an “environment” – Message sends to same environment (by default) • Method environments: 0 = GemStone/S (default) 1 = Maglev (reserved for Ruby) 2+ are for others – We use 2 for Pharo 27 Pharo Method Dictionaries Sequenceable Sequenceable Collection Collection class a MethodDictionary a MethodDictionary Arrayed Arrayed Collection Collection class a MethodDictionary a MethodDictionary Array Array class a MethodDictionary a MethodDictionary 28 GemStone Method Dictionaries - 1 Sequenceable Sequenceable Collection Collection class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] Array Array class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] 29 GemStone Method Dictionaries - 2 Sequenceable Sequenceable Collection Collection class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] Arrayed Arrayed Collection Collection class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] Array Array class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] 30 Sample Messages • Implicit – “in env 0” #(1 2 3) printString 'anArray( 1, 2, 3)' – “in env 2” #(1 2 3) printString '#(1 2 3)' • Explicit – #(1 2 3) @env0:printString 'anArray( 1, 2, 3)' – #(1 2 3) @env2:printString '#(1 2 3)’ Array printOn: aStream a MethodDict [0] “for GemStone” method nil dicts printOn: aStream a MethodDict [2] “for Pharo” 31 Problems with Using GemStone Classes • GemStone class might have different schema – OrderedCollection in Pharo • Instance variables: (array firstIndeX lastIndeX) – OrderedCollection in GemStone • No instance variables – Since OrderedCollection is not known to the compiler, we don’t have to use GemStone’s class 32 Options 1. Replace all classes But schema is wrong in some cases And we would like to use tools during development 2. Use parallel classes exclusively But compiler & VM know about some GemStone classes 3. Use parallel methods exclusively But schema is different for some classes 4. Hybrid Use GemStone classes when necessary Use parallel classes otherwise Use method environment 2 for everything 33 Tools • Topaz – GemStone’s command-line interface • GemBuilder for Smalltalk (GBS) – GemStone’s GUI IDE for VA & VW • Others – tODE – gt4gemstone – Jade 34 Issues • Incomplete Class Types – GemStone has in-memory Ephemerons, but no Weak references – No 32-bit objects • GemStone compiler is in VM – Not as easy to experiment with new syntaX • Low-level objects may need extensive rewrite – BlockClosure, Process, ProcessorScheduler, etc. 35 Demo • http://files.pharo.org/get-files/60/pharo-minimal.zip • http://downloads.gemtalksystems.com/pub/GemStone64/3.4.0-Alpha4/ • https://github.com/jgfoster/PharoGs/tree/james/james 36 Questions? James G. Foster VP of Finance & Operations ® GemTalk Systems LLC 15220 NW Greenbrier Pkwy., Suite 240 Beaverton, Oregon, 97006 Voice & Fax: +1 503 766 4714 [email protected] www.gemtalksystems.com 37.