Running on the GemStone VM James Foster VP of Finance & Operations, GemTalk Systems LLC ESUG 2017 – Maribor, Slovenia 4 September 2017 Agenda

• GemStone/S Introduction • Replacing Base Class Libraries • Questions

2 Limitations of traditional Smalltalks • Object space (image) must fit in (virtual) RAM • Object space visible to one (single-threaded) VM • Sharing objects between VMs is difficult – Convert to non-object format (binary, XML, SQL) – No built-in object identity for multiple exports • Object state is lost when VM exits

3 Welcome to the magical world of GemStone

• Object space limited by disk, not RAM • Object space shared across multiple VMs on multiple hosts • Transactional persistence

• Image from http://www.flickr.com/photos/laffy4k/182219003/ • Creative Commons License http://creativecommons.org/licenses/by-sa/2.0/ 4 GemStone/S Architecture

5 Gem Types

• Linked Gem OS Process Application & – Gem in application process space GCI Library

Gem • Remote Procedure Call (RPC) Gem – GCI library in application space – Gem has separate process

OS Process 1 OS Process 2 Application & GCI Library TCP/IP Gem

6 One-Machine Process Locations (Linked Gem)

Stone

SPC

Application & GCI Library Gem Repository

7 One-Machine Process Locations (RPC Gem)

NetLDI Stone

SPC

Application & GCI Gem Repository Library

8 Two-Machine Process Locations (Gem on Stone Host)

Client Host Stone Host NetLDI N Stone E T W SPC O R K Application & GCI Gem Repository Library

9 Two-Machine Process Locations (Gem Remote from Stone)

Gem Host Stone Host NetLDI N NetLDI E Stone T SPC Page W Server O R K Application Gem Repository & GCI Page Library Server

Remote SPC

10 Three-Machine Process Locations

Client Host Gem Host Stone Host NetLDI N NetLDI N Stone E E T T SPC W Page W O Server O R R K K Application Gem Repository & GCI Page Library Server

Remote SPC

11 Shared Page Cache

12 Agenda

• GemStone/S Introduction • Replacing Base Class Libraries • Questions

13 Replacing Base Class Libraries

• was intended to allow exploration • Portability – ANSI – Grease • Can we go to a lower level?

• One approach to learn more

14 Option 1: Complete Replacement

• Not possible in most Smalltalks – Creating methods requires code • GemStone schema editing with Topaz – Build process starts with a few base classes – But no methods for any class (initially) • Challenges – Difficult to debug (most tools are internal) – VM has knowledge of schema

15 Option 2: Parallel Class Hierarchy

• GemStone’s namespace model allows separate hierarchy

16 Pharo Global Lookup

Image

a CompiledMethod Smalltalk (a SmalltalkImage)

globals a GlobalVariable a SystemDictionary (Association)

key a Symbol

value an Object

17 GemStone/S Global Lookup

Repository

a CompiledMethod AllUsers a UserProfile (a UserProfileSet)

symbolList a SymbolList a SymbolDictionary a SymbolAssociation (Array)

key a Symbol

value an Object

18 Unique or Shared SymbolDictionary Instances

Repository

AllUsers

Bob Carol 1. UserGlobals 1. UserGlobals a SymbolList a SymbolList 2. Globals

3. Published

SymbolDictionary instances 19 Unique or Shared SymbolAssociation Instances

Bob’s SymbolList Carol’s SymbolList

1. UserGlobals 1. UserGlobals

2. Globals

assoc assoc key key #Array value value Array

20 Unique Keys

Bob’s SymbolList Carol’s SymbolList

1. UserGlobals 1. UserGlobals

2. Globals

assoc assoc key #Array #List key

value value Array

21 Unique Values

Bob’s SymbolList Carol’s SymbolList

1. UserGlobals 1. UserGlobals

2. Globals

assoc assoc key key #OrderedCollection

value OrderedCollection value OrderedCollection

22 Problems with Parallel Class Hierarchy

• GemStone’s namespace model allows separate hierarchy • But complications exist: – Literals – Classes known to the VM

23 Complication: Literals (and their superclasses)

• Array: #() • BlockClosure: [] • Boolean: true, false • ByteArray: #[1 2 3] • Character: $a • Float: 1.23 • SmallInteger: 42 • String: ‘Smalltalk’ • Symbol: #Array • UndefinedObject: nil

24 Complication: Classes Known to the VM

• Behavior, Class, Metaclass • Exception, MessageNotUnderstood, ZeroDivide, … • Pragma • Process • ProcessorScheduler

• So, we need to use (some) base classes

25 Problem: Conflicting Implementations

• Array>>printOn: – Pharo • #(1 2 3) printString '#(1 2 3)' – GemStone • #(1 2 3) printString 'anArray( 1, 2, 3)'

26 Option 3: Parallel Methods in GemStone Classes

• Each GemStone class has a collection of MethodDictionary instances – Methods are compiled into an “environment” – Message sends to same environment (by default) • Method environments: 0 = GemStone/S (default) 1 = Maglev (reserved for Ruby) 2+ are for others – We use 2 for Pharo

27 Pharo Method Dictionaries

Sequenceable Sequenceable Collection Collection class

a MethodDictionary a MethodDictionary

Arrayed Arrayed Collection Collection class

a MethodDictionary a MethodDictionary

Array Array class

a MethodDictionary a MethodDictionary

28 GemStone Method Dictionaries - 1

Sequenceable Sequenceable Collection Collection class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2]

Array Array class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] 29 GemStone Method Dictionaries - 2

Sequenceable Sequenceable Collection Collection class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2]

Arrayed Arrayed Collection Collection class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2]

Array Array class a MethodDict [0] a MethodDict [0] method nil method nil dicts dicts a MethodDict [2] a MethodDict [2] 30 Sample Messages

• Implicit – “in env 0” #(1 2 3) printString 'anArray( 1, 2, 3)' – “in env 2” #(1 2 3) printString '#(1 2 3)' • Explicit – #(1 2 3) @env0:printString 'anArray( 1, 2, 3)' – #(1 2 3) @env2:printString '#(1 2 3)’

Array printOn: aStream a MethodDict [0] “for GemStone” method nil dicts printOn: aStream a MethodDict [2] “for Pharo”

31 Problems with Using GemStone Classes

• GemStone class might have different schema – OrderedCollection in Pharo • Instance variables: (array firstIndex lastIndex) – OrderedCollection in GemStone • No instance variables – Since OrderedCollection is not known to the compiler, we don’t have to use GemStone’s class

32 Options

1. Replace all classes But schema is wrong in some cases And we would like to use tools during development 2. Use parallel classes exclusively But compiler & VM know about some GemStone classes 3. Use parallel methods exclusively But schema is different for some classes 4. Hybrid Use GemStone classes when necessary Use parallel classes otherwise Use method environment 2 for everything

33 Tools

• Topaz – GemStone’s command-line interface • GemBuilder for Smalltalk (GBS) – GemStone’s GUI IDE for VA & VW • Others – tODE – gt4gemstone – Jade

34 Issues

• Incomplete Class Types – GemStone has in-memory Ephemerons, but no Weak references – No 32-bit objects • GemStone compiler is in VM – Not as easy to experiment with new syntax • Low-level objects may need extensive rewrite – BlockClosure, Process, ProcessorScheduler, etc.

35 Demo

• http://files.pharo.org/get-files/60/pharo-minimal.zip • http://downloads.gemtalksystems.com/pub/GemStone64/3.4.0-Alpha4/ • https://github.com/jgfoster/PharoGs/tree/james/james

36 Questions?

James G. Foster VP of Finance & Operations

®

GemTalk Systems LLC 15220 NW Greenbrier Pkwy., Suite 240 Beaverton, Oregon, 97006 Voice & Fax: +1 503 766 4714 [email protected] www.gemtalksystems.com

37