Programming Languages as Op erating Systems

(or Revenge of the Son of the Lisp Machine)

Matthew Flatt Rob ert Bruce Findler Shriram Krishnamurthi Matthias Felleisen



Department of Computer Science

Rice University

Houston, Texas 77005-1892

reclaim the program's resources|even though the program Abstract

and DrScheme share a single virtual machine.

The MrEd virtual machine serves b oth as the implementa-

To address this problem, MrEd provides a small set of

tion platform for the DrScheme programming environment,

new language constructs. These constructs allow a program-

and as the underlying Scheme engine for executing expres-

running program, suchasDrScheme, to run nested programs

sions and programs entered into DrScheme's read-eval-print

directly on the MrEd virtual machine without sacri cing

lo op. We describ e the key elements of the MrEd virtual

control over the nested programs. As a result, DrScheme

machine for building a programming environment, and we

can execute a copyofDrScheme that is executing its own

step through the implementation of a miniature version of

copy of DrScheme (see Figure 1). The inner and middle

DrScheme in MrEd. More generally,weshowhow MrEd de-

DrSchemes cannot interfere with the op eration of the outer

nes a high-level op erating system for graphical programs.

DrScheme, and the middle DrScheme cannot interfere with

the outer DrScheme's control over the inner DrScheme.

In this pap er, we describ e the key elements of the MrEd

1 MrEd: A Scheme Machine

virtual machine, and we step through the implementation

of a miniature version of DrScheme in MrEd. More gen-

The DrScheme programming environment [10] provides stu-

erally,weshowhow MrEd de nes a high-level op erating

dents and programmers with a user-friendly environment

system (OS) for graphical programs. As in other high-level

for developing Scheme programs. Tomake programming

OSes, safety and securityinMrEdderive from prop erties of

accessible and attractivetonovices, DrScheme provides a

the underlying programming language. Mere safety,how-

thoroughly graphical environment and runs under several

ever, provides neither the level of protection b etween ap-

ma jor windowing systems (Windows, MacOS, and Unix/X).

plications nor the kind of pro cess control that conventional

More than 60 universities and high scho ols currently employ

OSes provide. Such protection and control is crucial for im-

DrScheme in their computing curriculum, and new scho ols

plementing many kinds of programs, includin g programming

adopt DrScheme every semester.

environments and scripting engines. By describing howwe

We implemented DrScheme by rst building MrEd [15],

implemented DrScheme in MrEd, we demonstrate howto

a p ortable Scheme [23] implementation with a graphical user

obtain key OS facilities through small extensions to a high-

interface (GUI) to olb ox. MrEd serves b oth as the implemen-

level programming language.

tation platform for DrScheme, and as the underlying Scheme

The remainder of the pap er is organized as follows. Sec-

engine for executing expressions and programs entered into

tion 2 sketches a miniature DrScheme, called SchemeEsq,

DrScheme's read-eval-printloop(repl). This strategy fol-

and explains in more detail the implementation challenges

lows a long tradition of meta-circular implementation that is

for creating a graphical programming environment. Sec-

virtually synonymous with Lisp, and generally understo o d

tion 3 provides a brief overview of MrEd. Section 4 steps

for high-level languages as a whole [20, 25 , 28 ].

through the implementation of SchemeEsq as a MrEd pro-

Since DrScheme exp oses MrEd's language constructs di-

gram. Section 5 explains how MrEd functions as a high-level

rectly to the repl,DrScheme can easily execute programs

OS. Section 6 discusses some problems for future work, and

that use the full MrEd language, includin g its GUI to ol-

Section 7 compares MrEd to related systems.

box. At the same time, DrScheme must protect its GUI

against interference from the programs it executes, and it

must b e able to halt a program that has gone awry and

2 SchemeEsq: The Challenge



This researchwas partially supp orted by a Lo dieska Sto ckbridge

SchemeEsq, depicted in Figure 2, is a simple programming

Vaughan Fellowship, NSF grants CCR-9619756, CDA-9713032,and

shell that captures the essential prop erties of DrScheme as

CCR-9708957, and a Texas ATP grant.

a program-running program. Roughly,SchemeEsq imple-

ments a read-eval-printloop(repl) that consumes expres-

sions and evaluates them:

(define (repl)

(print (eval (read)))

To app ear: Intl. Conf. on Functional Programming

(repl))

(ICFP) { Sept. 27-29 1999, Paris, France 1

DrScheme DrScheme DrScheme

4 4 4

> (run-drscheme) > (run-drscheme)

>

r r r

/ . / . / .

outer

inner middle

Figure 1: DrScheme in DrScheme in DrScheme

same wayasitwould execute within its own MrEd

virtual machine.

The crucial requirement for SchemeEsq is that it must run

any program securely, in the sense that the program can-

1

not interfere with SchemeEsq's op eration. Indeed, since

SchemeEsq is itself a MrEd program, SchemeEsq must b e

able to run copies of itself anynumb er of times and to

any nesting depth. No matter how manySchemeEsqs are

running and how deeply they are nested, each instance of

SchemeEsq must b e free from interference from its children.

At the same time, a single clicktoReset in the outermost

SchemeEsq must terminate all of the other instances.

3 MrEd Overview

Figure 2: SchemeEsq

MrEd acts as a high-level OS to service the GUI, security,

and scalabili ty needs of SchemeEsq. In the same waythata

current pro duction OS (e.g., Unix/X, Microsoft Windows)

de nes a GUI API, a pro cess mo del, and a library-li nki ng

This rough repl sketch relies on extensive underlying ma-

mechanism, MrEd de nes constructs within Scheme for cre-

chinery to implement each of the read, eval,and print steps.

ating GUIs, securely executing untrusted co de, and linking

SchemeEsq mo dels DrScheme in more detail, showing how

together co de mo dules:

read gets characters from the user, where print sends its out-

put, etc. Furthermore, unlike the repl function, SchemeEsq

 GUI construction | MrEd provides GUI elements via

demonstrates howtocatch errors in the user's program and

built-in classes that a programmer can instantiate or

how to stop a program's execution.

extend. Event dispatching is automatic and normally

 SchemeEsq's repl accepts expressions, evaluates them,

synchronous, but MrEd also p ermits controlled, asyn-

and displays the results, all within a GUI text edi-

chronous (i.e., parallel) event handling. In addition,

tor. For simplicity,we assume that the user submits a

MrEd provides sp ecial supp ort for multimedia editor

complete Scheme expression for evaluation by hitting

applicati ons (hence the Ed in MrEd ), suchasword

the Enter key.Ifarepl expression signals an error,

pro cessors and drawing programs.

SchemeEsq prints the error message and creates a new

 Emb edding and security | MrEd provides multiple

input prompt. At all times, the text preceding the

threads of execution, thread-sp eci c con guration, and

current prompt is lo cked so the user cannot mo dify it.

resource control. These constructs supp ort the secure

 In addition to standard Scheme, SchemeEsq's repl

emb edding of programs within other programs that

provides access to the entire MrEd to olb ox, p ermit-

control the emb edded execution environment and re-

ting the user's program to create threads and GUI ob-

source consumption.

jects. User-created GUI elements must not interfere

 Mo dularity and extensibil ity | MrEd's mo dule and

with SchemeEsq's own GUI. For example, the user's

class constructs enable the construction of reusable

program might create a mo dal dialog that disables all

comp onents. These comp onent constructs naturally

of the user's other windows, but it must not disable

complement MrEd's ob ject-oriented GUI to olb ox, al-

the SchemeEsq window.

lowing a programmer to implement reusable and com-

 SchemeEsq's Reset button stops the currentevalua-

p osable extensions of GUI elements.

tion, reclaiming all resources currently in use bythe

The following sections provide an overview of MrEd's con-

user's program. All active threads and GUI elements

structs. Section 3.1 describ es a few details of MrEd's GUI

created by the program must b e destroyed.

to olb ox, whichweprovide for the sake of de ning a concrete

 Although SchemeEsq redirects printed output from the

implementation of SchemeEsq. Section 3.2 describ es MrEd's

user's program to its repl window, the program must

1

Except by running out of memory; see Section 6.

otherwise execute within SchemeEsq in exactly the 2

constructs for program emb edding and security. The ideas MrEd's GUI classes also handle the graphical layout of

underlying these constructs form the main contribution of windows. Our example frame demonstrates a simple layout;

the pap er. Finally, Section 3.3 describ es MrEd's supp ort for the frame's elements are lined up top-to-b ottom. In general,

mo dular and ob ject-oriented programming, whichisintegral a programmer sp eci es the layout of a windowby assigning

to our mo del of programs and pro cesses. each GUI element to a parent container.Avertical con-

tainer, such as a frame, arranges its children in a column,

Recommendation: Skip the Exp erience and Rationale boxes for a

and a horizontal container arranges its children in a row. A

rst reading.

container can b e a child of another container; for example,

to place two buttons side-by-side in a frame, a programmer

3.1 GUI Construction

creates a horizontal panel for the buttons:

MrEd provides the basic building blo cks of GUI programs|

(define panel (make-object horizontal-panel% frame))

such as frames (top-level windows), mo dal dialogs, menus,

(make-object button% \Left" panel ...)

buttons, checkboxes, and text elds|via built-in classes

(make-object button% \Right" panel ...)

that the programmer can instantiate or extend. For ex-

A programmer can adjust the minimum width, minimum

ample, a programmer creates a frame by instantiating the

2

height, horizontal stretchability,andvertical stretchability

built-in frame% class:

of each GUI element. Using these settings, MrEd picks an

(define frame

initial size for each frame, and it rep ositions controls when

(make-object frame% \Example" #f 400 200))

the user resizes a frame.

Containers, Exp erience and Rationale: Existing GUI to ol-

MrEd's make-object pro cedure takes a class and returns an

boxes provide a varietyofmechanisms for geometry manage-

instance of the class. Extra arguments to make-object serve

ment, but our simple container mo del is intuitive and surpris-

as initial izati on arguments for the ob ject, similar to argu-

ingly p owerful. Although MrEd p ermits the de nition of new

ments provided with new in Java. For the frame% class, the

containers with arbitrary layout strategies, we implemented

DrScheme using only vertical and horizontal containers.

initiali zati on arguments sp ecify the frame's title, its parent

window(#f if none), and its initial size. The ab ove frame

In addition to the basic GUI building blo cks, MrEd pro-

is titled Example, has no parent, and is 400 pixels wide and

vides a collection of classes that supp ort a broad sp ectrum of

200 pixels tall.

editor programs, from word pro cessors to drawing programs.

The built-in classes provide various mechanisms for han-

The editor framework addresses a wide range of real-world

dling GUI events, which MrEd dispatches automatically.

issues for an editor|includ in g cut-and-paste, extensible le

For example, when instantiating the button% class, the pro-

formats, and layered text styles|while supp orting a high

grammer supplies an event callback pro cedure to b e invoked

level of extensibili ty through the class system.

when the user clicks the button. The following example cre-

ates a Close button that hides the frame when the user clicks

Editors, Exp erience and Rationale: MrEd's editor to olb ox pro-

the button:

vides a foundation for two common kinds of applications:

1. programs that include a sophisticated text editor:

(make-object button% \Close" frame

MrEd's simple text eld control is inadequate for text-

(lambda (button event)

intensive applications. Many programs need editors that

can handle multiple fonts and non-text items.

(send frame show #f)))

2. programs that include a canvas with dragable ob jects:

The button's callback pro cedure uses MrEd's send form,

MrEd's drawing to olb oxprovides a generic drawing sur-

which calls a metho d given an ob ject, the metho d's name,

face for plotting lines and b oxes, but many applications

need an interactivecanvas, where the user can drag and

and metho d arguments. A frame's show metho d takes one

resize individual ob jects.

argument, a Bo olean value that indicates whether to show

The p ower and exibility of the editor to olb ox make it fairly

or hide the frame.

complex, and using the to olb ox requires a solid understanding

If a window receives multiple kinds of events, MrEd dis-

of its structure and terminology.Nevertheless, enough appli-

patches events to metho ds of the window instead of to a

cations t one (or b oth) of the descriptions ab ove to justify the

depth and complexity of the to olb ox and the learning invest-

callback pro cedure. For example, a drawing canvas receives

ment required to use it.

up date events, mouse events, keyb oard events, and sizing

events; to handle them, a programmer must deriveanew

class from the built-in canvas% class and override the event-

3.2 Emb edding and Security

handling metho ds:

Conventional op erating systems supp ort multiple programs

(define my-canvas%

through a pro cess abstraction that gives each program its

(class canvas% ; my-canvas% extends canvas%

own control ow, I/O environment, and resource controls.

(override

A pro cess is distinguish ed primarily by its address space,

(on-char (lambda (event)(display \keyboard")))

where separate address spaces serve b oth as a protection

(on-scroll (lambda (event)(display \scroll"))))

barrier b etween programs and as a mechanism for de ning

...))

a program's environment; e.g., the stdout global variable in

a Unix C program contains a pro cess-sp eci c value.

Callbacks, Exp erience and Rationale: For simple controls, such

In MrEd, separate address spaces are unnecessary for

as buttons, the control's action is normally instance-sp eci c,

protection b etween programs, due to the safety prop erties

so the action is b est sp eci ed as a callbackinthemake-object

of the programming language. Nevertheless, separate pro-

expression. For more complex GUI elements, suchascan-

vases, event-handling is often common to a class of instances,

grams require separate control ow, I/O environments, and

so metho d overriding provides a more extensible mechanism

resource controls. Instead of providing an all-encompassi ng

for handling events.

pro cess abstraction, MrEd provides sp eci c mechanisms for

2

creating threads of control, dealing with graphical I/O, and

By convention, class names end with a p ercent sign (%) in MrEd.

The source co de in this pap er runs in MrEd version 100. managing resources. 3

3.2.1 Threads and Parameters 3.2.2 Eventspaces

MrEd's thread primitive consumes a pro cedure of no argu- An eventspace in MrEd is a context for pro cessing GUI

ments and invokes it in a new thread. The following example events in a sequential manner. Eacheventspace maintains

spawns a thread that prints \tick" every second: its own queue of events, its own collection of frames, and

its own handler thread. MrEd dispatches events within an

(define (tick-loop)

eventspace synchronously in the handler thread, while dis-

(sleep 1)(display \tick")(tick-loop))

patching events from di erenteventspaces asynchronously

(thread tick-loop)

in separate handler threads.

Creating an eventspace starts a handler thread for the

Each thread maintains its own collection of system set-

eventspace implicitly. Only the handler thread dispatches

tings, such as the current directory and the currentout-

events, but all threads that share an eventspace can queue

3

put p ort. These settings are called parameters. A pa-

events, and all threads (regardless of eventspace) can ma-

4

rameter is queried and mo di ed via a parameter pro cedure,

nipulate an accessible GUI ob ject. When a thread creates

suchascurrent-directory or current-output-p ort.For example,

a top-level window, it assigns the window to the current

(current-directory) returns the path of the current directory,

eventspace as determined by the current-eventspace param-

while (current-directory dir) sets the current directory to dir.

eter.

Mo difying a parameter changes its value in the current

Eventspaces, Exp erience and Rationale: Windows and BeOS

thread only. Therefore, by setting the current-output-p ort

also integrate threads with GUI ob jects, but in fundamentally

in the tick-loop thread, we can redirect the \tick" printouts

di erentways:

without mo difying tick-loop and without a ecting the out-

 Windows asso ciates an event queue with every thread,

put of any other thread:

and a thread can manipulate only those windows within

its own queue. A programmer can explicitly merge

the queues of two threads so that they share an

(thread (lambda ()

\eventspace," but the queues are merged p ermanently,

(current-output-port (open-output-file \ticks"))

so there is no waytochange the \eventspace" of a

(tick-loop)))

thread.

 BeOS creates a separate handler thread for every top-

A newly-created thread inherits the parameter values of the

level window. Programmers must explicitly implement

creating thread. Thus, if tick-loop creates its own threads,

synchronization among top-level windows, but monitors

protect many op erations on windows.

they also pro duce output to the \ticks" le.

Eventspaces are more exible than either of these designs.

Parameter inheritance provides an alternativemecha-

Compared to Windows, eventspaces more easily accommo date

nism for setting the output p ort in the \ticking" thread. In-

multiple threads that op erate on a single set of graphical ob-

stead of explicitl y setting the p ort within the ticking thread,

jects. Compared to BeOS, eventspaces more easily accommo-

we could temp orarily set the p ort in the main thread while

date single-threaded programs with multiple windows. In prin-

ciple, MrEd's lack of automatic synchronization on ob jects in-

creating the ticking thread:

creases the p otential for race conditions, but such race condi-

tions have o ccurred rarely in practice. While threads some-

(parameterize ((current-output-port

times manipulate GUI ob jects concurrently, they typically call

(open-output-file \ticks")))

thread-safe primitive metho ds.

(thread tick-loop))

For example, to call a graphical-tick-loop pro cedure that

A parameterize expression sets the value of a parameter

creates a ticking GUI, we parameterize the ticking thread

5

during the dynamic extent of its b o dy. In the ab ove exam-

with a new eventspace:

ple, parameterize restores the output p ort for the main

(parameterize ((current-eventspace (make-eventspace)))

thread after the ticking thread is created, but the ticking

(thread graphical-tick-loop))

thread inherits \ticks" as its current output p ort.

Since the output p ort is set b efore tick-loop is called, the

(define (graphical-tick-loop)

ticking thread has no way to access the original output p ort.

(letrec ([frame (make-object frame% \Tick")]

In this way, parameters p ermit securely con guring the en-

[msg (make-object message% \tick" frame)]

vironment of a nested program (or anyuntrusted thread).

[loop (lambda (now next)

(sleep/yield 1)

Parameters, Exp erience and Rationale: An early version of

(send msg set-label now)

MrEd supp orted bundles of parameter values as rst-class ob-

(loop next now))]) jects, called parameterizations.Two threads could share a pa-

rameterization, in which case mo difying a parameter in one

(send frame show #t)

thread would change the value in b oth threads.

(loop \tock" \tick")))

This generalization turns out to b e nearly useless in practice,

The rst expression ab ove creates two threads: the plain

since shared state is readily available through a parameter

thread explicitl y created by thread, and the handler thread

whose value is a mutable ob ject. Worse, parameterizations de-

implicitl y created by make-eventspace. Instead of creating

feat the essential purp ose of parameters for separating global

state from thread-sp eci c state. With parameterizations, a li-

the plain thread, we can use queue-callback to call graphical-

brary routine cannot, for example, freely adjust the current

tick-loop within the handler thread:

output p ort, b ecause even a temp orary change might a ect

evaluation in another thread.

(parameterize ((current-eventspace (make-eventspace)))

(queue-callback graphical-tick-loop))

3

The term parameter, the parameter pro cedure convention, and

the parameterize form in MrEd imitate those of Chez Scheme [9],

4

UnlikeJava, MrEd provides no automatic synchronization for the

although Chez do es not provide threads.

metho ds of a GUI ob ject. The primitive metho ds of an ob ject, how-

ever, are guaranteed to b e thread-safe.

5

The sleep/yield pro cedure is like sleep, except that it handles events

(such as window-up date events) while \sleeping." 4

The queue-callback primitive queues a pro cedure to b e in- 3.3 Mo dularity and Extensibility

voked by the handler thread of the currenteventspace. The

Parameters, eventspaces, and custo dians provide the nec-

pro cedure will b e invoked in the same wayasanevent call-

essary infrastructure for de ning pro cesses without sepa-

back for the eventspace.

rate address spaces. The resulting pro cess mo del p ermits

Each queued pro cedure is either a high-priorityorlow-

exible and ecient communication b etween programs via

priority callback, indicated by an optional second argument

pro cedures, metho ds, and other language constructs. This

to queue-callback. When a high-priority callback (the de-

exibili ty blurs the distinction b etween programs and li-

fault) and a GUI event are b oth ready for handling, MrEd

braries. For example, a picture-editing program could work

invokes the high-priority callback. In contrast, when a low-

either as a stand-alone application or as a part of a word-

priority callback and a GUI event are b oth ready for han-

pro cessing application . More generally, programmers can

dling, MrEd invokes the GUI event handler. A programmer

replace monolithic programs with exible software comp o-

can use prioritized callbacks to assign priorities to graphical

nents that are combined to de ne applications.

op erations, suchaslow-priority screen refreshing.

MrEd supp orts the de nition of units [14], whichare

separately compilable and reusable software comp onents. A

3.2.3 Custo dians

unit encapsulates a collection of de nitions and expressions

that are parameterized over imp orts, and some of the de ni-

In the same way that threads generalize p er-pro cess concur-

tions are exp orted. A programmer links together a collection

rency and eventspaces generalize p er-pro cess event sequenc-

6

of units to create a larger unit or a program. MrEd de nes

ing, custo dians generalize p er-pro cess resource control.

program to mean a unit with no imp orts, similar to the way

MrEd places every newly-created thread, eventspace, le

that conventional OSes with dynamic linking (via DLLs or

p ort, or network connection into the managementofthe

ELF ob jects) de ne a program as a certain form of linkable

current custo dian (as determined bythecurrent-custo dian

ob ject.

parameter). A program with access to the custo dian can

To p ermit comp onents that are as reusable as p ossible, a

terminate all of the custo dian's threads and eventspaces

unit linking graph can contain cycles for de ning mutually-

and close all of the custo dian's p orts and network connec-

recursive pro cedures across unit b oundaries. Furthermore, a

tions. The custo dian-shutdown-all pro cedure issues sucha

unit can contain a class de nition where the sup erclass is im-

shut-down command to a custo dian, immediately reclaim-

ported into the unit, even though the source of the imp orted

ing the resources consumed by the terminated and closed

class is not known at compile time. In the following exam-

ob jects.

ple, NoisyCanvasUnit de nes a noisy-canvas% class that is

Using a custo dian, we can start graphical-tick-loop and

derived from an imp orted plain-canvas% class:

p ermit it to run for only a certain duration, say 200 seconds,

b efore terminating the thread and reclaiming its graphical

(define NoisyCanvasUnit

resources:

(unit (import plain-canvas%)

(export noisy-canvas%)

(define cust (make-custodian))

(define noisy-canvas%

(parameterize ((current-custodian cust))

(class plain-canvas%

(parameterize ((current-eventspace (make-eventspace)))

...

(queue-callback graphical-tick-loop)))

(override

(sleep 200)

(on-event (lambda (e)

(custodian-shutdown-all cust)

(display \canvas event")

(super-on-event e))))

Although graphical-tick-loop could create new custo dians,

...))))

custo dians exist within a strict hierarchy.Every new custo-

dian is created as a sub-custo dian of the current custo dian,

Since the actual plain-canvas% class is not determined until

and when a custo dian receives a shut-down command, it

link time, NoisyCanvasUnit e ectively de nes a mixin [6,

propagates the shut-down command to its sub-custo dians.

16, 26], which is a class extension that is parameterized over

Thus, a program cannot evadeashut-down command by

its sup erclass. Using mixins, a programmer can mix and

migrating to a custo dian that it creates.

match extensions to pro duce a class with a set of desired

prop erties. This mo de of programming is particularly useful

Custo dians, Exp erience and Rationale: Custo dians manage

all ob jects that are protected from garbage collection byref-

for implementing GUIs, where each mixin encapsulates a

erences in the low-level system. For example, an active thread

small b ehavioral extension of a GUI element.

is always accessible via the scheduler's run queue|even if no

part of the program refers to the thread|and a visible frame

Units and Mixins, Exp erience and Rationale: Our work cited

is always accessible via the window manager. Such ob jects re-

ab ove for units and mixins provides a theoretical mo del of

quire explicit termination to remove the system's reference and

the constructs. In practice, MrEd's implementation of units

to free the ob ject's resources.

closely follows the theoretical mo del, except that units nor-

mally imp ort and exp ort bundles of names rather than indi-

An ob ject that has terminated maycontinue to o ccupy a small

vidual names. In contrast, MrEd's implementation of mixins

amount of memory. Custo dians rely on garbage collection to

is less expressive than the mo del, b ecause the implementation

reclaim the memory for a terminated ob ject, and a thread in a

do es not handle metho d name collisions. This di erence repre-

di erent custo dian might retain a reference to such an ob ject.

sents a signi cant compromise in our implementation of mix-

Each op eration on a terminable ob ject must therefore check

ins, but MrEd's weaker form is suciently p owerful for most

whether its op erand has terminated and signal an error if nec-

purp oses.

essary.For GUI ob jects in MrEd, primitive metho ds signal an

error when the ob ject has terminated.

6

Custo dians are similar to resource containers [4]. 5

4 Implementing SchemeEsq in MrEd 4.2 SchemeEsq Evaluation

When a user hits the Enter key,SchemeEsq evaluates the

Equipp ed with the MrEd constructs de ned in the previ-

expression following the current prompt. SchemeEsq ulti-

ous section, we can implement the SchemeEsq program de-

mately evaluates this expression by calling the built-in eval

scrib ed in Section 2. First, we create the SchemeEsq GUI

pro cedure. But b efore letting SchemeEsq call eval,wemust

using the MrEd to olb ox. Then, we use threads, eventspaces,

ensure that co de evaluated in the repl cannot interfere with

and custo dians to implement secure evaluation for repl ex-

SchemeEsq itself, since b oth SchemeEsq and the user's co de

pressions. Finally,we discuss how units and mixins let us

execute together in MrEd.

extend SchemeEsq to implement the full DrScheme environ-

Of course, user co de must not gain direct access to the

ment.

frame or editor of SchemeEsq, since it might call metho ds

of the ob jects inappropriatel y. We can hide SchemeEsq's

4.1 SchemeEsq GUI

implementation from the user's program by putting it into

a mo dule and making all de nitions private. For now, we

To implement the SchemeEsq GUI, we rst create a frame:

continue to de ne SchemeEsq through top-level de nitions,

(define frame

but the app endix shows the nal SchemeEsq program en-

(make-object frame% \SchemeEsq" #f 400 200))

capsulated in a mo dule.

The remaining problems concern the interaction of con-

and make it visible:

trol ow in the user's program and in SchemeEsq. Threads

(send frame show #t)

with parameters, eventspaces, and custo dians provide pre-

Next, we create the reset button to app ear at the top of the

cisely the mechanisms needed to solve these problems.

frame:

4.2.1 Threads in SchemeEsq

(define reset-button

(make-object button% \Reset" frame

An unb ounded computation in the user's program must not

(lambda (be)(reset-program))))

stall SchemeEsq's GUI. Otherwise, the program would pre-

vent the user from clicking SchemeEsq's reset button. To

The callback pro cedure for the reset button ignores its ar-

avoid blo cking SchemeEsq on a repl computation, weeval-

guments and calls reset-program, whichwe de ne later. Fi-

uate user expressions in a separate thread. The following is

nally,we create a display area for the repl, implemented as

a rst attempt at de ning the evaluate pro cedure for evalu-

an editor canvas:

8

ating user expressions:

(define repl-display-canvas

(make-object editor-canvas% frame))

(define (evaluate expr-str)

(thread

At this p oint, our SchemeEsq GUI already resembles Fig-

(lambda ()

ure 2, but the repl is not yet active. The actual repl is im-

7

(with-handlers ((exn?

plemented as a text editor that is displayed bythecanvas.

(lambda (exn)

The basic functionality needed in SchemeEsq's repl|

(display (exn-message exn)))))

includin g keyb oard event handling, scrolling, and cut and

(write (eval (read (open-input-string expr-str)))))

paste op erations|resid es in MrEd's text% editor class. The

(newline)

esq-text% class, de ned in the app endix, adapts the text%

(send repl-editor new-prompt))))

class to the needs of the repl byoverriding metho ds to

sp ecialize the editor's b ehavior. For example, when the edi-

Having created a thread to represent the user pro cess,

tor receives an Enter/Return key press, it calls the evaluate

wemust con gure the pro cess's environment. For simplicity,

pro cedure (whichwe de ne later).

we de ne con guration as redirecting output from the user's

In addition to handling input, the esq-text% class de-

program (via display or write)tothe repl editor. To redirect

nes an output metho d for printing output from the user's

output for the user's program, we set the output p ort in the

program into the repl editor. Since the user's program

evaluation thread:

can create many threads, the output metho d needs a sp ecial

wrapp er to convert multi-threaded output calls into single-

(define (evaluate expr-str)

threaded output. The queue-output wrapp er p erforms this

(thread

conversion bychanging a metho d call into a queued, low-

(lambda ()

priority GUI event:

added (current-output-port user-output-port);

(define esq-eventspace (current-eventspace))

(with-handlers ((exn?

(define (queue-outputproc)

(lambda (exn)

(parameterize ((current-eventspace esq-eventspace))

(display (exn-message exn)))))

(queue-callback proc #f)))

(write (eval (read (open-input-string expr-str)))))

(newline)

Using the new esq-text% class, we create an editor in-

(send repl-editor new-prompt))))

stance and install it into the displaycanvas:

(define repl-editor (make-object esq-text%))

The ab ove assumes that user-output-port p ort acts as a pip e

(send repl-display-canvas set-editor repl-editor)

to the repl editor. We can de ne user-output-port using

8

The SchemeEsq GUI is now complete, but wehave not yet

The with-handlers form sp eci es predicate-handler pairs that

implemented evaluate (used in esq-text%) and reset-program

are active during the evaluation of the with-handlers b o dy expres-

sion. In evaluate,the exn? predicate selects the (lambda (exn)(dis-

(used by reset-button's callback).

play (exn-message exn))) handler for all typ es of exceptions. Thus,

7

evaluate catches any exception, prints the error message contained in MrEd distinguishes b etween a display and its editor in the same

the exception, and resumes the repl. way that Emacs distinguishes b etween a window and its bu er. 6

make-output-p ort, a MrEd pro cedure that creates a p ort To implementthereset-program pro cedure for the reset but-

from arbitrary string-printin g and p ort-closing pro cedures: ton, weissueashut-down command on user-custodian and

then reset the repl editor:

(define user-output-port

(define (reset-program)

(make-output-port

(custodian-shutdown-all user-custodian)

(lambda (s)(send repl-editor output s))

(parameterize ((current-custodian user-custodian))

(lambda () 'nothing-to-close)))

(set! user-eventspace (make-eventspace)))

(send repl-editor reset))

In this use of make-output-p ort, the string-printing pro cedure

sends the string to the repl editor, and the p ort-closing

Each reset destroys user-eventspace (by issuing a shut-down

pro cedure do es nothing.

command to user-custodian), making the eventspace unus-

able. Therefore, reset-program creates a new eventspace

4.2.2 Eventspaces in SchemeEsq

after eachreset.

Since the user's program and SchemeEsq execute in separate

threads, the user's program and SchemeEsq must handle

4.3 Mo dularity and Extensibility in SchemeEsq

GUI events in parallel. To this end, SchemeEsq creates a

The app endix assembles the pieces that wehavedevelop ed

new eventspace for the user's program:

into a complete implementation of SchemeEsq. The most

striking asp ect of SchemeEsq's implementation|b esi des the

(define user-eventspace

fact that it ts on one page|is that half of the co de exists

(make-eventspace))

to drivetherepl editor. In the real DrScheme environment,

the repl is considerably more complex, and its implementa-

To execute user co de with user-eventspace,we mightre-

tion o ccupies a corresp ondingl y large p ortion of DrScheme's

vise evaluate to install the eventspace in the same way that

overall co de.

we installed user-output-port:

In implementing DrScheme, we tamed the complexityof

the GUI by making extensive use of units and mixins. For

(define (evaluate expr-str)

... example, the parenthesis-high li ghti ng extension for an editor

is implemented as a mixin in its own unit, and the interactive

(thread

(lambda () searchinterface is another mixin in a separate unit. Using

units and mixins in this way, the implementation strategy

(current-eventspace user-eventspace)

...))) that wehave demonstrated for SchemeEsq scales to the more

elab orate implementation of DrScheme.

Alternatively,we could eliminate the call to thread and eval-

DrScheme also exploits units for emb edding program-like

uate expressions in the handler thread of user-eventspace.

comp onents. For example, DrScheme's help system runs ei-

The handler thread is a more appropriate choice, b ecause

ther as a stand-alone application or emb edded within the

co de that creates and manipulates GUI ob jects should run

DrScheme programming environment. The help-system unit

in the event-handling thread to avoid race conditions. To

imp orts a class that de nes a basic frame for the help win-

evaluate expressions in the handler thread, we treat the

dow. In stand-alone mo de, the class implements a frame

evaluation of repl expressions as a kind of event, queuing

with a generic menu bar, but when the help system is em-

evaluation with queue-callback:

b edded in DrScheme, the imp orted class implements a menu

bar with DrScheme-sp eci c menus.

(define (evaluate expr-str)

(parameterize ((current-eventspace user-eventspace))

5 High-Level Op erating Systems

changed - added (queue-callback ;

The developmentofSchemeEsq in MrEd demonstrates how

(lambda ()

a few carefully de ned extensions can transform a high-level

(current-output-port user-output-port)

programming language into a high-level op erating system.

(with-handlers ((exn?

A high-level OS p ermits exible and ecientcommunica-

(lambda (exn)

tion b etween programs through common language mecha-

(display (exn-message exn)))))

nisms, such as pro cedures and metho ds. It also guarantees

(write (eval (read (open-input-string expr-str)))))

typ e and memory safety across programs through language

(newline)

mechanisms, eliminating the need for separate pro cess ad-

(send repl-editor new-prompt)))))

dress spaces and data marshaling. This exibility increases

the p otential for extensible and interop erating programs.

4.2.3 Custo dians in SchemeEsq

Mere safety,however, provides neither the level of pro-

We complete SchemeEsq by implementing the reset button's

tection b etween applicatio ns nor the kind of pro cess control

action with a custo dian. We de ne user-custodian and cre-

that conventional OSes provide. As an example, SchemeEsq

ate the user's eventspace under the managementofuser-

illustrates how a graphical programming environmentmust

custodian:

protect its GUI from interference from a program executing

within the environment. Although language-based safety

(define user-custodian (make-custodian))

can prevent a program from trampling on the environment's

data structures, it cannot prevent a program from starving

(define user-eventspace

the environment pro cess or from leaking resources.

(parameterize ((current-custodian user-custodian))

MrEd combines the programming exibility of a high-

(make-eventspace)))

level OS with the conventional pro cess controls of a conven-

tional OS. As wehaveshown, three key extensions makethis 7

therefore sacri ces the convenience of direct pro cedure calls combination p ossible: threads with parameters, eventspaces,

and direct data sharing. and custo dians. Our approach to buildin g a high-level OS on

Back and Hsieh [1] provide a detailed explanation of the top of Scheme should apply equally well to other languages,

di erence b etween pro cess control and mere safetyinaJava- such asMLorJava.

based op erating system. They emphasize the imp ortance of

the \red line" that separates user co de and kernel co de in

6 Problems forFuture Work

aconventional OS. This red line exists in MrEd, separating

low-level built-in primitives from the rest of the system. For

Although MrEd provides custo dians for resource reclama-

example, queue-callback is e ectively an atomic op eration to

tion, our current implementation do es not supp ort a priori

the calling thread. MrEd go es one step further, providing

resource limits on custo dians (analogous to memory use lim-

programs with the ability to de ne new layers of red lines.

its on a pro cess) or constraints that prevent a program from

In particular, SchemeEsq de nes a red line b etween itself

triggering frequent system-wide garbage collections. Cus-

and the programs that it executes.

to dians and parameters app ear to b e go o d abstractions for

Inferno [8] achieves many of the same goals as MrEd,

expressing these limits, but our memory managementtech-

but in a smaller language that is targeted for communica-

nology must b e improved to implementthem.

tions software rather than general-purp ose GUI implemen-

Our SchemeEsq example fails to illustrate certain kinds

tation. Balfanz and Gong [3] explore extensions to Javato

of protection problems, b ecause the communication b etween

supp ort multiple pro cesses, particularly multiple pro cesses

SchemeEsq and a user's program is rather limited. For ex-

owned by di erent users within a single JVM. They derive

ample, the user's program sends output to SchemeEsq by

some of the same constructs that are de ned by MrEd, no-

queueing a GUI event. Since the built-in queueing op er-

tably eventspaces.

ation is atomic and non-blo ckin g, there is no danger that

Haggis [11], eXene [18 ], and Fudgets [21] provide stream-

the user's program will break a communication invariantby

oriented graphical extensions of functional languages. None

killing its own thread. More sophisticated communication

provides a mechanism for pro cess and resource control, but

proto cols require stronger protection during the execution

the functional streams used by these systems makes them

of the proto col. Indeed, merely adding a limit to the size of

less susceptible to cross-pro cess interference than an imp er-

the output queue in SchemeEsq (so that the user's thread

ativeGUIlayer. Acombination of stream-oriented GUIs

blo cks when the queue is full) requires such protection.

with custo dians may b e p ossible.

One general solution to the protection problem is to cre-

ate a new thread|owned bySchemeEsq's custo dian|for

each communication. This techniques solves the problem

8 Conclusion

b ecause thread creation is an atomic op eration, and the

Wehave shown howkey constructs in MrEd|threads with newly-created thread can execute arbitrarily many instruc-

parameters, custo dians, and eventspaces|enabl ed the de- tions without the risk of b eing killed by the user's program.

velopment of a graphical programming environment. More Unfortunately, thread creation is an exp ensive op eration in

imp ortantly, the constructs that enabled DrScheme also ad- MrEd compared to pro cedure calls, as in many systems. To

dress problems in the design of a general-purp ose, high-level reduce this cost for common protection idioms, MrEd pro-

op erating system. vides a call-in-nested-thread pro cedure that creates a child

Although MrEd was sp eci call y created for DrScheme, thread, and then blo cks the parent thread until the child

MrEd serves as platform for many other applications as well. terminates. By exploiting the mutual exclusion b etween

These applications include a theorem prover [12], a theater the parent and child threads, MrEd can eliminate muchof

lighting system [30 ], and a mail client, which demonstrate the thread-creation and thread-swapping overhead for pro-

that MrEd's programming mo del extends to general GUI tection idioms. Using a similar technique, Ford and Lep-

programming tasks. reau [17] improved the p erformance of Mach RPC. Never-

theless, a signi cantoverhead remains.

Acknowledgements MrEd's GUI to olb ox is based on the

7 Related Work

wxWindows class library [29 ]. Thanks to Jay Lepreau, Go d-

mar Back, and Pat Tullmann for p ersp ective on existing

As a GUI-oriented, high-level language, MrEd shares much

work in high-level OSes, and thanks to the anonymous re-

in common with Smalltalk [19], Pilot [27], Cedar [32], the

viewers for helpful comments on the original draft of this

Lisp Machine [7], Ob eron [33], and JavaOS [31]. All of these

pap er.

systems simplify the implementation of co op erating graphi-

cal programs through a high-level language. Although most

References

of these systems supp ort multiple pro cesses, only MrEd pro-

vides the kind of pro cess controls that are necessary for im-

[1] Back, G. and W. Hsieh. Drawing the red line in Java.

9

plementing a SchemeEsq-like programming environment.

In Proc. IEEE Workshop on Hot Topics in Operating

Other related work aims to replicate the safety, secu-

Systems, March 1999.

rity, and resource control of conventional op erating systems

within a single address space. Architectures such as Alta [2],

[2] Back, G., P.Tullmann, L. Stoller, W. C. Hsieh and

SPIN [5], J-Kernel [22], and Nemesis [24], emphasize pro-

J. Lepreau. Java op erating systems: Design and imple-

tection within a single address space, but at the exp ense

mentation. Technical Rep ort UUCS-98-015, University

of program integration through indirect and inecient calls.

of Utah, 1998.

For example, the J-Kernel relies on explicit capabiliti es, and

[3] Balfanz, D. and L. Gong. Exp erience with secure

9

On the Lisp Machine, allowing programmers to tinker with the

multi-pro cessi ng in Java. Technical Rep ort TR-560-97,

OS on-the- y was considered an advantage [7, page 44].

Princeton University, Computer Science Department,

Septemb er 1997. 8

[4] Banga, G., P. Druschel and J. C. Mogul. Resource [21] Hallgren, T. and M. Carlsson. Programming with fud-

containers: A new facility for resource managementin gets. In Jeuring, J. and E. Meijer, editors, Advanced

server systems. In Proc. ACM Symposium on Operating Functional Programming,volume 925 of LectureNotes

System Design and Implementation,Feburary 1999. in Computer Science, pages 137{182. Springer, May

1995.

[5] Bershad, B. N., S. Savage, P.Pardyak, E. G. Sirer,

M. Fiuczynski, D. Becker, S. Eggers and C. Chambers.

[22] Hawblitzel, C., C.-C. Chang, G. Cza jkowski, D. Hu and

Extensibility, safety and p erformance in the SPIN op er-

T. von Eicken. Implementing multiple protection do-

ating system. In Proc. ACM Symposium on Operating

mains in Java. In Proc. of USENIX Annual Technical

Systems Principles, pages 267{284, Decemb er 1995.

Conference, pages 259{270, June 1998.

[6] Bracha, G. and W. Co ok. Mixin-based inheritance. In

[23] Kelsey, R., W. Clinger and J. Rees (Eds.). The rev-

5

Proc. Joint ACM Conf. on Object-OrientedProgram-

ised rep ort on the algorithmic language Scheme. ACM

ming, Systems, Languages and Applications and the

SIGPLAN Notices, 33(9), Septemb er 1998.

European Conference on Object-OrientedProgramming,

[24] Leslie, I. M., D. McAuley,R.J.Black, T. Rosco e, P.R.

Octob er 1990.

Barham, D. M. Evers, R. Fairburns and E. A. Hy-

[7] Bromley,H. Lisp Lore: A Guide to Programming the

den. The design and implementation of an op erating

Lisp Machine.Kluwer Academic Publishers, 1986.

system to supp ort distributed multimedia application s.

IEEE Journal on SelectedAreas in Communications,

[8] Dorward, S., R. Pike, D. L. Presotto, D. Ritchie,

14(7):1280{1297, Septemb er 1996.

H. Trickey and P. Winterb ottom. Inferno. In Proc.

IEEE Compcon Conference, pages 241{244, 1997.

[25] McCarthy, J. Recursive functions of symb olic expres-

sions and their computation bymachine (Part I). Com-

[9] Dybvig, R. K. Chez Scheme User's Guide. Cadence

munications of the ACM, 3(4):184{195, 1960.

Research Systems, 1998.

[26] Mo on, D. A. Ob ject-oriented programming with Fla-

[10] Findler, R. B., C. Flanagan, M. Flatt, S. Krishnamurthi

vors. In Proc. ACM Conference on Object-Oriented

and M. Felleisen. DrScheme: A p edagogic programming

Programming, Systems, Languages, and Applications,

environment for Scheme. In Proc. International Sym-

pages 1{8, Novemb er 1986.

posium on Programming Languages: Implementations,

Logics, and Programs, pages 369{388, Septemb er 1997.

[27] Redell, D., Y. Dalal, T. Horsley, H. Lauer, W. Lynch,

P. McJones, H. Murray and S. Purcell. Pilot: An op er-

[11] Finne, S. and S. P. Jones. Comp osing Haggis. In Proc.

ating system for a p ersonal computer. Communications

Eurographics Workshop on Programming Paradigms for

of the ACM, 23(2):81{92, Feburary 1980.

Computer Graphics, Septemb er 1995.

[28] Reynolds, J. De nitional interpreters for higher order

[12] Fisler, K., S. Krishnamurthi and K. Gray. Implement-

programming languages. In ACM ConferenceProceed-

ing extensible theorem provers. Technical Rep ort 99-

ings, pages 717{740, 1972.

340, Rice University,1999.

[29] Smart, J. et al. wxWindows.

[13] Flatt, M. PLT MzScheme: Language manual. Technical

http://web.ukonline.co.uk/julian.smart/wxwin/.

Rep ort TR97-280, Rice University, 1997.

[30] Sp erb er, M. The Lula system for theater lighting con-

[14] Flatt, M. and M. Felleisen. Units: Co ol mo dules for

trol. http://www-pu.informatik.uni-tuebingen.de/

HOT languages. In Proc. ACM ConferenceonPro-

users/sperber/lula/.

gramming Language Design and Implementation, pages

236{248, June 1998.

[31] Sun Microsystems, Inc. JavaOS: A standalone Java

environment, 1997.

[15] Flatt, M. and R. B. Findler. PLT MrEd: Graphical

http://www.javasoft.com/products/javaos/

to olb ox manual. Technical Rep ort TR97-279, Rice Uni-

javaos.white.html.

versity, 1997.

[32] Swinehart, D. C., P. T. Zellweger, R. J. Beach and R. B.

[16] Flatt, M., S. Krishnamurthi and M. Felleisen. Classes

Hagmann. A structural view of the Cedar programming

and mixins. In Proc. ACM Symposium on Principles

environment. ACM Transactions on Programming Lan-

of Programming Languages, pages 171{183, Janurary

guages and Systems, 8(4):419{490, Octob er 1986.

1998.

[17] Ford, B. and J. Lepreau. Evolving Mach 3.0 to a mi-

[33] Wirth, N. and J. Gutknecht. Project Oberon.ACM

grating thread mo del. In Proc. USENIX Technical Con-

Press, 1992.

ference and Exhibition, pages 97{114, Janurary 1994.

[18] Ganser, E. R. and J. H. Reppy. eXene. In Proc. of

the 1991 CMU Workshop on StandardML. Carnegie

Mellon University, Septemb er 1991.

[19] Goldb erg, A. and D. Robson. Smal ltalk 80: The Lan-

guage. Addison-Wesley, Reading, 1989.

[20] Hagiya, M. Meta-circular interpreter for a strongly

typ ed language. Journal of Symbolic Computation,

8(12):651{680, 1989. 9

App endix ;; Queueing repl output as an event

Syntax Note: The unit/sig mo dule form is like unit, except (define esq-eventspace (current-eventspace))

that imp ort and exp ort names are bundled together into signa-

(define (queue-output proc)

tures. The built-in mred^ signature contains all of the classes

(parameterize ((current-eventspace

and pro cedures provided by MrEd.

esq-eventspace))

(queue-callback proc #f)))

(define SchemeEsq

(unit/sig () ;; no exp orts

;; GUI creation

(import mred^)

(define frame

;; The repl editor class

(make-object frame% \SchemeEsq" #f 400 200))

(define esq-text%

(define reset-button

(class text% ()

(make-object button% \Reset" frame

;; lexical access to inherited metho ds:

(lambda (be)

(inherit insert last-position get-text erase)

(reset-program))))

;; lexical access to an overridden metho d:

(define repl-editor (make-object esq-text%))

(rename (super-on-char on-char))

(define repl-display-canvas

;; private elds:

(make-object editor-canvas% frame))

(private (prompt-pos 0)(locked? #t))

(send repl-display-canvas set-editor repl-editor)

(override

(send frame show #t)

;; override can-insert? to blo ck pre-prompt inserts:

(can-insert?(lambda (start len)

;; User space initialization

(and (>= start prompt-pos)

(not locked?))))

(define user-custodian (make-custodian))

;; override can-delete?toblock pre-prompt deletes:

(can-delete?(lambda (start end)

(define user-output-port

(and (>= start prompt-pos)

(make-output-port

(not locked?))))

;; string printer:

;; override on-char to detect Enter/Return:

(lambda (s)(send repl-editor output s))

(on-char (lambda (c)

;; closer:

(super-on-charc)

(lambda () 'nothing-to-close)))

(when (and (eq? (send c get-key-code)

#\return)

(define user-eventspace

(not locked?))

(parameterize ((current-custodian user-custodian))

(set! locked? #t)

(make-eventspace)))

(evaluate

(get-text prompt-pos

;; Evaluation and resetting

(last-position)))))))

(public

(define (evaluate expr-str)

;; metho d to insert a new prompt

(parameterize ((current-eventspace user-eventspace))

(new-prompt (lambda ()

(queue-callback

(queue-output

(lambda ()

(lambda ()

(current-output-port user-output-port)

(set! locked? #f)

(with-handlers ((exn?

(insert \> ")

(lambda (exn)

(set! prompt-pos (last-position))))))

(display

;; metho d to display output

(exn-message exn)))))

(output (lambda (str)

(write (eval (read (open-input-string

(queue-output

expr-str)))))

(lambda ()

(newline)

(let ((was-locked? locked?))

(send repl-editor new-prompt)))))

(set! locked? #f)

(insert str)

(define (reset-program)

(set! locked? was-locked?))))))

(custodian-shutdown-all user-custodian)

;; metho d to reset the repl:

(parameterize ((current-custodian user-custodian))

(reset (lambda ()

(set! user-eventspace (make-eventspace)))

(set! locked? #f)

(send repl-editor reset))))

(set! prompt-pos 0)

(erase)

;; Run the program mo dule

(new-prompt))))

(invoke-unit/sig SchemeEsq mred^)

(sequence

;; initialize sup erclass-de ned state:

Syntax Note: The invoke-unit/sig form executes the b o dy

(super-init)

of a unit given signatures for the unit's imp orts, and invoke-

;; create the initial prompt:

unit/sig satis es the imp orts from its lexical and top-level

(new-prompt))))

environment. 10