A Typechecking Prepro cessor for

a Multithreaded Language

by

Rob ert C Miller

Submitted to the Department of Electrical Engineering and Computer Science

in partial fulllment of the requirements for the degrees of

Bachelor of Science in Computer Science and Engineering

and

Master of Engineering in Electrical Engineering and Computer Science

at the

Massachusetts Institute of Technology

May

Copyright Massachusetts Institute of Technology All rights reserved

Author ::::::::::::::::::::::::::::::::::::::::::::::::::: ::::::::::::::::::::::::

Department of Electrical Engineering and Computer Science

May

Certied by ::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::

Charles E Leiserson

Thesis Sup ervisor

Accepted by ::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::::::::::::::

F R Morgenthaler

Chairman Department Committee on Graduate Theses

A Typechecking Prepro cessor for Cilk

a Multithreaded C Language

by

Rob ert C Miller

Submitted to the Department of Electrical Engineering and Computer Science

on May in partial fulllment of the

requirements for the degrees of

Bachelor of Science in Computer Science and Engineering

and

Master of Engineering in Electrical Engineering and Computer Science

Abstract

This thesis describ es the typechecking optimizing that translates Cilk a C ex

tension language for multithreaded parallel programming into C The CilktoC translator

is based on CtoC a to ol I developed that parses type checks analyzes and regenerates

a C program With a translator based on CtoC developers and users of C extension

languages can enjoy the b enets of a typechecking optimizing without the at

tendant development and p orting costs Like a compiler the CilktoC translator runs

quickly do es static checking and generates ecient co de making it t for everyday use

by ordinary programmers Unlike a compiler however the CilktoC translator is easy to

develop extend and p ort to other platforms Ab out of the development eort was

devoted to Cilk semantics translating Cilk into C which is where a language developer

wants to fo cus Only a small amount of work was sp ent extending CtoCs parsing type

checking and dataow analysis to recognize Cilk In return for a few weeks of fulltime

eort I obtained a typechecking optimizing translator for Cilk that targets any platform

with an ANSI C compiler With CtoC other C extension language developers can obtain

the same b enets

CtoC has a number of sp ecial features that it a go o framework for C extension

language translators First its C output is humanreadable so that a language developer or

programmer can read and debug it CtoC also provides op erational transparency repro

ducing nonsyntactic source features like line numbering and indentation which contributes

to the readability and p ortability of its C output and assists and prolers Cto

C p erforms dataow analysis directly on the highlevel abstract syntax tree of the source

program so that its C output is also highlevel not unreadable written

in C Finally CtoC provides a generic dataow analysis abstraction in which any mono

tonic dataow problem needed for optimization can b e sp ecied and solved automatically

These features greatly simplify the task of writing a typechecking optimizing translator

for an arbitrary C extension language

Thesis Sup ervisor Charles E Leiserson

Title Professor of Computer Science and Engineering

Contents

Introduction

One alternative a prepro cessor

Another alternative a full compiler

Related work

Outline

The Cilk

An example of Cilk fib

Type checking

Typedirected translations

Optimizations

The CtoC Typechecking Prepro cessor

The abstract syntax tree

Phases

Macro prepro cessing cpp

Parsing

Type checking

Analysis

Transform

Unparsing

Transparency in CtoC

Pragma directives

Line numbering and indentation

Constant expressions

Dataow Analysis in CtoC

Representing control ow in the AST

Dataow analysis frameworks

The iterative algorithm

Conclusions

Future work on CtoC

Getting CtoC

Acknowledgements

This research was supp orted in part by the Advanced Research Pro jects Agency under

Grant N

I am indebted to my advisor Charles E Leiserson Yuli Zhou provided advice and

supp ort throughout the development of the Cilk prepro cessor Thanks to Laura Cassenti

Charles Leiserson Anil Somaya ji and Yuli Zhou who were generous with their time in

agreeing to pro ofread this thesis any glaring errors that remain are solely the resp onsibili ty

of the author Thanks also to the entire Cilk team for supp ort and suggestions Bobby

Blumofe Matteo Frigo Chris Jo erg Bradley Kuszmaul Irena Kveraga Charles Leiserson

Howard Lu Phil Lisiecki Keith Randall Richard Tauriello Daricha Techopitayakul and

Yuli Zhou

I gratefully acknowledge the warm supp ort of Laura Cassenti and my parents Larry

and Marian Miller

Chapter

Introduction

Cilk pronounced silk is a parallel programming language under development at MIT

Lab oratory for Computer Science Cilk provides syntax for expressing control paral

lelism allowing a programmer to sp ecify that certain pro cedure calls should b e spawned or

run in parallel with the current thread of execution Cilk is an example of a C extension lan

guage a programming language that extends C with new keywords syntax or semantics

C extension languages have lately b ecome p opular in parallel and distributed computing

research b ecause of the wide p ortability of ANSI C the large p opulation of

C programmers and the extensive base of C applications and library software

Like many C extension languages Cilk is not translated directly into ob ject co de In

stead Cilk is translated into C with extension language features mapp ed into calls to a

runtime system Figure shows the translation pro cess schematically The translator

that converts a C extension language into C is called a b ecause its output

is a highlevel language rather than machine language The C postsource pro duced by the

prepro cessor is compiled into ob ject co de by the target machines C compiler also called the

backend compiler This compilation system a prepro cessor pip elined with a compiler

yields p ortability and ease of development b ecause the extension language developer need

not build a full compiler for every platform on which the language will run

In fact simple C extension languages can b e translated into C by lo cal syntactic trans

formations using a macro prepro cessor Cilk the rst incarnation of the Cilk language

was translated by a macro prepro cessor Though easy to write and debug macro prepro ces

sors suer a serious aw which manifested itself in Cilk A macro prepro cessor relies on

the backend C compilers type checking to detect and rep ort common programmer errors

Unfortunately type errors in Cilk programs are often missed by the backend compiler

b ecause the C p ostsource contains lowlevel co de that overrides type checking Even when

type errors are detected the C compilers error messages refer to the p ostsource which is

Cilk Cilk C C obj preprocessor compiler

source postsource object

Figure The mechanism for compiling a Cilk program Cilk is prepro cessed into C

which is then compiled into ob ject co de

unhelpful to a Cilk programmer trying to nd and x the error in the source

In addition macro prepro cessors are insucient for more abstract C extension languages

that require their translators to gather semantic information ab out the program Cilk

the latest version of the Cilk language includes features whose translation dep ends on the

types of the ob jects involved so the Cilk prepro cessor must determine those types An

ordinary macro prepro cessor has no access to type information

Finally macro prepro cessors oer no opp ortunity for global optimization The backend

C compiler p erforms global optimization when it generates ob ject co de but the compilers

optimizations are necessarily conservative missing p otential optimizations that the prepro

cessor can p erform Again Cilk serves as an example Gathering dataow information

enables the Cilk prepro cessor to emit faster co de than a macro prepro cessor could

This thesis studies typechecking optimizing prepro cessors for C extension languages

using the Cilk prepro cessor as an example In the course of building the Cilk prepro

cessor I have developed a generic prepro cessor framework for C extension languages called

CtoC CtoC parses a C program into an abstract syntax tree AST representation

p erforms type checking dataow analysis and tree transformations directly on the AST

then unparses the AST to recover the original C program To build a C extension language

prepro cessor based on CtoC a developer mo dies the frontend to recognize the extension

language then adds transformations that convert extension syntax into ordinary

The Cilk prepro cessor was built in precisely this manner

One alternative a macro prepro cessor

One goal of this thesis is to show that writing a typechecking optimizing prepro cessor

based on CtoC is a reasonable alternative to using a macro prepro cessor As the preceding

discussion demonstrated type checking and optimization increase the usability abstraction

level and p erformance of Cilk so the Cilk typechecking prepro cessor is sup erior to

the Cilk macro prepro cessor With CtoC these b enets were achieved at low cost

Our exp erience with the Cilk prepro cessor suggests that extending CtoC to build a type

checking prepro cessor is only a little more dicult than writing macros In fact only a

small fraction of the development time for the Cilk prepro cessor was sp ent extending

CtoCs type checking and dataow analysis Most of the development time was devoted

to translating Cilk into C essentially the same work that would b e required for a macro

prepro cessor CtoC translations are somewhat harder to write than macros since they

involve tree manipulations instead of textual substitutions but I b elieve that the extra

eort is more than justied by the rewards

Another alternative a full compiler

The second goal of this thesis is to argue that writing a prepro cessor based on CtoC is an

attractive alternative to another common approach to translating exp erimental C extension

languages extending a full compiler This approach b egins with a standard C compiler and

extends its front end to recognize the new extensions Compared to a full compiler

however a translator based on CtoC is p ortable and extensible highly desirable qualities

for language research In addition the output of a CtoC translator is readable highlevel

C not assembly language nor even assembly language written in C as might b e exp ected

from an optimizing translator This section will examine these arguments in more depth

First a translator based on CtoC is easier to p ort than a full compiler CtoC itself is

written in p ortable ANSI C and extension language features are translated into highlevel

C which can b e made p ortable For example the Cilk prepro cessor generates p ortable

C containing calls to a runtime system so moving Cilk to a new architecture entails no

changes to the prepro cessor whatso ever Only the relatively small runtime system must b e

p orted

Next a translator based on CtoC is easier to extend than existing publicdomain

C Its lexical scanner and parser are automatically generated from lexyacc

sp ecications which are easier to change than the handco ded frontends of and

the SUIF compiler framework CtoCs type checking and dataow analysis are table

driven so that new statements and expressions can b e sp ecied by just a few metho d

functions Transformations are expressed as op erations directly on the highlevel abstract

syntax tree which we found easier to manipulate than the intermediate representation

of compilers like gcc Also unlike most compilers CtoC provides an abstraction for

dataow analysis which can b e extended to solve any monotonic dataow analysis problem

needed for translation or optimization

Finally a translator based on CtoC pro duces output that is highlevel readable and

readily compared with its input since only extension language features are translated Cto

C provides op erational transparency preserving all highlevel C syntax line numbering

indentation and constant expressions used in the original program The readability of C

toCs output simplies the pro cess of checking that extension language constructs have

b een translated correctly

Related work

The Sage prepro cessor framework is similar to CtoC in that it provides a rich

to olkit for program analysis and transformation including source transparency and data

ow analysis Although the Sage frontend supp orts C C and it is more

dicult to extend to new languages Its lexical scanner is handco ded and its parser and

typechecker use optimized lowlevel interfaces to the AST rather than the highlevel C

classes Also Sage is written in C which is less widespread than ANSI C

Outline

The remainder of this thesis is organized as follows Chapter describ es the Cilk language

and explains why it requires type checking and dataow analysis from its prepro cessor

Chapter introduces the CtoC prepro cessor framework and describ es its architecture

Chapter explains how CtoC preserves nonsyntactic information like line numbering

and indentation from the source to the p ostsource Chapter describ es how CtoC p er

forms dataow analysis directly on the source program Chapter oers some conclusions

contrasting CtoCbased prepro cessors with other alternatives for translating C extension

languages and describ es our plans for future work

Chapter

The Cilk Programming Language

This chapter describ es the parallel C extension language Cilk The discussion b elow is

neither a complete sp ecication of the language nor a detailed description of how Cilk

extensions are translated into C That information is available elsewhere Rather the

rst section of this chapter will present an example of Cilk and describ e its sp ecial features

and the remaining sections will show how those features are translated into C in order to

explain why Cilk requires type checking and dataow analysis from its prepro cessor

An example of Cilk fib

Cilk programs are run in parallel on multiple pro cessors The basic unit of parallel compu

tation in a Cilk program is a procedure which is similar to a C function except that it can

run in parallel with other Cilk pro cedures

The b est way to explain Cilk is by example The Cilk pro cedure fib shown in

Figure computes Fib onacci numbers recursively with the recursive calls pro ceeding

in parallel Most of the co de is ordinary C which is passed through the Cilk prepro cessor

unchanged except for the statements and declarations containing keywords in b oldface

which must b e translated into C These keywords are briey explained b elow

pro cedure identies fib as a Cilk pro cedure Only Cilk pro cedures can b e spawned

or called in parallel

procedure int fib(int n) { if (n < 2) return n; else { int x, y; x = spawn fib(n-1); y = spawn fib(n-2); sync; return x+y; }

}

Figure The Fib onacci function computed by a recursive Cilk pro cedure Features in

b oldface must b e translated by the CilktoC prepro cessor

spawn calls a Cilk pro cedure in parallel with the current thread of execution The

pro cedure initiating the spawn is the parent and the pro cedure spawned is its child

A spawn statement must assign the childs return value to a lo cal variable whose

value is undened until the parent executes a sync statement to wait for the child to

return

sync waits for all children started with spawn to return After a sync statement the

return values of all previouslyspawned children are computed and available

return returns a value to the pro cedures parent A Cilk pro cedures return state

ment diers from the ordinary C return statement since the Cilk pro cedure may

need to communicate the return value to a dierent pro cessor

Cilk includes other features that provide the programmer with greater control over

synchronization and scheduling of parallel pro cedures which are describ ed elsewhere

The remaining sections of this chapter will use the fib example to illustrate the fol

lowing claims First the backend C compilers type checking fails to detect some static

type errors in Cilk programs and rep orts unhelpful error messages for others so the pre

pro cessor must p erform Cilksp ecic type checking Second some Cilk extensions require

typedirected translations in which the C translation dep ends on the types involved Thus

the prepro cessor must determine and store type information Third and nally some Cilk

extensions cannot b e optimized by the backend C compiler so the prepro cessor needs

global ow analysis to generate optimized C co de

Type checking

This section shows why the type checking provided by the backend C compiler is insucient

for Cilk considering as an example the spawn statement A typeincorrect spawn statement

translates into typecorrect C co de so the type error cannot b e detected by the backend

C compiler As a result the Cilk prepro cessor must p erform type checking directly on the

Cilk source

The C co de corresp onding to one of the spawn statements in the Fib onacci example is

shown in Figure This fragment of C co de calls the runtime system function NewFrame to

construct a frame which is a migratable activation record for the pro cedure b eing spawned

A frame is a C structure containing slots for the pro cedures formal arguments and lo cal

variables along with some housekeeping information required by the runtime system The

frame is initialize d with the pro cedures single argument n and a sp ecial p ointer called a

continuation The continuation p oints to the slot in the parent pro cedures frame which

{ struct fib_frame *new_fp; newfp = NewFrame(fib, fib_fsize); x = spawn fib(n-1); newfp->n = n-1; newfp->ret = MakeContinuation(fp, offsetof(fp, x)); PostFrame(newfp);

}

Figure Translation of Cilk spawn statement into C details omitted for clarity

should receive the childs return value in this case the slot representing the lo cal variable

x The continuation is constructed from a p ointer to the parent pro cedures frame structure

and the oset of x in that structure Finally PostFrame places the new frame in the

schedulers work queue It may b e executed later on the current pro cessor or migrated to

another pro cessor that needs work to do

Observe that the C co de deals with x on a low level using its byte oset in the frame to

construct a continuation Thus if the programmer accidentally assigns a spawn to a lo cal

variable of incorrect type the backend C compiler will fail to detect the type error As a

result a mistake that was statically detectable can slip through to runtime resulting in a

runtime error or worse an incorrect answer

Typedirected translations

This section gives an example of a typedirected translation to show why the Cilk prepro

cessor must collect type information from the Cilk source The example is the Cilk return

statement whose translation into C dep ends on the of the pro cedure in which

it app ears The return statement is also used to illustrate that even when the backend C

compiler succeeds in detecting a type error in a Cilk program it often rep orts an unhelpful

confusing error message This example rearms the need for type checking in the Cilk

prepro cessor

The C translation of one of fibs return statements is shown in Figure A Cilk

pro cedure returns a value to its parent using one of a family of runtime system primitives

SendType Argument where Type can b e any C arithmetic type Char Int Float Double

etc In order to choose the appropriate runtime system primitive to replace the return

statement the prepro cessor must determine the return type of the enclosing pro cedure

The return statement also illustrates how relying on the backend C compiler for type

checking can result in unhelpful error messages since the messages describ e the C p ost

source rather than the original Cilk source If the expression in the return statement is

incompatible with the return type of the pro cedure then the backend C compiler rep orts

an error message like type mismatch in argument of call to SendIntArgument

This message makes no reference to the fact that the real problem was a type error in the

Cilk return statement Admittedly even the simplest Cilk prepro cessor can cause the line

number of the SendIntArgument call in the p ostsource to corresp ond with the line number

of the return statement in the source so at least the error message correctly identies

the oending line in the source But a Cilk programmer unfamiliar with the output of the

prepro cessor is likely to b e confused by the reference to SendIntArgument and may have diculty guring out what is wrong with the oending line and how the mistake should b e

{ SendIntArgument(fp->ret, x+y); return x+y; FreeFrame(fp); return;

}

Figure Translation of a Cilk return statement into C This statement app eared in the

fib pro cedure of Figure { fp->x = x; fp->y = y; fp->n = n; /* UNNEEDED! */ fp->entry = 1; sync; return; sync1: x = fp->x; y = fp->y; n = fp->n; /* UNNEEDED! */

}

Figure Unoptimized translation of Cilk sync statement into C some details omitted for

clarity This sync app eared in the fib pro cedure shown in Figure In that pro cedure

n is not used after the sync so it do es not need to b e saved and restored

corrected

Optimizations

This section shows why dataow analysis is imp ortant to the Cilk prepro cessor using

the sync statement as an example Since the backend C compiler must b e conservative

in its optimizations it misses an imp ortant optimization opp ortunity in the translation

of the sync statement With dataow analysis the Cilk prepro cessor can p erform the

optimization itself

A naive unoptimized translation of the sync statement in fib is shown in Figure

The sync statement susp ends fib until its spawned children have returned To wait for its

children fib must return control to the scheduler which it do es with an ordinary C return

statement The scheduler works on other parallel pro cedures until the children return at

which p oint the scheduler calls fib in such a way that control resumes at the lab el sync

In order to preserve its state of execution fib saves its lo cal variables b efore returning to

the scheduler and restores them after resuming

The C co de shown in Figure is not ecient b ecause it saves and restores al l lo cal

variables In particular it unnecessarily saves and restores the formal parameter n A

glance at Figure conrms that n will not b e used after the sync so its value need not

b e preserved

Normally the Cilk prepro cessor is not concerned with generating carefully optimized C

co de as long as it can rely on the backend C compiler to generate optimized ob ject co de

In fact an optimizing C compiler can remove the restoring assignment n fpn b ecause

the lo cal variable n is not used subsequently But no C compiler in existence can optimize

out the saving assignment fpn n A generalpurp ose compiler cannot assume that the

frame slot fpn will not b e needed later b ecause the frame is stored in ordinary memory

p ossibly aliased by other p ointers in runtime system data structures and manipulated at

the byte level by the runtime system The frame can even b e migrated from pro cessor to

pro cessor in which case all b ets are o as far as the C compiler is concerned

The Cilk prepro cessor on the other hand knows that the runtime system do es not use

fpn or any lo cal variable slot in the frame Only the Cilk pro cedures co de reads or writes

those slots Knowledge of this invariant ab out Cilk allows the prepro cessor to remove the

saving assignment as well an optimization that is imp ossible for the C compiler

In order to p erform the optimization the prepro cessor must use dataow analysis to

determine which variables may b e live or needed later and emit C co de that saves and

restores only those variables

Chapter

The CtoC Typechecking

Prepro cessor

The preceding chapter motivated the need for type checking and dataow analysis in the

Cilk prepro cessor To partition the development pro cess and increase the p otential for co de

reuse I rst built a mo del prepro cessor called CtoC that translates C into C and then

I extended it to recognize and translate Cilk This chapter describ es the architecture of

CtoC fo cusing on how it can b e extended to build a typechecking prepro cesor for an

arbitrary C extension language

CtoC encapsulates the organization of a generic typechecking prepro cessor It consists

of a sequence of phases op erating on a shared data structure called an abstract syntax tree

AST Each phase annotates or transforms the AST The AST is constructed from a C

program by a parser annotated by type checking and dataow analysis transformed by tree

op erations and nally unparsed to pro duce the original C program To build a C extension

language prepro cessor based on CtoC a developer mo dies the parser typechecker and

dataow analyzer to recognize the extension language then adds transformations that

convert extensions into C The Cilk prepro cessor CilktoC was built in precisely this

manner

CtoC is based on cparser an ANSI C static checker written by Eric A Brewer and

Michael D Noakes at the MIT Lab oratory for Computer Science with a yacc grammar

for ANSI C written by James A Roskind This package is designed to parse a C source

le convert it into an abstract syntax tree and p erform static type checking on the tree

CtoCs parsing and type checking phases and the structure of its abstract syntax tree are

inherited from cparser

The rst section of this chapter describ es the structure of the abstract syntax tree The

remaining sections describ e the phases of translation in CtoC

The abstract syntax tree

This section describ es the abstract syntax tree explains why it is a go o d representation for

extension language prepro cessing and p oints out the sp ecial features of CtoCs abstract

syntax tree that enable type checking dataow analysis and typedirected translation

An abstract syntax tree AST represents the syntactic structure of a computer pro

gram in a tree data structure Each tree no de in an AST represents a declaration

statement or expression op erator Its children are the op erands or syntactic comp onents while

while (x < 10) < x = x + 1; =

x 10 x +

x 1

Figure An abstract syntax tree AST for a C program fragment

of the declaration statement or expression Some examples of AST no des are shown in

Figure

Abstract syntax trees are an ideal representation for syntaxdirected macro prepro ces

sors The AST allows a prepro cessor to p erform translations lo cally converting an extension

statement or expression no de into C while treating its op erands as black b oxes For instance

the Cilk macro prepro cessor could transform a spawn statement which is represented in

the AST by a no de with one child for each argument of the spawned pro cedure by gen

erating a replacement tree for the spawn with the argument expressions substituted see

Figure

Since the AST representation is so convenient for C extension language translation C

toC also uses an AST but extends the representation to include additional information

In order to p erform type checking and semanticsdirected translation CtoC annotates

the AST with usedeclaration p ointers that p oint from identier uses to their declarations

control ow p ointers that identify the destination of nonlo cal jumps like goto variable and

expression types constant expression values and dataow analysis results These semantic annotations are used to transform the source AST into an ordinary C AST

spawn ...

... arg1 argN

arg1 argN

Figure Translating a Cilk spawn statement into C by op erations on the abstract syntax

tree The spawn arguments shown here as gray circles are substituted directly into the

tree of C co de that replaces the spawn no de cpp parse ... preprocessed Cilk AST Cilk source Cilk source

type ... analyze ... check Cilk AST Cilk AST

... transform unparse C AST

C postsource

Figure Translation phases of the Cilk prepro cessor based on CtoC

Phases

This section describ es the phases of translation in the CtoC architecture shown schemat

ically in Figure Actually the gure shows the Cilk prepro cessor based on CtoC in

order to clearly distinguish the frontend phases which op erate on a C extension language

like Cilk from the backend phases which op erate on C In the unextended version of

CtoC the frontend and backend languages are b oth C so it would b e hard to nd the

dividing line The purp ose of each phase is summarized b elow The remaining sections of

this chapter will describ e each phase of CtoC in more detail using the Cilk prepro cessor

to illustrate how to extend each phase

Cpp p erforms standard C macro prepro cessing on the source le pro ducing a le with no

prepro cessor directives or comments

Parse parses the source program into an AST

Type check p erforms type checking on the AST annotating every declaration and ex

pression no de with type information

Analyze p erforms dataow analysis on the AST annotating every statement and expres

sion no de with dataow information

Transform p erforms tree transformations that convert the annotated sourcelanguage AST

into a pure C AST In CtoC this phase do es nothing b ecause the source language

and target language are b oth C

Unparse unparses the AST into text

Macro prepro cessing cpp

The cpp phase p erforms standard C macro prepro cessing During this phase macro

denitions are interpreted and expanded conditional compilation directives are recognized

and comments are removed from the source The output of prepro cessing is a le of text

with no prepro cessor directives or comments

The current implementation of CtoC invokes the backend compiler to p erform C macro

prepro cessing One consequence of this decision is that the cpp phase cannot b e extended It

also limits the p ortability of CtoC since ANSI did not standardize the connection b etween

the C macro prepro cessor and the C compiler some backend compilers may not oer a way

to invoke their macro prepro cessing pass by itself Even a compiler that allows separate

invocation of its macro prepro cessor may emit private nonstandard compiler directives in

the prepro cessor output To remedy these and other deciencies a future implementation

of CtoC may include its own cpp macro prepro cessor

Parsing

The parsing phase consists of a lexical analyzer generated automatically by lex and an

LR parser generated automatically from an LR grammar by yacc that together

parse the source program and construct an AST representing it If any syntax errors o ccur

in the source they are rep orted by the parsing phase The parsing phase also connects

identier uses to their declarations by usedeclaration p ointers and nonlo cal jumps to their

destinations by controlow p ointers The result of parsing is an AST annotated with

declaration and controlow p ointers

To extend the parsing phase to recognize Cilk the Cilk prepro cessor adds new tokens

describ ed by regular expressions to the lexical analyzer and new syntax describ ed by

contextfree pro ductions to the grammar

The parsing phase is based on the BrewerNoakes ANSI C static checker The lexyacc

parser is based on the ANSI C parser by James A Roskind

Type checking

The type checking phase makes a pass over the AST determining types for expressions

and computing values for constant expressions During the same pass it lo oks for type

mismatches and makes other static semantic checks required by ANSI C and rep orts any

errors to the user The result of type checking is an AST annotated with types and constant

values

Type checking is driven by a table of metho d functions one for each kind of AST no de

The checks required for a new statement or expression can b e dened by adding a metho d

to the table The Cilk prepro cessor adds metho ds that typecheck its unique statements

like spawn and sync In a C extension language that denes new types like the Cilk

procedure type it may also b e necessary to mo dify the typechecking metho ds of ordinary

C statements and expressions For instance Cilk allows a Cilk procedure to b e called like a

C function In order to allow this usage the Cilk prepro cessor extends the typechecking of

an ordinary C function call to accept a Cilk procedure as well a C function in the op erator

p osition

The type checking phase is also based originally on the BrewerNoakes static checker

though signicant changes have b een made

Analysis

The analysis phase p erforms intraprocedural dataow analysis on every pro cedure in the

AST The analysis phase can compute the solution of any dataow problem that can b e

represented in a monotonic dataow analysis framework including live variables reaching

denitions and constant propagation The algorithm is iterative making rep eated passes

over a pro cedure until the dataow equations converge to a solution Monotonic frameworks

and the iterative algorithm are discussed in detail in Chapter

The analysis phase also makes some semantic checks warning the user ab out unreachable

statements and functions which do not return a value The result of dataow analysis

is an AST annotated with dataow analysis information such as live variables at every

statement

Like type checking dataow analysis is also tabledriven In the Cilk prepro cessor the

control ow of Cilk statements was dened by adding metho ds to the table In addition

a developer can extend the analysis to solve additional dataow problems by dening

the problems as monotonic frameworks The Cilk prepro cessor for instance denes two

dataow problems live variable analysis which determines whether a variable may b e

used b efore its next assignment and dirty variable analysis which determines whether a

variable has b een assigned since it was last saved to the frame

Transform

The transform phase makes zero or more passes over the AST to convert C extensions into

pure C In CtoC this phase do es nothing since no C extensions are recognized by the

frontend In the Cilk prepro cessor however the transform phase converts the Cilk syntax

tree into a C syntax tree The output of the transform phase is a C AST

To an extension language developer the transform phase is the most imp ortant and most

interesting phase since it sp ecies how extensions are translated into C The transform

phase accounted for more than of the new co de needed to extend CtoC into the Cilk

prepro cessor and ab out of the development time

Unparsing

The last phase the unparsing phase walks over the C AST and emits C co de corresp onding

to it The unparsing phase is designed to b e the inverse of the parsing phase so that re

parsing the output with CtoC pro duces an identical AST

Since the Cilk prepro cessor generates ANSI C it do es not need to extend the unparsing

phase Other extension language prepro cessors might extend the unparsing phase if the

target language is a language other than ANSI C

Chapter

Transparency in CtoC

This chapter describ es the sp ecial care taken by CtoC to imitate one of the strengths of

simple macro prepro cessors transparency We say a translator is transparent to a feature

of the source program if the translator copies the feature unchanged from input to output

preserving it through any intermediate program representations Transparency is go o d for

three reasons First a transparent translator is more p ortable b ecause it avoids making

changes that could pro duce unp ortable co de Second a transparent translator interacts

well with other programming to ols that rely on nonsyntactic features of the source like

lineoriented debuggers and prolers Finally transparent translators are easier to debug

b ecause the p ostsource is highlevel and readable and extension translations app ear in

context in the p ostsource Textbased macro prepro cessors are transparent to all features

since they deal directly with the text of the program CtoC however converts the program

text to and from an abstract syntax tree representation which is not ideal for representing

all features of the source Nevertheless our implementation of CtoC is transparent to

pragma directives line numbering indentation and constant expressions

On a highlevel CtoC converts a C program the source into an AST then converts

the AST back to a C program the postsource In principle the p ostsource should b e

identical to the source a goal we call complete transparency Unfortunately the current

implementation of CtoC cannot satisfy complete transparency b ecause an external macro

prepro cessor removes comments and macros from the source b efore it reaches CtoC

In practice however the backend C compiler is the only consumer of the p ostsource

so not all information in the source is imp ortant Whitespace and comments intended for

human users are ignored by the backend compiler and macros and prepro cessor directives

can b e interpreted early without aecting the nal ob ject co de For the purp ose of extension

language prepro cessing it is unnecessary to capture and maintain all this extra information

in the AST

Thus CtoC meets a weaker sp ecication which we call operational transparency if

the source is a C program with no syntactic or semantic errors then the p ostsource is

a C program with no errors that is compiled to identical ob ject co de by the backend

compiler If the input contains errors then CtoC should issue appropriate error messages

without emitting the p ostsource Happily this sp ecication is as easy to test as complete

transparency we can compile the source and the p ostsource separately then p erform a

bytebybyte comparison of their ob ject co de les

Op erational transparency gives more freedom to an implementor of CtoC but it should

not give so much freedom that CtoC is useless as a prepro cessor framework CtoC should

still b e transparent to features of the source that do not aect ob ject co de such as line

numbering and highlevel syntax In particular we note the following desiderata for a go o d

implementation of CtoC

The p ostsource should b e portable In particular if the source is a strictly conforming

ANSI C program then the p ostsource should also b e a strictly conforming ANSI C

program Thus CtoC should not p erform transformations that dep end on a

particular target platform or backend C compiler An example of an unacceptable

transformation is constant expression folding which will b e discussed further b elow

Portability allows a CtoCbased prepro cessor to target any platform with a standard

ANSI C compiler

The p ostsource should have the same line numbering as the source so that debuggers

and prolers that rely on matching line numbers in the ob ject co de to line numbers

in the source can function correctly

The p ostsource should b e highlevel CtoC should not for instance transform while

lo ops into lab els and goto statements nor should it atten nested expressions into

sequences of simple op erations with temp oraries Preserving the highlevel features

of the source contributes to the readability of the p ostsource and allows an extension

language developer to view extension translations in the context of the original source

The p ostsource should b e formatted for reading since human programmers have to

read and debug it while developing a CtoCbased prepro cessor Rather than emit

all the p ostsource on one incredibly long line for instance CtoC follows the line

breaks and of the source as closely as p ossible

The remainder of this chapter examines how these transparency requirements aect the

manner in which CtoC handles pragma directives line numbering and indentation and

constant expressions all of which must b e preserved in the AST The next chapter will

explain how transparency requirements aect dataow analysis

Pragma directives

The ANSI standard p ermits a compiler to recognize private compiler directives preceded by

pragma This section describ es how CtoC preserves unrecognized pragma directives from

the source to the p ostsource so that the source can include compiler directives intended for

the backend C compiler

CtoC itself recognizes two pragma directives pragma lang C which puts the lex

ical analyzer into a sp ecial Conly mo de and pragma lang C which ends the sp ecial

mo de Of course these directives have no eect in the unextended version of CtoC since

CtoC recognizes only C anyway They are provided in anticipation that a C extension

prepro cessor based on CtoC would need a way to turn o its sp ecial reserved words and

treat part of the input program as strict C Many Cilk programs for example are mixes of

existing C co de and new Cilk co de With these pragma directives the Cilk prepro cessor

can accept existing C co de that happ ens to use Cilk reserved words as ordinary identiers

without forcing all such identiers to b e manually renamed

Other pragma directives in the source are assumed to b e intended for the backend

compiler and CtoC attempts to preserve them Unfortunately since pragma directives

stmt ::= if ( expr ) stmt else stmt

Figure A grammar pro duction for the ifelse statement in C

are like prepro cessor directives they are not constrained by the ANSI C grammar and

may app ear b etween any pair of tokens in the input program As a result they cannot b e

expressed in the grammar CtoC handles this problem by collecting pragma directives in

the lexical analyzer Then b efore starting a new statement or declaration the parser inserts

any waiting directives into the AST The drawback of this simple scheme is that pragma

directives that app eared within a statement or declaration are pushed to the end of the

statement or declaration p ossibly changing their meaning CtoC generates a warning

message when a pragma must b e moved in this manner In our exp erience

however these directives are generally used at the le level or statement level and rarely

app ear in the middle of a statement or declaration

Line numbering and indentation

CtoC preserves line numbering and indentation from the source to the p ostsource by

storing line numbers and character osets in the AST Preserving line numbering enables

lineoriented debuggers and prolers to match ob ject co de with original source lines and

preserving indentation makes the p ostsource easier to read

When it constructs the AST the parsing phase annotates each no de of the tree with

the source le line number and character oset within the line where the syntax construct

app eared Such a triple hle line oseti is referred to as the no des source coordinates

Source co ordinates are useful not only for identifying the oending source line in Cto

Cs error messages but also for providing the output phase with enough information to

reconstruct the line numbering and indentation of the source for the b enet of debuggers

prolers and human programmers using the p ostsource

Line numbering and indentation are preserved by saving the co ordinates of tokens in the

input program Every token in the input program is a terminal in some pro duction in the

grammar which is represented by a no de in the AST Consider for instance the pro duc

tion in Figure which shows the ifelse statement An if no de in the AST actually

represents four input tokens shown in b oldface in the gure if and else We could

repro duce the input p erfectly if we stored the co ordinates of every one of these tokens in the

ifelse no de Such precision is overkill however since our goal is not exact repro duction

of the input program but approximate repro duction with identical meaning In typical C

formatting styles the p ositions of semicolons commas and parentheses are xed relative to

their neighbor tokens always immediately following or immediately preceding so CtoC

saves storage space in the AST by throwing away the co ordinates of these tokens

As a result for most AST no des CtoC saves the co ordinates of only one token For

statement no des the co ordinates of the statement keyword are saved For op erators the

co ordinates of the op erator token are saved For twokeyword statements or op erators like

ifelse and dowhile the co ordinates of b oth keywords are saved For a blo ck a list of

statements enclosed in curly braces the co ordinates of b oth curly braces are stored since

dierent programmers prefer their braces in dierent places

Using the token co ordinates stored in the AST by the parsing phase the unparsing

phase attempts to reconstruct the line numbering and indentation of the input Since the

parsing phase didnt save co ordinates for every input token but only selected tokens the

reconstructed C co de is not identical to the input but it is close enough to meet the needs

of lineoriented debugging and readability

The transform phase of CtoC may have added and deleted parts of the AST and moved

other parts around As a result the unparsing phase cannot b e guaranteed that the token

co ordinates in the AST are in their original order Given a token T its source co ordinates

hlelineoseti and the co ordinates of the current output p osition hFileLineOseti Cto

C must b e able to emit T in an output p osition as close as p ossible to its source co ordinates

which may have a completely dierent lename or line number Fortunately C provides

a prepro cessing directive that sets the lename and line number arbitrarily line line

lename In order to prevent excessive line directives from obscuring the readability

of the p ostsource CtoC uses the following heuristic A line directive is emitted if and

only if one of the following conditions is true

le File to change the output lename

line Line to back up to an earlier line in the output

line Line to advance a large distance in the output The value is a small

constant typically This heuristic compresses the large runs of empty lines usually

left by the C macro prepro cessor after removing comments and conditionally compiled

co de With this heuristic runs of more than empty lines are replaced by a line

directive jumping immediately to the end of the run

After p ossibly emitting a line directive CtoC inserts sucient lines and spaces in the

output to make Line line and Oset oset This simple algorithm suces to match

the line numbering and indentation of the original source

Constant expressions

CtoC preserves all constant expressions from the source to the p ostsource For the sake of

p ortability it p erforms no constant folding even though it must evaluate constant expres

sions for semantic checking This section examines the problems with constant expressions

evaluation more closely

CtoC must evaluate constant expressions to p erform a variety of static semantic checks

For instance in ANSI C all array dimensions must b e p ositive the case expressions in a

switch must have distinct constant values and two array types are the same only if they

have the same dimensions

Evaluating constant expressions in CtoC is complicated by the fact that many constant

expressions in ANSI C are implementationdependent their values dep end strongly on the

backend compiler and the target architecture For example expressions dep end on

the machine word size and sizeofstruct foo dep ends on how the backend compiler

chooses to arrange the foo structure in memory Arithmetic on constant expressions can

also return implementationdependent results One example is the ones complement of

whose value dep ends on how many bits are in a machine word For maximum p ortability

to dierent backend compilers CtoC should not make any assumptions ab out the values

of such expressions

Still CtoC cannot simply omit semantic checks involving implementationdependent

constant expressions since the checks are imp ortant for discriminating strictly conforming

ANSI C programs from denitely erroneous inputs For instance all conforming ANSI C

compilers accept int foosizeofint in which the array dimension is certainly p osi

tive and reject or warn ab out int foosizeofintsizeofint in which the array

dimension is always zero In order to conform to the ANSI standard CtoC must make

the same discrimination

CtoC solves this problem by computing a value for every constant expression using

reasonable word sizes and structure packing rules solely for the purp ose of such semantic

checks The computed value is stored in the AST no de of the expression for later use but it

do es not replace the expression nor is it used in the unparsing phase to p erform constant

expression folding Although CtoC could conceivably recognize that an expression has a

welldened implementationindep ende nt value and fold the expression down to that value

this optimization is not worth the trouble since the backend C compiler is equally capa

ble of constant folding Furthermore folding constant expressions reduces the abstraction

level of the program converting a highlevel expression into a lowlevel expression CtoC

delegates constant folding to the backend compiler where it b elongs

Chapter

Dataow Analysis in CtoC

This chapter describ es CtoCs dataow analysis algorithm which works directly on the

highlevel abstract syntax tree CtoC implements a standard iterative algorithm but

with p erformance improvements enabled by the AST representation which include depth

rst order and testing for convergence only at lo ops and jumps It also provides an abstrac

tion for monotonic dataow analysis problems so that an extension language developer can

simply dene the dataow problem that needs to b e solved and CtoCs dataow anal

ysis algorithm automatically solves it The Cilk prepro cessor uses this dataow analysis

framework to determine the live variables at a sync statement

CtoC p erforms dataow analysis directly on the highlevel AST using an implicit

control ow graph dened by the semantics of the C language Because of the requirement

of transparency describ ed in the previous chapter CtoC cannot reduce the AST to a sim

pler intermediate language that contains no nested expressions CtoC cannot even treat

a nested expression as a basic blo ck of straightline co de b ecause many C expressions

do not have straightline control ow Expressions can contain conditional op erators or

shortcircuiting Bo olean op erators which may not evaluate their second op erand Worse

the GNU extensions to C allow arbitrary statements to app ear inside expressions so jumps

into and out of the middle of an expression are p ossible CtoC supp orts this state

ment expression extension b ecause it is o ccasionally useful in Cilk programs To avoid

these complications CtoC treats every AST no de as a basic blo ck so its control ow

graph CFG contains many more no des than a typical compilers control ow graph Each

iteration of CtoCs dataow analysis algorithm must propagate information through more

no des

CtoC osets this p erformance hit by taking advantage of prop erties of the implicit

CFG First the iterative algorithm converges fastest if the no des of the CFG are visited

in depthrst order on each iteration Since the AST is a depthrst spanning tree

of its implicit CFG we can visit CFG no des in depthrst order by simply walking the

tree so fast convergence is automatic Second CtoC tests for convergence only at no des

corresp onding to jumps and lab els rather than at all no des of the CFG as would a standard

algorithm based directly on controlow graphs

The iterative algorithm co ded in CtoC applies to a variety of dataow analysis prob

lems describ ed in general as monotonic frameworks Following the suggestions made by

Kildall CtoC denes a monotonic framework abstraction which enables an extension

language developer to dene a dataow analysis problem declaratively The monotonic

framework abstraction is describ ed in detail b elow if-else

entry exit expr else-stmt

then-stmt

Figure The implicit control ow of an ifelse no de The no de corresp onds to the C

co de if expr thenstmt else elsestmt

Representing control ow in the AST

CtoC p erforms dataow analysis on a control ow graph CFG represented implicitl y in

the AST Representing the control ow graph implicitl y rather than explicitly saves storage

and makes CtoC easier to extend

Every statement or expression no de in the AST has an implicit CFG For example

in the ifelse no de shown in Figure control passes rst to the expr child branches

to either thenstmt or elsestmt and then rejoins to leave the ifelse no de The implicit

control ow graphs for each AST no de together sp ecify an implicit control ow graph for

the entire AST

Most no des of the AST receive control from the parent initially and return to the

parent after execution Some no des however represent jumps or lab els such as goto

break continue case and switch The parsing phase annotates these no des with control

ow p ointers connecting for example each goto no de with its destination lab el no de

elsewhere in the AST The implicit control ow graph of jump and lab el no des makes use

of these nonlo cal links The parser do es not connect function calls with function denitions

however and so CtoCs ow analysis is presently limited to the scop e of a single function

Intraprocedural analysis is sucient for Cilk and thus this restriction is not severe Other

C extension languages could extend CtoC for interprocedural dataow analysis but the

work required would b e nontrivial

Representing the CFG implicitly saves space Most control ow follows existing AST

edges b etween parents and children in the tree and an implicit CFG reuses the AST edges

to save space Only the jump and lab el statements mentioned in the previous paragraph

have nonlo cal control ow that do es not follow AST edges requiring additional space to

store controlow p ointers

The implicit CFG is also easy to extend b ecause it is tabledriven Each statement or

expression no des implicit CFG is dened pro cedurally by a that passes data

ow information among the no des children following the implicit CFG edges The control

ow of a C extension language can b e dened simply by writing a new subroutine for each

new statement or expression

We should make one more observation ab out implicit control ow through expressions

CtoC assumes that expressions and function call arguments are evaluated strictly from left

to right but the backend compiler is free to choose any evaluation order when it generates

actual ob ject co de As a result if the b ehavior of the source program dep ends on evalu

ation order then CtoCs analysis may b e incorrect Fortunately programs that dep end

up on evaluation order are rare and inherently unp ortable since the order of evaluation of

expressions and sideeects in C has always b een undened Few programmers write

co de that dep ends on evaluation order and those that do deserve what they get

Dataow analysis frameworks

This section describ es CtoCs dataow analysis abstraction For the sake of extensibility

CtoC provides a monotonic dataow analysis framework abstraction that is applicable

to a wide variety of dataow problems including live variables constant propagation

reaching denitions and available expressions Monotonic dataow analysis frameworks

were originally prop osed by Kildall and developed by Kam and Ullman CtoCs

implementation follows the treatment of monotonic frameworks in the Dragon Bo ok

section

A monotonic dataow analysis framework consists of

Data type V a set of values to b e propagated A value in V represents an assertion ab out

the dynamic state of the program The V must contain a distinguished

element which represents undened or unknown The concrete data type used

to implement V must provide an equality op eration that tests whether two elements

of V are the same so that the iterative algorithm can detect convergence

Direction of propagation dataow information may b e propagated either forwards or

backwards through the control ow graph

Meet op eration a function mapping V V V which combines assertions ab out two

paths of execution where the paths join The meet op eration must b e commutative

asso ciative idemp otent and has as its identity element With these prop erties

V is a semilattice In particular we can dene the partial ordering on V by

x y x y x

Thus the distinguished undened element is the top of the semilattice since x

x for all x V

Also the semilattice V must b e bounded all chains x x in V must

1 2

have nite length If the set of values V is nite then this condition is automatically

satised

Transfer functions F a set of functions mapping V V which transform a value in V

as it passes through an AST no de Two functions from F are assigned to each no de

one to the entry p oint and one to the exit p oint The set F generally should contain

the identity function on V since many AST no des have no eect on data ow

All functions in F must b e monotonic

f x y f x f y for all f F and x y V

The monotonicity of F and b oundedness of V together imply that the iterative

algorithm of successive approximation used by CtoC eventually converges to a solu

tion

The functions in F should also satisfy strictness f for all f F Unlike

most compilers CtoC do es not remove obviously unreachable co de from its AST

b efore p erforming dataow analysis As a result if F contains nonstrict functions

unreachable paths in the control ow graph could propagate information to the rest

of the graph despite the fact that they would never b e visited at runtime Thus

with nonstrict functions CtoCs dataow analysis is to o conservative When all

functions in F are strict however this problem do es not arise All unreachable paths

remain undened which cannot aect the reachable paths of the program

For an example of a dataow analysis framework consider the problem of computing

live variables A variable x is live at a p oint P in a program if x may b e used b efore b eing

reassigned on some p ossible path of computation starting at P Livevariable analysis is

useful to the Cilk prepro cessor b ecause if a variable is dead ie not live at a sync p oint

then the prepro cessor need not preserve its value across the sync

Livevariable analysis ts into the framework as follows The data type V is a set of

variables represented by a bit vector A value of V asserts that the variables it contains are

live at that p oint in the program The direction of propagation is backwards since liveness

is a prop erty determined by the future execution of the program The meet op eration

is set union since a variable is live at a p oint in the program if and only if it is live

on at least one path leaving that p oint Finally the transfer function set F contains two

monotonic functions g en S S fxg which adds x to the set of live variables and

x

k il l S S fxg which removes x from the set of live variables Every AST no de

x

representing a use of the variable x has g en as its exit transfer function and every no de

x

representing an assignment to x has k il l

x

Monotonic dataow analysis frameworks are imp ortant to CtoCs extensibility By

dening a data type for V and writing for the meet and transfer op erations an

extension language developer can reuse CtoCs iterative algorithm to gather any dataow

information needed for translation or optimization

The iterative algorithm

Given a framework and an AST which together dene a set of dataow equations CtoC

computes a solution to the dataow equations using the standard iterative algorithm

algorithm The iterative algorithm b egins with all edges in the implicit control ow

graph set to and iteratively visits no des satisfying their dataow equations until the

solution converges to a xed p oint CtoCs algorithm diers from the standard algorithm

in two resp ects the control ow graph is traversed in depthrst order and convergence is

only tested at lo ops and jumps

Visiting CFG no des in depthrst order yields the fastest convergence of the standard

iterative algorithm If the dataow problem is a distributive framework or a monotonic

framework satisfying certain assumptions and if the CFG is reducible each lo op has at

most one entry p oint then the number of iterations b efore convergence is b ounded by the

loopconnectedness of the CFG which is dened as the largest number of back edges on any

cyclefree path of the graph Intuitively the lo opconnectedness is the nesting level of the

innermost lo op in the graph This result means that for most dataow frameworks and

wellstructured C functions which have only one entry p er lo op the iterative algorithm

requires only a few iterations roughly as many as the depth of the innermost lo op of the

function

To visit CFG no des in depthrst order other compilers must build a depthrst span

ning tree on the CFG In CtoC the AST is automatically a depthrst spanning tree

of its implicit control ow graph so CtoC visits CFG no des in depthrst order just by

walking over the AST

The standard iterative algorithm visits every no de in every iteration halting if and

only if the data ow at all no des remains constant across an iteration Testing all no des

for changes during the iteration would b e more exp ensive for CtoC b ecause it do es not

summarize expressions and straightline co de into a single basic blo ck

CtoC uses a faster termination condition that takes advantage of the highlevel AST

CtoCs algorithm halts if and only if the data ow at lo op no des and conuence no des

remains constant A loop node is an AST no de representing a lo op in the source langage In

C the lo op no des are dowhile for and while A conuence node is an AST no de that

may receive or pass control nonlo cally with another AST no de that is neither its parent

1

nor one of its children The conuence no des in C are goto switch continue break

return lab el case and default An acyclic no de is neither a lo op no de nor a conuence

no de The acyclic no des in C are expressions and conditional ifelse statements

Because CtoC visits no des in depthrst execution order the dataow equations at

acyclic no des are always satised after every iteration As a result if the data ow at the

lo op no des and conuence no des remains unchanged across an iteration then the data ow

at the acyclic no des also remains unchanged Thus it suces only to test for convergence

at the lo op no des and conuence no des

As a consequence of its termination condition CtoC p erforms only a single iteration

if the AST contains no lo op no des or conuence no des An AST consisting entirely of

acyclic no des has an acyclic implicit control ow graph so a single depthrst pass suces

to compute the solution to the dataow equations

1

A switch statement in C is not as structured in C as it is in other languages In C switch is actually

a computed jump to a lab el inside its b o dy Syntactically the lab el is not a direct child of the switch so

we regard this jump as a nonlo cal transfer of control

Chapter

Conclusions

Using the CtoC framework I have written a prepro cessor that translates b oth Cilk and

Cilk p erforms type checking on the Cilk source and uses live variable analysis to optimize

saving and restoring lo cal variables across sync statements

Building the Cilk prepro cessor on top of CtoC required a relatively small amount of

work the equivalent of a few weeks of eort by a fulltime programmer In fact ab out

of the programming eort was sp ent in guring out how to transform Cilk extensions into

C Extending type checking and dataow analysis to cover Cilk was comparatively simple

Our exp erience with the Cilk prepro cessor suggests that CtoC is a go o d prepro cessor

framework since it allows an extension language developer to fo cus on the syntax and

semantics of the extension language rather than on type checking or dataow analysis

The typechecking prepro cessor is now in everyday pro ductive use by the Cilk group who

prefer it to the older Cilk macro prepro cessor The typechecking prepro cessor translates

Cilk into C slower than the macro prepro cessor but just as quickly as our C compiler gcc

compiles C into ob ject co de Overall then users of the typechecking prepro cessor pay

only a factor of two in compile time in exchange for typechecking optimization and the

ability to use Cilk a higherlevel language than Cilk

With an extensible framework like CtoC we can aord to contemplate more ambitious

extensions to the Cilk language without cringing at the p otential development eort For

instance we are exp erimenting with a distributed shared memory system for Cilk which is

currently implemented with a lowlevel runtime library interface To simplify programming

the interface we have devised a global p ointer extension for Cilk which allows a program

mer to manipulate shared ob jects with familiar C p ointer op erations With the new Cilk

prepro cessor we anticipate that adding global p ointers to Cilk will take only a few days of

work

Future work on CtoC

A future version of CtoC may include its own standard C macro prepro cessor The current

implementation relies on the backend C compiler to p erform macro prepro cessing which

is neither p ortable since not all C compilers oer a separate macro prepro cessor nor

extensible

Once CtoC p erforms its own macro prepro cessing it will have access to the original

source program enabling a more transparent implementation of CtoC Enough informa

tion could b e saved in the abstract syntax tree to repro duce the source program precisely

including comments and macros

Precise source repro duction would make CtoC applicable to a wider variety of source

tosource pro cessing applications In C extension language prepro cessing the only consumer

of CtoCs output is a backend compiler But in many other applications of source

tosource translation a human programmer might b e another consumer in which case

preserving comments is extremely imp ortant Other sourcetosource translation problems

that may b e solvable with CtoC include automatic or programmerdirected sourcelevel

optimization program analysis source co de metrics error checking eg lint to ols and

instrumentation for proling or debugging

Getting CtoC

The source co de of CtoC is freely available from ftptheorylcsmitedupubcc

Building CtoC requires an ANSI C compiler and the free GNU to ols flex and bison

Bibliography

Alfred V Aho Ravi Sethi and Jerey D Ullman Compilers Principles Techniques

and Tools AddisonWesley

S P Amarasinghe J M Anderson M S Lam and C W Tseng The SUIF com

piler for scalable parallel machines In Proceedings th SIAM Conference on Parallel

Processing for Scientic Computing February

American National Standards Institute Inc NY American National Standards for

Information Systems Programming Language C ANSI X

Rob ert D Blumofe Christopher F Jo erg Bradley C Kuszmaul Charles E Leiserson

Phil Lisiecki Rob Miller Keith H Randall Andy Shaw and Yuli Zhou Cilk

Reference Manual MIT LCS Technology Square Cambridge MA May

Available via ftptheorylcsmitedupubcilkmanualpsZ

Rob ert D Blumofe Christopher F Jo erg Bradley C Kuszmaul Charles E Leiserson

Keith H Randall and Yuli Zhou Cilk An ecient multithreaded runtime system

In Proceedings th ACM SIGPLAN Symposium on Principles and Practice of Parallel

Programming Santa Barbara CA July To app ear

Francois Bo din Peter Beckman Dennis Gannon Jacob Gotwais Srinivas Narayana

Suresh Srinivas and Beata Winnicka Sage An ob jectoriented to olkit and class

library for building Fortran and C restructuring to ols In Proceedings of OONSKI

Also available as ftpftpextremeindianaedupubsageo onskipsgz

Francois Bo din Peter Beckman Dennis Gannon Srinivas Narayana and Shelby X

Yang Distributed pC Basic ideas for an ob ject parallel language Scientic

Programming Fall

K Mani Chandy and Carl Kesselman CC A declarative concurrent ob ject ori

ented programming notation Technical Rep ort CSTR California Institute of

Technology Pasadena CA

David E Culler Andrea Dusseau Seth Cop en Goldstein Arvind Krishnamurthy

Steven Lumetta Thorsten von Eicken and Katherine Yelick Parallel programming in

SplitC In Proceedings of Supercomputing the ACMIEEE Conference Portland

OR November

Chris Fraser and David Hanson A Retargetable C Compiler Design and Implemen

tation BenjaminCummings

John B Kam and Jerey D Ullman Global data ow analysis and iterative algorithms

Journal of the ACM

John B Kam and Jerey D Ullman Monotone data ow analysis frameworks Acta

Informatica

Brian W Kernighan and Dennis M Ritchie The C Programming Language nd

Edition Prentice Hall

Gary Kildall A unied approach to global program optimization In Proceedings st

ACM Symposium on Principles of Programming Languages pages

J R Rose and G L Steele Jr C An extended language for data parallel pro

gramming In Proceedings nd International Conference on Supercomputing volume

pages San Francisco CA May

Richard Stallman et al GNU C compiler GCC version Source co de and do cumen

tation available from prepaimitedu via anonymous ftp in directory pubgnu