APL# - an APL for Microsoft.Net Morten Kromberg, Jonathan Manktelow, John Scholes Dyalog Ltd Minchens Court, Minchens Lane, Bramley, RG26 5BH, United Kingdom [email protected] State of the art of array processing languages Abstract The paper discusses the design of APL# (pronounced “APL Sharp”), a new dialect of APL de- signed with object oriented / language-agnostic virtual machine platforms in mind, using Mi- crosoft.NET as the initial target. Microsoft.Net is a software platform which is based on Microsoft’s Common Language Runtime (CLR). Together with Sun’s Java Virtual Machine (JVM), the CLR is a popular member of a (relatively) new family of “application virtual machines” (AVMs) which are rapidly growing in im- portance as platforms for hardware-independent software development. These virtual machines provide high level components such as memory management, exception handling and other services which significantly simplify the task of implementing new programming languages. They also facilit- ate communication between modules written in different languages by providing a single memory manager and type system which is shared by every component which is built upon them. As the name suggests, the CLR was designed in co-operation with a diverse group of language implementers, and the services that it provides are designed to be “language agnostic”. APL# will also run under Microsoft SilverLight [SilverLight], a web browser plug-in which competes with products like Adobe Flash as a platform for “rich client” web applications. In fact, Sil- verLight is a cut-down version of the Microsoft.Net framework, capable of running many .Net applic- ations. SilverLight can be installed as a plug-in for most popular web browsers, under Microsoft Win- dows and Apple’s Mac OS X. The list of short-term target platforms also includes Mono and Moon- Light [Mono], which are open-source versions of the Microsoft.Net Framework and SilverLight, re- spectively – available under Linux and a growing number of other operating systems. There is reason to expect that SilverLight and Moonlight will achieve almost universal availability over the next few years, and that it will be possible to deploy applications written in APL# – including “rich” user inter- faces if required - on virtually any platform, possibly including the majority of mobile devices. Motivation Virtual Machines provide two very important benefits: Firstly, applications become (at least theoretically) universally portable, because the VM can be implemented on many different types of hardware. Modern AVMs include large libraries of high-level components, which make the develop- ment of cross-platform applications easier than it has ever been before. Secondly (but perhaps most significantly in the short term), the modern AVMs provide a secure environment, where the access that an application has to both local and remote resources can be strictly controlled. Such applications are said to be managed, or safe. Managed applications can be safely distributed across the internet, because it is possible to guarantee that they will not read or write data on the client computer onto which they have been downloaded – and that internet access is restricted to communication with the server from which they were downloaded. From a security per- 1 spective, the application can be considered to be running on the host computer, although it is making use of local resources for both presentation and data manipulation. Dyalog APL has had a bridge to the Microsoft.Net framework since 2002, and this interface has allowed APL developers to successfully tap in to many of the tools that the framework provides. However, there are a few glitches and discontinuities where the memory management model and type system of Dyalog APL conflict with the CLR. The design of APL# will resolve these conflicts, pro- ducing a language system which is “native” to the CLR. The immediate goal of the APL# project is to make the benefits of safe/managed computing available to users of APL. With APL#, it will be possible to run the same APL application on a vari- ety of client platforms, simply by directing users to a web page from which the software is retrieved and run – and it will be possible to do this with the blessings of most security departments. No actual “installation” of software is required on the client computer (the application is merely “cached”, in or- der to make it start faster the next time). Freedom from Upwards Compatibility Most APL interpreters request a single, large quantity of memory (the workspace) from the op- erating system, and manage user data and the APL execution stack without interference from the out- side world. On an AVM, we will be sharing the program stack and memory manager with other pro- gramming languages, and this more or less dictates some significant changes in APL language se- mantics. In addition, most of the system functions that provide interfaces to Application Program In- terfaces (APIs) will be abandoned in favour of tools provided by the AVM. For the first time in the history of the team behind Dyalog APL, we are designing an APL interpreter which does not need to run the same application code as the previous version1, and we have taken advantage of this opportun- ity to take a fresh look at the language. It is our hope that we will be able to take advantage of some of the work that has been done in- side the array language community in languages like J, and combine this with the improved under- standing of functional and dynamic languages that has emerged in recent decades. Our goal is to cre- ate a more competitive dialect of APL, that will be easier to market to a new generation of users – both to “domain experts” who have limited experience with software development, and to those “soft- ware engineers” who are looking for more dynamic tools. Although APL# must be a “first class cit- izen” of the Microsoft.Net framework, we want a language which will be portable to other VMs (like the Java VM), and can also be implemented on “real” computer hardware. In other words, there must not be any elements of the core language which require Microsoft.Net. In addition, APL# is expected to have performance characteristics which – at least initially – will be significantly different from those of Dyalog APL. We expect some things will be much faster; others will be slower, as many of the optimization techniques used by current APL interpreters cannot be translated to a strongly typed AVM memory manager, which does not provide reference counts for allocated memory [MicrosoftGCol]. We (or the APL application developers) will need to find new ways to write fast code. As a result, Dyalog will not be recommending that customers move quickly to translate existing applications to APL#. We think of APL# as a companion to traditional APL, aimed at new applications which can benefit from being native to an AVM – or as components of multi-tier applications which use APL# together with other APL systems. We also expect that some of the new language features pioneered in APL# will find their way “back” into Dyalog APL, and that the two languages will grow closer in the decade(s) to come. 1 Changes have not been introduced lightly; except for missing APIs, we believe that it remains possible to semi-automatically translate the bulk of existing APL algorithms into APL# and intend to build tools to do so. 2 Challenges The services provided by the CLR make it relatively easy to implement a new programming language, and the ability to inter-operate with solutions written in other languages makes a wide vari- ety of tools available to the application developer. However, this is a double-edged sword: when the framework is managing a heap (“workspace”) containing data which is shared between many pro- gramming languages, and also allows the stack to consist of functions written in different program- ming languages, APL# is not only forced to use the same type system – it should ideally also agree with other languages about what an application stack can contain. Under Microsoft.Net, it will be common for an APL# function to call a function written in a language like C# - which in turn calls another (or the original) APL# function. Unless we want to be faking a lot of things behind the cov- ers, and be a language which has strange discontinuities when debugging applications which use more than one programming language, we need to abandon some of the most central dogma of “classic” APL interpreters: ‐ The illusion that APL only has two basic data types: numbers and characters: APL# will still treat most numeric types (including Booleans) as a single logical type, but will also support all the data types available on the platform, as elements of an array. ‐ That arguments are always passed “by value”: in APL#, arrays are passed “by reference”, as they would be in most other programming languages. ‐ That user-defined names are global by default, and local variables are visible to all called functions: in APL#, all names created by a function are strictly local, unless they are placed within a pre-existing global container. The challenge is to retain the key advantages of APL, while transforming the language in ways which are dictated by the constraints of an “alien” software platform which is heavily influenced by scalar- and object-oriented compiled languages. Fortunately, the most important language changes, such as the strictly local scope for new names and the availability of arrays which are passed by refer- ence, are not new ideas; they have been proposed by many APL language theorists in the past [Seed- s1978].