Cover Venkatesh Prasad Ranganath Story Researcher at Microsoft Research, India

Exploring the Current Landscape of and Models

Until few years ago, of Programming Paradigms[44]” page assignments (e.g. i = i + 1). This is evident and object-oriented programming have at Wikipedia states that “None of the by the explicitly use and maintenance been two prevalent programming main programming paradigms have a of a counter variable i, dependence on paradigms in the industry. However, this precise, globally unanimous defi nition, let the length of nums, and direct access of has started to change over the past few alone an offi cial international standard.” elements of nums (via indexing syntax, years with the community exploring and Hence, instead of chasing defi nitions of i.e. nums[i]). slowly adopting . programming paradigms, let us try to In object-oriented style, the Similarly, there has been a renewed understand programming paradigms via program does not depend on the interest in exploring existing programming examples. internal details of list type. Instead, languages or creating new programming We shall start with three prevalent list type encapsulates its elements languages that support either tried-and- programming paradigms: imperative, (implementation details) and provides forgotten paradigms or a combination of object-oriented, and functional. an iterator abstraction (interface) to paradigms. Further, there has been a surge For this purpose, let us consider iterate over its elements. Consequently, in languages and programming systems to the following programs that identical in the program depends only on the support development of loosely coupled functionality but written in imperative, iterator abstraction. This style of applications such as web applications and object-oriented and functional using encapsulation to hide information distributed applications. programming styles. These programs and abstraction to provide appropriate To a large extent, these changes have accept a list of integers (nums) and views of information (decouple been fueled by two factors. The fi rst factor output a list of squares of these integers implementation from interface) are is the decision of the computer hardware (output). To simplify comparison, all the distinct features of object-oriented industry to move from faster single- of these programs are written in Python programming. core processors to slower multi-core [2]. list (an As for the part of specifying how to processors (to compensate for the failure ordered sequence of objects) is a primitive iterate through the elements, observe of Moore’s law[1]n).The second factor data type in Python. that the boilerplate code required to is the rise of internet-enabled mobile Imperative: use an abstraction such as iterators devices, cloud computing, and big data. output = list() is dependent only on the abstraction As a result of the fi rst factor, i = 0 and is parameterized only by the object applications cannot be scaled by merely while i

1Readers interested in concurrency and hardware changes may want to read Herb Sutter’s “Welcome to the Jungle” blog post[38]. 2As a result, the decisions regarding how to iterate and the order of iteration are deferred to the or the runtime system.

CSI Communications | February 2013 | 6 www.csi-india.org iterator abstraction provided by list Where’s that list of languages? type to iterate through nums. Hence, as There is a reason this article has been in mathematics, functional programming language agnostic thus far. Most existing is all about applying and composing main stream languages (such as #, functions to specify computationp. JavaScript, Python, and Ruby[3,4,2,5]) and Another distinct aspect of functional new up-and-coming languages (such programming is immutability of data – as Clojure, F#, and Scala[6,7,8]) support function applications and expression various aspects of multiple programming evaluations create values as opposed to paradigms. This is achieved by off ering modifying existing values. To understand language features that embody a this aspect in terms of its ramifi cation combination of aspects of multiple on programming, consider a function paradigms, e.g. F# supports object- append(l,e) that appends element e oriented programming by supporting Fig. 2: Summing network to list l. In terms of exposing the result of classes. On the other hand, there are program. So, square function can be append, there are two options: 1) modify languages that are pure in supporting a applied to the elements of nums in random list l in place (imperative) or 2) create a single paradigm, e.g. Haskell and SML/NJ order. This implies square function can be new list containing the elements of land are pure functional languages[9,10]. concurrently applied to elements of nums the element e (functional). The following While language support for a provided square and Seq.map functions pseudo code snippets exercise these programming paradigm certainly helps are side-eff ect free. Since these functions options to achieve the same goal – v1 programming in that paradigm, one could are indeed side-eff ect free, we can trivially should refer to the list[7,8,9]. program in a paradigm that is not directly parallelize the above programas follows supported by a language, e.g. one could v1 = [7,8] v2 = [7,8] by using PSeq.map, a parallel version of v2 = v1 v1 = append (v2,9) do object-oriented style programming in Seq.map. append(v1,9) a language (e.g. C) that is not ideal for object-oriented programming. Remember With the fi rst option (on the left), the nums |> PSeq.map square |> that programming paradigm is about the List.ofSeq change to listv1 will be visible via v2 style of programming (and not about any (an alias to v1) referring to v1’s list. This This example was a simple case specifi c programming language). phenomenon is known as side-eff ect. of data parallelism – same operation is With the second option (on the right), Rookies performed on various data items and due to the absence of side-eff ect, the list Besides prevalent programming these operations and their results are resulting from append is assigned to v1 paradigms, there are new language mutually independent – and functional to achieve the desired goal (as, without features (or paradigms?) that are inspired programming was ideal. the assignment, v1 will refer to list[7,8]). by specifi c aspects of problems. In the rest Now, consider the task of summing Fig. 1 illustrates eff ect of executing these of this article, we shall explore few such the squares of a given list of integers programs. problem aspects and related language (nums) with the following sequential F# While presence and absence of side- features and paradigms. program. In this program, Seq.reduce eff ect have their benefi ts and drawbacks, Concurrency / Parallelism reduces the input sequence to a single integer by applying sum to each element the absence of side-eff ect is a desirable Continuing with functional programming, of the sequence along with the result of aspect to program concurrency, and let us try to understand why side eff ect applying sum to the previous element; at we shall discuss this aspect in the next freedom and immutable data benefi ts the start, sum is applied to the fi rst two section. concurrent programming. Consider the elements of the sequence. following F# program to square a list of integers. nums |>Seq.map square |>Seq. As with the functional reduce sum |> List.ofSeq Python program, square To parallelize this program, observe function is applied to each that summation is dependent on each element of nums using input element. However, since summation Seq.map function (and is a commutative operation, summations the resulting sequence can be executed concurrently (possibly as is converted to a list via depicted in Fig. 2). Hence, as both Seq. List.ofSeq). reduce and sum are side eff ect free, we nums |> Seq.map can trivially parallelize the above program square |> List. as follows by using PSeq.reduce, a ofSeq parallel version of Seq.reduce. The order of iteration nums |>PSeq.map square |>PSeq. Fig. 1: Presence and absence of side eff ect is not specifi ed by the reduce sum |> List.ofSeq

3If you are intrigued by functional programming, then you should read about concatenative programming[42].

CSI Communications | February 2013 | 7 This example is a case of map-reduce the language-runtime combination to [evaluation[27]. Instead of writing callback parallelism and functional programming is concurrently apply the transformation functions and tying them to UI entities, well suited for this form of parallelism. to all units of data. Given the volume Flapjax allows the code to be executed in Another paradigm well suited for of data, concurrency needs to move reaction to an event to be embedded as concurrent programming is data fl ow beyond a single computing node and part of UI entities. This localizes the context (based) programming[11]. In this paradigm, be distributed and orchestrated across of the action (to either the source or target the program is modeled as a directed a cluster of nodes. To address this need, UI entity). For example, the following graph of data fl ow between various the language-runtime combination should Flapjax code snippet displays the current operations (and data stores) – each node ideally manage the distribution of work system time. In this snippet, timerB in the fl ow graph represents an operation load based on distribution schemes creates a “behavior” that generates a and each edge represents the data fl ow automatically derived from the program value every second (1000 milliseconds) path between operations. Consequently, (e.g. to optimize the available computing and currTime is updated with this value. operations corresponding to any pair of nodes or colocation of data). Every update to currTime then triggers nodes without a (data fl ow) path between This niche yet highly active scenario an update to the text fi eld (enclosed in

them can be executed in parallel (provided is handled by Hadoop, Dryad, and similar tag). Hence, the program sets up a pipeline that operations corresponding to all systems (along with extensions such of events strewn with reaction code that of their immediate predecessor shave as DryadLINQ) – program and execute can aff ect the UI and/or trigger new events. completed execution). data parallel programs on clusters of hence, need not explicitly programs is helpful to deal with large

identify or specify concurrent operations in volumes of data, it is not immediately The time is {! currTime !}. data fl ow programsq. Lustre, VHDL, and applicable in scenarios involving real-

Simulink are few programming languages time analysis of streaming data. Solutions Flapjax is implemented as a JavaScript that support data fl ow programming[12,13,14]. to perform near real-time analysis of framework; hence, it plays well with Data fl ow programming can also be voluminous data[24] do exist along with existing web technologies. In contrast, explored in Groovy using datafl ow solutions to analyze streaming data[25,26]. Elm, a functional reactive programing concurrency constructs provided by However, these solutions and their language, departs from HTML, CSS, and GPars [15,16]. combination need further exploration JavaScript trio and provides a unifi ed Similar to data fl ow programming and evaluation in the context of real language to declare UI entities and paradigm, languages such as Groovy (via world problems; specifi cally, in problem program the associated behaviors[28]. To GPars library), Erlang, Rust, and Scala (via instances involving real-time analysis of appreciate the diff erence, consider the Akka library)[15,16,17,18,8,19] support concurrent streaming data. Hence, the design and following Elm code snippet that displays programming via the Actor programming development of an apt programming the current system time. In this code model[20]. In this model, entities called model that enables solutions to cater snippet, asText function is applied (due actors can send and receive messages, to these scenarios is an interesting and to lift) to the value of the expression react locally to messages, and create relevant problem. (every second) and the return value is new actors. Further, multiple actors can Web displayed as the content of the page (due execute concurrently and the only means As web-enabled devices are becoming to assignment to main). of communication between actors is via ubiquitous, the need to program web asynchronous messaging passing. With main = lift asText (every second) applications for these devices is on side eff ect freedom and data immutability For the curious mind, similar to Elm, the rise. The most common approach requirements are trivially satisfi ed, this Dart is another language that tries to to program the front-end of these programming model is well-suited for depart from JavaScript[29]. applications is using a combination concurrent programming. Further, with of HTML, CSS, and JavaScript. While Numerical and Statistical Computing support for distribution and location tooling does immensely help alleviate The rise of big data and data analysis transparency (along with message passing issues stemming from reasoning about, has resulted in an increased interest in communication), actor programming assembling, and maintaining disparate languages for data analysis. In this context, model is also well-suited for distributed artifacts (i.e. declarative HTTP and general purpose programming languages programming (as demonstrated by Erlang CSS program fragments combined with such as Python has received attention due and Akka library). non-declarative JavaScript program to the rich libraries for number crunching, Distribution fragments), a programming model well- e.g. NumPy and Pandas[30,31]. Instead With large volumes of similar but suited to develop event-driven and of extending the language, the libraries distinct data that need to be transformed reactive applications would be more expose that embody operations on identically (e.g. for purpose of data helpful. domain-specifi c data types (e.g. vector, analytics), there is a growing need to In this context, consider Flapjax is a matrix). Hence, users can reap the program data transformation at the level programming language with support for benefi ts of such libraries without leaving of one unit of data and then to rely on event-driven programming and reactive the comfort of their favorite programming

4In the presence of side eff ects, programmers will have to ensure operations that can execute in parallel do not modify the same data. 5Interestingly, map-reduce architecture of most programs programmed using this programming model is reminiscent of map and reduce functions from functional programming. CSI Communications | February 2013 | 8 www.csi-india.org language/paradigm. On the downside, [2] "Python Programming Language," [Online]. Available: http://blog.cloudera.com/ users will have to wrestle with any Available: http://www.python.org. blog/2012/10/cloudera-impala-real-time- conceptual mismatch between general- [3] "CSharp Programming Language," [Online]. queries-in-apache-hadoop-for-real/. Available: http://msdn.microsoft.com/en- [25] "Streamdrill," [Online]. Available: https:// purpose languages and the domain. us/library/kx37x362.aspx. streamdrill.com/. At the other end, languages such [4] "JavaScript Programming Language," [26] "Microsoft StreamInsight," [Online]. as BUGS, Julia, and R are dedicated to [Online]. Available: http://en.wikipedia.org/ Available: http://blogs.msdn.com/b/ statistical computing, numerical computing, wiki/JavaScript. streaminsight/. and visualization[32,33,34]. These languages [5] "Ruby Programming Language," [Online]. [27] "Flapjax Programming Language," [Online]. Available: http://www.ruby-lang.org/. Available: http://www.fl apjax-lang.org/ provide a programming model along with [6] "Clojure Programming Language," [Online]. index.html. language features tailored for data types, Available: http://clojure.org. [28] "Reactive Programming on Wikipedia," values, and operations prevalent in the [7] "FSharp Programming Language," [Online]. [Online]. Available: http://en.wikipedia.org/ domain of data analysis (e.g. matrices, Available: http://msdn.microsoft.com/en- wiki/Reactive_programming. distributions). For example, in R, a sample us/library/dd233154.aspx. [29] "Dart Programming Language," [Online]. [8] "Scala Programming Language," [Online]. Available: http://www.dartlang.org/. of size 100 can be drawn from a normal Available: http://www.scala-lang.org/. [30] "Pythong Data Analysis Library (Pandas)," distribution N(10, 2) with mean of 10 and [9] "Haskell Programming Language," [Online]. [Online]. Available: http://pandas.pydata.org/. standard deviation of 2 using the simple Available: http://www.haskell.org/ [31] "NumPy, a fundamental package for expression rnorm(n=200,m=10,sd=2). haskellwiki/Haskell. scientifi c computing with Python," [Online]. Further, in BUGS, a can specify [10] "Standard ML Programming Language," Available: http://www.numpy.org/. [Online]. Available: http://www.smlnj.org/ [32] "BUGS, a software package for performing that a node xin a Bayesian network is sml97.html. Bayesian inference Using Gibbs Sampling," normally distributed by x ~ dnorm(mu, [11] J P Morrison, Flow-Based Programming, 2nd [Online]. Available: http://www.openbugs. tau)with mu and tau as mean and Edition: A New Approach to Application info/w/. precision, respectivelys. Development, CreateSpace Independent [33] "The Julia Language," [Online]. Available: In this sort of programming model, a Publishing Platform;, 2011. http://julialang.org/. program is a specifi cation of a model and [12] "Lustre Programming Language," [Online]. [34] "The R Project for Statistical Computing," Available: http://en.wikipedia.org/wiki/ [Online]. Available: http://www.r-project.org/. an execution of the program is a search Lustre_(programming_language). [35] "Probabilisitic-Programming.org," [Online]. for an instance of the model. This form of [13] "VHDL Programming Language," [Online]. Available: http://probabilistic-programming. programming using probabilistic modeling Available: http://www.eda.org/twiki/bin/ org/wiki/Home. to perform probabilistic reasoning is view.cgi/P1076/WebHome. [36] "Comparison of Programming Paradigms," referred to probabilistic programming[35]. [14] "Simulink Programming Language," [Online]. [Online]. Available: http://en.wikipedia. Available: http://www.mathworks.in/ org/wiki/Comparison_of_programming_ Where to Next? products/simulink/?s_cid=wiki_simulink_2. paradigms. This article provides a very basic [15] "Groovy Programming Language," [Online]. [37] "Strange Loop," [Online]. Available: https:// Available: http://groovy.codehaus.org. thestrangeloop.com/. introduction to prevalent programming [16] "Groovy Parallel Systems," [Online]. Available: [38] H Sutter, "Welcome to the Jungle," [Online]. paradigms while exploring other http://gpars.codehaus.org/Datafl ow. Available: http://herbsutter.com/welcome- programming paradigms and models in [17] "Erlang Programming Language," [Online]. to-the-jungle/. a problem and domain specifi c manner Available: http://www.erlang.org. [39] J M White, "Using JAGS in R with (possibly biased and based on the author’s [18] "Rust Programming Language," [Online]. rjags Package," [Online]. Available: personal experience and preferences). Available: http://www.rust-lang.org/. http://www.johnmyleswhite.com/ [19] "Akka, a toolkit and runtime for building notebook/2010/08/20/using-jags-in-r- So, if this article has piqued your interest highly concurrent, distributed, and fault with-the-rjags-package/. about programming paradigms, then tolerant event-driven applications on the [40] J C Reynolds, Theories of Programming please do one or more of the following: JVM," [Online]. Available: http://akka.io/. Languages, Cambridge University Press, • Read about various programming [20] "Actor Programming Model," [Online]. 1998. paradigms at Wikipedia[36], Available: http://en.wikipedia.org/wiki/ [41] B A Tate, Seven Languages in Seven Weeks: Actor_model. A Pragmatic Guide to Learning Programming • Explore few of the languages [21] "Hadoop," [Online]. Available: http:// Languages, 2010. mentioned in this article, and hadoop.apache.org/. [42] "Concatenative Language," [Online]. • Watch videos of various lectures at [22] "Dryad," [Online]. Available: http://research. Available: http://concatenative.org/wiki/ Strange Loop’13[37]t. microsoft.com/en-us/projects/Dryad/. view/Concatenative%20language. [23] "DryadLINQ," [Online]. Available: http:// [43] "Programming Paradigm, http://en.wikipedia. References / Sources: research.microsoft.com/en-us/projects/ org/wiki/Programming_paradigm". [1] H Sutter, "The Free Lunch Is Over," [Online]. dryadlinq/default.aspx. [44] "Comparison of Programming Paradigms, Available: http://www.gotw.ca/publications/ [24] "Cloudera Impala: Real-Time Queries http://en.wikipedia.org/wiki/Comparison_ concurrency-ddj.htm. in Apache Hadoop, For Real," [Online]. of_programming_paradigms". n

6John Myles White provides a detailed example of such programming in R[39]. 7Bruce A Tate provides a good introduction to a small yet distinct set of programming languages[41].

Venkatesh Prasad Ranganath is a researcher at Microsoft Research, India. Currently, he explores empirical approaches to software engineering tasks with focus on development and maintenance tasks. In the past, he has worked on program slicing, program verifi cation, and model driven development. He has Ph.D. and M.S. degrees in Computer Science from Kansas State University and B.E. degree in Computer Science from Bangalore University. You can learn more about his work at http://research.microsoft.com/~rvprasad and follow him on Twitter via @rvprasadTweet. About the Author

CSI Communications | February 2013 | 9