Webworlds - Web-Based Learning Environments for Software Engineering

Webworlds - Web-based modelling environments for learning software engineering

Peter D Chalk University of North London (from 1/8/02 London Metropolitan University) [email protected] ([email protected]) in Journal of Computer Science Education 10.1, April 2000

Abstract This article presents a survey of Web-based learning environments ('Webworlds') such as diagramming tools and program animators that support the teaching of some standard topics in a software engineering course. In addition it proposes an approach to classification based on how the student explores or expresses the model represented in the software. It is suggested that such a classification, based on the student learning view, can assist in evaluating claims for enhanced learning effects. For each Webworld discussed, various attributes are analysed to assist in its classification. Finally, some technical concerns and issues for further evaluation are raised.

Keywords: Software engineering education, Webworlds, Microworlds, Modelling, Web-based learning.

Introduction There are an increasing number of Web-based learning environments ('Webworlds') created for software engineering education. The term 'Webworlds' is suggested in Chalk (1999), based on Seymour Papert's concept of a microworld which "can be manipulated by the learner, providing an environment for active learning" (Papert, 1980, p.126). Papert's definition is applied here to a range of example software artefacts, such as program animators and modelling tools, which simulate aspects of the software engineering curriculum. Laurillard (1993, p.138) comments that "microworlds are very similar to simulations in the sense that they allow the user to act in a simulated world". This paper presents a partial survey and classification of the use of software in teaching situations that satisfy this definition of Webworlds and that have been variously described by their authors as interactive animations, visualisations, simulations and laboratories. Similar surveys have been published for other areas of the computer science curriculum. Boroni et al. (1998, p. 146) describe the 'Web paradigm' where "instructors will tend away from a traditional lecture style towards the role of a facilitator, and students will become more active in exploratory learning". They provide an overview of the work of many university departments in the field of algorithm animation and of the new ways in which students benefit from Web-based learning. This includes the benefits of having many different Webworlds to choose from, giving the student a chance to evaluate their

P Chalk Webworld Survey 09/01/18 Page 1 of 17 findings comparatively. Additionally, the Web provides much greater accessibility to CAL (Computer-Aided Learning). These factors alone must affect the way students learn and more research is needed into how to use them to best advantage.

Identification and classification of Webworlds The programs cited in this paper have been identified by a variety of methods including: reading ACM SIGCSE and ITiCSE proceedings, Web searches, monitoring repositories and by creating them ourselves. Several repositories of software to support computer science education exist (ACM SIGCSE, 1997, Brummond, 1998 and CSTC, 1997). The programs, or Webworlds, mentioned in this article are chosen because they support a software engineering unit, based on Pressman (1997), and taught at South Bank University by the author. In a previous issue of Computer Science Education, Laurie Werth (1998) described a CS1/2 curriculum that integrated software engineering into an introduction to computer science. These examples could support such a course with topics covered including performance analysis, flowgraphs, testing, design and control and data structures. Webworlds are continually emerging for other topics in this curriculum area. Previous authors have proposed various methods of classification. Bergin (1997), of the Web-based Visualisation Group in SIGCSE, classifies their programs as non- interactive, interactive, complete visualisation systems with automatic creation and complete visualisation systems requiring program annotation. Boroni (1998, p.148) classifies animations as depicting the workings of either a program, an algorithm or a concept. Stasko et al. (1998) include contributions on similar themes, for example considering whether the display is static or dynamic. Others, for example Brummond (1998) and the CSTC (1997) describe the control of inputs, starting, stopping etc and the degree of interactivity. These are important factors that also contribute to the analysis presented here. The approach in this paper will be to propose a method based on control over the model that is the object of interaction. A variety of Webworlds will be presented to illustrate each category as it applies to an aspect of software engineering. This approach to categorising the use of models in education distinguishes between exploratory and expressive modes (Mellar et al., 1994). In the first (exploratory) case, the student is presented with a model to explore and attempts to understand it (usually through a form of interaction), or conducts experiments using it. In the second (expressive) case, the student is able to express their own model, and get feedback on its 'correctness', developing a deeper understanding of how the model works. For each category a range of different types of examples are described. This method of categorising, from the standpoint of the student's learning, adds another dimension to previous work on the subject and is currently the subject of ongoing evaluative research.

Exploratory models - visualisation and experiments Visualisation is argued to be a powerful method for understanding how a process works or for studying its effects (although there are important concerns relating to cognitive issues raised by Petre et al., 1998). For example, sort algorithm animators can illustrate the performance of different sorting algorithms and give students the chance to observe and measure their comparative speeds. One such example, Sort Algorithm Animations, is shown in Figure 1 (Begole, 1996). Naps & Bressler (1998) have also produced a

P Chalk Webworld Survey 09/01/18 Page 2 of 17 system that displays algorithm animations in parallel. Dynamic visualisation can also be considered as viewing the execution of a model, in this case in the exploratory mode as the model is given.

Fig. 1. Sort Algorithm Animations - algorithm visualisation and performance comparison.

The only control given to the user by this animation is over starting and stopping. Yet, because the three sorts start at the same time on similar random sets of data, it offers a powerful visual example of comparative speeds. A student learning the topic of performance analysis could use a stopwatch to measure timings and correlate them with the order of each algorithm. Source code for each sort is also available. Other similar examples include Gosling & Harrison (1999) and Kitchen (1999). Both also include source code and an indication of the order of each algorithm.

P Chalk Webworld Survey 09/01/18 Page 3 of 17 Fig. 2. Illustrating White Box Testing with JAWAA - concept visualisation and graphical modelling environment.

An example of concept visualisation is shown in Figure 2 (Ahmed, 1999). It illustrates the animation of a program flowgraph using the JAWAA (Java And Web-based Algorithm Animation) environment developed by Pierson & Rodger (1998). The start/ stop control has been augmented by pause/ unpause - a set of sequencing controls that are offered by most simulations. The student can also view the source code modelled by the flowgraph. (The JAWAA code that drives the animation is also available to be viewed or edited and this expressive aspect will be described later). This example is intended to allow students to visualise the different paths through the code. Each path is dynamically animated using a moving ball of different colours to represent each possible path of execution. The paths can then be used by a student to construct a white box test plan or to calculate the complexity metric for the code. It also serves as an alternative to, or introduction to, the Java Flowgraph Editor below. It could be argued that visualisation per se does not necessarily support model exploration and an additional category explanatory may be needed in this case.

P Chalk Webworld Survey 09/01/18 Page 4 of 17 Fig. 3. Sort Animator - interactive algorithm animator.

The next example, the Sort Animator in Figure 3, also fits into the category of visualisation of the execution of a model, again that of a sort algorithm (Dershem & Brummond, 1998). The student can explore the model by varying the controls (seven menus, three buttons and a number entry box) and watching the highlighting of data items and source code lines as it executes. This is typical of a large number of such algorithm animators; indeed the 'gateway' site maintained by Brummond (1998) lists 43 different examples. They provide a rich set of laboratory experiments for performance analysis as well as for data and program structure exercises. In addition to the controls already shown (sequencing) the user can also: vary the speed, choose the algorithm (or model) type, view the algorithm code and an explanation, change various colours (background, foreground, highlighting) and vary the size and order (random or pre-sorted) of the data set. Thus it readily fits the category described by Bergin (1998) as interactive visualisation. In addition, as will be described below, it allows the user to input their own code (or express their own model). The outputs consist of highlighted (animated) code and data and a count of the total number of comparisons and swaps. For recursive algorithms it also includes a graphic representation of the depth and location of the recursion. Students can use this Webworld to see how different sort algorithms work, how they perform on their chosen data sets and, hence, carry out performance analysis (asynchronously). Of all the algorithm animators viewed by the author, this one seems to have the largest number of opportunities for interaction (as well as supporting both exploratory and expressive modes of modelling). Other similar examples of sort algorithm animators include Dildy & Poonen (1996), Hausner (1996) and Mitra (1996).

P Chalk Webworld Survey 09/01/18 Page 5 of 17 Fig. 4. Learning Black Box Testing using Simulation - fault finding.

The next example presented in this section on exploratory models is in Figure 4, an exercise in fault finding using black box testing (Aziz, 1999). Four menus are presented and the student can choose one of five functions (or models) to study. Each function has a specification that can be viewed and has at least one deliberate error - the task is to find them. Input values can be entered by the student and the output is calculated and added to a test log. The expected outcome can be chosen by the student or generated automatically and, when compared to the specification, it allows the faults (or failures) to be found. The help menu includes an option that lists the faults, so that the program is fully self-contained. Another type of algorithm or data structure animator also encourages interactivity by allowing the student to enter or manipulate data values according to their view of how the algorithm works. Jarc (1999) describes a set of Interactive Data Structure Visualisations and a research study into its effects on learning. An example, showing the Heap Sort, is shown in Figure 5. The radio button labelled 'I'll try' enables the student to enter the interactive 'emulate' mode.

P Chalk Webworld Survey 09/01/18 Page 6 of 17 Fig 5. Interactive Data Structure Visualisations - algorithm emulation.

In summary, Webworlds for exploratory (and, as a superset, for expressive) modelling activity can include the following attributes:  Sequence controls: start, stop, pause, step, reverse direction, change speed,  Colour controls: back/ foreground, animation highlighting, data,  Choice of model: algorithm, data structure, function,  Object modelled: graphical, textual, conceptual,  Data input: values, size, organisation, pattern, emulation,  Model: source code, design or specification of the model,  Counters: comparisons, moves, swaps, timing,  Results: log, illustration of important side effects, annotation,  Display: animated, highlighted,  Output: measurable, calculated,  Help: solutions, exposition, FAQ, analysis,  Storage: saveable model state, printable, shareable, multi-user. These attributes will need to be considered by teaching staff when choosing or evaluating any particular Webworld for inclusion in their course. The list does not include other usability heuristics such as those concerned with navigation, memory, etc. The degree of interactivity in each Webworld can be 'measured' using these attributes and research is underway to determine any relationship to learning. The type of exploratory Webworld can be categorised as animation, simulation, emulator, fault finder or performance comparator. Other types not illustrated here may include games, role play scenarios or real time control environments. Evidence for improved learning from the use of exploratory Webworlds is limited, as noted by Jim Foley in his foreword to the book by Stasko et al. (1998) - he notes that "Empirical evaluation... is for me the most critical remaining step in fulfilling the promises of software visualisation" (p. xiii). There are some positive results, for

P Chalk Webworld Survey 09/01/18 Page 7 of 17 example, Naps & Bressler (1998, p. 278) claim that their 'learn-by-example environment' produced "marked improvements in students' performance on those test questions that asked them to trace the execution of an algorithm". They concede that "it is not yet clear how algorithm animation can help in better learning the more conceptual type of material". Perhaps one solution is to engage the student in a wider variety of tasks, included their own model building exercises, that integrate every stage of the software engineering process. This approach fits the category expressive and is described in the next section.

Expressive modelling and feedback In each of the preceding examples the model is presented to the user. In the case of JAWAA and the Sort Animator, the student can also express their own model, either as an animation or as program code, and have it executed in the Webworld environment. This approach to learning more closely resembles that advocated by Papert (1980) in his classic treatment of the microworld concept using the Logo programming language. The illustration of a flowgraph in Figure 2 was produced by a final year student at South Bank University for his project (Ahmed, 1999). The animation was coded in the JAWAA graphics language and run using the JAWAA engine. Feedback from each drawing was used to determine the correctness of the code model. The Sort Animator is accompanied by the Sort Animation Builder, the source code that enables a student to recompile the Webworld with their own code for the algorithm under study and watch the result. In both cases, the student can express their model, albeit in the specified form of a specialised language code. In his report Ahmed (1999) states that "using the JAWAA script for programming purposes has proved to be most educational in the development of my understanding of different methods of coding". JAWAA authors Pierson & Rodger (1998) describe similar examples of students benefiting from creating their own animations for a variety of computing concepts. They claim that JAWAA animations are used "for demonstration use in the classroom or by students generating quick animations of their programs" (p. 267). The experience of Ahmed (1999) raises some doubts about the use of the word 'quick' but the animations produced have been described by students as visually appealing.

P Chalk Webworld Survey 09/01/18 Page 8 of 17 Fig. 6. Java Flowgraph Editor - (expressive) model diagramming tool.

Webworlds can provide an opportunity for simplified CASE tools to be made widely available to students. Figure 6 shows a tool (Cumberbatch, 1999) which can be used in a follow-up or alternative activity to the flowgraph animator in Figure 2. In this case, the student has a chance to draw the flowgraph (express their model) for any piece of code, or for one of the examples provided by the program. In the latter case, it acts in a similar way to the testing fault finder in Figure 4, showing, in the right hand window, the correct flowgraph for the code presented on the left. The student's model can thus be compared to a correct version and the resulting feedback used to assist them to correct any mistakes. The expression of the student's model in this case is a passive representation, but the example that follows is one that executes the diagram model.

P Chalk Webworld Survey 09/01/18 Page 9 of 17 Fig. 7. JSP Editor - CASE tool executes design model to generate code.

A similar simplified type of CASE tool is the JSP Editor, shown in Figure 7 (Engström, 1997). The student can enter a diagram, using controls that enforce the correct JSP syntax, and generate skeletal code in C or Pascal. This feedback can then be used, if necessary, to check the correctness of the diagram or to help with writing the resulting program. By executing the user's model in a graphical environment, this Webworld again exhibits some of the characteristics of Papert's microworlds. Other expressive modelling environments for computer science education exist. Boroni et al. (1998) describe WebLab, the successor to DynaLab, and the 'paradigm shift' that Web-based learning entails. Their Webworlds include a program animator (see Figure 8) which animates program code and data structures in a similar way to the algorithm animators above and allows input of the user's own code using an editor and compiler. In the latter case, acting as an algorithm checker, the program fits the definition of an expressive modelling environment that executes the model. This could also support software engineering topics like profiling and debugging. An on-line Java Development Environment (JaDE) has been produced by Stegink et al. (1999) to support their course in computer graphics. This facility could prove very useful in a software engineering course or in supporting Webworlds that require remote on-demand code compilation. A similar approach has been adopted by Rowe & Thorburn (1999) for VINCE - a C program animator that allows editing and compiling of code and checking of values in memory in step mode.

P Chalk Webworld Survey 09/01/18 Page 10 of 17 Fig. 8. WebLab - program control and data structure animation and algorithm checker.

In summary, the additional attributes required for expressive modelling environments can include:  Model input facility: graphical editor, modelling language editor, file input,  Execution: model transformation, runnable, choice of method of highlighting,  Storage: local, server, shareable, printable,  Output: display model, execute it, compare to solution, choose transformation,  Assistance with creation: syntax checker, on-line help, example solutions. The examples of expressive modelling environments described so far in this section are all specific to the kind of model being created, whether an algorithm animation or a design diagram, and could be called 'content-specific' Webworlds. Another kind of modelling environment would be 'content-free', for example a spreadsheet or graph plotter. Both of these already exist as Webworlds (Kurtz & O'Neal, 1998, see Figure 9, and Zobel, 1999, respectively) that could be used for software engineering topics such as COCOMO cost estimation, performance analysis and the Personal Software Process (Humphrey, 1997). Semi-quantitative or qualitative modelling tools (not yet found on the Web) could be used to model the software development process itself as a dynamic system.

P Chalk Webworld Survey 09/01/18 Page 11 of 17 Fig. 9. Watson Project Spreadsheet Lab - a 'content-free' modelling environment.

The approach in this section is similar to that of Werth (1998, p. 8) who argues that "lessons can be effectively driven home by applying the software tool(s) to help evaluate the student's own programs". As Mellar et al. (1994) argue, the expressive use of models can be a powerful way of learning and Webworlds that fit into this category need to be identified or developed. Additional research is on-going into how their availability on the Web will affect patterns of learning. One major issue that needs to be addressed concerns the technical problems of using interactive software on the Web, in particular the problem of saving and loading student models that is essential to the expressive mode of modelling. This will be addressed in the next section. To complete this section, a summary classification of the use of Webworlds presented in this paper is shown in Table 1. The intention is to demonstrate the increasing complexity of and interactivity with the model. Note that the Webworlds in Figures 2 and 3 are listed twice as the category exploratory or expressive depends on how they are used. A brief description of the type of Webworld is also given to assist with the taxonomy, which is itself undergoing further research and development. At least the description should assist teachers with selecting and integrating the Webworlds into their teaching.

Table 1. Summary classification of the use of software engineering Webworlds Title, URL F Explorat Description of i ory/ Webworld activity in g Expressiv this mode . e Sort Algorithm Animations, 1 Explorato Algorithm http://simon.cs.vt.edu/~begole ry visualisation and j/Java/Sort performance comparator.

P Chalk Webworld Survey 09/01/18 Page 12 of 17 Illustrating White Box Testing 2 Explorato Concept visualisation. Using JAWAA, ry http://www.sbu.ac.uk/~chalkp /proj/fayyez Sort Animator, 3 Explorato Interactive algorithm http://www.cs.hope.edu/~alga ry animator. nim/ccaa/ccaa.html Learning Black Box Testing 4 Explorato Fault finder. using Simulation, ry http://www.sbu.ac.uk/~chalkp /proj/bbt/ Interactive Data Structure 5 Explorato Algorithm Visualisations, ry visualisation and http://tangle.seas.gwu.edu/~id emulation. sv JAWAA, 2 Expressiv Graphical modelling http://www.cs.duke.edu/~wcp e environment. /JAWAA.html Sort Animation Builder, 3 Expressiv Animation builder http://www.cs.hope.edu/~alga e using student code nim/ccaa/ccaa.html model. Java Flowgraph Editor, 6 Expressiv Model diagramming http://www.sbu.ac.uk/~chalkp e tool. /proj/sharron JSP Editor, 7 Expressiv CASE tool executes http://www.ida.his.se/ida/~he e design model to nrike/JSP/example.chtml generate code. WebLab, 8 Expressiv Program animator and http://www.cs.montana.edu/~ e algorithm checker. weblab Watson Project Spreadsheet 9 Expressiv 'Content-free' Lab, e modelling http://www.latech.edu/~watso environment. n/overview.html

Technical issues concerning Webworlds Technical concerns fall into the following areas: general concerns about the stability of Web sites, methods for processing files, performance issues and Webworld production issues. Any evaluation of the use of Webworlds for a particular course has to take some or all of these into account. First, the stability of any Web site cannot be guaranteed, either by the author of the Webworld or by the site provider. URLs change or become permanently or temporarily unavailable and the software may not be maintained. There are two possible solutions to this: one, use a secure repository (for example, one of the several set up by ACM SIGCSE, 1997) or, better, ask the author for a copy of the code. This also alleviates the problem of latency, but not that of multiple copies. In

P Chalk Webworld Survey 09/01/18 Page 13 of 17 future, payment and access rights may become more of an issue. (All of the Webworlds cited in this paper are currently accessible free of charge). Secondly, and most importantly for education, is the issue of how Webworlds process students' models. Because of the current security restrictions on using Java applets in Web browsers it is not usually possible to load from, save to or print using the client machine. Webworld authors have used some ingenious methods to overcome the problem of file input. For example, a student wanting to view their animation in JAWAA sets up two Web pages. The first HTML page calls up the JAWAA engine code base stored at Duke University passing as a parameter the URL of their local animation code text file. The applet then starts up, interpreting the JAWAA statements in the second file and producing the animation in a window (as in Figure 2). In another example, the Sort Animation Builder is available as Java source code which has to be downloaded, recompiled with the student's own algorithm and run as a local application. Other Webworlds use a combination of CGI scripting and FTP. For printing the usual method to use with Java applets is to copy and paste screen displays into local files and print them. These 'solutions' are thoroughly discouraging to serious educational use. The next generations of browsers and of Java may allow an easing of this problem but security will then become a much bigger issue. Already university laboratories are plagued by student-spread viruses and the prospect of more coming from Web applets is not appealing. Again, the solution may lie in using reliable repositories or holding a local copy. There is little evidence of the widespread take-up of computer aided learning (CAL) programs predicted in the past. One reason could be the problems of developing for obsolete machines and of distribution: often the student only has access to CAL in a particular lab with the teacher present. The ubiquitous nature of a Web artefact could change that. Only one copy (theoretically) needs to be produced and stored anywhere in the world for everyone to have access, anywhere at any time on any platform. Maintenance becomes simply a matter of amending that copy. However, anyone with a Web account can load and publish any program they want to write. In terms of maintaining standards and guaranteeing quality this could present problems! Hence, another role for a repository is to provide peer review and other measures of quality assurance (ACM SIGCSE, 1997).

Conclusions The aim of this article is to draw attention to the emerging variety of Web-based learning environments (Webworlds) that are suitable for use in the software engineering curriculum. The topic of algorithm animation is now supported by a rich set of programs demonstrating incredible ingenuity and skill by many university teaching staff and researchers. Some claims for improved learning are made and other factors, such as the degree of interactivity and the usefulness of the exploratory/ expressive modelling categories need further research to establish the limits of their validity. The availability of similar programs that support other areas of the curriculum now means that resource-based learning anywhere, any time, can be a reality for our students. The challenge for educators now is to find ways of integrating Webworlds into taught courses and to evaluate the learning that they produce. There are many

P Chalk Webworld Survey 09/01/18 Page 14 of 17 theorists of education who advocate learning by doing and experiential or problem- solving activities (Kolb, 1984, Laurillard, 1993, Papert, 1980 to name just three). Experiments, investigations and tasks, based on a variety of Webworlds, may provide a rich set of experiences for reflective learning. A pilot study at South Bank University is currently evolving a unit in software engineering based on these learning principles and a variety of Webworlds including those described in this article. An initial attempt at classifying the Webworlds according to the stage of the modelling process they support has also been made. This should assist teachers in identifying the amount of preparation and depth of study that is required before they can be usefully deployed. Some technical issues have also been raised that have a bearing on the potential problems involved in Web-based learning and that provide a challenge to the future developers of new Webworlds. The next task is to evaluate systematically the learning outcomes that may be achieved by the use of Webworlds in software engineering education. This should include the contribution made by the different categories of modelling environment or task and the unique contribution made by delivery over the Web.

References

ACM SIGCSE (1997). Computer Laboratory Repository. http://springfield.tcnj.edu:8000/~compsci/peer/index_hi.html, College of New Jersey, last modified 25/2/97. Ahmed, S. F. (1999). Online Tutorial on a Fundamental Computing Concept. Final year project report, London: South Bank University, http://www.sbu.ac.uk/~chalkp/proj/fayyez. Aziz, T. (1999). Learning Black Box Testing using Simulations. Final year project report, London: South Bank University, http://www.sbu.ac.uk/~chalkp/proj/bbt/. Begole, J. (1996). Sort Algorithm Animations. http://simon.cs.vt.edu/~begolej/Java/Sort, Virginia Tech., last modified 16/5/96. Bergin, J. (1997). Web Based Visualisation. Web Document http://sol.pace.edu/webvis/. Boroni, C. M., Goosey, F. W., Grinder, M. T. & Ross, R. J. (1998). A paradigm shift! The Internet, the Web, browsers, Java, and the future of computer science education. Proceedings of the 1998 SIGCSE Symposium, Atlanta, 145-152. Brummond, P. (1998). The Complete Collection of Algorithm Animations. http://www.cs.hope.edu/~alganim/ccaa/site.html. Chalk, P. (1999). Survey of Webworlds for Software Engineering Education. Proceedings of the 7th Annual Conference on the Teaching of Computing, Belfast Aug 25-27, 37-41. CSTC (1997). Computer Science Teaching Centre. http://www.cstc.org/~cstc/, last modified 28/5/99. Cumberbatch, S. (1999). Java Flowgraph Editor. Final year project report, London: South Bank Univerity, http://www.sbu.ac.uk/~chalkp/proj/sharron. Dershem, H. L. & Brummond, P. (1998). Tools for web-based sorting animation. Proceedings of the 1998 SIGCSE Symposium, Atlanta, 222-226. Dildy, J. & Poonen, S. (1996). Sort Algorithm Animator. http://blackcat.brynmawr.edu/~spoonen/JavaProject/sorter.html.

P Chalk Webworld Survey 09/01/18 Page 15 of 17 Engström, H. (1997). JSP Editor. http://www.ida.his.se/ida/~henrike/JSP/example.chtml, last accessed 12/2/99. Gosling, J. & Harrison, J. (1999). Sorting Algorithms. http//:www.cs.ubc.ca/spider/harrison/Java/sorting-demo.html, last modified 16/3/99. Hausner, H. (1996). Bubble Sort. http://www.cs.princeton.edu/~ah/alg_anim/version1/BubbleSort.html, Princeton University, last modified 23/7/96. Humphrey, W. S. (1997). Introduction to the Personal Software Process. Addison- Wesley. Jarc, D. J. (1999). Interactive Data Structure Visualisations. http://tangle.seas.gwu.edu/~idsv, George Washington University, last accessed 17/6/99. Kitchen, A. (1999). Sorting Algorithms. http://www.cs.rit.edu/~atk/Java/Sorting/sorting.html, accessed 22/5/99. Kolb, D. A. (1984). Experiential Learning. Englewood Cliffs, N. J.: Prentice-Hall. Kurtz, B. & O'Neal, M. (1998). Developing Educational Materials in Java: A Report From the Front Lines. National Educational Computing Conference Proceedings, San Diego, CA, 234-246. Laurillard, D. (1993). Rethinking University Teaching: A Framework for the Effective Use of Educational Technology. Routledge. Mellar, H., Bliss, J., Boohan, R., Ogborn, J., & Tompsett, C. (Eds.). (1994). Learning with Artificial Worlds: Computer Based Modelling in the Curriculum. The Falmer Press. Mitra, S. (1996). Java Sorting Animation. http://www.cs.brockport.edu/cs/java/apps/sorters/, last modified 19/9/96. Naps, T. L. & Bressler, E. (1998). A multi-windowed environment for simultaneous visualisation of related algorithms on the World Wide Web. Proceedings of the 1998 SIGCSE Symposium, Atlanta, 277-281. Papert, S. (1980). Mindstorms: Children, Computers and Powerful Ideas. Brighton, Sussex: Harvester Press. Petre, M., Blackwell, A. & Green, T. (1998). Cognitive Questions in Software Visualization. in Stasko et al., 453-480. Pierson, W. C. & Rodger, S. H. (1998). Web-based Animation of Data Structures Using JAWAA. Proceedings of the 1998 SIGCSE Symposium, Atlanta, 267-271. Pressman, R. (1997). Software Engineering: A Practioner's Approach. McGraw-Hill. Rowe, G. & Thorburn, G. (1999). Evaluation of VINCE - a Tool for Teaching Introductory Programming. Proceedings of the 7th Annual Conference on the Teaching of Computing, Belfast Aug 25-27, 98-102. Stasko, J. Domingue, J. Brown, M. H. & Price, B. A. (Eds.). (1998). Software Visualization: Programming as a Multimedia Experience. MIT Press. Stegink, G., Pater, J. & Vroon, D. (1999). Computer Science and General Education: Java, Graphics, and the Web. Proceedings of the 1999 SIGCSE Symposium, New Orleans, 146-149. Werth, L. H. (1998). Integrating Software Engineering into Introductory Computer Science Courses. Computer Science Education, 8.1, 2-15. Zobel E. A. (1999). EZ Graph. http://id.mind.net/~zona/ezGraph/ezGraph.html, last accessed 10/2/99.

P Chalk Webworld Survey 09/01/18 Page 16 of 17 Words, including references, 0 plus 9 figures at 150 words each (say, 1350) equals an equivalent word total of approximately 6200.

P Chalk Webworld Survey 09/01/18 Page 17 of 17