Computer Science and Systems Analysis Computer Science and Systems Analysis Technical Reports Miami University Year Interface-based Programming Assignments and Automatic Grading of Java Programs Michael T. Helmick Miami University This paper is posted at Scholarly Commons at Miami University. http://sc.lib.muohio.edu/csa techreports/2 Interface-based Programming Assignments and Automatic Grading of Java Programs Michael T Helmick Miami University Department of Computer Science and Systems Analysis Oxford, Ohio USA [email protected] ABSTRACT Preliminary analysis of Java coding style can be handled AutoGrader is a framework developed at Miami University automatically using either the open source Checkstyle [1] or for the automatic grading of student programming assign- PMD [4] packages (AutoGrader is integrated with PMD). ments written in the Java programming language. Auto- While it might be possible to handle the evaluation of pro- Grader leverages the abstract concept of interfaces, brought gram design in an automated fashion, we feel that this task out by the Java interface language construct, in both the is best left to an experienced instructor or teaching assis- assignment and grading of programming assignments. The tant. There is a large body of program design knowledge use of interfaces reinforces the role of procedural abstraction that comes from experience, and this experience can not in object-oriented programming and allows for a common easily be programmed into an automated system. API to all student code. This common API then enables Functional testing, on the other hand, is a task that pro- automatic grading of program functionality. AutoGrader duces a boolean answer. A program either works and pro- provides a simple instructor API and enables the automatic vides the correct answer based on predetermined inputs, or testing of student code through the Java language features of it does not. Since software systems tend to be built in lay- interfaces and reflection1. AutoGrader also supports static ers, we can further sub-divide assignments and target spe- code analysis using PMD [4] to detect possible bugs, dead cific functionalities for testing. Our goal in automatically code, suboptimal, and overcomplicated code. While Auto- testing student code is two-fold: (1) to adequately exercise Grader is written in and only handles Java programs, this student code; and (2) to segment the tests such that pieces style of automated grading is adaptable to any language that of functionality are isolated and tested as independently as supports (or can mimic) named interfaces and/or abstract possible. This not only makes automatic assessment eas- functions and that also supports runtime reflection. ier, it is also a valuable teaching technique for instruction of software construction. We have introduced a pedagogy for our data structures 1. INTRODUCTION course that enables and utilizes automatic grading for pro- Automatic grading of student programs has been of in- gramming assignment functionality. Interface-based pro- terest to computer science educators for some time [8] and gramming can raise the level of abstraction within a program continues to gain attention today [7, 5, 9, 12, 14, 10]. There and is often underutilized [15]. The use of interfaces with have been various approaches with most leaning towards an- the Java programming languages enforces an extra level of alyzing the output of the entire program with some mention indirection and allows the instructors to set a certain level of of techniques and tools such as JUnit [16]. However, there commonality in a what is to be done manner, without regard seems to be no established consensus on the best way to to the individualized implementation details. While seem- automatically grade student code. ing restrictive on the surface, this still leaves implementa- We take the position that there are two tasks involved in tion considerations up to individual students and does not grading of student assignments: impose a common design, just a common interface. This 1. Functional testing also unifies grading of student work since each individual solution adheres to a prescribed interface, allowing the in- 2. Evaluation of design and style structor to save time by constructing the tests a single time 1The Java reflection API allows the runtime system to be and applying them to all students in the course. inspected and manipulated dynamically. http://java.sun. This paper describes how we use interfaces in our data com/docs/books/tutorial/reflect/index.html structures class, how this enables discrete functional testing of student code, and how this is integrated into the Auto- Grader framework. Section 2 describes how interfaces are used in the data structures curriculum. Section 3 gives Permission to make digital or hard copies of all or part of this work for an overview of the AutoGrader framework, how it is con- personal or classroom use is granted without fee provided that copies are structed, and how grading packages are written by an in- not made or distributed for profit or commercial advantage and that copies structor. Section 4 places this framework in the context of bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific related work in this field. permission and/or a fee. Copyright 2006 Miami University Technical Report: MU-SEAS-CSA- 2006-002 . 2. INTERFACE-BASED ASSIGNMENTS For programming assignments in the context of a data 13 getList( String. class ); structures course, there is a certain level of commonality between all student projects. When talking about abstract 15 assertNotNull( strList ); data structures there is a natural tendency towards using assertEquals( 0, strList.size() ); a common interface. For instance, data structures for lists 17 } can have multiple implementations: array-backed list, singly linked list, and doubly linked list. It doesn’t make sense that 19 } each of these implementations would present a different in- terface to the clients of list services. In fact, the usefulness of these data structures increases if they implement common Listing 2: ArrayListTest.java interface[11], emphasizing the flexibility of switching imple- 1 import java.util.List; mentations as long as the client only utilizes the interface. import java. util .Random; Presenting data structures in this fashion is a worthwhile 3 educational objective [15]. public class ArrayListTest We emphasize the use of the Java language construct of 5 extends GeneralListTest { interface in order to increase student awareness of compo- nent software and to show that we can test software with- 7 @Override out knowing the implementation details. Black box testing public <T> List <T> g e t L i s t ( is the process of testing software where we craft tests based 9 Class<T> c l a z z ) { solely on the interface of the software, ignoring what we return new studentid .ArrayList<T>(); know about the implementation details. For the list assign- 11 //return new java.util.ArrayList<T>(); ment in our data structures course, we assign students the } task of implementing the standard java.util.List<E> in- 13 } terface [2] by providing both array-backed and linked list implementations. Because of the complexity involved, the The code from Listing 1 is then completed by the student, subList method is excluded. fully testing the list interface. Listing 2 shows how easy Students are also required to write unit tests using JUnit it is to then either test the student developed code (new [3] for all aspects of their programs (White box testing). This studentid.ArrayList<T>()) or test the built in Java im- serves two purposes: (1) to get students to test their code; plementation (new java.util.ArrayList<T>()). This en- (2) again, to emphasize the use of interfaces in procedural ables students to run their tests against an implementation abstraction, getting them to focus on what is being done, of an object that implements the same interface and is pre- rather than how it is being done. If tests are properly written viously known to be in full working order. While the abil- against the List<E> interface and not directed towards a ity to do this is extremely helpful to someone who is new specific implementation, they are able to reuse their tests to unit testing, it is not always possible to test an exist- for both list implementations, reducing their overall time ing implementation. For assignments where there is not a spent on the assignment. built in data structure, an instructor provided implementa- Configuring a JUnit test class to be able to use multiple tion can be used as the reference implementation. On more implementations can be enabled through the use of an ab- advanced assignments, students are generally not provided stract class containing all of the tests, and two sub-classes with a working implementation to compare against. containing the code to instantiate the list classes. This sepa- Interfaced-based assignments have been used successfully ration further emphasizes good unit testing in that it makes in our data structures course for lists, binary trees, heaps, it more difficult to make one test dependent on another test sets, maps, and graphs. We emphasize the portability of in the test package. Students are provided with some starter client code using an interface and thus the ability for a stu- code to demonstrate constructing tests in this way. An ex- dent to write one test suite which can then test multiple cerpt of this starter code is shown in Listings 1 and 2. This implementations. The use of assignments of this type and example setup works well when students are to provide more subsequent automatic grading of student code is not limited than one implementation of the same interface and also pro- to the scope of a data structures course.