CALIFORNIA STATE UNIVERSITY, NORTHRIDGE
Automation for CSUN Computer Science Department Course Scheduling
A thesis submitted in partial fulfillment of the requirements
For the degree of Master of Science in Computer Science
By
Rayna Burgess
May 2017 SIGNATURE PAGE
The thesis of Rayna Burgess is approved:
Dr. Adam Kaplan Date
Dr. John Noga Date
Dr. Rick Covington, Chair Date
California State University, Northridge
ii
TABLE OF CONTENTS
Signature Page ...... ii List of Figures ...... v Abstract ...... vii 1 INTRODUCTION ...... 1 1.1 The Problem ...... 1 2 LITERATURE REVIEW ...... 5 2.1 Timetabling Strategies...... 5 2.2 Available Timetabling Systems ...... 6 2.3 Time Table Tool Selection ...... 8 2.4 OptaPlanner Constraint Satisfaction Engine ...... 9 2.4.1 Constraint Satisfaction Problem ...... 9 2.4.2 OptaPlanner Problem ...... 10 2.4.3 CSUN Scheduler Problem ...... 11 2.4.4 OptaPlanner Constraint Rules ...... 12 2.4.5 OptaPlanner Algorithm Configuration ...... 13 2.4.6 OptaPlanner Engine Integration ...... 15 2.5 Web Application Design and Development ...... 16 2.5.1 Spring Boot – Java Server Framework ...... 20 2.5.2 AngularJS – JavaScript Browser Framework ...... 23 3 ANALYSIS AND REQUIREMENTS ...... 24 3.1 Current Manual Process for Schedule Creation ...... 24 3.2 Instructor Preferences Automation...... 25 3.3 Schedule Generation Automation ...... 26 3.3.1 Schedule of Classes Hard Constraints ...... 26 3.3.2 Schedule of Classes Soft Constraints ...... 27 4 DESIGN OF SOLUTION ...... 28 4.1 Web Architecture ...... 28 4.1.1 Java Spring Boot ...... 29 4.1.2 MySQL Database ...... 29 4.1.3 AngularJS ...... 30 4.2 Database Design ...... 30
iii
4.3 Web Server Design...... 33 4.3.1 Web Server File Structure ...... 35 4.4 Front End Single Page App Design...... 36 5 IMPLEMENTATION ...... 39 5.1 Web Front End Implementation ...... 39 5.1.1 Instructor Page ...... 39 5.1.2 Home Page ...... 44 5.2 Web Server Implementation...... 44 5.2.1 Typical Data Request ...... 45 5.2.2 OptaPlanner...... 51 5.3 Challenges and Lessons Learned ...... 61 5.3.1 Selecting the Development Stack ...... 61 5.3.2 Acquiring adequate hardware ...... 62 6 CONCLUSIONS ...... 63 6.1 Practical Schedule Generation ...... 63 6.2 Instructor Preferences Automation...... 63 7 FUTURE ENHANCEMENTS ...... 64 7.1 Authentication and Authorization ...... 64 7.2 Web Sockets for Engine Controls ...... 65 7.3 Remaining Constraints ...... 65 7.4 Feature Tags for Rooms and Courses ...... 66 7.5 More Flexible Instructor Time Preferences ...... 66 References ...... 67 Appendix A – User Stories ...... 69 Appendix B – Current Instructor Preferences Paper Memo ...... 71
iv
LIST OF FIGURES
Figure 1 Published Schedule of Classes (SOLAR screenshot) ...... 1
Figure 2 Timetabling Solution Space Size...... 3
Figure 3 Timetabling Tool Choices ...... 7
Figure 4 General Constraint Satisfaction Problem and Solution ...... 10
Figure 5 OptaPlanner Problem and Solution ...... 11
Figure 6 CSUN Scheduler Problem and Solution...... 12
Figure 7 Drools File and Rule Format Definitions ...... 13
Figure 8 OptaPlanner Phases and Configurable Algorithms ...... 14
Figure 9 OptaPlanner Engine Integration ...... 16
Figure 10 Web Application Terms and Concepts ...... 19
Figure 11 Spring Boot Relation to Spring ...... 20
Figure 12 Current Method of Generating Schedules ...... 25
Figure 13 High Level Architecture ...... 29
Figure 14 Database High Level Entity Relationship Diagram ...... 32
Figure 15 Database Solution Entity Relationship Diagram ...... 32
Figure 16 Database Instructor Entity Relationship Diagram ...... 33
Figure 17 Web Server Architecture ...... 35
Figure 18 Web Server Code File Structure ...... 36
Figure 19 Front End Architecture ...... 38
Figure 20 CSUN Scheduler Navigation Bar ...... 39
Figure 21 User Time Preference Grid ...... 41
v
Figure 22 Instructor Course Preference Selection with No Courses ...... 42
Figure 23 Add Courses Modal ...... 42
Figure 24 Add Courses Modal Search Box ...... 43
Figure 25 Get List of Terms Sequence Diagram ...... 45
Figure 26 Instructor Preferences Memo Page 1 ...... 71
Figure 27 Instructor Preferences Memo Page 2 ...... 72
vi
ABSTRACT
Automation for CSUN Computer Science Department Course Scheduling
By
Rayna Burgess
Master of Science in Computer Science
This project implements a solution to the semester class scheduling problem faced by the
CSUN Computer Science Department each term. I wrote a web application following typical Agile software development practices including project requirement elicitation to define the major stories for the backlog, system level design to select the architecture and stack, and scrum-like sprints to cycle through feature development. The app uses
MySQL as the database, Spring Boot for the web server, and AngularJS for the single page application (SPA) that runs in the browser. The two major features of the app are the instructor preference page and the course schedule generation. Course schedule generation is of the class of problems called Timetabling problems, which are considered
NP Complete (not solvable in realistic time). This app uses a constraint engine with custom rules to generate a score for a given schedule and a local search algorithm to traverse the solution space looking for the schedule with the best score. The OptaPlanner open source Java library is used to perform the initialization, the heuristics, the local search, and the score calculation according to detailed data preparation, rules definitions and engine configuration which I coded specifically for this app. This project shows that it is possible to automatically generate the CSUN Computer Science department schedule.
vii
1 INTRODUCTION
1.1 THE PROBLEM
At California State University Northridge (CSUN), each department is responsible for creating a course schedule every semester. These schedules are entered into CSUN’s administrative software package called SOLAR, where they become available for student registration. SOLAR provides no assistance in creating this schedule but will inform the user when an inconsistent item is entered. Departments generate these schedules manually using a spreadsheet, a rotating plan, or a piece of paper and pencil. This project provides a user friendly, online method of generating, validating and updating a semester schedule for the CSUN Computer Science department.
Figure 1 Published Schedule of Classes (SOLAR screenshot) The course schedule (also called the schedule of classes) consists of a list of sections. A section is one offering of a course set at a particular time, in a particular room, with a 1 particular instructor. Figure 1 shows the completed and published version of a course schedule. Note that multiple instructors can teach the same course and that instructors can teach multiple courses.
Several areas of the current manual scheduling process lend themselves to automation.
Automation can help with communicating instructor preferences (such as courses to teach and time slots available), to the chair. Automation can also aid in maintaining lists.
There is the rotating list of upper division classes so that students can graduate within 4 years. There are also lists of courses in a curriculum such as the 1st year (or 2nd year) computer science required courses so that these classes are not accidentally offered at conflicting times. Finally, automation can be employed to do the actual mapping of sections of a course to the instructor, time slot and room that will appear on the final schedule of classes.
The problem of generating an optimal course schedule, matching course sections, instructors, rooms and times is of the class of problems called timetabling problems (E.
Burke, 1997). These problems are considered NP-complete, which means that as the size of the problem increases, the time needed to find the exact best solution increases to the point of becoming impractical.
The CSUN Computer Science Department has around 200 sections in the typical
Schedule of Classes. Each possible Schedule of Classes represents a solution. It is comprised of a mapping of around 70 courses, 50 instructors, 20 rooms and 60 timeslots.
Most working examples of this type of problem assume that instructors have already been mapped to courses. Introducing instructors as a separate independent variable greatly increases the solution space. Consider the simpler problem of mapping a section only to
2 a room and timeslot. In the OptaPlanner tutorial (DeSmet, 2017), they call the timeslot a period, and the section a lecture. The figure below from the tutorial depicts the size of the solution space. It gives the formula as (푝×푟)푙 which in our case would be (60×
20)200 = 8.28×10206, a number too large to show on a simple calculator, a number larger than the number of atoms in the known universe. And that is without including the mapping of instructors. Clearly, a simple brute force approach will not be feasible.
Figure 2 Timetabling Solution Space Size
(DeSmet, 2017)
Since an exact solution is impractical, a heuristic approach is needed to search for an optimal solution. There are several libraries and packages which can generate good
3 solutions for timetabling problems. A timetabling tool was selected which generates an approximate solution and tries to maximize a score based on hard and soft criteria.
After examining the literature and currently available timetabling tools, the OptaPlanner open source library was selected to provide the timetabling function. Requirements for this problem were elicited and documented. A solution architecture was designed and the program was implemented using OptaPlanner, MySQL, Java Spring Boot and AngularJS.
The implementation can successfully generate a schedule of classes given an initial set of data as extracted from SOLAR and additional configuration data for the rooms, courses, instructors and time slots. There are many exciting enhancements that can be made to this program to make it even more useful, and to further save the department chair some time and energy.
4
2 LITERATURE REVIEW
2.1 TIMETABLING STRATEGIES
As exhaustive search for the optimal schedule is impractical due to time constraints, most strategies involve mechanisms to find a non-optimal but very good solution in a practical amount of time. An initial solution can be created using one of several construction heuristics. These heuristics alone rarely locate a good enough solution but used in combination with a local search algorithm, they can help to cut the time of the search.
Construction heuristics build an initial solution based on a simple strategy (E. Burke,
1997). There are several types. First Fit involves mapping each planning entity (section) into the first place (room and timeslot) that it will fit. First Fit Decreasing is similar but it sorts the planning entities first so the most difficult ones are placed first. Best Fit involves mapping each planning entity into the “best” place (room and timeslot) that it will fit. Of course, “best” must be defined specifically for the given search. None of these construction heuristics are likely to build an adequate schedule, so once the initial construction is complete, it is typically followed up with a local search or genetic algorithm to refine the solution.
There are several local search algorithms (jboss, 2017) which attempt to take a given solution and move it toward a better solution. Hill Climbing tries all the possible adjacent moves and chooses the best move. It has the problem of getting stuck on local maximums. Tabu Search is like Hill Climbing but it remembers paths that are “tabu” and won’t take them again. Unlike Hill Climbing, Simulated Annealing does not look at all
5 the possible moves but only selects a few so that it can move faster and it adds a random element so that it can hop out of local maxima.
Genetic Algorithms use the model of evolution to evolve a solution population to a good solution (E. Burke, 1997). The algorithm starts with an initial population of solutions and combines selected members of the population in various ways to generate the next generation. Different algorithms have been proposed for the “cross-over,” the combining of the two parent solutions to create a child solution. A fitness function is defined to determine the “fittest” solutions that will survive. Although genetic algorithms have been successfully applied to the Timetabling problem, using a local Tabu search has been found to be faster and better (S. C. Chu, 1999). This project will therefore focus on the local search approach.
2.2 AVAILABLE TIMETABLING SYSTEMS
There are several timetabling systems available. The desired timetabling system will search the solution space for as long as the user wants it to, attempting to meet all hard criteria, minimize soft negative criteria (penalties) and maximize soft positive criteria
(rewards). The user can then choose to accept the generated solution, alter the generated solution manually, give the timetabling tool more time to work on the generated solution, or simply discard the generated solution.
The two main Java contenders are OptaPlanner, by jBoss (jboss, 2017), which is a part of the Drools Business Rules Management System, and UniTime (Unitime, 2017) which came out of Purdue University and other universities as an open source project and has since taken on its own identity as UniTime.org. In addition, there is a JavaScript based
6 solver called timetabling-solver. Figure 3 shows the three tools. These three tools were evaluated based on ease of use, application, and integration with the CSUN Scheduler.
Figure 3 Timetabling Tool Choices UniTime is a comprehensive University Timetabling system. It is specific to the generation of course and exam schedules for universities. It is a very large Java package and includes the user interface, the persistence and the data model. One problem I had fitting this tool to the CSUN model is that the timetabling does not map instructors to the section, instead it expects the instructor to already be assigned to the given course. This is not how our current manual process works. Most instructors can teach multiple courses and the assignments are based on where an instructor is most needed in addition instructor preference. The usefulness of this app to our chair is in the ability to create the schedule of classes including mapping the instructors to the courses.
The timetabling-solver, written in JavaScript, would imply a JavaScript based server instead of the Java based server which I was leaning toward. The more important problem with the timetabling-solver is that it uses a genetic search algorithm, which was shown earlier to be not as effective as local search on the timetabling problem.
7
The OptaPlanner tool, unlike the others, is a general-purpose constraint engine and while there are many examples of implementing course timetabling using OptaPlanner, it is also currently being used on traveling salesman problems, load balancing problems, bin packing, the n-Queens problem and several other problems that fit into this constraint resolution category.
2.3 TIME TABLE TOOL SELECTION
After investigating alternatives such as UniTime and timetabling-solver, I selected
OptaPlanner because it is the most configurable, and even the search algorithm can be changed. UniTime and timetabling-solver are both specific to timetabling, so are not as flexible as OptaPlanner.
This configurability was important for applying this problem to the CSUN Scheduling problem because unlike the examples (given in all three engines), the CSUN problem has to solve for three planning variables: instructor, time slot and room. The other solutions assume that the instructor is already mapped to the course and then only solve for the time slot and room. This limitation would be inadequate for the CSUN scheduling problem.
Tool Search Language Configurability Learning Algorithm Curve OptaPlanner Configurable Java High Medium
Timetabling-solver Genetic JavaScript Low Medium
UniTime Local search Java Medium High
Table 1 Time Tabling Engine Selection
8
2.4 OPTAPLANNER CONSTRAINT SATISFACTION ENGINE
This section begins with a general definition of terms and then dives deeper into the
OptaPlanner Constraint Satisfaction Engine. There are several key terms and concepts necessary to fully understand the project design and implementation.
2.4.1 Constraint Satisfaction Problem
The OptaPlanner engine is applicable to situations that fall into the constraint satisfaction problem (CSP) definition. A formal definition is given below by (Bulatov, Krokhin, &
Jeavons, 2000).
Definition 1. The constraint satisfaction problem (CSP) is the combinatorial decision problem with Instance: a triple (V, A, C) where V is a set of variables; A is a domain of values; and C is a set of constraints, {C1,...,Cq}.
Each constraint Ci ∈ C is a pair
A solution is a mapping from V to A. A solution is complete if it maps all of the variables in V and a solution is consistent if it passes all of the constraints in C. Figure 4 depicts this general constraint satisfaction problem and solution.
9
Figure 4 General Constraint Satisfaction Problem and Solution
2.4.2 OptaPlanner Problem
OptaPlanner problems are constraint satisfaction problems, however OptaPlanner uses its own set of terms and options. The general OptaPlanner problem and solution are depicted in Figure 5. In the OptaPlanner documentation, the term planning entity is given to the set V of variables. The term planning variables is used for the set A of domain values. Instead of using a set of boolean constraint functions, C, to provide thumbs up or thumbs down on the consistency of a solution, OptaPlanner uses the concept of a solution score. There are typically two sets of constraint rules: hard constraints and soft constraints. Each constraint rule, instead of simply passing or failing, returns a positive or negative value which will be added to the solution score. A score has two parts: the hard part and the soft part. A solution is considered feasible if it has a hard score of 0,
10 implying that all of the hard constraints were met. The optimal solution will have the highest soft score value. The OptaPlanner engine can be configured to run until either score value reaches a certain level or it can be run for a certain amount of time. While running, OptaPlanner attempts to maximize its hard and soft scores.
Figure 5 OptaPlanner Problem and Solution
2.4.3 CSUN Scheduler Problem
For this project, the CSUN Scheduler, the planning entity (or set V in the formal notation) is the list of sections that will be listed in the schedule of classes. Each section needs to be assigned a room, an instructor and a time slot, so our planning variables (set
A of domain objects in the formal notation) includes the list of all rooms, the list of all instructors and the list of all time slots. The rules (set C of Constraints in the formal
11 notation) come from our Drools (J-Boss, 2017) definition file which is discussed in section 2.4.4 . Figure 6 shows the CSUN Scheduler problem and solution.
Figure 6 CSUN Scheduler Problem and Solution
2.4.4 OptaPlanner Constraint Rules
OptaPlanner accommodates several methods of defining constraint rules but the preferred method, and the method used in the CSUN Scheduler, is to define rules using the Drools business rules management system. The rules are written in the Drools rule language and stored in a Drools .drl file in the server’s resources folder. The formats for the file and for the rules are simple and shown in Figure 7. The top of the file resembles a Java file.
The Java classes for the domain objects are imported just as they would be in a Java file.
12
The OptaPlanner Constraints, both hard and soft are defined according to the rule format.
The complete documentation for the rule language is found in the Drools Rule Language
Reference (J-Boss, 2017)
Figure 7 Drools File and Rule Format Definitions
Diagrams from (J-Boss, 2017)
2.4.5 OptaPlanner Algorithm Configuration
The OptaPlanner engine moves through a series of phases to arrive at the optimal solution quickly. Figure 8 shows the basic flow of the OptaPlanner engine. The first phase is to generate an initial mapping. The next phase is to move that mapping toward a solution with a better score. The final phase is to stop the engine; otherwise it could keep searching for years. The algorithms used at each of these phases are configurable.
13
Figure 8 OptaPlanner Phases and Configurable Algorithms The first phase is to use an algorithm called a construction heuristic to generate an initial mapping that covers all of the variables in the planning entity. This heuristic can be specific to the model. Algorithms such as best fit, best fit decreasing and strongest fit are available as construction heuristics. The Java model will need to define “best” and the sorting order if required. Algorithm configuration is done with simple XML statements such as:
OptaPlanner XML Configuration: Construction Heuristic
14
The next phase is to move that complete solution toward a better solution using a metaheuristic. This configurable algorithm can be model independent, hence the meta.
There are two categories of metaheuristics available in OptaPlanner: local search algorithms such as hill climbing, tabu search and simulated annealing; as well as evolutionary algorithms such as genetic algorithms.
OptaPlanner XML Configuration: Local Search
OptaPlanner XML Configuration: Termination
2.4.6 OptaPlanner Engine Integration
In order to run the OptaPlanner engine, all of the above mentioned must be implemented and integrated. Figure 9 shows the high-level view of how each part is used by the
OptaPlanner engine. The XML file in the resources folder contains the algorithm settings and references to the Java domain objects and the Drools rule file. The calling application creates a solver factory passing in the reference to the XML configuration file, then uses the solver factory to build the solver. The solver takes in a list of planning
15 entities, which need to be mapped onto one or more lists of problem variables. The solver generates the solution, which is a mapping of those planning entities to the problem variables, and the corresponding score for that configuration.
Figure 9 OptaPlanner Engine Integration
2.5 WEB APPLICATION DESIGN AND DEVELOPMENT
Jumping into web development can be overwhelming due to the variety of terms and concepts used and the velocity at which these terms and concepts are evolving. At its lowest level, a web application uses HTTP to deliver content from a web server to a web browser upon request by the browser. Most non-trivial web applications need to store data somewhere and so include a database component as well. Instead of having all of
16 the logic in one big program, modern web apps usually separate the web server, which deals directly with the HTTP, from the web application server, which implements the application logic.
This paragraph is not a comprehensive list of web components and stacks but merely provides an overview. Typical web servers are Apache, Apache Tomcat, Microsoft IIS, nginx, Jetty etc. Some of these are specific to a single language or platform. Apache
Tomcat, for example, can be used to serve static content, but can also be used as a Java
Servlet Container to provide that interface to a Java based web application. Some web servers typically host a single web app, but others such as Tomcat can host any number of separate web applications. A common web architecture is to have client code which runs in the browser, a web server to serve the pages, a web application that processes the requests and a database to persist the data. The selection of all four components together is commonly called a web stack. There are several popular web stacks such as the LAMP family of stacks (LAMP, WAMP, XAMPP…) which are all PHP based, the MEAN stack which is JavaScript based, the RoR stack which uses Ruby on Rails, the IIS stack which is Microsoft based, and various other stacks with no cute acronym or specific logo.
Figure 10 shows several stacks and partial stacks. It can be confusing if it is not clear which terms are referring to a full stack and which terms are referring to a specific framework. Most people are familiar with the LAMP stack, so it is included in the diagram for context. The diagram should clarify that Spring Boot can be used as a web application framework, but it is flexible and can be used with many diverse front end frameworks, web servers or databases. Likewise, AngularJS is a front-end framework that runs in the browser and it can be used with any number of web servers, web
17 applications and databases. The term “node server” is commonly used to describe a
Node/Express server, which is a light-weight server that is written in JavaScript and runs in the node runtime environment (available for all of the major platforms: Windows, Mac,
Linux).
18
Figure 10 Web Application Terms and Concepts
19
2.5.1 Spring Boot – Java Server Framework
Spring is a lightweight Java Framework. Spring Boot is a collection of the Spring core framework and other Spring packages that can be used with Spring core (Pivitol, 2017).
Figure 11 shows Spring Boot’s relationship to Spring. Spring Boot makes the whole
Spring environment into one configurable system and makes many configuration choices easier by hiding boilerplate details and presenting the developer with bundled higher level packages. For example, instead of configuring all of the various packages used to persist data, Spring Boot provides a Spring Data Starter configuration that chooses
“sensible” default configuration settings for each persistence related package. This makes for much easier startup and configuration. The Spring Boot Initializr program asks a few basic questions and then creates a complete file and folder structure for the base of a new Spring Boot project.
Figure 11 Spring Boot Relation to Spring
20
2.5.1.1 Inversion of Control – the Spring Container
The Spring framework is a container. The developer provides Spring with class definitions and configuration information, and Spring manages and runs those class objects. This concept is called Inversion of Control (IoC). Martin Fowler (Fowler,
InversionOfControl, 2005) explains that Inversion of Control (IoC) is simply a general principle where the code that one writes is called by the framework, unlike a library where the code one writes calls the library (Don’t call us, we’ll call you).
2.5.1.2 Spring beans
A bean in Spring is simply a plain old Java object (POJO) class definition along with some configuration data. A bean is managed by Spring: it is instantiated by Spring; its methods are called by Spring; and it is destroyed by Spring. In the past, developers primarily used XML files to provide the configuration for the beans. Today there is a trend toward preferring Java configuration using annotations and/or Java code to provide this information. Typically, Spring instantiates a bean to handle some type of event such as the receipt of an HTTP request, and destroys that bean when the processing for the event is complete (the HTTP response is sent).
2.5.1.3 Dependency Injection
It would be rare for a Spring event to be completely handled by a single class object.
Well written and decoupled code suggests a layered approach, where classes are layered and each layer has its own specific responsibility. A class at one layer calls one or more classes at a different layer to provide the more detailed functionality. For example, a controller layer class may be instantiated by Spring to handle an HTTP request, but that controller will in turn be calling one or more service layer classes to handle the actual
21 business logic for the request. It is the controller layer’s job to handle validation of inputs but not to handle business logic. The controller could be coded up to instantiate its own service classes to process the request at compile time. This, however, tightly couples the controller layer to the service layer. Because the controller needs a service object to process the request, we say that the controller has a dependency on that service class. Instead of binding that service to the controller at compile time, Spring uses
Dependency Injection (DI) to inject a managed service bean at run time.
Spring uses the @Autowired annotation to identify which dependencies need to be injected. The code snippet below shows how Spring will inject the termService, the userService and the scheduleService into the TermController when the bean is created.
Spring @Autowired Dependency Injection Example private final TermService termService; private final ScheduleSolver scheduleSolver; private final UserService userService;
@Autowired TermController(TermService termService, UserService userService, ScheduleSolver scheduleSolver) { this.termService = termService; this.scheduleSolver = scheduleSolver; this.userService = userService; } When Spring is ready to create the controller bean, it will also create beans for all of that controller’s dependencies and inject them into the controller, usually through the controller’s constructor. Thus, a bean’s dependencies are not compiled into the bean, instead they are configured for the bean (Fowler, Inversion of Control Containers and the
22
Dependency Injection pattern, 2004) and Spring creates those dependent beans (service etc) and injects them at run time. Dependency injection allows for cleaner decoupling of the code. It allows Spring to manage all of the beans being used to process the event.
And perhaps the most important benefit of dependency injection is that each bean can be cleanly unit tested, because the unit test can instantiate the bean and inject mock classes for the dependencies, allowing the bean unit to be tested in isolation.
2.5.1.4 Spring Boot and Apache Tomcat
Spring web implements a Spring container that can run inside of a Java Servlet web server such as Apache Tomcat (or Jetty, or Glassfish, or Wildfly…). It receives HTTP requests from and sends HTTP responses to the Java servlet. Typically, in a large server installation, the entire Spring application is compressed (zipped) into a web archive file
(war file) which can be deployed onto a Java servlet web server. A web server such as
Tomcat can host several different war files for several different web apps simultaneously.
Spring Boot introduced the concept of an embedded web server. For debugging or for smaller apps, the Spring web application can be run on the command line using its own internal version of Apache Tomcat as the Java servlet.
2.5.2 AngularJS – JavaScript Browser Framework
AngularJS is a front-end framework for developing Single Page Applications. The server simply serves the static html files, and the AngularJS framework running on the client controls the navigation and code execution. Like Spring, AngularJS provides dependency injection. The main reason for this dependency injection is for ease of testing. In an automated test, a mock service can be injected in place of the actual service, thereby allowing a unit to be tested in isolation.
23
3 ANALYSIS AND REQUIREMENTS
The primary method for requirements elicitation was direct interview with the Computer
Science Department Chair, who is the main user of the program. We discussed the manual method being used today and potential enhancements once the system is automated. We also discussed the criteria required to create an acceptable schedule (hard constraints) and an optimal schedule (soft constraints).
3.1 CURRENT MANUAL PROCESS FOR SCHEDULE CREATION
Currently, schedules are created manually using a spreadsheet. The initial Schedule of
Classes (SOC) is created each semester within the SOLAR system. It contains sections that were present in the prior year’s semester, but it leaves the times, rooms and instructors unassigned. This schedule can be exported as a comma separated values (csv) file and imported to the custom spreadsheet. In addition, each semester, all instructors submit paper forms indicating their preferences. They list their desired courses in priority order, as well as their desired times and times which would be impossible for them to teach. In addition, they indicate their teaching load, which is the number of units they will be teaching this semester. Because the rare upper division classes need to be scheduled every few semesters in order for students to be able to graduate within the desired time frame, a spreadsheet containing the rotating schedule for these courses is also considered while refining the SOC. Other constraints reside in the chair’s knowledge base such as which courses require Windows computers and which rooms have those computers, or which courses require Macs or other specific hardware and which rooms match those requirements. In addition, he knows that certain professors
24 prefer tables to desks and which rooms contain tables.
Figure 12 Current Method of Generating Schedules The custom spreadsheet allows the chair to calculate the full-time equivalent students
(FTES) for each course and to calculate the percentage of Friday and Saturday FTES, which needs to be greater than 15%. Trial and error and sorting by room, instructor and time slot help the chair to generate an optimal SOC each semester. As the registration process proceeds, this schedule needs to be updated to accommodate the addition of new classes as existing classes fill up and waiting lists grow.
3.2 INSTRUCTOR PREFERENCES AUTOMATION
Each semester, the instructors are asked to fill out and turn in a paper form to indicate their preferences for the term. The paper memorandum includes the listing of upper
25 division courses being offered for the semester. There is an area for the instructor to list the courses they are willing to teach in decreasing priority order. There is an area to give the load requirements for the semester. There is a grid for the instructor to shade in their preferred times and their impossible times, which shows Monday through Saturday across the top and 9 time slots down the left side. By implementing this paper sheet in an online form, the hope is that there can be faster feedback, as instructors do not have to remember to fill in the paper and turn it in. Also, the online form can prefill the fields with the prior semesters values so that only a very few clicks are required by the instructors to modify the form for the current semester.
3.3 SCHEDULE GENERATION AUTOMATION
The automatic generation of the schedule by mapping instructors, rooms and time slots to each section for the semester is the NP-complete part of the app requiring the time tabling tool. OptaPlanner is a general constraint engine, which can solve time tabling problems as well as other constraint-based problems. Although OptaPlanner comes with several working examples of course scheduling for courses and conferences, none of these examples is completely applicable to this project. Using these demos as a guide, I came up with a different data model and different constraints which more accurately reflect the
CSUN Computer Science course scheduling requirements.
3.3.1 Schedule of Classes Hard Constraints
The outcome of numerous discussions with the department chair about the manual process is the following desired list of hard constraints (Table 2). A solution that fails a hard constraint is considered unfeasible.
26
Hard Constraint Priority
Two sections with the same instructor with overlapping time slots High
Two sections in the same room with overlapping time slots High
An instructor cannot be assigned a course they are not willing to Medium teach An instructor cannot be assigned a time slot in which they are High unavailable An instructor cannot be assigned more than the maximum allowable High faculty teaching units Assign courses to time slots with a weekly number of hours matching Medium the weekly hours required by the course units and type (Lab/Lecture) Table 2 Hard Constraint Requirements
3.3.2 Schedule of Classes Soft Constraints
Soft constraints can either be negative (a penalty) or positive (a reward). The constraints are constructed to optimize the schedule for everyone. During requirements elicitation, many soft constraints were identified (Table 2). They were prioritized so that I could implement the highest priority constraints first.
Soft Constraint Priority
Prefer rooms which can seat all the students in the section’s High enrollment cap Prefer sections scheduled for an instructor’s highest priority High time slot Prefer sections scheduled for an instructor’s highest priority Medium courses Prefer a schedule in which Friday/Saturday full-time Medium equivalent students (FTES) units are at least 15% of total FTES units Prefer assigning sections to an instructor that meets that Medium instructors load requirements Prefer scheduling courses in the same curriculum in non- Low overlapping time slots Prefer scheduling courses marked evening only (600 level Low courses) in the evening Table 3 Soft Constraint Requirements
27
4 DESIGN OF SOLUTION
The initial design of the solution revealed several points requiring resolution. How would the CSUN Scheduler be integrated with the existing SOLAR system? How would the user data persist? Which timetabling tool would be used? What would the overall architecture of the tool be?
4.1 WEB ARCHITECTURE
As the vision for the tool is to expand to additional departments, a standard web architecture was chosen. In this initial version, the tool can be deployed locally within the Computer Science department and be accessible to the chair and the instructors via the Internet. In the future, it could be deployed by the university and accessible over the
Internet by multiple departments. Figure 13 shows the high-level web architecture including the AngularJS web single page app, the Java Spring Boot server and the
MySQL database. The current manual method of creating a schedule involves manual entry of data into SOLAR and manual export of data from SOLAR using a comma separated values (csv) file. I decided to keep this method of communication with
SOLAR for this version, in the future perhaps a more direct communication path might be identified and implemented.
28
Figure 13 High Level Architecture
4.1.1 Java Spring Boot
Because the timetabling tool is Java based, Spring Boot was a natural choice. Spring
Boot is easy to build with, as it has a builder that asks some simple configuration questions and then creates the folder structure and files required for the basic app. Spring
Boot comes with components for data, security and web services, and is often the choice for building enterprise web applications. It is a technology that I wanted to learn.
4.1.2 MySQL Database
I selected MySQL for persistence because it works well with Spring Boot, it is open source and it is relational. At first, I thought that I could get away with a non-relational database, but it soon became clear that the CSUN scheduler data model is highly relational and different relations are important to different users. For example, an instructor would want to see their own view of the data as it relates to them: their
29 schedule, their preferences, their terms; while a chair would want to see the overall set of data as it relates to a particular term: all sections, all instructors, all rooms etc.
4.1.3 AngularJS
AngularJS is a good front end framework for an enterprise level web application. I already understood angularJS so it was a natural choice. Alternaitve frameworks include
React and Angular 2, but I wanted to focus my learning on the server with Spring Boot and OptaPlanner.
4.2 DATABASE DESIGN
The database tables are designed to follow the natural relationships between the entities.
There is a table for users of the system. There is a table for departments (even though this first version is only for the Computer Science department). There is a table for terms
(semesters for CSUN, quarters for other universities). It is anticipated that schedules will be created each term, and that it would be desirable to refer to prior terms or perhaps even work on more than one future term at the same time, hence each department can have many terms, but any given term object will belong to a single department. There are tables for rooms, instructors, courses, and time slots. Each of these has a natural many to many relationship with terms. For example, a term certainly has more than one instructor, and the same instructor usually teaches many terms. A mapping table linking terms with instructors was created to model this relationship (used to get the list of instructors available for the given term). Similarly, mapping tables were used for courses, rooms and time slots. The goal of this project is to generate an optimal schedule of classes.
This might end up being a process where multiple possible schedules might be worked on
30 at the same time. Starting with a base schedule, the chair may want to work on a few
“what if” scenarios.
31
Figure 14 Database High Level Entity Relationship Diagram For this reason, there is a list of solutions (schedules and scores) related to any given term.
An entity relationship diagram for this high-level view can be seen in Figure 14.
A solution is represented as a table with three score values: hard score, soft score and a boolean value: feasible. In addition, there is a mapping table to relate the list of sections making up the schedule to the solution object. The section is the OptaPlanner Planning
Entity, each section is mapped to a single course entity, and OptaPlanner will map it to the time slot, instructor, and room. Figure 15 below shows the tables related to the solution object.
Figure 15 Database Solution Entity Relationship Diagram As was indicated during the requirements elicitation, instructors are the driving force for this app. Instructors select and prioritize their own courses and time slots. Figure 16 shows the tables used to store the instructor preferences.
32
Figure 16 Database Instructor Entity Relationship Diagram
4.3 WEB SERVER DESIGN
The web server uses the Spring Boot design shown in Figure 17. The entire web server application runs in the Spring framework and follows the typical layered architecture depicted in the figure. The controller layer receives HTTP requests and returns HTTP responses. Data Transfer Objects (DTOs) provide the external format for data in the
HTTP requests and responses so that the external API is not too tightly coupled with the internal implementation. The service layer provides the business logic and usually converts the DTO to its internal format as a domain object. The data layer handles the persistence of the domain objects. Security, logging and exception handling are aspects which cut across all of the layers.
The following classes were developed for this design. At the controller layer, there are three controllers: UserController, TermController and InstructorController. There are
33 four Data Transfer Objects. There should be more so that the database entities are not so tightly coupled with the user interface, this is an area that can be improved in the future.
There are six services: course, instructor, room, schedule, term and user. There are ten repositories. The domain objects used by OptaPlanner ought to be decoupled from the database entities in the future. There are 19 domain entities representing the various tables and relationships in the database. There are 8 Java domain classes that are used for the OptaPlanner configuration and pre-calculations. This app utilizes Spring Security for security and one class was defined to configure the security settings. The log4j logger is used for logging across all of the layers. A common exception handler also crossing all layers is in the plans for the future.
34
Figure 17 Web Server Architecture
4.3.1 Web Server File Structure
The file structure is shown in Figure 18. It even shows the OptaPlanner configuration
XML file kept in the resources folder along with the Drools file containing the rules definitions.
35
Figure 18 Web Server Code File Structure
4.4 FRONT END SINGLE PAGE APP DESIGN
The AngularJS front end software is designed to follow the John Papa style guide for
AngularJS (Papa, 2017). This style guide recommends organizing the files and modules
36 by object type rather than by layer type. So instead of having a package that contains all of the controllers for the app, there is a module which contains the controllers, route, service, etc for a given object type. In this app, there are modules such as home, instructor, course, timeslot, etc. Within each module there is typically a partial HTML file, a Less (CSS like) style file, a controller, a directive or two, a service or two, a route and a module definition. The high-level architecture is depicted in Figure 19. In addition, there is a core module containing common utilities, services and directives.
The user interacts with the app through the view. AngularJS handles the interaction between the view, the model and the controller. AngularJS provides services to make
AJAX (asynchronous JavaScript and XML) calls to the server to retrieve data. Instead of the server controlling which files are pushed out to the browser, in a single page app
(SPA), the browser framework pulls the files it wants asynchronously.
37
Figure 19 Front End Architecture
38
5 IMPLEMENTATION
The implemented solution is running on Windows using MySQL for the database, Java
Spring Boot as the server, and AngularJS as the single page web app. The implementation follows the design described in Section 4.
5.1 WEB FRONT END IMPLEMENTATION
There are several pages on the web front end. The home page contains the current schedule of classes, but there are navigation menu items across the top of the page (see
Figure 20) to view and edit the lists of courses, instructors, rooms and time slots. Most of
Figure 20 CSUN Scheduler Navigation Bar the relevant functionality, however, can be found on the home page when the chair is the user and on the instructor page when an instructor is the user. The chair can also get to any instructor’s page by drilling down in the list of instructors.
5.1.1 Instructor Page
The primary purpose of this page is to communicate instructor preferences to the chair for use in assembling the schedule of classes. This page is modeled directly after the paper memo being used currently for this purpose. This memo is attached in Appendix B. The memo lists the upper division courses scheduled for the given semester. On the first page of the memo, there is an area for entering name, entering a list of prioritized courses and entering the teaching load for the semester. On the second page is a grid with the days
Mon through Sat across the top and 1½ hour time slots running down the left side.
39
Instructors shade in the empty boxes in the grid to indicate times they prefer and times that are impossible for them.
5.1.1.1 Time Preference Selection
I modeled the time preference selector after the paper form, using the same grid. Each grid cell can be set to unavailable or given a preference level of 1, 2, or 3 stars, with 3 stars being the highest preference and 1 star the lowest possible preference. The default for each professor is unavailable on Friday and Saturday and one star on all the other days. Once a professor sets their time preferences, they will persist from semester to semester, so there is no need to change them if the instructor’s preferences have not changed. To change an entire column or row, there are controls in the headers. At present, there are no controls on the settings each instructor can select. It should be encouraged that instructors try to minimize their unavailable times. A maximum number of selected unavailable time slots could be added if instructors abuse this privilege.
Notice that the instructor time preference cells do not necessarily correspond cleanly with the selected time slots for the semester. Those mappings will be made on the server. The front end just captures the instructors’ time preferences in a simple consistent manner that can be applied to different time slot patterns each semester. Also, unavailable time slots are a hard “do not schedule me for even 5 minutes into this time,” while preference levels will be averaged to determine an instructor’s preference for scheduling a course that spans multiple preference cells. The details of this calculation are discussed in Section
5.2.2.4 Instructor Time Preferences in the Web Server Implementation portion of this document.
40
Figure 21 User Time Preference Grid
5.1.1.2 Course Preference Selection
The course preference area allows an instructor to prioritize as many courses as desired.
They can even include courses that are not offered every semester. The engine will just check the priority number for the courses that it is trying to place into the current schedule. When an instructor is first added to the system, they do not have any courses in
41 their list. They will see a screen asking them to add some courses (Figure 22).
Figure 22 Instructor Course Preference Selection with No Courses Selecting the “Add courses” button brings up a modal where the instructor can select courses to add to their list. The instructor can add as many courses as they would like.
The order in which they click to select the courses will be the order that those courses are appended to their current list (if they have one).
Figure 23 Add Courses Modal
42
Clicking the “Add” button will append the selected courses to the instructor’s course preference list. Clicking “Cancel” will discard the selections, leaving the instructor’s course preference list as it was before opening the modal. The search box at the top of the modal allows the instructor to filter the list by any text string to find specific courses.
Figure 24 shows looking up a course by number. Alternatively, one could enter part of a course title such as “data” into the search box to look up all of the data structure courses, or “lab” to see all of the lab courses.
Figure 24 Add Courses Modal Search Box Once the instructor has a list of preferred courses, these courses can be dragged and dropped to change their priority. Preferred courses can be deleted from the list by clicking the red X. (They can be added back by clicking the Add button).
43
5.1.1.3 Load Selection
This section allows the user to select their desired load in terms of Faculty Teaching
Units (FTUs). It defaults to 12 FTUs and can take a value from 0 to the max, which is currently hard-coded to be 15. In the future, the maximum allowed FTUs may be configurable by the chair.
5.1.2 Home Page
The home page allows the chair to manage terms, import and export data, view the current solutions and run the schedule generation engine. Each solution is displayed as a list of sections in a grid. The grid can be sorted and filtered as desired to interact with the schedule.
5.2 WEB SERVER IMPLEMENTATION
Following the design discussed in Section 4.3, the server implementation makes use of
Spring Boot for processing API requests. The code is divided into two main parts: the
44
REST api allowing access to Create, Read, Update and Delete (CRUD) the domain objects, and the schedule solver allowing access to start, terminate and fetch results from the schedule solver engine.
5.2.1 Typical Data Request
Each REST API request follows a similar path through the server. Examining one such request in detail should establish the pattern that was used for the others. The request to get the list of terms is shown in Figure 25. It assumes that the user is authenticated and has a current session1.
Figure 25 Get List of Terms Sequence Diagram
5.2.1.1 Server to Spring
The web server (in this case embedded Tomcat) sends incoming HTTP requests to the
Spring framework. Spring Security filters out those requests that the current user is not authorized to see. An HTTP request can be a POST (create this object), GET (read this object), PUT (update this object) or DELETE (delete this object). The associated URL
1 An HTTP session starts with a login, spans all of the request and responses until it ends with a logout. 45 specifies the particular object. Following typical REST URL naming conventions, the
URL corresponds to the object hierarchy and an individual object of a given type can be accessed by appending its id to the URL. Thus /api/term refers to the list of terms while
/api/term/8 refers to the single term with the id of 8.
5.2.1.2 Spring to Controller
The first thing that Spring Web needs to do is to identify which controller to instantiate to handle this request. Spring Web’s configuration will provide this information.
Configuration can be supplied to Spring Web via a separate XML configuration file or by embedding annotations into the source code itself. This project uses the annotation method. Classes are identified by Spring as controllers by adding the @RestController annotation before the class definition. Controllers and methods are mapped to particular
HTTP requests by including the @RequestMapping annotation which can include HTTP request type (POST, GET, PUT, DELETE) and URL pattern. The @requestMapping before a class definition tells Spring that this is the base URL for this class, all of the method level @requestMapping URLs are appended to this base URL. The code snippet in Error! Reference source not found. shows that the term controller handles URLs with a base of /api/term.
Spring Web instantiates the identified controller and all its injected dependent classes
(recursively). The @Autowired annotation tells Spring which dependent classes to instantiate and inject. In the term controller code snippet shown below, it shows that the
TermService, the UserService and the ScheduleSolver services are being injected.
46
Finally, Spring calls the appropriate method on the controller. In the code snippet, it shows that the getTerms method has been mapped to the GET, /api/term request. Spring calls this function and provides useful parameters such as the HTTP request and the principal object (the authenticated entity making the request).
TermController @RestController @RequestMapping("/api/term") public class TermController {
private static final Logger LOGGER = LoggerFactory.getLogger(TermController.class);
private final TermService termService; private final ScheduleSolver scheduleSolver; private final UserService userService;
@Autowired TermController(TermService termService, UserService userService, ScheduleSolver scheduleSolver) { this.termService = termService; this.scheduleSolver = scheduleSolver; this.userService = userService; }
@RequestMapping(method = RequestMethod.GET) public List
LOGGER.info("Found {} terms for department {}", termList.size(), csunUser.getDepartment().getLabel()); return termList; } … more code
5.2.1.3 Controller to Service
The controller is like a switchboard operator wiring together the HTTP request with business logic provided by the services. In this way, the public API is decoupled from the implementation logic. In this example, the getTerms method gets the department from the session and calls the findByDepartment method on the termService to retrieve the list of terms for the given department. Since this is a simple call, the termService merely forwards the request on to the repository layer. If there was business logic to be implemented, the service would implement it.
Examining the term service code shown below, it shows five @Autowired classes which
Spring will inject upon instantiation. It injects both the base and the detailed term object repositories and also the services for instructors, courses and rooms which are all referenced by lists contained in the term object.
TermService @Service public class TermServiceImpl implements TermService { private static final Logger LOGGER = LoggerFactory.getLogger(TermServiceImpl.class); private final TermRepository termRepository; private final TermTypeRepository termTypeRepository; private final TermDetailsRepository termDetailsRepository; private final InstructorService instructorService; private final CourseService courseService;
48
private final RoomService roomService;
@Autowired TermServiceImpl(TermRepository termRepository, TermTypeRepository termTypeRepository, TermDetailsRepository termDetailsRepository, InstructorService instructorService, CourseService courseService, RoomService roomService) { this.termRepository = termRepository; this.termTypeRepository = termTypeRepository; this.termDetailsRepository = termDetailsRepository; this.instructorService = instructorService; this.courseService = courseService; this.roomService = roomService; } public List
5.2.1.4 Service to Repository
The repository is where Spring Data makes the code look magical! The code snippet below shows the entire amount of code I had to write to implement this layer. Spring
Data (Pivotal Software) automatically implements the basic CRUD (create, read, update, delete) calls for the given entity, so there is no need to write any further code for them.
Simply defining the interface makes these calls available. For particular get requests, simply defining the abstract method is enough for Spring Data to implement the method.
49
Spring Data looks at the name of the method to determine the query parameters. For example, findByDepartment generates a data query filtered by objects where the department field equals the department passed in as a parameter. Spring Data sends the request to the database and returns the response as a Java List.
TermRepository public interface TermRepository extends CrudRepository
5.2.1.5 Configuring Spring Data
In order to perform the seemingly magical queries, Spring Data needs configuration information about the entities and relationships in the underlying database. This could be done using a configuration file, but I chose to use annotations for this configuration.
Spring JPA defines a standard interface and standard annotations for the data access layer.
The two snippets below show the annotations for the basic term object.
Term Class @Entity @Table(name = "term") public class Term extends TermBase {}
Term Base Class @MappedSuperclass public class TermBase { @Id @Column(name = "term_id") @GeneratedValue(strategy = GenerationType.AUTO) private Integer termId;
50
@OneToOne @JoinColumn(name = "term_type_id") private TermType termType;
@Column(name = "term_year") private Integer termYear;
@Column(name = "term_state") private String termState = "new";
@OneToOne(cascade = CascadeType.ALL) @JoinColumn(name = "note_id") private Note note;
@ManyToOne @JoinColumn(name = "department_id", nullable = true) private Department department;
5.2.1.6 The Return Trip
The result of the query is returned to the service, which returns it to the controller, which returns it to Spring. Spring Boot uses the Jackson library to convert the Java List into a json object and return that to the Server to be sent as an HTTP response to the browser.
5.2.2 OptaPlanner
The OptaPlanner engine runs the given problem to generate an optimal solution. The key to good results is the configuration of the engine and domain objects. The engine is driven by the hard and soft scores associated with each solution set (mapping of sections to rooms, instructors and courses). The constraint rules are used to generate the hard and soft scores. The rules are written in Drools and must not require a lot of time to process due to the compute intensive nature of these NP-complete problems. So, the key to a good implementation is to pre-calculate when possible to speed up and simplify the rules.
51
Much time was spent in determining the rules and then the pre-calculated classes needed to process them quickly.
5.2.2.1 OptaPlanner Examples Insufficient
OptaPlanner comes with an extensive set of examples, including several pertaining to course scheduling. I downloaded, ran and modified three of them. These were fantastic as a guideline and helped greatly in understanding how the rules engine worked, however in the end I generated my own data model, my own pre-calculated classes and my own rules. There were several model differences which were significant. The instructor in all of the models was pre-assigned to each course. In the CSUN Scheduler model, a course can be taught by multiple instructors, and an instructor can teach a variety of courses. In the demo programs, the time slots were simpler (one day a week), non-overlapping, and all of the same length. In the CSUN scheduler model, there are time slots for 2 hours per week and time slots for 3 hours per week. Time slots can be for one day a week, or they can be for two days a week. There are both daytime and evening time slots. The consequence of all of these differences is that completely new Java classes and Drools rules were written for the CSUN Scheduler.
5.2.2.2 Time Conflict
An obvious hard constraint for any course scheduling problem is that there can’t be two courses in the same room at the same time. In the examples given by OptaPlanner this is a fairly simple rule:
OptaPlanner Example Rule for 2 lectures in the same room at the same time // RoomOccupancy: Two lectures in the same room at the same period. // Any extra lecture in the same period and room counts as one more violation.
52 rule "roomOccupancy" when Lecture($leftId : id, period != null, $period : period, room != null, $room : room) // $leftLecture has lowest id of the period+room combo not Lecture(period == $period, room == $room, id < $leftId) // rightLecture has the same period Lecture(period == $period, room == $room, id > $leftId, $rightId : id) then scoreHolder.addHardConstraintMatch(kcontext, -1); end Note that OptaPlanner uses different names: a section is called a lecture and a time slot is called a period. It simply checks each lecture for the same room and same period to determine a conflict. This is insufficient for the CSUN scheduler. For the CSUN scheduler, it needs to check if the two time slots are overlapping at all. My first attempt at fixing this was to add a public method to the timeSlot class named conflictsWith() to be run when checking solutions (See the code below). However, I found that I could not access that method when comparing two sections within the rules engine. Also, it was a little time consuming, and it is recommended to pre-process such calculations. conflictsWith() Method in TimeSlot class private boolean sameDays(TimeSlot other) { // Look for any matching day in either one int len = other.days.length(); for (int i = 0; i < len; i++) { if (this.days.contains(other.days.substring(i, i + 1))) { return true; } } return false; } private boolean overlappingTimes(TimeSlot other) {
53
if (this.startTime <= other.endTime && this.endTime >= other.startTime) { return true; } return false; } public boolean conflictsWith(TimeSlot other) { if (sameDays(other)) { return overlappingTimes(other); } return false; }
Following the pattern for course curriculum found in the OptaPlanner examples, I created a class named TimeConflict to represent a pair of time slots that are in conflict.
TimeConflict Class public class TimeConflict implements Serializable, Comparable
public TimeConflict(TimeSlot leftTimeSlot, TimeSlot rightTimeSlot, int conflictCount) { this.leftTimeSlot = leftTimeSlot; this.rightTimeSlot = rightTimeSlot; this.conflictCount = conflictCount; } Etc… Prior to starting the OptaPlanner engine, the ScheduleOfClasses Java object pre- calculates the entire list of time slots in conflict to make that information available to the solver and rules.
54 preCalculateTimeConflictList() method in ScheduleOfClasses class // Time slots that overlap are used to detect time conflicts public List
CSUN Scheuler Rule for room conflict // ROOM CONFLICT: Two sections in the same room with overlapping time slots. rule "roomConflict" when $timeConflict : TimeConflict($leftTimeSlot : leftTimeSlot, $rightTimeSlot : rightTimeSlot) $leftSection : Section($leftId: sectionId, timeSlot == $leftTimeSlot, $leftRoom: room) $rightSection: Section(sectionId > $leftId, timeSlot == $rightTimeSlot, room == $leftRoom) then scoreHolder.addHardConstraintMatch(kcontext, -1); end
To explain the rule a little further, the when clause considers 3 things at the same time two sections and one timeConflict object. I arbitrarily call one section the left section and
55 the other the right. It doesn’t need to check the same 2 sections in the other direction
(with the left swapped for the right) hence the requirement that the left id is bigger than the right id (it also it shouldn’t check a section with itself). It is a conflict if both sections are in the same room. Since both sections are on the list of time conflicts, their times are overlapping. It will reduce the hard score by 1 for each course that is scheduled at a time overlapping with another course in the same room.
5.2.2.3 Instructor Course Preferences
The instructor course preference rules are unique to the CSUN Scheduler, as all of the
OptaPlanner examples had the instructor already assigned to courses. Recall that our model for keeping the instructor preferences is to keep a list of courses for each instructor along with a sort order (preference order). This instructor preference is a property of each Instructor object. In order to implement this for the rules engine, again a new class was created named InstructorCoursePreference that associates the instructor with the course and the associated soft constraint reward. The reward needs to be larger for the higher preference courses (which have a lower sort order). Also, the reward needs to be of a comparable weight to the other soft score rewards and penalties. Thus, a minimum value of 1 reward point was used, with a maximum of 9 reward points for the instructors first choice.
InstructorCoursePreference public class InstructorCoursePreference implements Serializable, Comparable
56
public InstructorCoursePreference(Instructor instructor, Course course, Integer sortOrder) { this.instructor = instructor; this.course = course; if (sortOrder >= 10) { this.reward = 1; } else { this.reward = 10 - sortOrder; } }
Etc…
This list is calculated in the ScheduleOfClasses object prior to running the OptaPlanner engine.
PreCalculateInstructorCoursePreferenceList() method in ScheduleOfClasses class public List
57 is not on their list; and there is a soft constraint to prefer a schedule which assigns instructors their top-rated courses.
Instructor/Course Conflict Rule: Hard Constraint // INSTRUCTOR/COURSE CONFLICT: Instructor is not willing to teach this course rule "instructorCourseConflict" when Section($instructor : instructor, $course: course) not InstructorCoursePreference(instructor == $instructor, course == $course) then scoreHolder.addHardConstraintMatch(kcontext, -1); end Notice that the “not” allows the rule to match only courses that are not on the instructor preference list.
Instructor/Course Preference: Soft Constraint // INSTRUCTOR/COURSE PREFERENCE rule "instructorCoursePreference" when Section($instructor : instructor, $course: course) InstructorCoursePreference(instructor == $instructor, course == $course, $reward : reward) then scoreHolder.addSoftConstraintMatch(kcontext, $reward); end
5.2.2.4 Instructor Time Preferences
The instructor time preference was implemented in a similar manner. The
InstructorTimePreference class was defined as a relation between and instructor and a time slot with an associated reward value. However, there is no one to one mapping of
58 the time preference cells used by the instructor to state their preferences and the actual time slots being considered for this semester. So, the following methods were added to the Instructor class to figure out the score reward. It averages the preference ranks of all of the time preferences which overlap with the given time slot. This is an area that could be studied and enhanced, but it seems sufficient for an initial launch. Note that if any part of the time slot in question overlaps with a preference block marked as unacceptable the entire score is 0. Again, I tried to keep the scores in the same range as other rewards, hence multiplying the sum by three which gives the best choice a value of 9, just like the most preferred course. getReward() method in the Instructor class private boolean overlappingTimes(TimeSlot timeSlot, TimePreference timePreference) { if (timeSlot.getStartTime() <= timePreference.getEndTime() && timeSlot.getEndTime() >= timePreference.getStartTime()) { return true; } return false; }
public Integer getReward(TimeSlot timeSlot) { String days = timeSlot.getDays(); int count = 0; int sum = 0; for (TimePreference timePreference : timePreferences) { if (overlappingTimes(timeSlot, timePreference)) { for (DayPreference dayPreference : timePreference.getDayPreferences()) { if (days.contains(dayPreference.getDay())) { if (dayPreference.isUnacceptable()) { return 0; }
59
count++; sum += dayPreference.getRank(); } } } } return (sum * 3) / count; }
The list of InstructorTimePreference objects is calculated in the ScheduleOfClasses before running the OptaPlanner engine. Note that time slots with a reward of 0 (meaning they overlapped with an unacceptable region) are not included in the list. preCalculateInstructorTimePreferenceList() method in ScheduleOfClasses class public List
60
Finally, the hard and soft rules for instructor time preferences are shown below.
Instructor/Time slot conflict rule: hard constraint // INSTRUCTOR/TIME SLOT CONFLICT: Instructor is not available during this time slot rule "instructorTimeConflict" when Section($instructor : instructor, $timeSlot: timeSlot) not InstructorTimePreference(instructor == $instructor, timeSlot == $timeSlot) then scoreHolder.addHardConstraintMatch(kcontext, -1); end
Instructor/Time Slot Preference rule: soft constraint // INSTRUCTOR/TIME PREFERENCE rule "instructorTimePreference" when Section($instructor : instructor, $timeSlot: timeSlot) InstructorTimePreference(instructor == $instructor, timeSlot == $timeSlot, $reward : reward) then scoreHolder.addSoftConstraintMatch(kcontext, $reward); end
5.3 CHALLENGES AND LESSONS LEARNED
5.3.1 Selecting the Development Stack
I believe that my major challenge was selecting the development stack. Initially, I began the implementation using the MEAN stack with MongoDB, Express, AngularJS and node.
After selecting the timetabling engine, I realized I needed to move to a Java based server, because OptaPlanner is implemented in Java. So, I migrated to Spring Boot with
61
AngularJS and MongoDB. I made great progress with this stack until I started to implement the instructor preference page and the corresponding engine rules. Up to that point, I had considered the entire hierarchical term object to be one MongoDB document.
At that time, I realized I really needed a relational database because I needed to be able to get to each instructor’s data independently and because I wanted to model the natural relationships within the data. Thus, my third iteration was to convert the MongoDB to
MySQL. Using Spring Data meant that the repository layer didn’t really need to change much, but the entity layer required radical changes to annotate each domain object for
MySQL. In addition, the seemingly magical Spring Data repository worked differently for the relational model than for the simple document model, and it took time to resolve some anomalous responses from the repository layer. Nearly all of these were fixed by adjusting the entity annotations related to table joins.
5.3.2 Acquiring adequate hardware
At the time that I switched databases from MongoDB to MySQL, I really started to feel the pain of developing on a laptop with only 4GB of RAM. Progress slowed remarkably when I needed to have all of the development tools open at the same time. Changing to a laptop with adequate RAM improved productivity incredibly. Running the database and the server on the original laptop was fine, however running the development tools ground performance to a standstill.
62
6 CONCLUSIONS
This project has made a solid first step toward a practical solution for generating schedules of classes each semester for the Department of Computer Science at CSUN.
6.1 PRACTICAL SCHEDULE GENERATION
This project has shown that automatically generating a schedule of classes for the CSUN
Computer Science department is feasible.
6.2 INSTRUCTOR PREFERENCES AUTOMATION
The instructor preferences page provides an easier method of communicating schedule information from the instructors to the chair. Instructors can quickly and easily update their information, and there is no need to “reinvent the wheel” every semester by filling in the same paper chart for preferences that haven’t changed. This page alone has value even separate from the schedule generation engine. The chair can even extract instructor information more easily, and it means one less stack of papers to keep track of in the chair’s office.
63
7 FUTURE ENHANCEMENTS
There are many possible future enhancements for this app.
7.1 AUTHENTICATION AND AUTHORIZATION
Currently the app is using Spring Security with hard coded values for authentication and authorization (See code snippet below). Spring Security is easily used with single sign on (SSO) systems. Providing integration with an external SSO for authentication and authorization seems to be a better choice than attempting to implement it as part of the app! Security issues are complex and continually changing. The Open Web Application
Security Project (OWASP) provides detailed guidelines on setting up authentication and authorization for a website (OWASP, 2017). For this project, it seems optimal to allow the installer to configure the desired SSO.
Hard-coded Values in Spring Security Configuration @Autowired public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception { auth.inMemoryAuthentication() .withUser("cov").password("****").roles("USER", "CHAIR"); auth.inMemoryAuthentication() .withUser("noga").password("****").roles("USER", "INSTRUCTOR"); auth.inMemoryAuthentication() .withUser("kaplan").password("****").roles("USER", "INSTRUCTOR"); }
64
7.2 WEB SOCKETS FOR ENGINE CONTROLS
The user interface to control the schedule generation engine was done quickly in order to complete this proof of concept. It is piggy backing a stateful engine communication over our RESTful (Representational state transfer) API (Application Programming Interface).
RESTful API’s are designed intentionally so that the server does not have to maintain state but instead each call is independent. Roy Fielding originated the idea of REST in his dissertation (Fielding, 2000) where he defined being stateless as one of the major constraints. He states, “each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is therefore kept entirely on the client.”
Our engine control is stateful. The server session object is used to store handles to the
OptaPlanner engine and there are API calls to start, stop and get the result of the engine.
A better design would be to implement web sockets to interact with the engine and keep the RESTful API only for stateless data-related calls. The socket would allow two-way communication between the engine on the server and the UI on the front end. As soon as the engine completes a run, it could inform the UI, and the UI would not need to rely on polling to get its state.
7.3 REMAINING CONSTRAINTS
I have only begun to implement the constraints in the list. It would be good to implement the lower priority constraints as well.
65
7.4 FEATURE TAGS FOR ROOMS AND COURSES
Another criterion for mapping a course to a room is the feature set of the room and the feature requirements of the course. Instead of hard-coding these features such as “has
Windows PCs” or “has Macs,” I envision a more generic tagging type of solution.
Rooms could be tagged and courses could be tagged. A constraint could be added to make sure the selected room has the feature tags required by the course.
7.5 MORE FLEXIBLE INSTRUCTOR TIME PREFERENCES
The initial version of representing instructor time preferences was challenging enough, but I can envision the desire for more complex preferences. During requirements elicitation, the following challenging conditions were discussed.
• I want to work MW or TR
• I want to work either an early shift or a late shift
Some of these could be a useful extension to this project. This would be very challenging.
66
REFERENCES
Bulatov, A., Krokhin, A., & Jeavons, P. (2000). Constraint Satisfaction Problems and
Finite Algebras. Automata, Languages and Programming (pp. 272-282). Berlin:
ICALP . doi:10.1007/3-540-45022-X_24
DeSmet, G. (2017). OptaPlanner Slides. (JBoss Community) Retrieved September 9,
2016, from OptaPlanner: https://www.optaplanner.org/learn/slides.html
E. Burke, K. J. (1997). Automated University Timetabling: The State of the Art. 40(9),
565-571. doi:10.1093/comjnl/40.9.565
Fielding, R. (2000). Architectural Styles and the Design of Network-based Software
Architectures. Retrieved from University of California, Irvine:
https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm
Fowler, M. (2004). Inversion of Control Containers and the Dependency Injection
pattern. Retrieved from MartinFowler.com:
https://martinfowler.com/articles/injection.html
Fowler, M. (2005). InversionOfControl. Retrieved from MartinFowler.com:
https://martinfowler.com/bliki/InversionOfControl.html
Gierke, O., Darimont, T., Strobl, C., & Paluch, M. (2017, 4 19). Spring Data JPA -
Reference Documentation. Retrieved from Spring: http://docs.spring.io/spring-
data/jpa/docs/current/reference/html/
J-Boss. (2017). Drools. Retrieved from Drools: https://www.drools.org/
67 jboss. (2017). OptaPlanner User Guide. Retrieved October 3, 2016, from OptaPlanner:
https://docs.optaplanner.org/6.5.0.Final/optaplanner-docs/html_single/index.html
OWASP. (2017, 4 21). Authentication Cheat Sheet. Retrieved from The Open Web
Application Security Project:
https://www.owasp.org/index.php/Authentication_Cheat_Sheet
Papa, J. (2017). Angular 1 Style Guide. Retrieved from John Papa:
https://github.com/johnpapa/angular-styleguide/blob/master/a1/README.md
Pivitol. (2017). Spring Boot Reference Guide. Retrieved from Spring:
https://docs.spring.io/spring-boot/docs/current-
SNAPSHOT/reference/htmlsingle/#getting-started-introducing-spring-boot
Pivotal Software. (n.d.). Spring Data. Retrieved from Spring:
http://projects.spring.io/spring-data/
S. C. Chu, H. F. (1999). Genetic Algorithms vs. Tabu Search in Timetable Scheduling.
1999 Third International Conference on Knowledge-Based Intelligent
Information Engineering Systems. Adelaide: IEEE.
doi:10.1109/KES.1999.820230
Unitime. (2017). University Timetabling. Retrieved September 5, 2016, from unitime.org:
http://www.unitime.org/
68
APPENDIX A – USER STORIES
Id Epic Title Description Priority Status 1 Import Import CSV As a dept chair I want to import 1-Must 1- from SOLAR schedule data from a csv file Completed generated by SOLAR so that I can run it against the timetabling engine. 2 Import Import directly As a dept chair I want to import 4-Wont 5-Future from SOLAR schedule data directly from SOLAR so that I can quickly run it against the timetabling engine. 3 Export Export CSV file As a dept chair I want to export 3-Could 4-Release schedule data to a csv file so that I can Backlog save it, work with it in excel or feed it into another program. 4 Export Export to As a dept chair I want to export the 4-Wont 5-Future SOLAR generated schedule directly into SOLAR, so that I do not have to enter the information manually. 5 Persistence Save to As a dept chair I want to be able to 2- 1- MongoDB create, read, update and delete my Should Completed work so that I can manage my schedule information. 6 Persistence Instructor As a dept chair I want to be able to 2- 1- CRUD create, read, update and delete the Should Completed instructor information so that I can keep my list of instructors and their properties up to date. 7 Persistence Course CRUD As a dept chair I want to be able to 2- 1- create, read, update and delete the Should Completed course information so that I can keep my list of courses and their properties up to date. 8 Persistence Room CRUD As a dept chair I want to be able to 2- 1- create, read, update and delete the Should Completed room information so that I can keep my list of rooms and their properties up to date. 9 Persistence Time Slot As a dept chair I want to be able to 2- 1- CRUD create, read, update and delete the Should Completed time slot information so that I can keep my list of time slots and their properties up to date. 10 OptaPlanner OptaPlanner As a developer, I want to understand 2- 1- examples Example (Spike) how the OptaPlanner engine works by Should Completed implementing and experimenting with the OptaPlanner example files so that I can change, compile, build and run the scheduling engine.
69
11 Optaconf Optaconf As a developer, I want to understand 2- 1- Example (Spike) how the OptaPlanner engine works by Should Completed implementing and experimenting with the optaconf example files so that I can see a different model for implementation. 12 Time slots Time slots As a dept chair, I want to be able to 2- 1- instead of schedule sections to flexible time slots Should Completed Periods which potentially overlap rather than fixed periods so that I can more closely model my real scheduling needs. 13 Engine Solve button As a dept chair, I want to be able to 2- 1- Control start the engine solving the problem Should Completed from its current state, so that I can let it generate a better schedule. 14 Engine Stop button As a dept chair, I want to be able to 2- 1- Control stop the running engine so that I can Should Completed see the partial results at any time that I want. 15 Engine Show button As a dept chair, I want to be able to 2- 1- Control see the results of the engines solution, Should Completed so that I can update the schedule of classes. 16 Solve for Add Instructor As a dept chair, I want the engine to 2- 3-Sprint Instructor map instructors to sections as well as Should Backlog rooms and timeslots, so that the instructor preference can be considered in schedule generation. 17 Refine Refine As a dept chair, I want the engine to 2- 3-Sprint Constraints Constraints generate a "good" schedule in a Should Backlog relatively short time, so that the tool is actually useful in generating useable schedules. 18 Clean UI Clean UI As a dept chair, I want the user 2- 3-Sprint experience of the app to be decent, so Should Backlog that I prefer using the tool to just going back to excel.
70
APPENDIX B – CURRENT INSTRUCTOR PREFERENCES PAPER MEMO
.
Figure 26 Instructor Preferences Memo Page 1
71
.
Figure 27 Instructor Preferences Memo Page 2
72