Distributed Event Handler Creation of a distributed event handler with priority handling and synchronized timing ROBIN OLAUSSON JIMMY WESTERLUND Degree project in Information and Software systems Stockholm, Sweden 2013 TRITA-ICT-EX-2013:163 Distributed Event Handler Creation of a distributed event handler with priority handling and synchronized timing ROBIN OLAUSSON, JIMMY WESTERLUND Bachelor Thesis at ICT Examiner: Johan Montelius (School of ICT) Abstract During the start up of a project, we found out that an event handler was needed. This event handler needed to be redundant and able to ensure that events with dependencies to each other, where executed in a specified correct order. Policies where written to ensure that the end result, if able to fulfill these, was the product that was needed. The result is an event handler written in Erlang, which ensures that all events given to it will be executed in the order specified. Even though not fulfilling all policies there are plans for future work with solutions on how this will be fixed. In conclusion, even though the event handler is not a complete product, this report presents a solution to the general event handling problem. Referat Ditribuerad Händelsehanterare Under uppstarten av ett projekt fick vi reda p˚aatt det beh¨ovdes en h¨andelsehanterare. H¨andelsehanteraren skulle vara redundant och kunna s¨akerst¨alla att h¨andelser beroende av varandra, k¨ordes i den ordning som specifiserats. Policies skrevs ner f¨or att f¨ors¨akra att om h¨andelsehanteraren kunde uppfylla dessa, var detta produkten som efters¨oktes. Re- sultatet blev en h¨andelsehanterare skriven i Erlang, vilken f¨ors¨akrar att samtliga event givna, blev utf¨orda i den spec- ifierade ordningen. Trots att den inte uppfyller samtliga policies, finns det planer f¨or hur man i framtiden skall kun- na l¨osa dessa problem. Sammanfattningsvis, trots att den framtagna h¨andelsehanteraren inte ¨ar den kompletta pro- dukten, presenterar ¨and˚adenna rapport en l¨osning p˚adet generella h¨andelsehanteringsproblemet. Contents 1 Introduction 1 1.1 Background . 1 1.2 Problem . 2 1.2.1 Specification . 2 1.2.2 Policies . 2 1.3 Purpose . 3 1.4 Goal . 3 1.5 Boundaries . 3 1.6 Outline . 4 2 Background 5 2.1 Erlang . 5 2.1.1 Distribution . 5 2.1.2 Concurrency . 5 2.1.3 Fault tolerance . 5 2.1.4 Hot swapping . 5 2.2 Erlang OTP - Open Telecom Platform . 6 2.2.1 OTP behaviors . 6 2.3 Erlang supervisors . 6 2.4 Mnesia . 7 3 Event handler 9 3.1 Overview . 9 3.1.1 Event definition . 9 3.1.2 Event handler . 10 3.2 Clock component . 13 3.2.1 Module: clock . 13 3.3 Pipe component . 17 3.3.1 Module: event queue . 18 3.3.2 Module: scheduler . 21 3.3.3 Module: load balancer . 24 4 Analysis 29 4.1 Questions that need answers . 29 5 Conclusion 33 5.1 Comparing the product to the policies . 33 5.1.1 Event handling . 33 5.1.2 Fault tolerance and Maintenance . 33 5.1.3 Auto scalability . 34 5.1.4 Time and Timing . 34 5.2 Future work . 34 5.2.1 Supervision . 34 5.2.2 Scalable Event Handler . 35 5.2.3 Worker node . 35 5.2.4 Release . 35 Appendices 35 A Source code for module: clock.erl 37 B Source code for module: event queue.erl 43 C Source code for module: scheduler.erl 47 D Source code for module: load balancer.erl 51 Chapter 1 Introduction This introduction will inform the reader on the difference between game servers and ordinary servers. It will also bring up and explain the problem we are trying to solve, with the specification and policies of the task handling server module we see as a solution. 1.1 Background Today many game developers make their games as online games. These online games make it possible for more than one person to interact with the same game. The online games of today used a centralized server [3]. The centralized server connects the players through the Internet which enables them to play the game together. As in a centralized case the focus in this project will be to be able to handle a lot of players. This sort of servers are also sometimes referred to as a multiplayer server. The architecture of a multiplayer server follows the Client-Server architecture, meaning it is a setup of clients and a centralized server. Clients send requests to the server and it responses to the clients. Even though ordinary online servers and online game servers are similar, there are some differences. An ordinary online server, in example a web server, only handles clients request, and event though it may be a complex state machine, it will only change between states when it's requested by a client request. Game servers may also be complex state machines, but it can be able to change states without having to get a request from a client, since game servers can have internal events that changes the state. These internal events can be caused by player interaction through the client, game mechanisms or game rules. The work required to handle these, often interactive events, is complex since there are rules which requires events to be executed in specific orders. For the server to be able to work as intended a redundant event handling module have to exist. 1 CHAPTER 1. INTRODUCTION 1.2 Problem In another project we are a part of, the groups are trying to create a large scale multiplayer online web browser game. To create a game like this we soon realized that we would need to build or find some sort of event handler. We came up with the idea of trying to build the event handler ourselves, since this would give us good knowledge about event handling while enabling us to form the event handler after our own needs. The first problem we need to solve is the event modules general work flow. Since it's impossible to figure out the amount of people that will play the game, the event handler will probably need some sort of scalability. The scalability will enable us to easily scale up or down the module if needed. Another problem we will need to figure out is how to make the module redundant. 1.2.1 Specification The specification that follows is a general description of the module's functionality. The module must follow this description in order to work as intended. Since the event handler is a module that is part of a bigger system, it needs to have some sort of connection to the rest of the server application. This connection must be reachable at all times and will be used for gathering events, sent by the rest of the system. Events that are sent to the module are received through an internal front-end. The events must contain information specifying the time when the event shall be executed. The front-end will place received events in a queue, which will keep events until the specified time occurs. When this happens, the events will be sent on to the worker nodes. A worker node is an internal service which will execute any sort of event given to it. For the queue to know what worker node that shall have each event, there must be a scheduler to schedule events over worker nodes. This scheduler will work as a load balancer and both divide work between the nodes and place events that are dependent of each other to be executed in the same worker node. 1.2.2 Policies The distributed real time event handler we decided to create, had the following policies. Event handling The module have to ensure that all events it has been given will be executed. Every event has to be executed at the time specified in it. The module have to ensure that all events will be executed in synchronized order. The synchro- nized order is based on events relations and priority. 2 1.3. PURPOSE Fault tolerance and Maintenance The systems redundancy must fulfill the requirement of the \no downtime" policy. \No downtime" policy means that the system must be fully fault tolerant. During a maintenance phase, the system must still be able to work as it did before the module was taken down for maintenance. Meaning that another module must be able to take over the work of the module taken down. Auto scalability Scalability makes it possible for the system to handle large amount of work [2]. Due to distribution over several machines. This scalability should also work dynamically so that machines can be called out or in during run time. This corresponds to an elastic distributed system [14]. The system must behave like this due to the variation of workload. Time and timing All events that have been scheduled for execution, must be executed at the correct time. If an event would be late, all other event must wait until the late event have been executed correctly. This is to make sure that events, related to earlier scheduled events, can execute correctly. An example of such a relation is a gun being loaded and then fired. The gun cannot be fired if the loading event have not yet been completed. 1.3 Purpose The purpose of this report is to present how a scalable redundant event handling module has been built, which can be used in any large scale online strategy game. The event handling module's capabilities will be focused around three different parts.