A Proxy Server-Network for Real-Time Computer Games Jens M¨uller1,StefanFischer2,SergeiGorlatch1, and Martin Mauve3 1 Westf¨alische Wilhelms-Universit¨at M¨unster, Germany 2 Technische Universit¨at Braunschweig, Germany 3 Heinrich-Heine-Universit¨at, D¨usseldorf, Germany Abstract. Computer games played over the Internet have recently be- come an important class of distributed applications. In this paper we present a novel proxy server-network topology aiming at improved scala- bility of multiplayer games and low latency in client-server data transmis- sion. We present a mechanism to efficiently synchronize the distributed state of a game based on the concept of eventual consistency. We analyse the benefits of our approach compared to commonly used client-server and peer-to-peer topologies, and present first experimental results. 1 Introduction Computer games played over a network are categorized in the three main genres: First Person Shooter Games (FPS), Real-time Strategy Games (RTS) and Mas- sive Multiplayer Online Roleplaying Games (MMORPG). For MMORPG, there exist several advanced networking topologies, which can support a high number of participating clients [1]. In contrast, the basic concepts of network commu- nication for FPS and RTS games have not been altered for years. Commonly used architectures like client-server and peer-to-peer do not provide the required scalability for massive-multiplayer sessions. Although other communication ar- chitectures like peer-to-peer server-networks or hierarchical server-networks ex- ist, their efficient usage for FPS and RTS games is still not well understood (an overview can be found in [2]). In this paper, we present our work on a Proxy Server-Network for Real-time Computer Games. Our main goal is to improve network scalability for FPS and RTS games in comparison to traditional topologies, thus allowing games with higher numbers of players. A further objective is to enhance responsiveness of massive-multiplayer, Internet based games by providing low communication la- tency between clients and servers. The paper is organized as follows: In Sect. 2 we introduce our architecture and propose a concept for an efficient synchro- nisation of the distributed game state. We describe our implementation of the architecture as a C++ API and present experimental results on its scalability in Sect. 3. Section 4 compares our approach with related work and concludes the paper. M. Danelutto, D. Laforenza, M. Vanneschi (Eds.): Euro-Par 2004, LNCS 3149, pp. 606–613, 2004. c Springer-Verlag Berlin Heidelberg 2004 A Proxy Server-Network for Real-Time Computer Games 607 2 Architecture Commonly used communication architectures like client-server or peer-to-peer topologies provide a variety of positive aspects and can be implemented quite easily, which leads to their high acceptance for computer games. However, there are several disadvantages in both concepts, resulting in poor quality of service and inhibiting novel game designs with a higher number of participating users. Client-server and peer-to-peer topologies do not scale well when the number of users increases. Additionally, the server in a client-server setup forms a single point of failure for the whole session. While the peer-to-peer approach eliminates the problem of a single point of failure, it allows a hacked client to cheat, since submitted game updates are not filtered by a server instance and hidden information can be made visible to the player [3]. 2.1 The Proxy Server-Network The proposed proxy server architecture, which is derived from a peer-to-peer server-network [2], is shown in Fig. 1. The initial idea was sketched in [4]. clients clients [...] [...] proxy server proxy server ISP 1 client proxyserver communication inter proxyserver communication [...] [...] clientsproxy server proxy server clients ISP 2 ISP 3 Fig. 1. Example session using the proxy server architecture. The key idea of our architecture is the use of several interconnected proxy servers for a single game session. Each proxy server has a full view of the global game state, allowing it to serve arbitrary clients. Each client communicates with a single proxy to send user actions and receive updates of the game state. Proxy servers process user actions and forward them to other proxies using multicast (IP or application level) for synchronizing the distributed game state. For the sake of low latency in Internet based sessions, proxy servers should be located at different Internet Service Providers (ISP), such that each client connects to a proxy at its ISP. In our current implementation, the setup of 608 J. M¨uller et al. servers is done manually, but future work includes a dynamic setup of proxies respecting current user demand. MMORPG commonly use an approach orthogonal to ours: Instead of repli- cating the game world, they partition it among servers, which forces clients to change the server when the user moves into another region. While this approach works well for the relatively slow game flow in MMORPG, it cannot be applied to fast paced games like FPS and RTS, in which recurrent pauses resulting from server changes are annoying for users. Since each proxy has full view of the game state, our proxy server-network avoids reconnects during a session. As we show in Sect. 3.1, the described proxy network scales better with an increasing number of clients than traditional topologies. 2.2 Managing the Distributed Game State Since the game state is replicated at the proxy servers, our architecture requires mechanisms to provide adequate consistency of the replicated data. There is a classical trade off between scalability, responsiveness, and con- sistency in distributed simulations in general. Implementing strong consistency models like causal or even sequential consistency [5] would increase the amount of inter-proxy communication: Proxies would have to order changes of the game state using Lamport timestamps or physical clock synchronisation. This would delay the transmission of acknowledgements of user actions to clients, thereby reducing the responsiveness of the game. As reported in [6], a scalable distributed system with real-time performance is only possible if not all participating pro- cesses share the same view on common data at one time. Therefore, we propose to use the concept of eventual consistency [5] for the game state. To implement eventual consistency, only a single process is allowed to alter specific parts of replicated data. Changes have to be propagated to other processes, ensuring consistency of the replicated state as soon as the update notification arrives. (5) remote proxy updates (5) remote proxy updates local game state local game state and checks interactions and checks interactions (6) proxies submit results of interaction evaluation [...] (4) proxy notifies [...] other servers (7) proxy informs (2) proxy validates (7) proxy informs affected clients user input and affected clients updates local game state on success (1) client submits (3) proxy sends user action acknowledgement Fig. 2. Acknowledging and forwarding a single user action. A Proxy Server-Network for Real-Time Computer Games 609 The main steps in our concept of transmitting and processing a single user action in the proxy server-network are depicted in Fig. 2. User actions trans- ferred from clients (1) are categorized in movements and interactions.Incaseof movement commands, the server has to check if the user input is legal to prevent cheating before changing the state (2), and to inform other proxies about up- dated position values (4), which in turn update the local copies accordingly (5). In this way eventual consistency for the changed part of the game state is guar- anteed and the clients receive acknowledgments for movement commands in a relatively short time (3). In case of interactions, notified proxies additionally have to check whether local clients are affected. If so, the responsible proxy up- dates the game state of its local client, and informs other servers in turn (6). For all state updates received from other proxy servers, each proxy evaluates which local clients are affected and informs them (7). In our architecture, general parts of the game state that are not associated with a specific client are assigned to single proxy servers for maintenance, which ensures eventual consistency for these parts of the game state as well. Such data contain manipulable parts of the gameworld like collectible items or avatars not controlled by the users (Non-Player Characters). To avoid a single point of failure, applications have to implement a proper election method to reassign responsibility to another proxy in case of a server failure. For data representing states of clients, a reassignment of responsibility is not required: If a proxy process fails, clients connected to it will loose connection to the session anyway, while for all other processes the game will continue to be playable. The presented concept for managing the distributed state allows an efficient synchronisation of the game state in combination with fast acknowledgment of user actions. However, due to the use of eventual consistency, execution of commands can occur in a different order than their initation by the users. In the following, we discuss the impact of this fact on the playing conditions. 2.3 Discussion of Game Correctness A computer game provides equal game opportunities for all users, if the order of processed state updates is the same as the order of their initiation by the users. Additionally, in continuous simulations like real-time games, the point in time at which actions are processed has to be considered, too. Let us consider an example interaction between two clients, c1 and c2, in a real-time game: The user at c1 issue an action affecting c2 (e. g. shooting at the avatar managed by c2 with an instant-hit weapon), while the user at c2 issues a movement command some time later which would nevertheless prevent his avatar from being affected if processed before the action of the user at c1.