#LyX 1.6.7 created this file. For more info see http://www.lyx.org/ \lyxformat 345 \begin_document \begin_header \textclass report \begin_preamble \usepackage{indentfirst} \usepackage{tocloft} \usepackage{calc} \date{} \usepackage[section] {placeins} \def\myClearpage{% \ifvmode \ifnum \@dbltopnum =\m@ne \ifdim \pagetotal <\topskip \hbox{}% \fi \fi \fi % \newpage \write\m@ne{}% \vbox{}% \penalty -\@Mi } \def\myCleardoublepage{\myClearpage\if@twoside \ifodd\c@page\else \hbox{}\if@twocolumn\hbox{}\fi\fi\fi} \usepackage{morefloats} \usepackage{graphicx} \usepackage{subfig} \usepackage{tocloft} \renewcommand{\cftchapfont}{\bfseries} \renewcommand{\cftchappagefont}{\bfseries} \renewcommand{\cftchappresnum}{Chapter } \renewcommand{\cftchapnumwidth}{6em} \oddsidemargin 0.5in \textwidth 6in \topmargin 0.0in \textheight 8.0in \setlength\topskip{24pt} \footskip 0.75in \usepackage[compact]{titlesec} \titleformat{\chapter}[display] {\vskip-8em\normalfont\bfseries} {\LARGE\raggedright\thechapter} {14ex} {\vspace{-20ex}% \LARGE\raggedleft} [\vspace{1ex}% {\titlerule[1pt]}] \usepackage[absolute]{textpos} \usepackage{fancyheadings} \pagestyle{fancy} \lhead{\thechapter} \usepackage[hmargin=3cm,vmargin=3.5cm]{geometry} \usepackage{algorithmic} \end_preamble \use_default_options true \language english \inputencoding auto \font_roman lmodern \font_sans default \font_typewriter default \font_default_family rmdefault \font_sc false \font_osf false \font_sf_scale 100 \font_tt_scale 100 \graphics default \paperfontsize default \spacing onehalf \use_hyperref false \papersize a4paper \use_geometry false \use_amsmath 1 \use_esint 1 \cite_engine natbib_authoryear \use_bibtopic false \paperorientation portrait \paperwidth 11in \paperheight 8.5in \leftmargin 1.25in \topmargin 0in \rightmargin 1in \bottommargin 1.7in \secnumdepth 2 \tocdepth 2 \paragraph_separation indent \defskip medskip \quotes_language english \papercolumns 1 \papersides 1 \paperpagestyle empty \tracking_changes false \output_changes false \author "" \author "" \end_header \begin_body \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{textblock*}{ \backslash paperwidth}(0mm,40mm) \end_layout \begin_layout Plain Layout \backslash begin{center} \end_layout \begin_layout Plain Layout \backslash includegraphics[width= \backslash paperwidth/2]{graphics/logo} \end_layout \begin_layout Plain Layout \backslash end{center} \end_layout \begin_layout Plain Layout \backslash end{textblock*} \end_layout \end_inset \end_layout \begin_layout Title Reliable Generic Game Server \end_layout \begin_layout Author Niklas Landin \begin_inset Newline newline \end_inset Richard Pannek \begin_inset Newline newline \end_inset Mattias Pettersson \begin_inset Newline newline \end_inset Jonatan Pålsson \end_layout \begin_layout Abstract This is the abstract! \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash pagenumbering{roman} \end_layout \begin_layout Plain Layout \backslash setcounter{page}{3} \end_layout \begin_layout Plain Layout \backslash renewcommand \backslash contentsname{Table of Contents} \end_layout \begin_layout Plain Layout \backslash renewcommand{ \backslash cfttoctitlefont}{ \backslash hfill \backslash Large} \backslash renewcommand{ \backslash cftaftertoctitle}{ \backslash hfill} \end_layout \begin_layout Plain Layout \backslash renewcommand \backslash cftpartdotsep{6.6} \end_layout \begin_layout Plain Layout \backslash renewcommand \backslash cftchapdotsep{6.6} \end_layout \end_inset \begin_inset CommandInset toc LatexCommand tableofcontents \end_inset \end_layout \begin_layout Standard \begin_inset Newpage newpage \end_inset \end_layout \begin_layout Standard \begin_inset ERT status open \begin_layout Plain Layout \backslash pagenumbering{arabic} \end_layout \begin_layout Plain Layout \backslash setcounter{page}{1} \end_layout \end_inset \end_layout \begin_layout Chapter Introduction \end_layout \begin_layout Standard Online gaming, and computer gaming in general has become an important part in many peoples day-to day lives. A few years ago, computer games were not at all as popular as they are today. With the advances in computer graphics and computer hardware today's games are much more sophisticated then they were in the days of \emph on Hack \emph default , \emph on Zork, \emph default or \emph on Pac Man. \emph default \begin_inset Note Note status open \begin_layout Plain Layout Was it really called Zork? \end_layout \end_inset \end_layout \begin_layout Standard The early computer games featured simple, or no graphics at all. \begin_inset Note Note status open \begin_layout Plain Layout Drop a reference to Hack \end_layout \end_inset The games often took place in a textual world, leaving the task of picturing the world up to the player. Multi-player games were not as common as they are today, whereas most games today are expected to have a multi-player mode, most early games did not. \end_layout \begin_layout Standard Since these early games, the gaming industry have become much more influential in many ways. Many advanced in computer hardware are thought to come from pressure from the computer game industry. More powerful games require more powerful, and more easily available hardware \begin_inset Note Note status open \begin_layout Plain Layout Drop a reference to the gaming industry pressuring more advanced hardware \end_layout \end_inset . Due to the high entertainment value of modern computer games, gaming has become a huge industry, where large amounts of money are invested. The gaming industry is today, in some places even larger than the motion picture industry. \begin_inset Note Note status open \begin_layout Plain Layout Drop a reference to that graphic we found, about the motion picture industry VS. gaming industry \end_layout \end_inset \end_layout \begin_layout Standard Due to the increasing importance of computer gaming, more focus should be spent on improving the quality of the gaming service. As more and more computer games are gaining multi-player capabilities, the demands for multiplayer networking software rises. This thesis is about techniques for improving the quality if this networking software. \end_layout \begin_layout Standard The reliable generic game server, hereafter known as GGS, is a computer program designed to \emph on host \emph default network games on one or more server computers. Hosting, in a network software setting, means allowing client software connect to the server software, for the purpose of utilizing services provided by the server. The GGS software provides games as a service, and the clients connecting to the GGS can play these games on the GGS. \end_layout \begin_layout Standard The idea of game servers is not new, network games have been played for decades. Early, popular examples of network games include the \emph on Quake \emph default series, or the \emph on Doom \begin_inset Note Note status collapsed \begin_layout Plain Layout \emph on Come up w/ better game \end_layout \end_inset \emph default games. Newer examples of network games include \emph on World of Warcraft \emph default , and \emph on Counter-Strike \emph default . The difference between the GGS and the servers for these games is that the servers for Doom, Quake, and the others listed, were designed with these specific games in mind. \end_layout \begin_layout Standard What GGS does is to provide a \emph on generic \emph default framework for developing network games. The framework is generic in the sense that it is not bound to a specific game. There are many different types of games, some are inheritly more time sensitive than others, strategy games are examples of games which are not very sensitive to time delays, first-person shooters however, can be very sensitive. \end_layout \begin_layout Standard The generic nature of the GGS allows the creation of many different types of games, the motivation behind this is to remove the neccessity of writing new game servers when developing new games. \end_layout \begin_layout Standard The GGS is in addition to being generic, also \emph on reliable \emph default in the sense that the gaming service provided is consistant and available. A consistant and available server is a server that handles hardware failiures and software failiures gracefully. In the event of a component breaking within the GGS, the error is handled by fault recovery processes, thereby creating a more reliable system. \end_layout \begin_layout Section Background \end_layout \begin_layout Standard The game industry is a quickly growing industry where the need for new technique s is large. One specific section where the development has stalled is the game server section. \begin_inset Note Note status open \begin_layout Plain Layout Citation needed \end_layout \end_inset The existing game servers are functional but they lack good fault tolerance and the ability to scale well \begin_inset Note Note status open \begin_layout Plain Layout Citation needed \end_layout \end_inset . Users will notice this in low uptime and many crashes. This is a problem that has existed and been resolved in other industries. In the telecom industry solutins to similar problems have been found. \end_layout \begin_layout Standard A common figure often used in telecoms is that of \emph on the nine nines \emph default , referring to \begin_inset Formula $99.999999999\%$ \end_inset of availability, or rougly \begin_inset Formula $15ms$ \end_inset downtime in a year. The level of instability and bad fault tolerance seen in the game server industry would not have been accepted in the telecoms industry. This level of instability should not be accepted in the game server industry either. An unavailabvle phone system could potentially have life threatening consequenc es, leaving the public unable to contant emergency services. The same can not be said about an unavailable game server. The statement that game servers are less important than phone systems is not a reason not to draw wisdom from what the telecoms have already learnt. \end_layout \begin_layout Standard Moving back to the gaming industry. The main reason to develop reliable servers are monetary, it is important for game companies to expand its customer base. Reliable game servers are one improvement that will create a good image of a company. In general the downtime of game servers are much higher than the downtime of telecom system. The structure of the system is similar in many ways and it should be possible to reuse solutions from the telecom system to improve game servers. \end_layout \begin_layout Standard In the current state game servers are developed on a per-game basis, in many cases this seems like a bad solution. Developers of network game need to understand network programming. A way to change this is a generic game server which give the game developers a server which they implement their game towards. This approach would not only make it easier to develop network games, it would also allow games in different programming languages to be implemented using the same server. \end_layout \begin_layout Standard Some factors key to the development of GGS have been isolated. Many of these come from the telecom sector. The factors are \emph on scalability, fault tolerance \emph default and being \emph on generic \emph default . These terms are defined below. \end_layout \begin_layout Standard Scalability (see \begin_inset CommandInset ref LatexCommand ref reference "sec:Scalability" \end_inset ) in computer science is a large topic and is commonly divided into sub-fields, two of which are \emph on structural scalability \emph default and \emph on load scalability \emph default \begin_inset CommandInset citation LatexCommand citet key "Bondi:2000:CSI:350391.350432" \end_inset . These two issues are addressed in this thesis. Structural scalability means expanding an architecture, e.g. adding nodes to a system, without requiring modification of the system. Load scalability means using the available resources in a way which allows handling increasing load, e.g more users, gracefully. \end_layout \begin_layout Standard Fault tolerance (see \begin_inset CommandInset ref LatexCommand ref reference "sec:Fault-Tolerance" \end_inset ) is used to raise the level of \emph on dependability \emph default in a system, so that the dependability is high even in presence of errors. Dependability is defined as the statistical probability of the system functioni ng as intended at a given point in time. Fault tolerance is defined as the property of a system to always follow a specification, even in the presence of errors. The specification could take the form of error handling procedures which activate when an error occurs. This means that a fault tolerant, dependable system, will have a very high probability of functioning at a given point in time, and is exactly what is desired. \begin_inset CommandInset citation LatexCommand citet key "Gartner:1999:FFD:311531.311532" \end_inset \end_layout \begin_layout Standard A generic (see \begin_inset CommandInset ref LatexCommand ref reference "sec:Generic" \end_inset ) game server has to be able to run different client-server network games regardless of the platform the clients are running on. It runs network games of different type. A very rough separation of games is real time games and turn based games. \end_layout \begin_layout Standard The server behaves in a way similar to an application server, but is designed to help running games. An application server provides processing ability and time, therefore it is different from a file- or print-server, which only serves resources to the clients. \end_layout \begin_layout Standard The most common type of application servers are web servers, where you run a web application within the server. The application server provides an environment and interfaces to the outer world, in which applications run. Hooks and helpers are provided to use the resources of the server. Some examples for web application servers are the \emph on Glassfish \emph default server which allows running applications written in Java or the \emph on Google App Engine \emph default where you can run applications written in Python or some language which runs in the \emph on Java Virtual Machine \emph default . An example of an application server not powering web applications, but instead regular business logic, is Oracle’s \emph on TUXEDO \emph default application server, which can be used to run applications written in COBOL, C++ and other languages. \end_layout \begin_layout Standard A database server can also be seen as an application server. Scripts, for example SQL queries or JavaScript, are sent to the server, which runs them and returns the evaluated data to the clients. \end_layout \begin_layout Standard One of the purposes of this thesis is to investigate how we can make a game server as generic as possible. Some important helpers are discussed, such as abstraction of the network layer, data store and game specific features. \end_layout \begin_layout Standard As an aid in discussing the theoretical parts of the GGS a prototype has been developed. The prototype does not feature all of the characteristics described in this thesis. A selection has been made among the features, and the most important ones have been implemented either full or in part in the prototype. \end_layout \begin_layout Standard The choice of implementation language for the prototype of the GGS was made with inspiration from the telecom industry. The Erlang language was developed by the telecom company Ericsson to develop highly available and dependable telecom switches. One of the most reliable systems ever developed by Ericsson, the AXD301 was developed using Erlang. The AXD301 is also possibly has the largest code base even written in a functional language \begin_inset CommandInset citation LatexCommand citep key "Armstrong03" \end_inset . The same language is used to develop the prototype of the GGS. Chapter \begin_inset CommandInset ref LatexCommand vref reference "cha:Implementation-of-a" \end_inset provides a description of the prototype developed for this thesis. \begin_inset Note Note status open \begin_layout Plain Layout We could go on and on about erlang.. \end_layout \end_inset \end_layout \begin_layout Section Purpose \end_layout \begin_layout Standard \begin_inset Note Note status collapsed \begin_layout Plain Layout The purpose of the GGS project. What is the purpose of creating a fault tolerant server, why is it generic, and what good does it do to have a scalable system? In the background, we should give the motivations behind creating this software, but here we should outline why the software is needed. This section should be shortened, and the bulk of the text should be moved to theory or background. \end_layout \end_inset \end_layout \begin_layout Standard The purpose of creating a generic and fault tolerant game server is to provide a good framework for the development of many different types of games. Allowing the system to scale up and down is a powerful way to maximize the usage of physical resources. By scaling up to new machines when load increases, and scaling down from machines when load decreases costs and energy consumption can be optimized. \end_layout \begin_layout Standard Fault tolerance is important for the GGS in order to create a reliable service. The purpose of a reliable game server is to provide a consistant service to people using the server. Going back to the telecom example, the purpose of creating a reliable telecom system is to allow calls, possibly emergency calls, at any time. Should the telecom network be unavailable at any time, emergency services may become unavailable, furthermore the consumer image of the telecom system degrades. \end_layout \begin_layout Standard Returning to the game industry, emergency services will not be contacted using a game server, however an unavailable server will degrade the consumer image of the system. Consider an online casino company. The online casino company's servers must be available at all times to allow customers to play. If the servers are unavailable, customers can not play, and the company loses money. In this scenario, an unavailable server can be compared to a closed real-world casino. \end_layout \begin_layout Section Challenges in developing the prototype \end_layout \begin_layout Standard The word \emph on generic \emph default in the game of the GGS implies that the system is able to run a very broad range of different code, for instance code written in different programming languages, in addition to a broad range of different game types. In order to support this, a virtual machine (VM) for each \emph on game development language \emph default (hereafter GDL for brevity) is used. \end_layout \begin_layout Standard No hard limit has been set on which languages can be used for game development on the GGS, but there are several factors which decide the feasibility of a language; \end_layout \begin_layout Itemize How well it integrates with Erlang, which is used in the core the GGS system \end_layout \begin_layout Itemize How easy it is to send messages to the virtual machine of the GDL from the GGS \end_layout \begin_layout Itemize How easy it is to send messages from the GDL VM to the GGS \begin_inset Note Note status open \begin_layout Plain Layout Add more like threads, events, etc. \end_layout \end_inset \end_layout \begin_layout Standard Internally, the GDL VM needs to interface with the GGS to make use of the helpers and tools that the GGS provides. Thus an internal API has to be designed for use in interacting with the GGS. This API is ideally completely independent of the GDL, and reusable for any GDL. \end_layout \begin_layout Standard The communication with gaming clients has to take place over a protocol. Ideally a standard protocol should be used, in order to shorten the learning curve for developers, and also make the system as a whole less obscure. A large challenge during this project is to decide whether an existing protocol can be used, and if not, how a new protocol can be designed which performs technically as desired, while still being familiar enough to existing developers. \end_layout \begin_layout Standard A great deal of work is devoted to make the GGS \emph on reliable \emph default . This includes ensuring that the system scales well, and to make sure it is fault tolerant. In order to facilitate scalability, we need a storage platform which is accessible and consistent among all of the GGS, this is also investigated. \end_layout \begin_layout Section Limitations of the prototype \end_layout \begin_layout Standard The implementation of the GGS protocol, together with storage possibilities, server capacity, and game language support imposes some limitations on the project. To get a functional prototype some limits must be set on the types games that can be played on the prototype. \end_layout \begin_layout Standard The UDP protocol is not supported for communication between client and server. The TCP protocol was chosen in favour of UDP, due to the fact that the implementation process using TCP was faster than if UDP would have been used. UDP is generally considered to be faster than TCP for the transfer of game (and other) related data, this is discussed in more depth in \begin_inset CommandInset ref LatexCommand vref reference "sec:Choice-of-network" \end_inset . In short, the decision of using TCP means that games that requires a high speed protocol will not be supported by the GGS prototype. Another limitation necessary to set on the system is the possibility to have huge game worlds due to the implementation of the scaling mechanism in the prototype. \end_layout \begin_layout Standard In real time games all players are playing together at the same time. Latency is a huge problem in real time games, a typical round trip time for such games is one of \begin_inset Formula $50$ \end_inset to \begin_inset Formula $150ms$ \end_inset and everything above \begin_inset Formula $200ms$ \end_inset is reported to be intolerable \begin_inset CommandInset citation LatexCommand citet key "Farber:2002:NGT:566500.566508" \end_inset . Latency sensitive games include most of the first person shooters with multiplayer ability, for example \emph on Counter Strike \emph default or massively multiplayer online role playing games (MMORPG:s), for example \emph on World of Warcraft \emph default . \end_layout \begin_layout Standard In turn based games each player has to wait for their turn. Latency is not a problem since the gameplay does not require fast interactions between the players, long round trip times will not be noticed. Examples of turn based games include board and card games, as well as multiplay er games like \emph on Jeopardy \emph default . Both game types have varying difficulties and needs when it comes to implementi ng them, a Generic Game Server should address all of them and help the developer to accomplish his goal. \end_layout \begin_layout Standard Due to the limited capability of threading in many GDL VM:s, the GGS prototype will not support MMORPG:s. \end_layout \begin_layout Standard The implementation of the GGS described in this thesis is only a small prototype and tests will be performed on simple games like pong or chess, thus there are no need to implement more advanced features in the system. It is important to note that these limitations only apply for the prototype of the project, and that further developments to the GGS could be to implement these features. \end_layout \begin_layout Section Method \end_layout \begin_layout Subsection Development process \end_layout \begin_layout Standard A prototype was developed early on in the project in order to carry out experiments. Using this prototype, the system was divided into modules. A demand specification was created, using this specification, the modules were easily identifiable. \end_layout \begin_layout Standard The first prototype of the GGS consisted of simple modules, however, due to the separation of concerns between the modules, they were easily independant ly modified and improved. \end_layout \begin_layout Standard Once the basic structure of the GGS had been established, the first prototype was removed, remaining was the structure of the modules and the internal flow of the application. This could be seen as an interative workflow, with the first prototype being the first iteration. The second iteration later became the final result of the GGS \end_layout \begin_layout Subsection Design \end_layout \begin_layout Standard The layout of the GGS is both layered and modular. The first layer handles the most primitive data and produces a higher level representation of the data, passing it along to different modules of the GGS. The modular structure of the GGS plays an important role in making the system fault tolerant. The approach to fault tolerance is by replication, and restarting faulting modules with the last known good data. \end_layout \begin_layout Standard \begin_inset Note Note status open \begin_layout Plain Layout Perhaps a graphic of this modular layout..? \end_layout \end_inset \end_layout \begin_layout Subsection Testing and evaluation \end_layout \begin_layout Standard Can we use quickcheck? \end_layout \begin_layout Chapter Theory behind the GGS \begin_inset CommandInset label LatexCommand label name "cha:Theory" \end_inset \end_layout \begin_layout Standard In this chapter, the theory behind the techniques used in the GGS are discussed. Performance issues and the measuring of performance is discussed. Benchmarking techniques are discussed. The options when choosing network protocols are given, along with a discussion of each alternative. Finally, a overview of scalability, fault tolerance and availability is presented. \end_layout \begin_layout Section Design of the GGS system \begin_inset CommandInset label LatexCommand label name "sec:Design-of-the" \end_inset \end_layout \begin_layout Standard The GGS is modelled after a real world system performing much of the same duties as GGS. This is common practice \begin_inset CommandInset citation LatexCommand citep key "armstrong2011" \end_inset in the computer software world, in order to understand complex problems more easily. While there may not always be a real world example of a system performing th exact duties of the system being modelled in the computer, it is often easier to create and analyze requirements for real world systems and processes than systems existing soley in a computer. The requirements and limitations imposed on the real-world system can, using the proper tools, be transferred in to the software. \end_layout \begin_layout Standard The real world system chosen for the GGS is a \begin_inset Quotes eld \end_inset Chess club \begin_inset Quotes erd \end_inset - a building where chess players can meet and play chess. Since a real-world scenario is readily available, and to such a large extent resembles the computer software required for the GGS, the next step in developing the GGS system is to duplicate this real world scenario in a software setting. \end_layout \begin_layout Standard Some requirements, limitations and additions were made to the chess club system, so that the system would more easily and efficiently be replicated in a software setting. \end_layout \begin_layout Standard In the text below, two examples will be presented. On example is that of a real-world chess club, in which players meet to play chess against each other, the other example is the GGS, and how it corresponds to this chess club. \begin_inset Float figure wide false sideways false status collapsed \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Graphics filename graphics/theory_layout.eps scale 40 \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash end{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Caption \begin_layout Plain Layout \begin_inset CommandInset label LatexCommand label name "fig:theory-layout" \end_inset The layout of a physical \begin_inset Quotes eld \end_inset Chess club \begin_inset Quotes erd \end_inset with two players (P) sitting by each chess table (Table), a coordinator keeps track of all moves and players in the building. A player has to pass by the entrance to enter or exit the building. The building is represented by the outermost box. \end_layout \end_inset \end_layout \end_inset In figure \begin_inset CommandInset ref LatexCommand vref reference "fig:theory-layout" \end_inset a graphical representation for the chess club is presented. The club is seen from above. The outermost box represents the building. In the GGS setting, the building would represent one instance of GGS. Several buildings linked together would represent a cluster of GGS instances. In order for a player (the P symbol in the graphic) to enter the theoretical chess club, the player must pass by the entrance. By having each player pass by the entrance, a tally \begin_inset Note Note status open \begin_layout Plain Layout Does this mean what I think it does? \begin_inset Quotes eld \end_inset Räkning \begin_inset Quotes erd \end_inset ? \end_layout \end_inset can be kept, ensuring that there are not too many players within the building. In the GGS setting, too many players entering would mean too many connections have been accepted to the GGS system, and that the structure of the system thus must be modified, adding additional servers. \end_layout \begin_layout Standard Once a player has been allowed in to the chess club the player is greeted by the host of the chess club, in the GGS setting represented by the \emph on Coordinator \emph default , and is seated by a table. The coordinator keeps track of all the players in the building, and all moved made by the players. The information available to the coordinator means that cheating can be monitored and book keeping can be performed by this entity. \end_layout \begin_layout Standard Moves by players are made using the tables present in the chess club. Every game is isolated to a table, just as expected. This means that communication during a game only has to pass by the players of that particular game, and the coordinator, making sure that no cheating takes place. \end_layout \begin_layout Standard This isolation of the games plays an imporant part in many properties of the GGS, the isolation means that games can for example be transferred between different chess clubs, furthermore, if cheating takes place, corruption can only occur in the particular table where it was found, and can not spread. \end_layout \begin_layout Standard Moving chess players from one location to another is one of the alterations made to the real world chess club system to make the system more appropriate for a software setting. Allowing games to be transferred is not a property usually desired in a real world chess club, where transferring players would mean moving the players from one building to another. In the software setting, moving players means moving the game processes from one system to another, perhaps to balance the system load. This transfer of players can occur transparently, without notifying the players. \end_layout \begin_layout Standard The simplified life cycle of a game in GGS can be viewed using algorithm \begin_inset CommandInset ref LatexCommand vref reference "alg:game-lifecycle" \end_inset . In order to make this life cycle as efficient and useful as possible, the scalability, fault tolerant and generic traits are added to the GGS. These are not shown in the algorithm, as these traits are tools in making the algorithm behave as efficient as possible, and are not the main focus when studying the life cycle of a game. \end_layout \begin_layout Standard The limits imposed in \begin_inset CommandInset ref LatexCommand vref reference "alg:game-lifecycle" \end_inset are arbitrary for this example, there are no limits in the GGS on the number of players connecting, for example. \end_layout \begin_layout Standard \begin_inset Float algorithm wide false sideways false status collapsed \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{algorithmic}[1] \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash newcommand{ \backslash INDSTATE}[1][1]{ \backslash STATE \backslash hspace{#1 \backslash algorithmicindent}} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold while \series default \begin_inset Formula $players<2$ \end_inset : \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset \series bold if \series default a player connects, call \emph on connected \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold while \series default the game commences: \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset call the function \emph on game \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold when \series default the game has stopped \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset call the function \emph on endGame \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold function \series default connected: \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset assign the new player an id \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset alert the coordinator of the new player \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset \series bold if \series default a free table does not exist: \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE[2] \end_layout \end_inset the coordinator creates a new table \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset the coordinator places the player by the table, and begins watching the player \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold function \series default game \series bold : \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset perform game-specific functions. In chess, the rules of chess are placed here \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold function \series default endGame: \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset alert the coordinator, de-registering the players \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset disconnect the players from the system, freeing system resources \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash end{algorithmic} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Caption \begin_layout Plain Layout \begin_inset CommandInset label LatexCommand label name "alg:game-lifecycle" \end_inset A very simple example of the flow through the GGS system when a game played. \end_layout \end_inset \end_layout \end_inset \end_layout \begin_layout Standard \end_layout \begin_layout Section Performance \end_layout \begin_layout Standard There are many ways in which performance could be measured. For the clients, time and response times are useful measurements in time critical settings. In non-time critical settings, the reliability of message delivery may be an even more important factor than speed. \end_layout \begin_layout Standard In a first person shooter game, the speed of delivery of messages is essential. Failiure to deliver messages in time results in choppy gameplay for the players. In strategy games, the reliability of delivery may be more important than the speed, since the game is not percieved as choppy even if the messages are delayed. \end_layout \begin_layout Standard For someone operating a GGS, it is perhaps more interesting to measure the system load, memory consumption, energy consumption and network saturation. These topics are discussed in theory in this section. The practical results for the prototype are discussed in chapter \begin_inset CommandInset ref LatexCommand vref reference "cha:Implementation-of-a" \end_inset . \end_layout \begin_layout Subsection Performance measurements \end_layout \begin_layout Standard \begin_inset Note Note status open \begin_layout Plain Layout Tue apr 26, 9:15. Continue from here on. Discuss which results we may expect in a fully fledged GGS system. What impedes the speeds, what raises the CPU load (and therefore the temperetur es & power consumption). What factors are there in the network saturation problem? \end_layout \begin_layout Plain Layout Which games are affected by what, and what does this mean for the number of players a GGS can handle? \end_layout \end_inset \end_layout \begin_layout Standard How many players can we have on a server? Performance differences between games? e.g can one game have thousands players on a server and another only have hundreds? Questions \end_layout \begin_layout Standard In the current state game servers is coded for each game that needs it, in many cases this seems like a bad solution. Developers that want to make a network game need to understand network programming. A way to change this is a generic game server which give the game developers a server which they implement their game towards. This approach would not only make it easier to develop network games, it would also allow games in different programming languages to be implemented using the same server. to be discussed here. \begin_inset Note Note status open \begin_layout Plain Layout Create a game with several thousand players, see how our server scales, how can we improve the performance? Sharding isn’t very nice.. alternatives? Improve the speed of sharding? \end_layout \begin_layout Itemize See how the server scales \end_layout \begin_deeper \begin_layout Itemize When adding many clients \end_layout \begin_deeper \begin_layout Itemize Measure in \begin_inset Formula $ms$ \end_inset (ping to clients) \end_layout \begin_layout Itemize measure in system load \end_layout \begin_layout Itemize Measure in loss of messages \end_layout \begin_layout Itemize Measure in # of timeouts? (if any??) \end_layout \begin_layout Itemize Measure in time-to-crash \end_layout \end_deeper \begin_layout Itemize When adding new machines to the pool \end_layout \begin_deeper \begin_layout Itemize % increase of performance per machine \end_layout \end_deeper \begin_layout Itemize Single-core CPU vs multi-core CPU \end_layout \begin_deeper \begin_layout Itemize It's very important to scale well on multi-core systems, since this is where the industry is going. Multicore is the future. \end_layout \end_deeper \end_deeper \begin_layout Plain Layout Find reference on how to benchmark distributed, multiprocess systems \end_layout \end_inset \end_layout \begin_layout Section Choice of network protocol \begin_inset CommandInset label LatexCommand label name "sec:Choice-of-network" \end_inset \end_layout \begin_layout Standard There are three main ways in which computer communication over the Internet usually takes place; TCP, UDP and HTTP. The first two are transport layer protocols, which are commonly used to transport application layer protocols, such as HTTP. TCP and UDP can not be used on their own, without an application layer protocol on top. Application layer protocols such as HTTP on the other hand needs a transport layer protocol in order to work. \end_layout \begin_layout Subsection HTTP \end_layout \begin_layout Standard Since HTTP is so widely used on the Internet today in web servers, it is available on most Internet connected devices. This means that if HTTP is used in the GGS, firewalls will not pose problems, which is a great benefit. However, due to the intended usage of HTTP in web servers, the protocol was designed to be stateless and client-initiated. In order to maintain a state during a game session using HTTP, some sort of token would have to be passed between client and server at all times, much like how a web server works. These facts combined makes HTTP unsuitable for our purposes, since the GGS requires a state to be maintained throughout a session, and also needs to push data from the server to clients without the clients requesting data. It should also be mentioned that HTTP uses the TCP protocol for transport, and what is said about TCP also applies to HTTP. \end_layout \begin_layout Subsection UDP \end_layout \begin_layout Standard Many online games use UDP as the carrier for their application layer protocol. UDP moves data across a network very quickly, however it does not ensure that the data transferred arrives in consistent manner. Data sent via UDP may be repeated, lost or out of order. To ensure the data transferred is in good shape, some sort of error checking mechanisms must be implemented. UDP is a good choice for applications where it is more important that data arrives in a timely manner than that all data arrives undamaged, it is thus very suitable for media streaming, for example. In the GGS reliability of transfer was chosen before the speed of the transfer, ruling out UDP as the transport later protocol. \begin_inset Note Note status open \begin_layout Plain Layout Perhaps we should only say that we chose TCP just for our GGS prototype and why. If we leave it like that it seems that we think it is not suitable. \end_layout \end_inset \end_layout \begin_layout Subsection TCP \end_layout \begin_layout Standard For reliable transfers, TCP is often used on the Internet. Built in to the protocol are the error checking and correction mechanisms missing in UDP. This ensures the consistency of data, but also makes the transfer slower than if UDP had been used. In the GGS, data consistency is more important than transfer speeds, and thus TCP is a better alternative than UDP. \begin_inset Note Note status open \begin_layout Plain Layout Same here it is simply not true for a generic server to chose one or the other. We should rephrase it so it is clear that we only state it about the GGS prototype. \end_layout \end_inset \end_layout \begin_layout Section Generic \begin_inset CommandInset label LatexCommand label name "sec:Generic" \end_inset \end_layout \begin_layout Standard \begin_inset Note Note status open \begin_layout Plain Layout Bad name of the chapter, but here we should give the theory of how the server is generic \end_layout \end_inset \end_layout \begin_layout Section Fault Tolerance \begin_inset CommandInset label LatexCommand label name "sec:Fault-Tolerance" \end_inset \end_layout \begin_layout Standard Fault tolerance is an important factor in all servers, a server that is fault tolerant should be able to follow a given specification when parts of the system failures. This means that fault tolerance is different in each system depending on what specification they have. A system could be fault tolerant in different aspects, one is where the system is guaranteed to be available but not safe and it could also be reversed, that the system is safe but not guaranteed to be available. Depending on the system one property may be more important(some example here). A system could also have non existent fault tolerance or it could be both safe and guaranteed to be available. It should be noted that it is not possible to achieve complete fault tolerance, a system will always have a certain risk of failure. With this in mind the goal is to make the GGS prototype as fault tolerant as possible. \end_layout \begin_layout Standard In order to make the GGS prototype fault tolerant the programming language Erlang has been used. Erlang will not guarantee a fault tolerant system but it has features that support and encourage the development of fault tolerant systems. In the GGS it is important that the system overall is fault tolerant. Crashes of the whole system should be avoided as this would make the system unusable for a time. By using supervisor structures it is possible to crash and restart small parts of the system, this is convenient as fault can be handled within small modules thus never forcing a crash of the system. \end_layout \begin_layout Standard The need for fault tolerance in game servers is not as obvious as it may be for other typ of servers. In general all servers strive to be fault tolerant as fault tolerance means more uptime and a safer system. This applies to game servers aswell, in brief good fault tolerance is a way of satisfying customers. In general, game servers differ from many other fault tolerant systems in that high-availability is more important than the safety of the system. For example a simple calculation error will not be critical for a game server but it may be in a life-critical system and then it is better that the system crashes than works with the faulty data. There are cases where safety may be critical in game servers, one example is in games where in-game money exist. \end_layout \begin_layout Subsubsection Performance penalties \end_layout \begin_layout Section Availability \begin_inset CommandInset label LatexCommand label name "sec:Availability" \end_inset \end_layout \begin_layout Standard One important factor of any server is the availability. A server to which you are unable to connect to is a useless server. Other then within telecomunication, their uptime is of about 99,9999999%, the game developer community hasn't approched this problem very genuinely yet so there is much room for improvement. \end_layout \begin_layout Standard There are several good papers on how to migrate whole virtual machines between nodes to replicate them \begin_inset Note Note status open \begin_layout Plain Layout Add more information about that \end_layout \end_inset but for the GGS a different approach has been chosen. Instead of just duplicating a virtual machine, the programming language Erlang has been used which offers several features to increase the availability. Some of them are \emph on hot code replacement \emph default , where code can be updated while the application is running and without the need to restart it, the \emph on supervisor structure \emph default provided by \emph on OTP \emph default and the inter node and process communication via \emph on messages \emph default instead of shared memory. We will discuss each of them later on. \end_layout \begin_layout Section Scalability \begin_inset CommandInset label LatexCommand label name "sec:Scalability" \end_inset \end_layout \begin_layout Standard Each instance of the GGS contains several tables. Each table is an isolated instance of a game, for example a chess game or a poker game. The way that the GGS scales is to distribute these tables on different servers. In many games it is not necessary for a player to move between tables during games. This is for example not a common occurrence in chess, where it would be represented as a player standing up from her current table and sitting down at a new table, all within the same game session. With this in mind, the main focus of the GGS is not to move players between tables, but to keep a player in a table, and to start new tables instead. When a server has reached a certain amount of players the performance will start to decrease. To avoid this the GGS will start new tables on another server, using this technique the players will be close to evenly distributed between the servers. It is important to investigate and find out how many players that are optimal for each server. This approach makes it possible to utilize all resources with moderate load, instead of having some resources with heavy load and some with almost no load. \end_layout \begin_layout Standard As mentioned in the purpose section there are two different types of scalability , structural scalability and load scalability. To make the GGS scalable both types of scalability are needed. Structural scalability means in our case that it should be possible to add more servers to an existing cluster of servers. By adding more servers the limits of how many users a system can have is increased. Load scalability in contrast to structural scalability is not about how to increase the actual limits of the system. Instead it means how good the system handles increased load. The GGS should be able to scale well in both categories. \end_layout \begin_layout Subsection Load balancing \end_layout \begin_layout Standard The need for load balancing varies between different kind of systems. Small systems that only use one or a couple of servers can cope with a simple implementation of it, while in large systems it is critical to have extensive and well working load balancing. The need also depends on what kind of server structure that the system works on. A static structure where the number of servers are predefined or a dynamic structure where the number varies. \end_layout \begin_layout Standard Load balancing and scaling is difficult in different scenarios. When running in a separate server park, there are a set number of servers available, this means that an even distribution on all servers is preferable. When running the GGS in a cloud, such as Amazon EC2, it is possible to add an almost infinite number of servers as execution goes on. In this cloud setting, it may be more important to evenly distribute load on newly added servers. \end_layout \begin_layout Standard Two methods of balancing load (increasing structure): \end_layout \begin_layout Itemize Fill up the capacity of one server completely, and then move over to the next server \end_layout \begin_layout Itemize Evenly distribute all clients to all servers from the beginning, when load becomes too high on all of them, then comes a new problem: \end_layout \begin_deeper \begin_layout Itemize How do we distribute load on these new servers? \end_layout \end_deeper \begin_layout Standard Load balancing is a key component to achieve scalability in network systems. The GGS is a good example of a system that needs to be scalable, to attain this load balancing is necessary. Optimization of the load balancing for a system is an important task to provide a stable and fast load balancer. There are certain persistence problems that can occur with load balancing, if a player moves from a server to another data loss may occur. This is an important aspect to consider when the load balancer is designed and implemented. \end_layout \begin_layout Standard \begin_inset Note Note status open \begin_layout Plain Layout Because P2P game architectures are a constant goal for cheaters and because “Cheating is a major concern in network games as it degrades the experience of the majority of players who are honest” and preventing cheating in P2P game architectures is very difficult game developers try to use Client - Server architectures which have a natural problem to scale. In this paper we want to show some strategies to achieve scalability. \end_layout \end_inset \end_layout \begin_layout Subsection UUID \end_layout \begin_layout Standard \begin_inset Float algorithm wide false sideways false status open \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{algorithmic}[1] \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash newcommand{ \backslash INDSTATE}[1][1]{ \backslash STATE \backslash hspace{#1 \backslash algorithmicindent}} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset global variable \begin_inset Formula $state:=0$ \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash STATE \end_layout \end_inset \series bold function \series default unique \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset \begin_inset Formula $state:=state+1$ \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash INDSTATE \end_layout \end_inset \series bold return \begin_inset Formula $state$ \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash end{algorithmic} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Caption \begin_layout Plain Layout \begin_inset CommandInset label LatexCommand label name "alg:A-simple-generator" \end_inset A simple (insufficient) generator for identifiers \end_layout \end_inset \end_layout \end_inset \end_layout \begin_layout Standard Inside the GGS, everything has a unique identifier. There are identifiers for players, tables and other resources. When players communicate amongst each other, or communicate with tables, they need to be able to uniquely identify all of these resources. Within one machine, this is mostly not a problem. A simple system with a counter can be imagined, where each request for a new ID increments the previous identifier and returns the new identifier based off the old one, see algorithm \begin_inset CommandInset ref LatexCommand ref reference "alg:A-simple-generator" \end_inset . This solution poses problems when dealing with concurrent and distributed systems. In concurrent systems, the simple solution in algorithm \begin_inset CommandInset ref LatexCommand ref reference "alg:A-simple-generator" \end_inset may yield non-unique identifiers due to the lack of mutual exclusion. \end_layout \begin_layout Standard The obvious solution to this problem is to ensure mutual exclusion by using some sort of lock, which may work well in many concurrent systems. In a distributed system, this lock, along with the state, would have to be distributed. If the lock is not distributed, no guarantee can be made that two nodes in the distributed system do not generate the same number. A different approach is to give each node the ability to generate Universally Unique Identifiers (UUID), where the state of one machine does not interfere with the state of another. \end_layout \begin_layout Standard According to \begin_inset CommandInset citation LatexCommand citet key "Leach98uuidsand" \end_inset , \begin_inset Quotes eld \end_inset A UUID is 128 bits long, and if generated according to the one of the mechanisms in this document, is either guaranteed to be different from all other UUIDs/GUI Ds generated until 3400 A.D. or extremely likely to be different \begin_inset Quotes erd \end_inset . This is accomplished by gathering several different sources of information, such as: time, MAC addresses of network cards, and operating system data, such as percentage of memory in use, mouse cursor position and process ID:s. The gathered data is then \emph on hashed \emph default \begin_inset space ~ \end_inset using an algorithm such as SHA-1. \end_layout \begin_layout Standard When using system wide unique identifiers, such as the ones generated by algorithm \begin_inset CommandInset ref LatexCommand ref reference "alg:A-simple-generator" \end_inset with mutual exclusion, it is not possible to have identifier collisions when recovering from network splits between the GGS clusters. Consider figure \begin_inset CommandInset ref LatexCommand ref reference "fig:network-split" \end_inset , where \emph on Site A \emph default is separated from \emph on Site B \emph default by a faulty network (illustrated by the cloud and lightening bolt). When \emph on Site A \emph default and \emph on Site B \emph default later re-establish communications, they may have generated the same ID:s if using algorithm \begin_inset CommandInset ref LatexCommand ref reference "alg:A-simple-generator" \end_inset , even when mutual system-wide exclusion is implemented. This is exactly the problem UUID:s solve. \end_layout \begin_layout Standard \begin_inset Float figure wide false sideways false status collapsed \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Graphics filename graphics/NetworkSPlit.eps scale 40 \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash end{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Note Note status open \begin_layout Plain Layout Add clients on each side, and replace the cloud with phole-landlines being cut by a pair of scissors \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Caption \begin_layout Plain Layout \begin_inset CommandInset label LatexCommand label name "fig:network-split" \end_inset An example of a network split \end_layout \end_inset \end_layout \end_inset \end_layout \begin_layout Section Security \end_layout \begin_layout Standard We only support languages running in a sandboxed environment. Each game session is started in its own sandbox. The sandboxing isolates the games in such a way that they can not interfere with each other. If sandboxing was not in place, one game could potentially modify the contents of a different game. A similar approach is taken with the persistent storage we provide. In the storage each game has its own namespace, much like a table in a relational database. A game is not allowed to venture outside this namespace, and can because of this not modify the persistent data of other games. \end_layout \begin_layout Subsection Encryption \end_layout \begin_layout Section Game Development Language in a Virtual Machine \end_layout \begin_layout Standard There is only a very limited number of game developers who would like to write their games in Erlang, therefore we had to come up with something to resolve this problem. The main idea was to offer a replacable module which would introduce a interface to different virtual machines which would run the game code. This way a game developer can write the game in his favourite language while the server part still is written in Erlang and can benefit from all of its advantages. \end_layout \begin_layout Subsection JavaScript \end_layout \begin_layout Standard JavaScript has gained a lot of popularity lately, it is used in large projects such as \emph on Riak \emph default \begin_inset Foot status open \begin_layout Plain Layout \begin_inset Flex URL status collapsed \begin_layout Plain Layout http://wiki.basho.com/An-Introduction-to-Riak.html \end_layout \end_inset \end_layout \end_inset , \emph on CouchDB \emph default \begin_inset Foot status open \begin_layout Plain Layout \begin_inset Flex URL status collapsed \begin_layout Plain Layout http://couchdb.apache.org \end_layout \end_inset \end_layout \end_inset . On the popular social coding site \emph on GitHub.com \emph default , 18% \begin_inset Foot status open \begin_layout Plain Layout during the writing of the thesis the percentage went up to 19% \begin_inset Flex URL status collapsed \begin_layout Plain Layout https://github.com/languages/ \end_layout \end_inset \end_layout \end_inset of all code is written in JavaScript. The popularity of JavaScript in the programming community, in combination with the availability of several different JavaScript virtual machines was an important influence in choosing JavaScript as the main control language for our GGS prototype. \end_layout \begin_layout Subsection Other languages \end_layout \begin_layout Standard Other languages like \emph on lua \emph default , \emph on ActionScript \emph default are suitable as well because there is a virtual machine for each of them which can be \begin_inset Quotes eld \end_inset plugged in \begin_inset Quotes erd \end_inset into our GDL VM interface. With help of the \emph on Java Virtual Machine \emph default or the \emph on .NET \emph default environment it is even possible to run nearly every available programming language in a sandbox as a GDL. \end_layout \begin_layout Standard Due lack of time we have decided to use just the Erlang <-> JavaScript bridge with our interface. \end_layout \begin_layout Section Testing \end_layout \begin_layout Standard NOTE: This is not a part of the final text. There has been some work on the area of testing game servers, see \begin_inset CommandInset citation LatexCommand citet key "Lidholt02designand" \end_inset , who describes a test bench using \emph on bots \emph default for testing his generic hazard-gaming server. Lidholt describes how his server, capable of running several different casino games is tested using artificial players, so called bots. Performance is measured in \begin_inset Quotes eld \end_inset number of clients \begin_inset Quotes erd \end_inset able to connect to the server, and the system load. \end_layout \begin_layout Chapter Implementation of a prototype \begin_inset CommandInset label LatexCommand label name "cha:Implementation-of-a" \end_inset \end_layout \begin_layout Standard This chapter contains the realization of much of the principles and techniques described in chapter \begin_inset CommandInset ref LatexCommand vref reference "cha:Theory" \end_inset . Here the problems and their solutions are discussed in greater detail, and at times the text becomes more specific to GGS. \end_layout \begin_layout Standard Much of what is discussed in this chapter has been implemented in the Erlang GGS prototype. Specific solutions such as \emph on supervisor structures \emph default and distribution of erlang nodes on physical nodes. The different means of communications within the GGS and outside the GGS with third parties are also discussed here. \end_layout \begin_layout Section Overview of the prototype \end_layout \begin_layout Standard The prototype of the GGS was developed using the Erlang language. The functional and concurrent style of Erlang facilitates devlopment of software based on a real-world model \begin_inset CommandInset citation LatexCommand citep key "armstrong2011" \end_inset . In Erlang, most things are processes. The software running the Erlang code is known as the Erlang machine, or an Erlang node. Each Erlang node is capable of running several \emph on threads \emph default (also known as \emph on Light Weight Processes; LWP \emph default ) \emph on , \emph default much like the threads in an operating system. Threads in a Linux system, for example, are treated much like operating system processes in different systems. Due to the size of datastructures related to each process, swapping one process for another (known as \emph on context switching \emph default ) is an expensive task in many systems. \begin_inset Note Note status open \begin_layout Plain Layout Citation needed, perhaps the book for the \emph on Unix Internals \emph default course? \end_layout \end_inset \end_layout \begin_layout Standard The cost of swapping operating system processes becomes a problem when many processes are involved. If the GGS system had been developed using regular operating system processes, it would have had to be designed in a way to minimize the number of processes. Using Erlang, which is capable of running very many processes, several times more than an operating system can, the mapping between the real world system (described in \begin_inset CommandInset ref LatexCommand vref reference "sec:Design-of-the" \end_inset ) becomes clearer. \end_layout \begin_layout Standard Erlang allows the GGS to create several process for each player connecting, these processes can handle a multitude of different tasks, parsing data for example. Since each task is handled by a different process, the tasks are clearly separated and the failiure of one is easily recovered without affecting the others. \end_layout \begin_layout Standard In addition to creating (or \emph on spawning \emph default ) processes specifically to handle new players connecting, the GGS has more permanent processes running at all times. The constantly running processes in the GGS system are called \emph on modules \emph default . An example of a module in the GGS is the \emph on dispatcher module \emph default , which handles the initial connection made by a client, passing the connection along further in to the system. \end_layout \begin_layout Standard \begin_inset Float figure wide false sideways false status collapsed \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Graphics filename graphics/Chess_no_text.eps width 100text% \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash end{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Caption \begin_layout Plain Layout \begin_inset CommandInset label LatexCommand label name "fig:The-layout-of" \end_inset The layout of the GGS. The circles marked with 'C' topmost in the picture represent clients. The cloud marked 'network' pictured directly below the clients can be any network, for example the Internet. The barell figure marked 'backup' is a process being fed backup data from the coordinator. The barell marked 'State' contains the state of a table, and this is fed into the box marked 'Mnesia' which is database. Finally the figure shaped as a shield marked 'GameVM' contains the actual game process. \end_layout \end_inset \end_layout \end_inset \end_layout \begin_layout Standard In figure \begin_inset CommandInset ref LatexCommand vref reference "fig:The-layout-of" \end_inset the entire GGS system is represented graphically. The circles marked with 'C' topmost in the picture represent game clients. These circles represent processes running on gamers' computers, and not on the GGS machine. If a game of chess is to be played on the server, the clients on the gamers' machines will be chess game clients. Clients connect through a network, pictured as a cloud, to the dispatcher process in the GGS. The dispatcher process and all other modules are discussed in \begin_inset CommandInset ref LatexCommand vref reference "sec:The-modular-structure" \end_inset . For each connection, a new player process is spawned, which immediately after spawning is integrated in to the GGS by the coordinator process. \end_layout \begin_layout Section The usage of Erlang in the GGS \end_layout \begin_layout Standard \begin_inset Note Note status open \begin_layout Plain Layout Here we can have a more in-depth look at why Erlang was used, and outline the characteristics of EWrlang that we make use of in the GGS. \end_layout \end_inset \end_layout \begin_layout Section The modular structure of the GGS prototype \begin_inset CommandInset label LatexCommand label name "sec:The-modular-structure" \end_inset \end_layout \begin_layout Standard The separation of concerns, and principle of single responsibility \begin_inset Foot status collapsed \begin_layout Plain Layout More information on the SRP is available at: \begin_inset Note Note status open \begin_layout Plain Layout Insert adress! \end_layout \end_inset \end_layout \end_inset are widely respected as good practices in the world of software engineering and development. By dividing the GGS up into modules each part of the GGS can be modified without damaging the rest of the system. \end_layout \begin_layout Standard The responsibility and concern of each module comes from the responsibility and concern of the real-world entity the model represents. The modelling of the GGS after a real world system was discussed in chapter \begin_inset CommandInset ref LatexCommand vref reference "cha:Theory" \end_inset . \end_layout \begin_layout Standard In the text below the word module refers to the actual code of the discussed feature, while the word process is used when referring to a running instance of the code. Those familiar to object oriented programming may be helped by thinking in the lines of classes and objects. \begin_inset Note Note status open \begin_layout Plain Layout Am I right here? \end_layout \end_inset \end_layout \begin_layout Subsection The dispatcher module \end_layout \begin_layout Standard \begin_inset Note Note status open \begin_layout Plain Layout The discussion of the modules is divided into the following parts: \end_layout \begin_layout Itemize What does the module do? \end_layout \begin_layout Itemize What happens when the module fails? \end_layout \begin_layout Itemize How does the module correspond to the real-world scenario of the chess club? \end_layout \end_inset \end_layout \begin_layout Standard The dispatcher module is the first module to have contact with a player. When a player connects to the GGS, it is first greeted by the dispatcher module, which sets up an accepting socket for each player. \begin_inset Note Note status collapsed \begin_layout Plain Layout Is this the proper way to day the dispatcher greets connecting players? \end_layout \end_inset The dispatcher is the module which handles the interfacing to the operating system when working with sockets. Operating system limits concering the number of open files, or number of open sockets are handled here. The operating system limits can impose problems on the GGS, this is discussed more in detail in chapter \begin_inset CommandInset ref LatexCommand vref reference "cha:Problems-of-implementation" \end_inset . \end_layout \begin_layout Standard Should the dispatcher module fail to function, no new connections to the GGS can be made. In the event of a crash in the dispatcher module, a supervisor process immediately restarts the dispatcher. There exists a window of time between the crashing of the dispatcher and the restarting of the dispatcher, this window is very short, and only during this window is the GGS unable to process new connection requests. Due to the modular structure of the GGS, the rest of the system is not harmed by the dispatcher process not functioning. The process does not contain a state, therefore a simple restart of the process is sufficient in restoring the GGS to a pristine state after a dispatcher crash \begin_inset Note Note status open \begin_layout Plain Layout Well.. In theory.. \end_layout \end_inset . \end_layout \begin_layout Standard Returning to scenario of the chess club, the dispatcher module is the doorman of the club. When a player enters the chess club, the player is greeted by the doorman, letting the player in to the club. The actual letting in to the club is in the GGS represented by the creation of a player process discussed in \begin_inset CommandInset ref LatexCommand vref reference "sub:The-player-module" \end_inset . The newly created player process is handed, and granted rights to, the socket of the newly connected player. \end_layout \begin_layout Subsection The player module \begin_inset CommandInset label LatexCommand label name "sub:The-player-module" \end_inset \end_layout \begin_layout Standard The player module is responsible for representing a player in the system. Each connected player has its own player process. The player process has access to the connection of the player it represents, and can communicate with this player. In order to communicate with a player, the data to and from the player object must pass through a protocol parser module, discussed in \begin_inset CommandInset ref LatexCommand vref reference "sub:The-protocol-parser" \end_inset . Raw communication, witout passing the data through a protocol parser is in theory possible, but is not useful. \end_layout \begin_layout Standard In the creation of a player process, the coordinator process, discussed in \begin_inset CommandInset ref LatexCommand vref reference "sub:The-coordinator-module" \end_inset , is notified by the newly connected process. \end_layout \begin_layout Standard In the event of a crash in a player process, several things happen. \end_layout \begin_layout Enumerate The player process, which is the only process with a reference to the socket leading to the remote client software, passes this reference of the socket to the coordinator process temporarily. \end_layout \begin_layout Enumerate The player process exits. \end_layout \begin_layout Enumerate The coordinator spawns a new player process, with the same socket reference as the old player process had. \end_layout \begin_layout Enumerate The player process resumes operation, immediately starting a new protocol parser process, and begind receiving and sending network messaged again. \end_layout \begin_layout Standard The window of time between the crash of the player process and the starting of a new player process is, as with the dispatcher, very short. Since the connection changes owners for a short period of time, but is never dropped, the implications of a crash are only noticed, at worst, as choppy gameplay for the client. Note however that this crash recovery scheme is not implemented in the GGS prototype. \begin_inset Note Note status open \begin_layout Plain Layout Can we do this..? Seems a bit sneaky. \end_layout \end_inset \end_layout \begin_layout Standard Moving back to the real world example, the player process represent an actual person in the chess club. When a person sits down at a table in the chess club, the person does so by requesting a seat from some coordinating person, and is then seated by the same coordinator. Once seated, the player may make moves on the table he or she is seated by, this corresponds clearly to how the GGS is structured, as can be seen in the following sections. \end_layout \begin_layout Subsection The protocol parser module \begin_inset CommandInset label LatexCommand label name "sub:The-protocol-parser" \end_inset \end_layout \begin_layout Subsection The coordinator module \begin_inset CommandInset label LatexCommand label name "sub:The-coordinator-module" \end_inset \end_layout \begin_layout Subsection The table module \end_layout \begin_layout Subsection The game virtual machine module \end_layout \begin_layout Subsection The database module \end_layout \begin_layout Section Techniques for ensuring reliability \end_layout \begin_layout Standard One of the main goals of the project is to achieve high reliability. A highly reliable application is one that crashes very, very rarely \begin_inset Note Note status open \begin_layout Plain Layout CITATION NEEDED \end_layout \end_inset . There are some tools for creating reliable applications built in to Erlang. \end_layout \begin_layout Itemize Links between processes. When a process spawns a new child process, and the child process later exits, the parent process is notified of the exit. \end_layout \begin_layout Itemize Transparent distribution over a network of processors. When several nodes participate in a network, it does not matter on which of these machines a process is run. Communication between processes does not depend on the node in which each process is run. \end_layout \begin_layout Itemize Hot code replacements. Two versions of the same module can reside in the memory of Erlang at any time. This means that a simple swap between these versions can take place very quickly, and without stopping the machine. \end_layout \begin_layout Standard These three features are some of the basic building blocks for more sophisticate d reliability systems in Erlang. Many times it is not necessary to use these features directly, but rather through the design patterns described below. \end_layout \begin_layout Subsection Supervisor structure \begin_inset Note Note status open \begin_layout Plain Layout This entire section is bad. \end_layout \end_inset \end_layout \begin_layout Standard By linking processes together and notifying parents when children exit, we can create supervisors. A supervisor is a common approach in ensuring that an application functions in the way it was intended. When a process misbehaves, the supervisor takes some action to restore the process to a functional state. \end_layout \begin_layout Standard There are several approaches to supervisor design in general (when not just considering how they work in Erlang). One common approach is to have the supervisor look in to the state of the process(es) it supervises, and let the supervisor make decisions based on this state. The supervisor has a specification of how the process it supervises should function, and this is how it makes decisions. \end_layout \begin_layout Standard In Erlang, we have a simple version of supervisors. We do not inspect the state of the processes being supervised. We do have a specification of how the supervised processes should behave, but on a higher level. The specification describes things such as how many times in a given time interval a child process may crash, which processes need restarting when crashes occur, and so forth. \end_layout \begin_layout Standard When the linking of processes in order to monitor exit behaviour is coupled with the transparent distribution of Erlang, a very powerful supervision system is created. For instance, we can restart a failing process on a different, new node, with minimal impact on the system as a whole. \end_layout \begin_layout Standard In the GGS, we have separated the system in to two large supervised parts. We try to restart a crashing child separately, if this fails too many \begin_inset Foot status collapsed \begin_layout Plain Layout Exactly how many \begin_inset Quotes eld \end_inset too many \begin_inset Quotes erd \end_inset is depends on a setting in the supervisor, ten crashes per second is a reasonable upper limit. \end_layout \end_inset times, we restart the nearest supervisor of this child. This ensures separation of the subsystems so that a crash is as isolated as possible. \begin_inset Float figure wide false sideways false status open \begin_layout Plain Layout \begin_inset Note Note status open \begin_layout Plain Layout We should really do this graphic in EPS instead of PNG \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash begin{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Graphics filename graphics/Supervisor_tree_GGS.eps scale 40 \end_inset \end_layout \begin_layout Plain Layout \begin_inset ERT status open \begin_layout Plain Layout \backslash end{centering} \end_layout \end_inset \end_layout \begin_layout Plain Layout \begin_inset Caption \begin_layout Plain Layout The supervisor structure of GGS \end_layout \end_inset \end_layout \end_inset \end_layout \begin_layout Standard The graphic above shows our two subsystems, the coordinator subsystem and the dispatcher subsystem. Since these two systems perform very different tasks they have been separated. Each subsystem has one worker process, the coordinator or the dispatcher. The worker process keeps a state which should not be lost upon a crash. \end_layout \begin_layout Standard We have chosen to let faulty processes crash very easily when they receive bad data, or something unexpected happens. The alternative to crashing would have been to try and fix this faulty data, or to foresee the unexpected events. We chose not to do this because it is so simple to monitor and restart processes, and so difficult to try and mend broken states. This approach is something widely deployed in the Erlang world, and developers are often encouraged to “Let it crash”. \end_layout \begin_layout Standard To prevent any data loss, the good state of the worker processes is stored in their respective backup processes. When a worker process (re)starts, it asks the backup process for any previous state, if there is any that state is loaded in to the worker and it proceeds where it left off. If on the other hand no state is available, a special message is delivered instead, making the worker create a new state, this is what happens when the workers are first created. \end_layout \begin_layout Subsection Distribution \end_layout \begin_layout Subsection Hot code replacement \end_layout \begin_layout Section Implementation \end_layout \begin_layout Subsubsection User interface \end_layout \begin_layout Chapter Problems of implementation \begin_inset CommandInset label LatexCommand label name "cha:Problems-of-implementation" \end_inset \end_layout \begin_layout Section erlang_js \end_layout \begin_layout Standard To be able to run JavaScript on our server we needed to embed a JavaScript engine within the server. After a thorough investigation erlang_js became our choice. Erlang_js provides direct communication with a JavaScript VM (Virtual Machine). This was exactly what we wanted, but we also needed the possibility to communicate from erlang_js to Erlang. This functionality was not yet implemented in erlang_js, due to lack of time. \end_layout \begin_layout Standard There were two possible solutions to the problem. We could rewrite some part of erlang_js, or we could switch erlang_js for some other JavaScript engine. Searching for other engines we found erlv8 and beam.js which provided the functionality that we wanted. As we tested beam.js it occurred random crashes of the whole Erlang environment. These crashes were related to the use of erlv8 in beam.js and we decided that the use of erlv8 was not an alternative due to the stability issues. \end_layout \begin_layout Standard To get the functionality needed we decided to implement this in erlang_js. \end_layout \begin_layout Subsection UUID \end_layout \begin_layout Standard Erlang identifies processes uniquely throughout the entire Erlang network using process IDs (PID). When we wish to refer to erlang processes from outside our erlang system, for example in a virtual machine for a different language, possibly on a different machine, these PID:s are no longer useful. \end_layout \begin_layout Standard This problem is not new, and a common solution is to use a Universally Unique Identifier, a UUID. These identifiers are generated both using randomization and using time. A reasonably large number of UUID:s can be generated before a collision should occur. There are standard tools in many UNIX systems to generate UUID:s, we chose to use the uuidgen command, which employs an equidistributed combined Tausworth e generator. \end_layout \begin_layout Section Design choices \end_layout \begin_layout Standard When designing concurrent applications, it is useful to picture them as real world scenarios, and to model each actor as a real world process. A real world process is a process which performs some action in the real world, such as a mailbox receiving a letter, a door being opened, a person translating a text, a soccer player kicking the ball, just to name a few examples. Since we focus on games in this project, it is suitable to model our system as a place where games take place. We imagined a chess club. \end_layout \begin_layout Standard The clients pictured as green circles can be thought of as the physical chess players. \end_layout \begin_layout Standard When a player wants to enter the our particular chess club, he must first be let in by the doorman, called the \emph on Dispatcher \emph default in the GGS. \end_layout \begin_layout Standard He then gets a name badge, and thus becomes a \emph on Player \emph default process in the system. He is also guided in to the lobby by the \emph on Coordinator \emph default , which has the role of the host of the chess club. \end_layout \begin_layout Standard When players wish to play against each other, they talk to the \emph on Coordinator \emph default who pairs them up, and places them at a table. Once they have sat down at the table, they no longer need the assistance of the \emph on Coordinator \emph default , all further communication takes place via the table. This can be thought of as the actual chess game commencing. \end_layout \begin_layout Standard All the moves made in the game are recorded by the table, such that the table can restore the game in case something would happen, such as the table tipping over, which would represent the table process crashing. \end_layout \begin_layout Standard Once a player wishes to leave a game, or the entire facility, he should contact the \emph on Coordinator \emph default , who revokes his name badge and the \emph on Dispatcher \emph default will let the player out. \end_layout \begin_layout Standard With the information kept in the tables and the \emph on Coordinator \emph default combined, we can rebuild the entire state of the server at a different location. This can be thought of the chess club catching fire, and the \emph on Coordinator \emph default rounding up all the tables, running to a new location and building the club up in the exact state it was prior to the fire. \end_layout \begin_layout Section Understanding OTP \end_layout \begin_layout Section Usability \end_layout \begin_layout Chapter Results and discussion \end_layout \begin_layout Section Software development methodology \end_layout \begin_layout Standard The project has not followed any specific software development methodology. All work has been based on a predefined schedule and the specifications are made by team members rather than an outside customer or stakeholder. The process can be described as a plan-driven development method going from brainstorming to design, then implementation and finally testing. Yet there have been cycles in the process such as redesign and code refactoring. \end_layout \begin_layout Section Statistics \end_layout \begin_layout Chapter Conclusion \end_layout \begin_layout Chapter* Glossary \end_layout \begin_layout Standard Here we could add some important words and their definitions.. \end_layout \begin_layout Standard \end_layout \begin_layout Standard \begin_inset CommandInset bibtex LatexCommand bibtex bibfiles "bibliography" options "plainnat" \end_inset \end_layout \end_body \end_document