Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Critical success factors to improve the game development process from a developer’s perspective Saiqa Aleem 1, Luiz Fernando Capretz 1, and Faheem Ahmed2 1Department of Electrical and Computer Engineering, University of Western Ontario, London, ON, Canada.

2 Department of Computing Science, Thompson River University, Kamloops, BC, Canada. E-mail: {saleem4, lcapretz}@uwo.ca, [email protected]

Abstract The growth of the software game development industry is enormous and is gaining importance day by day. This growth imposes severe pressure and a number of issues and challenges on the game de- velopment community. Game development is a complex process, and one important game development choice is to consider the developer perspective to produce good-quality software games by improving the game development process. The objective of this study is to provide a better understanding of the devel- oper’s dimension as a factor in software game success. It focusses mainly on an empirical investigation of the effect of key developer factors on the software game development process and eventually on the qual- ity of the resulting game. A quantitative survey was developed and conducted to identify key developer factors for an enhanced game development process. For this study, the developed survey was used to test the research model and hypotheses. The results provide evidence that game development organizations must deal with multiple key factors to remain competitive and to handle high pressure in the software game industry. The main contribution of this paper is to investigate empirically the influence of key de- veloper factors on the game development process.

Keywords Developer’s perspective, Software games, Empirical investigation, Good-quality games, Game development process, Game developer’s factors.

1. Introduction to mature over time [1]. The game development process has also had an impact on the industry, The first software game was created half a century ago. In the world of software gaming, which now counts on special methodologies and many things have changed during this time period. mature processes for its development, ultimately Now the software game industry has reached the leading to an enhanced game development pro- point that it rivals other well-established industries cess. Game developers try to produce games that are different from any other game in the market. such as music and cinema. As a result, the soft- ware gaming business has grown enormously, has This difference can be achieved through by intro- made billions of dollars in profit, and has started ducing new perspectives, new gameplays, new

Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. genre combinations, enhanced graphics, or new product. Fundamentally, game development is a characters. Therefore, almost all games must be form of software development process with sever- novel, and their success depends on their overall al additional requirements such as creative design, quality [2]. Only good-quality games are able to artistic aspects, and visual presentation [4], [5]. In retain their players, and this has become an im- this context, game development organizations can portant factor for any software game to succeed apply the same software engineering principles to commercially. In other words, if a game is not of improve their development processes. However, good quality, players can easily switch to another many studies have discussed the challenges of ap- game. Hence, it has become mandatory for the plying software engineering principles to the game software game industry to try to morph and adapt development process [5]. to the preferences and demand of its players. Kultima [6] highlighted these challenges One of the main concerns in game develop- from the game design perspective. Blow [4] dis- ment process is that developers need to follow best cussed their implications from the perspective of practices and procedures from software engineer- technical frameworks and development tech- ing discipline to develop good quality games. The niques. Blow [4] and McGill [7] discussed issues game development process involves four main even for the required technical skills for game de- phases: concept, pre-production, production, and velopment. Software game development also re- post-production [3]. It consists of various activities quires a range of skills that include design, project such as synopsis, background research, script writ- management, development, and asset creation. It ing, visualization and concept art, level and inter- also involves team members from heterogeneous action design, animation, programming, media disciplines, e.g., game designers, artists, pro- editing, integration, testing, and publishing. Soft- grammers, and software developers. Knowledge of ware games are also characterized based on the best practices for game development is very im- category into which they fall, which is called the portant and has become crucial to sustain the genre of the game. Genres include action, shoot- growth of the software game industry. Finally, this ers, fighting, racing, adventure, sports, role play- knowledge will help game developers make cor- ing, strategy, simulations, puzzles, dance, music, rect game development decisions at the right time. and others. Each genre has its own requirements An investigation of key success factors from a de- which must be taken into consideration during the veloper’s perspective will contribute to the under- pre-production phase. For this reason, software standing of current development process implica- game development is considered as a complex tions and will help developers improve the game process that involves multidisciplinary collabora- development process. tive team efforts and processes (including sound, Exploring diverse developers’ preferences for gameplay, art, artificial intelligence, control sys- software game development will provide a signifi- tems and human factors) to develop a creative - 2 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. cant benefit to improve the development process Issues of game production, development, and test- by generating valuable insights. No research has ing reflect only the general state of the art in soft- been done to date on including developer-centred factors in the software game development process. ware engineering. Pressman [12] stated that a This study will help identify key factors empirical- game is a kind of software which entertains its us- ly from the developer’s perspective, an effort that ers, but game development faces many challenges will ultimately help improve the software game and issues if only a traditional software develop- development process to produce good-quality ment process is followed [5],[13]. software games. To identify key factors, a quanti- tative survey was conducted, and the results are Many researchers have discussed game de- reported here. The survey was used to test the re- velopment challenges. Pertillo et al. [13] surveyed search model and several hypotheses. Finally, the the problems faced by game development organi- results show that consideration of key factors from a developer’s perspective helps identify important zations. The overall game development process game development choices and their implications combines both an engineering process and the cre- for the current process. ation of artistic assets. Ramadan and Widyani [14]

1.1 Research Background compared various game development strategies

The software game domain covers a great va- from a management perspective, and some re- riety of player modes and genres [8], [9], [10]. The searchers [15], [16], [17] have proposed frame- complexity of digital games has posed many chal- works for game development. To effectively man- lenges and issues in software development be- age and improve the game development, key de- cause it involves diverse activities in creative arts veloper’s factors are required. Tschang [18] and disciplines (storyboarding, design, refinement of Petrillo et al. [13] highlighted the issues in the animations, artificial intelligence, video produc- game development process and its differences tion, scenarios, sound effects, marketing, and fi- from traditional software development practices. nally sales) besides technological and functional In traditional software engineering, the devel- requirements [11]. This inherent diversity leads to opment phase usually involves activities like ap- a greatly fragmented domain from the perspectives plication design and implementation, and the pro- of both underlying theory and design methodolo- duction phase is when the software actually runs gy. The software game literature published in re- and is ready for use. However, in the game devel- cent years has focussed mainly on technical issues. opment, the production phase includes the devel- Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. opment process, which is the pre-production phase software process models used for game develop- of the software engineering process, and the pro- ment. They concluded that agile and hybrid ap- duction phase of software engineering is actually proaches are used by most organizations for game the post-production phase of the game develop- development. They also reported that Scrum [5], ment life cycle [20]. Therefore, the game devel- Kanban [23], Rapid Development Application opment is different from the traditional software (RAD) [24], XP [25], and incremental [5] meth- engineering process, and many researchers [5] odologies are used by game development organi- have studied the challenges faced by this domain. zations. The major difference in software devel-

Kanode and Haddad [5] stated that an important opment and game development is in the design incorrect assumption has been made that game de- phase because design of game may undergo major velopment follows the waterfall method. More re- change in late development. The other differences cently, researchers have agreed that it must follow are content development and quality criteria. Man- the incremental model because it combines the aging game development has become a much waterfall method with an iterative process. Petrillo harder process that anyone could have initially im- et al. [13] reported a major concern, that develop- agined, and because of the fragmented nature of ers for software creation in the game industry the domain, no clear picture of its advancement commonly use very poor development methodolo- can be found in the literature. gies. The game development life cycle (GDLC) is From the above discussion, it can be easily the object of questions on many forms, which at- concluded that game development process is dif- ferent from traditional software development pro- tempt to determine what types of practices are cess. Kasurinen et al. [26] argued that current used. However, this question has no single answer. software engineering knowledge is unable to The most prominent observation made in these bridge the gap between software engineering and studies is that to address the challenges faced by certain aspect of game development. The overall development process to produce a game includes the GDLC, more rigorous software engineering art, audio and gameplay other than software de- strategies must be used. However, the proposed velopment discussed above. In the game develop- GDLCs [14], [19], [20], [21] do not ensure the ment process, the content and production activities quality of the development process. Hagan et al. are performed in tandem with the development and engineering activities. Further, it is well [22] published a systematic literature review of agreed that the game development process is a - 4 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. multidisciplinary activity that involves the merg- out by small team members and involved simple ing of creative and technical talent to bring a con- architectures consisting of 2D graphics, sound, cept to life, where the main activities can be cate- simulation, and input/output streaming. The first gorized into content and production, and engineer- software games were developed by a few talented ing at each phase of the development process. individuals from diverse backgrounds like mathe-

Moreover, sometime game development or- matics, computer science, and physics with no ed- ganizations reduce their development process due ucational background in engineering or computer science. At that time, developers were mainly fo- to of high competition and extreme market de- mand so they can be first to market [27]. This re- cussed on how to develop interesting games rather duction of the development process definitely af- than on architecture or software engineering prin- fects game quality. Therefore, they do not strictly ciples. The current success of the game industry, follow the software engineering standards and continuous enhancements in game technology, and the need to meet the ever-higher expectations of practices. Because of these types of complex pro- ject-management tasks, the game development the players resulted in a complex game develop- process diverges from traditional software devel- ment process. opment. Nevertheless, the differences between The main research motivations behind this software engineering and games development are study are the rapid and continual changes in tech- not exclusive; it seems that traditional software nology and the severity of competition in game development does not fully support game devel- development organizations. Ultimately, these fac- opment activities and provide process assessment tors will not only affect the business, but also will procedures [28]. So, we need key success factors have a major impact on the game development to improve game development process that may process. Nowadays, games are developed by large overlap with traditional software development fac- teams because game projects have grown in size tors or just exclusive to game development. There- and complexity [4]. Various stakeholders are in- fore, it has become important now to investigate volved in the development process and have dif- the critical success factors for game development ferent expectations and world views. For example, organizations in developing good quality games the game designer does not know the level of from developer’s perspective. complexity involved in implementing artificial intelligence to represent the behaviour of a non- 1.2 Research Motivation player character. A software engineer may think Game development has become incredibly that some features in the game design document challenging due to rapid changes in game technol- are infeasible to implement due to time deadlines ogy such as game platforms, game engines, and or technical constraints. Another important re- reuse of code modules for different genres. During quirement that must be part of the game is the fun, the 1990s, game development was usually carried flow, and enjoyment factors. The game develop- Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. ment processes have different phases and are in- rience. This indicates that there is a need for col- fluenced by many factors. Identifying the key suc- laboration between researchers and developers that cess factors in a game development process is ex- will be ultimately beneficial to game industry tremely important for sustaining the economic standards. This study also attempts to fill this growth of the software game industry. communication gap between researchers and de- velopers. Above discussed facts, motivated us to However, very little research has been reported in carry out empirical investigation of key success the academic literature about key success factors factors that can help developers to improve their for the game development process. Many topics in development practices. It will be ultimately enable software games need attention from researchers them to develop good quality games. and highlighted by some studies [22], [29], [30]. Moreover, researchers and game developers have The rest of the paper is organized as follows: different points of view. Basically, game develop- Section 2 provides a literature review of identified ers prioritize the game development process by factors, Section 3 describes the research method- rapid creation and implementation of content. On ology used for this study, Section 4 presents the the other hand, scientists and researchers prioritize results of the empirical investigation, Section 5 investigation and research into the individual provides a discussion, and finally, Section 6 con- cludes the study.

Table 1 Identified factors from a developer’s perspective

Factors References

Team Configuration & Management Claypool & Claypool [31]; Eric et al. [32]; Musil et al. [25]; Tran and Biddle [35]; Stacey et al. [36]; Barros et al. [37]. Kasurinen et al. [38]; Bosser [39]; Callele et al. [40]; Callele et al. [41]; Reyno Game Design Document Management and Cubel [42]; Almeida and da Silva [43]; Ahmed and Jaafar [44]; Bringula et al. [45]. Robins [3]; Sherrod [46]; Cowan and Kapralos [47]; Hudlicka [48]; Yan-Hui et al. Game Engine Development [49]; Rodkaew [50]; Vanhutupa [51]; Sousa & Garlan [52]; Aitenbichler et al. [53]; Pimenta et al. [54]; Neto et al. [55]; Peker and Can [56]. Llopis [57]; Hendrikx et al. [58]; De Carli et al. [59]; Phelps [60]; Pranatio and Game Asset Management Kosala [61]; Lasseter [62]; Xu and CuiPing [64]; Chehimi et al. [65]; Manocha et al. [66]; Pichlmair [67]; Migneco et al. [68]. Wang and Nordmark [69]; Amendola et al. [70]; Lukashev et al. [72]; Rhalibi et Quality of Game Architecture al. [73]; Jhingut et al. [74]; Kosmopoulos et al. [75]; Al-Azawi et al. [76]; Segun- do et al. [77]. Redavid and Farid [78]; Helppi [79]; Charles et al. [80]; Wilson [81]; Marri & Game Test Management Sundaresaubramanian [82]; Kasurinen and Smolander [83]; Al-Azawi et al. [84]; Omar and Jaafar [85]; Straat and Warpefelt [86]. Robins [3]; Sarinho and Apolinario [87]; Czarnecki and Kim [88]; Chen et al. Programming Practices [89]; Anderson [90]; Xu and Rajlich [91]; Zhang et al. [92]; Wang and Norum [93]; Meng et al. [94]. components of a system. Researchers do not have 2. Literature review and proposed hypothesis resources to develop a standard game, whereas In recent times, the Software Game Industry developers never publish the results of their expe- (SGI) has seen unprecedented growth. To succeed

- 6 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. in a highly competitive environment, game devel- The term “collaboration” can be defined as the opers must bring innovative, good-quality games level of shared understanding and coordination to the table. Identifying key success factors to im- among teams and the maintenance of this level prove the game development process will help de- [34]. Very few research studies have investigated velopers maintain the pace. Key factors in the the importance of multidisciplinary team configu- game development process are the least addressed ration and management in software game devel- area in software game research. Various factors opment. Musil et al. [25] highlighted the im- have been identified from a literature review of portance of heterogeneous team collaboration in published articles on software games as a basis for the video game development process. They pro- discussion of the game development process. posed a method based on the Scrum methodology

Table 1 briefly presents the identified factors, to improve workflow integration and collaboration with references for each. The identified factors and between heterogeneous game development team members. The proposed process separates the pre- the related literature are described in the following sub-sections. production, production, and post-production phas- es. Management through collaboration and inte- 2.1 Team Configuration and Management gration of heterogeneous disciplines in game de- The development of software games involves mul- velopment is achieved by executing daily hetero- ti-disciplinary team configuration and manage- geneous discipline-specific workflows in a sprint ment. More specifically, team configuration and iteration adjusted by daily scrums. They claimed management are considered critical to the success that this approach will enable each discipline to of any game development project. Game devel- use the workflows in which they are most profi- opment requires intensive team management [31]. cient in accordance with the demands and pace of Team management can be defined as the process other involved disciplines. of administration and coordination between groups Tran and Biddle [35] discussed the collabora- of individuals who are performing specific tasks tion factor for team management in serious game [32]. It involves forming different groups, estab- development. They explained that the collabora- lishing collaboration among them, setting objec- tive process is based on ethnography and a qualita- tives for a common set of interpersonal dynamics tive approach. The proposed model includes many among team members, and performance apprais- factors such as physical resources, social relation- als. The game development process also involves ships, organizational goals, and team knowledge. configuration and management of multidiscipli- They conducted a case study that determined that nary teams or teamwork projects and management collaboration between multidisciplinary team re- of the collaboration among them. The term quires teams to communicate frequently, to respect “teamwork” refers to group of individuals who are each other’s contributions, and to share the same completing a specific task [33]. model and goals for game development. Stacey et Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. al. [36] and Barros et al. [37] also investigated the a basis for requirements engineering in the game collaboration factor in multidisciplinary game de- development process. Game designers can trace velopment teams and the development of comput- back all their efforts to the requirement analysis in er games. the GDD.

To determine whether proper team configura- In the game development process literature, tion and its management has any impact on the researchers have explored the importance of the game development process, “team configuration game design document and its management in var- and management” was selected as an independent ious ways. Some of them have highlighted the im- variable, as shown in Fig. 1. Hence, Hypothesis 1 portance of the GDD by discussing the importance and corresponding null hypothesis can be stated as of requirements engineering in game development. follows: For example, Kasurinen et al. [38] highlighted the

Hypothesis 1: Team configuration and man- importance of requirements engineering in the agement have a positive influence on the enhanced game development process. They interviewed 27 game development process. software professionals from game development organizations to obtain insight into their develop- Null Hypothesis: Team configuration and ment process. The findings of the study showed management have no influence on the enhanced that the professionals follow approaches or meth- game development process. ods that are somewhat comparable to requirements

2.2 Game Design Document Management management and engineering, but not to particular requirements engineering practices. Bosser [39] The Game Design Document (GDD) has also suggested that massively multi-player game design been identified as an important factor in improving needs a prototyping tool and proposed a frame- the game development process. The GDD is the work model to facilitate its design. They also sug- outcome of the pre-production phase of game de- gested that game prototyping is important and velopment. It is developed and edited by the game helpful for better game design. Callele et al. [40] design team to organize their efforts and their de- also investigated the importance of requirements velopment process. The form of the GDD varies engineering in the video game development pro- widely across studios and genres. Basically, the cess. They suggested that the reasons for the fail- GDD includes the goals of the game, the genre of ure of any game may be rooted in problems of the game, the overall flow, the story behind the transforming the pre-production phase document, game, the characters and their dialogue, special i.e., the GDD, with any implied information and effects, the number of elements and feature fits application of domain knowledge from the pre- within the game, and feature creeping information production phase into the production phase. if required. Typically, this document is developed to express the concept of the game and to provide An understanding of upcoming media and

- 8 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. technology developments, game play, and non- Null Hypothesis: Proper management of the functional requirements is also considered im- game design document has no effect on the overall portant for the GDD. Callele et al. [41] described game development process. how the GDD is helpful in obtaining a better un- 2.3 Game Engine Development derstanding of the game design process and ex- plained the definition of gameplay process in cog- Game engines are considered to be a powerful nitive game development. Reyno and Cubel [42] tool by game developers and have been in use for proposed a model-driven game development more than two decades. A game engine is a soft- method that ultimately accelerates game design. ware layer that helps in the development process Almeida and da Silva [43] performed a systematic by enabling developers to focus solely on game review of game design methods and of various logic and experimentation [3]. Many commercial available tools. They emphasized the use of stand- game engines are available to help game develop- ardized tools to develop the GDD. Other research- ers with advanced rendering technologies and ers have emphasized inclusion of the user perspec- code reuse, resulting in shorter development time tive and have provided game design guidelines. and reduced cost. Sherrod [46] defined the game Ahmed and Jaafar [44] emphasized the importance engine as a “framework comprised of a collection of user-centered game design and proposed that it of different tools, utilities, and interfaces that hide should be considered at the concept phase of game the low-level details of the various tasks that make development. Bringula et al. [45] gathered user up a video game”. Overall, the game engine repre- perceptions to determine how a serious game sents the basic structure of the game as it appears should be developed. Based on their study, they in the middle layer, between the application layer suggested some design guidelines for four- and the various underlying platforms. dimensional game design, including storyline, aes- In the literature, most researchers often use thetics, reward systems, and the game objective. the terms “game engine” and “game development

To develop a good-quality game, the GDD framework” interchangeably. This study uses the must be properly managed so that production team term “game engine” to refer to the development members can easily move it into game production. tool that includes most of the functionality and GDD management has also been selected as an features that become part of any software game. independent variable in this study, and therefore The list of primary features that can be part of any the following hypothesis and corresponding null modern game engine includes scripting, rendering, hypothesis are proposed: animation, artificial intelligence, physics, audio, and networking. Cowan and Kapralos [47] per- Hypothesis 2: Proper management of the formed a survey on frameworks and game engines game design document has a positive and signifi- for serious game development only. They com- cant effect on the overall game development pro- pared all the commercially available game engines cess. Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. and their various features. The results of their sign goals suggested by them were usability, effi- study suggested that most of the game engines that ciency, portability, and adaptability. To determine have been developed to create entertainment whether standard game engine development has a games can also be used for serious game devel- positive impact on the overall game development opment. Hudlicka [48] suggested a set of require- process, game engine development was considered ments that are necessary for game engine devel- as an independent variable in this study. Hence, opment, specifically for affective games. Research the following hypothesis and corresponding null has been also done on development of game en- hypothesis are proposed: gines specific to different platforms, such as for Hypothesis 3: Game engine development has the Android platform [49], a 3D role-playing game a significant impact on the game development for cross-platform development [50], and the process. Browser games [51]. Null Hypothesis: Game engine development A few researchers have explored the means of has no impact on the game development process. addressing the challenges faced by developers in supporting and building development tools [52], 2.4 Game Asset Management [53]. However, they were not successful in achiev- Anything can be considered as a game asset ing the required feature and design flexibility. Re- that contributes to the visual appearance of a searchers proposed different solutions for game game, whether artwork (including 3D elements or engines to address the challenges they faced. Pi- textures), music, sound effects, dialogue, text, or menta et al. [54] proposed that game engines ena- anything else. Llopis [57] stated that “game assets ble fast learning for game developers and include include everything that is not code: models, tex- the ubiquitous characteristics of the game design ture, materials, sound, animations, cinematics, and development process. Neto et al. [55] dis- scripts, etc.” Actually, game assets include any cussed the issue of game engine standardization in piece of data that can be used by a game engine software game development. Game developers are aside from code, scripts, and documentation. The interested in producing the same game for differ- elementary unit of game assets can be referred to ent platforms and rely mostly on the same game as a game bit [58] and typically has no value when engine. They suggested that commonality and var- considered independently. There are two catego- iability assessment must be done to enable game ries of bits: characters, which can be an asset that engine reuse. Peker and Can [56] proposed a interacts in a simulated environment, and abstract methodology for developing game engines for bits, which are kinds of sound and texture that can mobile platforms based on design goals and design be use together to produce a concrete bit. The patterns. They emphasized the need to design main six kinds of game bit are texture, sound, veg- goals and strategies for implementation in the etation, buildings, fire, water, stone, clouds (con- game engine. For mobile platforms, the basic de- crete), and behaviour. Game space definition is - 10 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. another game asset, which is part of content gen- that can be triggered during any game event [66]. eration for any game. It provides a kind of game These can be managed through dynamic audio environment where game bits can be placed. processing libraries. Researchers have also studied

In the literature covering game asset creation the use of audio processing libraries in software and management, researchers have explored game game development. Pichlmair [67] studied music assets in term of animation, audio processing li- games and determined that they can be classified braries for different genres, and content generation into two categories, rhythm and instrument games. Their analysis showed that music in video games for games. De Carli et al. [59] and Hendrikx et al. [58] carried out a survey of procedural content has seven qualities: rhythm, active score, quantiza- generation techniques for game development. An- tion, synesthesia, play as performance, sound imation in games is considered an important asset agents, and free-form play. Migneco et al. [68] because it has a great impact on game perfor- proposed an audio processing library to enable use of sound in Web-based games using a Flash devel- mance [60]. Studies have been done to explore animation models for different genre of games. opment tool. They claimed that this approach pro- Pranatio and Kosala [61] performed a comparative vided flexibility and great functionality for devel- study of keyframes [62] and skeletal animations oping games using Flash technology. [66] for multiplayer games. Their results indicated For the reasons discussed above, creation and that skeletal or bone-based frames are better than management of the number of assets required for keyframe models in term of memory load and game development has become challenging. frames per second. Xu and CuiPing [64] reviewed Mechanisms are needed to control the different currently used 3D accelerators for graphics anima- versions of assets that are developed for games. tion. A wide variety of graphics cards are available Commercially, a number of tools are available, to programmers. Hence, they discussed the current such as 3D Studio Max, Maya, and Adobe Pho- benefits and limitations of APIs such as OpenGL toshop, which can also create various assets like and DirectX. Chehimi et al. [65] described the textures, 3D models, animations, sound effects, evolution of 3D graphics for mobile platforms. music, voice recordings, levels, and scenes. Mod- They concluded that the current market presents ern game engines also include modules for asset challenges regarding graphics quality and battery management. Based on a literature review of game life of mobile devices. These need to be addressed asset management, this study has considered game by standardizing successful game development for asset management as another independent variable mobile platforms. that is considered important for the game devel-

Sound within a game is one of the game as- opment process. Hence, the following hypothesis sets that enable developers to build responsive, and corresponding null hypothesis are proposed: interactive, and attractive games. Currently, game Hypothesis 4: Game asset management is development relies on pre-recorded sound clips important for enhancing the game development Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. process. and model), a graphic processing system (graphic

Null Hypothesis: Game asset management is interface and view interface), and an input pro- not important for enhancing the game develop- cessing system (event manager, controller, and ment process. event publisher). These sub-systems must be clear- ly separated so that they can work independently. 2.5 Quality of Game Architecture The authors suggested a Model-View-Controller The primary function of game architecture is (MVC) [71] pattern for the architecture. Basically, to support game play. It helps to define challenges this pattern divides the application into three com- by using constraints, concealment, exploration, ponents: model, view, and controller. The defined and obstacles or skill testing. Game architecture is relations and collaboration among these compo- a kind of blueprint for the underlying complex nents helps in game deployment because ultimate- software modules. It is used to delineate design, ly the code associated with each sub-system’s log- perform trade-off analysis, and investigate system ic will operate in the desired manner. properties before implementation and potential reuse. Basically, it draws together gameplay fac- Lukashev et al. [72] proposed a mobile plat- tors and technical requirements. A perfect game form development framework specifically for 3D architecture would have modularity, reusability, application. They claimed that their suggested ap- robustness, and tractability features. proach will help developers improve the develop- ment process. The first stage of the proposed The importance of software architecture in framework is the design phase for creation of the game development has rarely been researched. initial model (2D or 3D) and selection of the right Only Wang and Nordmark [69] have explored this modeling tool and graphic format. The second topic. Their finding was that software architecture stage, the integration stage, enables developers to plays an important role in game development, with put together already-created models into scenes the focus mainly on achieving high performance and create animation. The authors suggested that a and modifiability. They also stated that most de- structural optimization technique can be used to velopers use game-specific engines, middleware, create scenes. The next stage is the utilization and tools for game development. A number of stage, in which the created models are converted studies have explored these various development to mobile format. Implementation is the final step frameworks. The proposed game development of the framework, where developer put together frameworks can help game developers to define source code, auto-generated source code, and cre- their game architecture. Amendola et al. [70] pro- ated resources. Several other studies have also posed a framework for experimental game devel- been performed to propose development frame- opment called GLIESE. They proposed that a works for various platforms based on different game architecture should have at least three sub- technologies for defining the system architecture. systems: a game logic processing system (view For example, Rhalibi et al. [73] proposed a 3D - 12 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Java framework for Web-based games, Jhingut et tion of testing helps developers fix problems more al. [74] and Kosmopoulos et al. [75] proposed a quickly and cheaply. Delays in testing can result in framework for mobile platforms, Al-Azawi et al. project failure.

[76] proposed an agent-based agile methodology Helppi [79] discussed many game test meth- for game development, and Segundo et al. [77] ods that can be used during the development proposed a game development framework specific phase, such as smoke testing that is used to test the to the Ginga middleware. user interface logic. Regression testing is per- From the preceding discussion, it is clear that formed to check that game quality is still good af- the quality of the game architecture is important ter any change such as addition of features or add- for the game development process, and therefore it ons. Connectivity testing is used for networking was considered as another independent variable in games and mobile games to test client-server in- this study. The following hypothesis and corre- teraction. Performance testing can ensure the real sponding null hypothesis are therefore presented: performance of the game. Abuse testing is per-

Hypothesis 5: Quality of game architecture formed by giving multiple inherent inputs through has a positive impact on the enhanced game de- the controller and determining game performance. velopment process. Compliance testing makes sure that any compli- ance standards enforced by any stakeholder are Null Hypothesis: Quality of game architec- met. Finally, functional testing verifies overall ture has no impact on the enhanced game devel- game play and reveals issues related to stability, opment process. game flow, game mechanics, integration of graph-

2.6 Game Test Management ic assets, and user interface. Redavid and Farid [78] also discussed game testing methods used to Game testing is a very important phase of detect interactions failures and listed them under game development. A game can be tested at differ- the term combinatorial testing [79]. The second ent levels of development because game testing is approach involves test flow diagrams, which are different from software testing [78]. There are used to develop models of game behaviour from a many steps involved in game testing other than player’s perspective. Third is cleanroom testing, test-case definition because most game testing is which helps to determine game reliability. The test based on black-box testing. Hence, management tree is another testing method discussed by the au- of overall game test methods becomes crucial. In thors, which can used to organize test cases. the pre-production phase, a test plan document should be established to set standards for the game Wilson [81] also argued that no one testing software. Game quality can be evaluated accord- method is better than another. He suggested that ing to the graphics, sounds, and code that are good testing is a combination of 30% of ad-hoc compiled into the game code. Proper documenta- testing, 40% test cases, and 30% alternating be- tween the two until the strengths of both are de- Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. termined. Marri and Sundaresaubramanian [82] 2.7 Programming Practices discussed game test methods and suggested that Good programming practices are a very im- the game tester should test game quality by verify- portant factor in successful game development. A ing game play, logical consistency, observability, programming team with the necessary skills is def- progressive thinking, and reasoning ability, as well initely considered as the backbone of the game as exhaustively testing features, game strategy, development process. The programmer must select and functionality. Kasurinen and Smolander [83] the right coding architecture for each game pro- interviewed seven game development teams from ject. Basically, the lead programmer must select different organizations and studied how they test between two types of coding style: either game- their games based on grounded theory. They con- specific code (the programmer has to develop eve- cluded that all participating organizations had the rything by him/herself) or game-engine code resources to perform technical testing, but that (where the game engine is the foundation for a they relied mostly on exploratory and usability game-specific code). The game code can then be testing rather than using a pre-planned approach. organized in various ways [3], such as an ad-hoc Al-Azawi et al. [84] proposed a set of evaluation architecture where the programmer must deal with heuristics that could be used in game development tightly coupled code. Another choice is a modular methodologies for most game genres. Omar and architecture-based coding style, where the pro- Jaafar [85] proposed a tool to evaluate the usabil- grammer identifies and separates the code into dif- ity of educational games, and Straat and Warpefelt ferent modules or libraries. In this type of pro- [86] suggested use of the two-factor theory to gramming, reuse and maintainability are improved evaluate game usability. over ad-hoc-based coding. However, dependencies Management of game testing during the game between different modules cannot be controlled, development process has clearly come to be of which may lead to tight coupling. The directed crucial importance for game developers. Hence, acyclic graph (DAG) is another way of organizing test management was selected as another inde- code. This is also a modular architecture-based pendent variable in this study, and the following coding scheme in which dependencies between hypothesis and corresponding null hypothesis modules are tightly controlled. Layered-style cod- were proposed: ing is also based on a DAG architecture, but mod- Hypothesis 6: Game test management has a ules are arranged in rigid layers, and each can in- positive impact on the enhanced game develop- teract only with the modules in the layer directly ment process. below.

Null Hypothesis: Game test management has Game programming involves a wide range of no impact on the enhanced game development issues and considerations. Most researchers have process. tried to address these individually. The first is the

- 14 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. issue of coupling between different modules. Sari- Zhang et al. [92] performed experiments on five nho and Apolinario [87] tried to address this prob- industrial RPG mobile games developed using the lem using a proposed generative programming ap- object-oriented programming paradigm. Optimiza- proach. Generative programming aims to automate tion strategies with structural programming were the software development process using a number applied to the same code. The results of the study of static and dynamic technologies including re- showed that object-oriented programming must be flection, meta-programming, and program and used with great care and that structural program- model analysis [88]. The proposed method was ming is also a good option for mobile game devel- based on a game feature model that could repre- opment. Another study [93] highlighted the issues sent both common and variable implementation for game development posed by wireless peer-to aspects of software games. Meta-programming peer games in a J2ME environment using an resources were used to generate and represent available Bluetooth API. The issues discussed in- compatible source code for available game cluded slow device discovery, Bluetooth transfer frameworks and game engines. The authors con- speed, extra resource consumption, and Bluetooth cluded that the proposed approach would result in topology. Meng et al. [94] developed a peer-to- loss of the coupling development strategy between peer online multiplayer game using DirectX and game implementation and its domain software ar- C# to achieve playability in a .Net environment. tifacts. Code cloning in open-source games is an- According to the above discussion, program- other issue discussed by Chen et al. [89]. They ming practices were selected as an independent provided a detailed study of the issues of code variable in this study, and the following hypothesis clones in more than twenty open-source game pro- and corresponding null hypothesis were proposed: jects based on C, Python, and Java for various Hypothesis 7: Good programming practices game genres. Selection of a scripting language is are important for the enhanced game development another issue in game programming. Anderson process. [90] discussed the classification of scripting sys- tems used for software games. Xu and Rajlich [91] Null Hypothesis: Good programming prac- described a study that explored pair programming tices are not considered important for the en- practices and concluded that paired programmers hanced game development process. completed their task faster with higher quality. They suggested that pair programming is a good approach for game development.

Selection of a programming language is an- 3. Research Model other challenge for today’s game developers. The main objective of the proposed research Many studies have been done to explore different model is to analyze the interrelationship between programming languages for different platforms. Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. key factors and game development and also to un- game design document management, game engine derstand the influence of these factors on overall development, game asset management, quality of game quality in the SGI market. The model’s theo- game architecture, game test management, and retical foundation is based on existing concepts programming practices, and one dependent varia- found in the game development literature. Note ble: the enhanced game development process. that most studies in the literature discuss one or The multiple linear regression equation of the two of the factors mentioned above for software model is given as Equation 1: games and their impact on the overall game devel- Enhanced game development process = opment process. To the best of the authors’

α0 + α1f1 + α2f2 + α3f3 + α4f4 + α5f5 + α6f6 + α7f7, knowledge, this is the first study in the game de- velopment literature that highlights key factors in Where α0, α1, α2, α3, α4, α5, α6, α7 are coefficients game development. This study proposes to inves- and f1–f7 are the seven independent variables. tigate empirically the influence and association of 4. Research Methodology key game development factors. Figure 1 presents a Developing a software game involves phases theoretical research model used in this study, such as pre-production, production and which will be empirically investigated. The theo- post-production, in which each phase contains a retical model evaluates the relationships of various number of activities. Some of these activities are independent variables emerging from software en- dependent on others, whereas some are independ- gineering and management concepts such as pro- ent. Employees of game development organiza- ject management, theory, and behaviour with the tions or studios were selected as the targeted re- dependent variable, enhanced game development, spondents of this study. In this study, the term in the context of the game development process. “developer” is used to refer to any game develop- This study mainly investigates and addresses the ment team member. For purposes of data collec- following research question: tion, the authors initially joined various game de- Research Question: How can game develop- velopment community forums. ers improve the game development process?

The research model includes seven independent variables: team configuration and management,

- 16 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Key Developer’s Factors

(H1 +), (α1) Team Configuration & Management (H2 +), (α ) 2 Game Design Document

Management (H3 +), (α3) Game Engine Development Enhanced Game Development Process (H4 +), (α4) Game Asset Management

(H5 +), (α5) Quality of Game Architec- ture (H6 +), (α6)

Game Test Management

(H7 +), (α7) Programming Practices

Fig 1. Research model

The respondents participating in the study were development methodologies used by respondents part of multinational organizations in Asia, Eu- for any particular game project. rope, and North America; statistics describing The participants in the study were mainly part them are presented in Fig. 2 of game projects that were developed for different

The organizational participants agreed to take platforms such as kiosks and standalone devices, part in the study based on a mutual agreement that the Web, social networks, consoles, PC/Macs, and their identities would be kept confidential. The mobile phones. The game genres implemented in size of the game project development teams varied most of their projects included action or adven- from 10 to 50. Fig. 3 shows the total time period ture, racing, puzzles, strategy/role playing, sports, of the game development projects considered by music-based, and other categories. The qualifica- respondents while answering the measuring in- tions for this study were that the respondent must strument. Figure 4 represents the number of re- be a part of a development team that had at least spondents based on their development role in the three full-time developers; that the respondent game project. Figure 5 shows the percentage of worked on the project for at least one- third of its

total duration; Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Fig. 2 Number of respondents by continent

Percentage of respondents

Fig. 3 Total software game development duration of particular game projects considered by respondents.

- 18 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Roles of respondents Fig. 4 Percentage of respondents based on their role in the development process.

Development methodologies used for game projects

Fig. 5 Percentage of development methodologies used by respondents.

Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. and that the project was either completed or can- point Likert scale was used in the questionnaire, celled within the last three years. and with each statement, the respondents were re-

Finally, respondents must have worked in the quired to specify their level of agreement or disa- development team in some sort of development greement. Thirty-four items were used to measure role, such as a designer, artist, animator, pro- the independent variables (the key factors), and for grammer, producer, or sound designer. The survey the dependent variable (enhanced game develop- respondents worked in various capacities such as ment process), nine items were used. The literature game designer, artist, programmer, audio designer, related to key developer’s factors was reviewed in and producer. The total number of survey re- detail to ensure a comprehensive list of measure- spondents was 118, including a minimum of one ment items for each factor from the literature. A and a maximum of four responses from each or- multi-item, five-point Likert scale was used to ganization. Although the collected sample size is measure the extent to which each key developer relatively large but it is still considered small sam- factor was practiced for the game development ple as compared to the population size. project. The Likert scale ranged from (1) meaning “strongly disagree” to (5) meaning “strongly 4.1 Measuring instrument agree” and was associated with each item. The This study gathered data on the key develop- items for each identified factor were numbered er’s factors and the perceived level of enhanced from 1 to 34 in Appendix A and also labelled se- game development process identified in the re- quentially. They were measured for each project search model depicted in Fig. 1. To learn about that was completed within the last three years these two topics, the questionnaire presented in based on a multi-item five-point Likert scale. The Appendix A was used as a data collection instru- enhanced game development process was the de- ment. First, organizations involved in the game pendent variable, and designated items for the de- development process were asked to what extent pendent variable were numbered separately from they practiced the identified key developer factors one to nine and labelled sequentially. All the items for the game development project in question. specifically written for this study are presented in Second, they were asked what they thought of the Appendix A. To the best of the authors’ enhanced game development process for different knowledge, this is the first empirical study of key games in the software game industry. The five- - 20 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Table 2 Cronbach’s alpha coefficient and principal component analysis of seven variables.

Developers factor Item no. Coefficient α PC eigenval- ue Team Configuration & Management 1–6 (excluded 1) 0.63 1.48 Game Design Document Management 7–11(excluded 10) 0.60 1.51 Game Engine Development 12–18(excluded 18) 0.68 1.49 Game Asset Management 19–22(excluded 22) 0.81 1.57 Quality of Game Architecture 23–25 0.84 1.01 Game Test Management 26–29 0.64 1.79 Programming Practices 30–34(excluded 30) 0.86 1.25 software game developer’s factors for the en- hanced game development process in the SGI. -actory criteria for Cronbach’s alpha based on

their findings. Osterhof [96] suggested that a value 4.2 Reliability and validity analysis of 0.60 or higher was satisfactory for reliability To perform reliable and valid research, quan- coefficients based on his findings. Nunnally and titative analysis was carried out. Two integral Brenste [97] reported that a value of 0.70 or higher measure of precision, reliability and validity anal- for a reliability coefficient can be considered satis- ysis, were used to conduct empirical studies. The factory for any measuring instrument. consistency or reproducibility of a measurement is referred to as reliability. On the other hand, valid Van de Ven and Ferry [98] recommended that inference or agreement between the measured and a value of 0.55 or higher of the reliability coeffi- true values is referred to as validity. The measur- cient could be considered satisfactory. A first cal- culation was performed on a sample dataset to de- ing instrument designed for this study was also tested by reliability and validity analysis. The test termine the reliability of the dataset using was based on common practices usually used for Cronbach’s alpha coefficient. Some of the as- empirical analysis. Reliability analysis was per- sessment items for each factor were excluded if formed to determine the internal consistency of the they affected the desired value of Cronbach’s al- multi-scale measurement items designed for the pha coefficient. In the sample dataset, other than seven identified factors. To evaluate internal con- item no. 1 of team & configuration management, sistency, Cronbach’s alpha [95] coefficient was item no. 10 of game design document manage- used. Criteria for Cronbach’s alpha ranging from ment, item no. 18 of game engine development,

0.55 to 0.70 were considered satisfactory. Re- item no. 22 of game asset management and item searchers have reported different ranges of satisf- no. 30 of programming practices, all assessment Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. items were found reliable. So, we removed item design document management and programming no. 1, item no. 10, item no. 18, item no. 22 and practices loaded on a second factor, and both ei- item no. 30 from the instrument. After this, the genvalues were greater than one. The reported whole dataset was evaluated using Cronbach’s al- convergent validity of this study was considered pha coefficient. The results of these calculations adequate. showed that reliability coefficients for the seven 4.3 Data analysis techniques factors ranged from 0.61 to 0.76. These coeffi- To perform the empirical investigation for this cients are reported in Table 2. Hence, all variables study, various statistical approaches were used. developed for this study could be considered relia- Initially, the research activity was divided into ble. three phases to evaluate the significance of the

Validity analysis was performed for the da- proposed hypotheses H1–H7. In phase I, paramet- taset using principal component analysis (PCA) ric statistical and normal distribution tests were

[99]. PCA is usually used for convergent validity performed. A non-parametric statistical approach analysis and was calculated here for seven factors. was used in phase II, and for the analysis, a Partial

Campbell and Fiske [100] suggested that conver- Least Squares (PLS) analysis was carried out. gent validity has occurred in a given case only if To address external threats to validity, both the scale items in a measurement instrument are parametric and non-parametric approaches were highly correlated and if they move in the same di- used. Parametric approach is used to measures the strength of the linear relationship between normal- rection in a given assembly. The construct validity ly distributed variables. When the relationship be- of PCA-based analysis was determined using the tween the variables is not linear or the variables eigenvalue criterion [101]. Here, a criterion value are not normally distributed then it may be more greater than one was used to retain any component appropriate to use the non-parametric approach. Due to the small sample size, both parametric and based on the Kaiser criterion [102]. Eigenvalue nonparametric approaches were used to address analysis showed that out of the seven variables, the threat to external validity and we found results five together formed a single factor, whereas game of both approaches are consistent.

- 22 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

The measuring instrument contains multiple The PLS technique was used in Phase III to items for each independent and dependent varia- increase the reliability of the results and deal with ble, and respondent ratings were aggregated to ob- the limitation of small sample size. For statistical tain a composite value. Using a parametric statisti- calculations, the Minitab 17 software was used. cal approach in phase I, the Pearson correlation 5. Data Analysis and Results coefficient was calculated for the tests, with a one- 5.1 Phase I of hypothesis testing tailed t-test for each hypothesis H1–H7. For phase To test hypotheses H1–H7, parametric statis- II, the Spearman correlation coefficient was used tics were used in this phase. The Pearson correla- to test hypotheses H1–H7 using a non-parametric tion coefficient was determined between the inde- statistical approach. Phase III of the empirical in- pendent variables (developer’s factors) and the vestigation was carried out to address issues of dependent variable (the enhanced game develop- non-normal distribution and complexity or small ment process) of the research model, as illustrated sample size of the dataset. Fornell and Bookstein in Fig. 1. The level of significance to accept or [103] and Joreskog and Wold [104] reported that reject the hypotheses was then selected. Each hy- if non-normal distribution, complexity, small sam- pothesis was accepted if its p-value was less than ple size, and low theoretical information are is- sues, then partial least squares (PLS) analysis will 0.05 and rejected if its p-value [105] was greater be helpful. than 0.05. In Table 3, calculated results for the Pearson correlation coefficient are listed.

Table 3 Hypothesis testing using parametric and non-parametric correlation coefficients.

Hypothesis Key factors Pearson correlation Spearman correlation coefficient Coefficient H1 Team configuration and management 0.29* 0.29* H2 Game design document management 0.79* 0.74* H3 Game engine development 0.59* 0.64* H4 Game asset management 0.45* 0.47* H5 Quality of game architecture 0.13** 0.19** H6 Game test management 0.42* 0.37* H7 Programming practices 0.52* 0.48*

*Significant at p<0.05 **Insignificant at p>0.05

Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Hypothesis H1 was accepted because the Hypothesis H6 regarding game test management

Pearson correlation coefficient between team con- and the enhanced game development process was figuration and management and the enhanced accepted due to its positive Pearson correlation game development process was positive (0.29) at coefficient (0.42) at p<0.05. The last hypothesis p<0.05.For hypothesis H2 concerning game design (H7) relating programming practices to the en- document management and the enhanced game hanced game development process was also found development process, the Pearson correla- tion to be significant (0.52) at p<0.05 and was there- coefficient was also positive (0.79) at p<0.05, and fore accepted. Hence, in summary, hypotheses H1, therefore hypothesis H2 was also accepted. Hy- H2, H3, H4, H6, and H7 were accepted and found pothesis H3 concerning game engine development to be statistically significant. Hypothesis H5 was and the enhanced game development process was not supported statistically and was therefore re- accepted due to a positive (0.59) correlation coef- jected. ficient at p<0.05. Hypothesis H4 concerning game 5.2 Phase II of hypothesis testing asset management and the enhanced game devel- Hypotheses H1–H7 were tested based on the opment process was accepted based on its positive non-parametric Spearman correlation coefficient Pearson correlation coefficient (0.45) at p<0.05. in phase II. Table 3 reports the results for the

Hypothesis H5 concerning quality of game archi- Spearman correlation coefficient. Hypothesis H1 tecture and the enhanced game development pro- regarding team configuration and management cess was rejected based on its positive correlation was accepted because of its positive Spearman coefficient (0.13), but higher p>0.05. correlation coefficient (0.29) at p<0.05.

Table 4 PLS regression results for hypothesis testing

Hypothesis Factors Path coefficient R2 F-Ratio H1 Team configuration and management 0.29 0.08 11.35* H2 Game design document management 0.74 0.56 148.9* H3 Game engine development 0.59 0.34 62.09* H4 Game asset management 0.07 0.006 0.72* H5 Quality of game architecture 0.13 0.02 2.3** H6 Game test management 0.42 0.18 26..20* H7 Programming practices 0.52 0.27 44.45*

*Significant at p<0.05 **Insignificant at p>0.05

- 24 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

The Spearman correlation coefficient for 5.3 Phase III of hypothesis testing game design document management and the en- Hypothesis testing in phase III was performed hanced game development process (hypothesis using the partial least squares (PLS) technique.

H2) was also positive (0.74) at p<0.05 and was The main reason for using the PLS method in this also found to be significant. The relationship be- phase was to cross-validate the results obtained tween game engine development and the enhanced from the parametric and non-parametric statistical game development process game (hypothesis H3) approaches used in Phases I and II and to over- was found to be statistically significant due to its come their associated limitations. Spearman correlation coefficient (0.64) at p<0.05 Tests were also per formed on hypotheses H1–H7 and was accepted. For hypothesis H4 regarding to check their direction and significance. The de- game asset management, the Spearman correlation pendent variable, i.e., the enhanced game devel- coefficient was positive at p<0.05, and therefore opment process, was designated as the response H4 was accepted. Hypothesis H5 concerning qual- variable and other individual factors (independent ity of game architecture and the enhanced game variables) as the predicate variables for PLS exam- development process was rejected due to its posi- ination. The observed results of the structural hy- tive coefficient (0.19) at p>0.05. pothesis tests are presented in Table 4. The table

Hypothesis H6 concerning game test man- also includes the values of the path coefficient, R2, agement and the enhanced game development and the F-ratio. The path coefficient for team con- process was accepted due to its positive Spearman -figuration and management (H1) was observed to correlation coefficient (0.37) at p<0.05. The last be 0.29, with an R2 of 0.08 and an F-ratio of 11.35, hypothesis (H7) relating programming practices to and H1 was therefore found to be significant at the enhanced game development process was also p<0.05. Game design document management found to be significant (0.48) at p<0.05. In sum- (H2) had a positive path coefficient of 0.74, R2 = mary, hypotheses H1, H2, H3, H4, H6, and H7 0.56, and F-ratio = 148.9 and was also found to be were accepted and found to be statistically signifi- statistically significant at p<0.05. Game engine cant. Hypothesis H5 was not supported statistical- development (H3) had a path coefficient of 0.59, a ly and was therefore rejected. very low R2 of 0.34, and an F-ratio of 62.09 and

Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. was found to be significant at p<0.05. Game asset process in the SGI. The test procedure examined management (H4) had a positive path coefficient the regression analysis, the model coefficient val- of 0.07, a low R2 of 0.06, and an F-ratio of 0.72 ues, and the direction of the associations. The de- and was judged to be significant because the p- pendent variable (the enhanced game development value was less than 0.05. Quality of game archit- process) was designated as the response variable

-ecture (H5) (path coefficient: 0.13, R2: 0.02, and and the other independent variables (all the key

F-ratio: 2.3) was found to be statistically insignifi- developer factors) as predicate variables. The re- cant at p<0.05. Game test management (H6) (path gression analysis model results are reported in Ta- coefficient: 0.42, R2: 0.18, and F-ratio: 26.20) and ble 5. The path coefficients of six of the seven var- programming practices (path coefficient: 0.52, R2: iables (team configuration and management, game

0.27, and F-ratio: 44.45) were found to be signifi- design document management, development of a cant at p<0.05. game engine, game asset management, game test management, and programming practices) were 5.4 Research model testing positive and were found to be statistically signifi- The linear regression equation for the research cant at p<0.05. The path coefficient for quality of model is given by Eq. 1. The research model was game architecture was positive, but was found not tested to provide empirical evidence that factors to be statistically significant at p<0.05. The overall important to game developers play a considerable R2 value of the research model was 0.83, and the role in improving the overall game development adjusted R2 value was 0.68 with an F-ratio of

36.97, which was significant at p<0.05.

Table 5 Linear regression analysis of the research model Model coefficient name Model coefficient Coefficient value t-value

Team configuration and management α1 0.06 1.14*

Game design document management α2 0.50 7.44*

Game engine development α3 0.31 5.19*

Game asset management α4 0.21 0.38*

Quality of game architecture α5 0.03 6.57**

Game test management α6 0.13 2.24*

Programming practices α7 0.10 1.58*

Constant α0 0.01 1.13* R2 0.83 Adjusted R2 0.68 F-ratio 36.97*

*Significant at p<0.05 **Insignificant at p>0.05 - 26 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

6. Discussion served results support the theoretical assertions

made here and provide the very first evidence that Software game development is a multidisci- plinary activity that has its roots in the manage- consideration of key developers’ factors while de- ment and software engineering disciplines. The veloping games is important for software game software game industry has become a mass phe- success. This could well result in institutionalizing nomenon, supplemented by a number of possible the software game development approach, which strategies and exciting questions for game devel- in turn has a high potential to maximize profits. opment companies. More and more companies are Especially in the game development process, multidisciplinary team configuration and man- entering the market, and hence the intensity of agement is a huge challenge. Basically, producing competition is increasing. Established and new high-quality games relies on a high level of plan- entrants must both pay attention to the key factors ning, communication, and organization of multi- that help to improve their game development pro- disciplinary teams to avoid costly delays and fail- ures. Many factors have been identified by re- cesses and keep them competitive in the market. searchers as important to implementing a success- Now it is time to understand the perspective of ful collaboration between any kinds of multidisci- game developers and to learn what they think is plinary team. These factors include interpersonal important to improve software game quality and factors such as trust among team members and ability to communicate [34], willingness to collab- how the developed game can become successful in orate, and mutual respect [106]. Others are organi- the market. This research is a first step towards zational factors, including establishing appropriate this understanding because it will help developers protocols and supporting collaboration [107] and game development organizations to under- These factors can be implemented by using vari- ous software applications that are specifically de- stand the relationships and interdependences be- signed for collaborating on commercial software tween key factors from a developer’s perspective development projects. The main concern when and to understand the enhanced game develop- using these software applications is that they must ment process. This research is the first empirical fit in with the existing computing and workflow investigation of factors important to developers in environment [108]. Management of the members of various multidisciplinary teams can be evaluat- relation to improving the current development ed and maintained mainly by examining values process and provides an opportunity to explore and practices, for example, what each individual associations between them empirically. The ob- team member brings to the table, how they use Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. material or assets produced by other team mem- review section. Management of the game design bers, how they reconcile conflicting priorities, and document and its transition into a requirements finally how their personal relations influence the and specifications document is challenging. collaboration. The multidisciplinary team can use One way to handle this during the pre- management or collaboration software for task production phase is to produce two documents. tracking, version control, file sharing, and contin- The first one is the GDD, and the second one is a uous integration. Successful collaboration between requirements and specifications document based team members enables them to manage easily all on the GDD. Managing and transforming the phases of game development from start-up, creat- GDD into a production document is complex be- ing a concept, creating a proof of concept, the pro- cause the two require different documentation duction phase, and so forth until the game is pub- styles. Supportive documentation is also required lished. to help the development team in its transition from This study has explored the importance of pre-production to the production phase. The author team configuration and management factors from of the GDD may not have the requisite writing a developer’s perspective. It has found positive skills to produce a document that is understanda- associations between team configuration and man- ble by the production team (technical people). Ba- agement and the enhanced game development sically, there is a long list of required skills for a process. Hence, proper configuration and man- GDD developer, such as knowledge of game de- agement of multidisciplinary teams is a crucial sign, technical communication, and requirements part of the game development process. However, it engineering. Hence, a formal process is needed to must be balanced with other development issues in support the transition and would likely increase the game development process. the reliability of the game development process. Game design document management has been The results of this study have shown that devel- found to be positively associated with the en- opment and transformation of the GDD is very hanced game development process. The GDD is important and also requires strong management mainly a pre-production artifact which is defined skills to reduce documentation effort. Hence, the by the pre-production phase team to capture a cre- results presented here have shown that a good ative vision of the game. Game developers gener- GDD is the greatest contributor to the success or ally feel that imposing too much structure at the failure of a game development project. start of a game may be highly detrimental [40], Game asset management was also found to resulting in reduced creativity, constraining ex- have a positive association with the enhanced pression, and risking the intangibles that create an game development process. Game assets, defined enjoyable feeling or experience. At the same time, as any piece of data that is in a format that can be the importance of structure has been highlighted used by the game engine, will be presented to the by many researchers, as discussed in the literature user. To create and manage game assets, a realistic

- 28 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. content generator must be developed that can fill [109]. Most researchers in the area of game devel- in the missing bits. Trade-offs between realism and opment tools have proposed their own architec- performance and between realism and control tures for specific genres and platforms. Anderson must also be investigated for any asset created. For et al. [110] raised some important open questions graphical animation, a number of 3D model for- for the academic community that are specific to mats can be used by game developers. These can the game engine development research field. The generally be divided into two categories: frame- first is the main issue of the lack of a development based animation and skeletal-based animation. De- language. The second question is how to define termination of the perfect animation model for a the boundaries between the game loop and the game has become crucial because a diversity of game engine. For example, what technical aspects format types for graphics are available. Eventual- should a game engine cover in a game? The third ly, a poor choice could limit the performance of problem is that there is no standardization for the game itself. For sound effects in games, certain game engines because most of them are specific to problems are faced by developers because of un- a particular game genre and game project. The expected or complex scene configurations. A fourth issue involves design dependencies, and the number of asset management tools exist, but se- last the need for best practices when creating game lecting the appropriate one is a challenge because engines. It was generally agreed that a game en- each has its own limitations and benefits. gine should handle a diversity of inputs and out- Improvements in the game development pro- puts, a restricted set of customizations based on cess have been greatly aided by the emergence of each genre, and an asset and resource management game development tools, specifically game en- system. The results of this study have also showed gines. A game engine facilitates the game devel- that development of a game engine has a positive opment process by providing various sets of fea- impact on the enhanced game development pro- tures that help decrease development time and cess. In other words, game engine development is cost. These are available for most game genres an important factor that needs more consideration (e.g., role-playing games or serious games for from a developer perspective. training) and vary in cost and complexity. Not all It is a common perception that a good-quality game engines support the entire feature set of all or even perfect game architecture is a very im- the game genres. Hence, integrating all the techno- portant part of the game development process be- logical aspects into one framework is a prohibi- cause reworking architecture afterwards is always tively difficult task. It is understandable, therefore, hard. A game architecture identifies the main that confusion exists among game developers with structural components of the underlying software regard to selecting the appropriate game engine. and their relationships. In the game development Game development tools should be selected only literature, many researchers have proposed differ- after determining the game concept and the GDD ent frameworks for different platforms and based Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. on different technologies. As a developer it is dif- cluded that this approach can improve the outcome ficult to select among these because all provide a of games and result as a more robust end-product. kind of reference architecture and their validity is Therefore, testing plays an important role in each still in question. The findings of the study do not step of the development phase, and its manage- support a statistically positive relation between ment throughout the game development process is quality of game architecture and the enhanced important. The results of this study have also sup- game development process. The direction of asso- ported the hypothesis that game testing manage- ciation was found to be positive, but the required ment is important for the enhanced game devel- level of confidence was not supported. Hence, the opment process. At the same time, testing tech- hypothesis that quality of game architecture has a niques have matured over time, but still need im- positive impact on the enhanced game develop- provement. ment process was statistically rejected. Game programming strategy has a direct ef- fect on game performance. There are many con- Testing in game development is done mainly at a very late stage or the end of development to cerns associated with today’s game programming ensure the quality and functionality of the finished practices. Game developers must look for solu- product. Typically, in a particular game project, tions to common problems in game programming the leader dedicates a specific amount of time for such as coupling of modules, availability of differ- quality assurance or a beta tester to test the game. ent scripting and programming languages, plat- Various development methodologies are used to form compatibility issues, memory management, develop games, such as the agile methodology and and code optimization strategy, specifically to im- the waterfall model, but testing must form part of prove game performance and quality. Hence, game all processes. Every aspect of a game should be developers must consider various aspects of the tested during the development and production game such as speed, flexibility, portability, and phases. In addition, certain foundational elements maintainability while still coding. Ultimately, the skilled programming team will be able to develop should also be tested during the pre-production phase, such as frameworks and platform set-up. and implement the full functional game. Matching The most important aspect of testing for game de- of required skills to the abilities of developers is velopers is to integrate testing as part of the pro- very important to improve the overall game devel- duction phase to improve efficiency. To ensure opment process, a conclusion also supported by this study. continuous quality and delivery of good games to the market, developers must consider majority Overall, the findings of this study are im- testing options during the production phase. portant for the development of good-quality soft- Helppi [80] also researched the possibility that mobile game robustness can be improved by con- ware games. Rapid and continual changes in tech- tinuous integration, delivery, and testing and con- nology and intense competition not only affect the - 30 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. business, but also have a great impact on devel- as well. Easterbrooks et al. [112] suggested four opment activities. To deal with this strong compe- criteria for validating empirical studies: internal tition and high pressure, game development organ- validity, construct validity, external validity, and izations must continually assess their activities and reliability. Wohlin et al. [113] stated that general- adopt an appropriate evaluation methodology. Use izing experimental results to industrial practice by of a proper assessment methodology will help the researchers is mostly limited by threats to external organization identify its strengths and weaknesses validity. In this study, measures were taken to ad- and provide guidance for improvement. However, dress external threats to validity. The random the fragmented nature of the game development sampling method was used to select respondents process requires a comprehensive evaluation strat- from all around the world. Open-ended questions egy, which has not yet been entirely explored. The were also included in the questionnaire. findings of this study will help game development The choice and selection of independent vari- organizations to look for contributing key success ables was one of the limitations of this study. To factors from a developer’s perspective. This study analyze the association and impact of factors af- is a part of a larger project aiming to propose a fecting software game success, seven independent software game maturity assessment model. The variables were included. However, other key fac- developer’s perspective was one of the important tors may exist which have a positive association dimensions identified among the consumer, the with and impact on the game development pro- business [111], and the process itself. The findings cess, but due to the presence of the selected seven of this study also provide a justification to include variables in the literature, they were included in these factors in the process assessment methodol- the study. In addition, other key factors may exist, ogy. such as regionally or environmentally based

6.1 Limitations of the study choices, which may have a positive impact on the

For software engineering processes or product game development process, but were not consid- investigations, various empirical approaches are ered in this study. Furthermore, the focus of this used, such as case studies, metrics, surveys, and study was only on developers’ factors affecting the experiments. However, certain limitations are as- enhanced game development process. sociated with empirical studies and with this study Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Another notable limitation of the study is the need to be interpreted carefully [114]. The results small sample size. Although, collected number of of small studies should be used to design larger responses are large in number but they can still confirmatory studies which is the case of this consider small as compared to overall population study as well. size. The vast majority of game developers work In software engineering, the increased popu- in one- to three-person teams and did not have the larity of empirical methodologies has raised con- required level of experience (three years) and were cerns about ethics. This study has adhered to all therefore excluded from this empirical investiga- applicable ethical principles to ensure that it would tion. Most respondents refused to answer the ques- not violate any experimental ethics guidelines. tionnaire because they were too busy in the game Regardless of its limitations, this study has con- development process or launching their games in tributed to the software game development process the market. Therefore, data collection from the and has helped game development organizations game industry was limited, resulting in small sam- understand the developer’s dimension of software ple size. There are some approaches discussed by games. researchers such as by Zhang and Zhang [117] to 7. Conclusions handle the small sample size by using different Game development is a complex process, and machine learning techniques. However, one of the for successful development of good-quality soft- objective to divide data analysis section into three ware games, game developers must consider and phases is to address the small sample size issue. explore all related dimensions as well as discuss- The main effect of small sample size is on its sta- ing them with all the stakeholders involved. This tistical power, Type II error, significance and on study provides a better understanding of the fac- distribution [115]. Therefore, the important thing tors important to developers in the software game is while making conclusion avoid strong state- development process and explores the impact of ments. As, the small samples size studies results key factors on the success of software games from can be difficult to replicate or generalize [116] but a developer’s perspective. This study has mainly they do provide some interplay between variables. tried to answer the research question that was The well designed small studies are seems ok to posed earlier in this paper and to analyze the im- conduct as they provide quick results but they pact of developers’ key factors for game develop- - 32 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. ment process improvement. The results of this kind and will help game developers and game de- empirical investigation have demonstrated that velopment organizations achieve a better under- developers’ key factors are very important and standing of key factors for improving the game play a key role in improving the software game development process. To improve the current development process. The results showed that game development process and develop good- team configuration and management, game design quality games, it is important for developers to document management, game engine develop- consider the identified key factors as well as oth- ment, game test management, and programming ers. Currently, the authors are working on devel- practices are positively associated with the en- oping a software game maturity model for game hanced game development process. The empirical development process assessment. This study has investigation found no strong association or im- provided the empirical evidence and justification pact between quality of game architecture and the to include factors from the developers’ perspective enhanced game development process. In the game in evaluating the developer dimension of game development field, this research is the first of its development process maturity.

[3] Robins, S., (2010). Introduction to Game References Development, Second Edition,

[1] Petrillo, F. & Pimenta, M. (2010). Is agility Independence, KY, USA: Cengage

out there? Agile practices in game Learning.

development, in Proceedings of 28th ACM [4] Blow, J., (2004). Game development:

International Conference on Design of harder than you think, ACM Queue

Communication (SIDOC ‘10), 9–15. Magazine, 1 (10), 28–37.

[2] Delmestri, G., Montanari, F., Usai, A., [5] Kanode, M. C., & Haddad H. M., (2009).

(2005). Reputation and strength of ties in Software engineering challenges in game

predicting commercial success and artistic development, in Proceedings of Sixth

merit of independents in the Italian film International Conference on Information

industry. Journal of Management Studies, Technology: New Generations (ITNG ‘09),

42, 975–1002. 260–265. Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

[6] Kultima, A., (2010). The organic nature of Learning, Cambridge University Press,

game ideation: game ideas arise from U.K., pp. 549–567.

solitude and mature by bouncing, in [11] Keith, C. (2010). Agile Game

Proceedings of the International Academic Development with Scrum. Boston:

Conference on the Future of Game Design Addison-Wesley.

and Technology, ACM Press Future Play, [12] Pressman, R.S. (2001). Software

May 6–7, Vancouver, Canada, 33–39. Engineering: A Practitioner Approach, 5th

[7] McGill, M., (2009). Defining the ed., New York: Wiley.

expectation gap: a comparison of industry [13] Petrillo, F., Pimenta, M., Trindade, F.

needs and existing game development (2009). What went wrong? A survey of

curriculum, in Proceedings of the 4th problems in game development. Computers

International Conference on Foundations in Entertainment, ACM Digital Library,

of Digital Games (FDG ‘09). ACM, New Vol. 7, No. 1, pp. 13.1–13.22.

York, NY, USA, 129–136. [14] Ramadan, R., Widyani, Y. (2013). Game

DOI=10.1145/1536513.1536542. development life-cycle guidelines.

[8] Gredler, M.E. (1995). Designing and Proceedings of 5th International Conference

evaluating games and simulations. on Advanced Computer Science and

Behavioral Science, Wiley Online Library, Information Systems (ICACIS), IEEE

Vol. 40, No. 1, pp. 76–77. Computer Society, Jakarta, Indonesia,

[9] Gredler, M.E. (2004). Games and September 28–29, pp. 95–100.

simulations and their relationship to [15] Lee, S.H., Lee, G.H., Cho, H.H., Song,

learning. Handbook of Research on D.H., Rhew, S.Y. (2006). An empirical

Educational Communications and model of the game software development

Technology, pp. 571–581. process. Proceedings, Fourth International

[10] Rieber, L.P. (2005). Multimedia learning Conference on Software Engineering

in games, simulations, and micro-worlds. Research, Management. and Applications,

Cambridge Handbook of Multimedia Seattle, WA, August 9–11, pp. 371–377. Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

[16] Ayyad, R., Masood, M. (2010). An [20] Bethke, E. (2003). Game Development

optimization of CPS model in computer and Production. Wordware.

game development for non-programmers. [21] Hendrick, A. (2009). Project Management

Second International Conference on for Game Development. June 15, Available

Computer Intelligence, Modelling, and at:

Simulation, IEEE Computer Society, Bali, http://mmotidbits.com/2009/06/15/project-

Indonesia, September 28–30, pp. 125–128. management-for-game-development/.

[17] Pena, J. (2011). Collaborative framework Accessed on May 20, 2014.

for browser game development, [22] Hagan, A.O., Coleman, G., O’Connor,

Proceedings, 2011 Workshop on Open R.V. (2014). Software development

Source and Design of Communication, processes for games: A systematic

ACM Digital Library, Lisbon, Portugal, literature review. 21st European

July 11, pp. 65–72. Conference on Systems, Software, and

[18] Tschang, F.T. (2003). Beyond normal Services Process Improvement (EuroSPI

products and development process: 2014), CCIS Vol. 425, Springer-Verlag,

computer games as interactive experimental June, pp. 182–193.

goods and their manner of development. In: [23] Polk, R. (2011). Agile and Kanban in

What do we Know about Innovation? A coordination. In: Agile Conference

conference in honor of Keith Pavitt, (AGILE), pp. 263–268.

University of Sussex, November 13–15, pp. [24] Birchall, J., Gatzidis, C. (2013). The

1–31. periodic table of elements via an

[19] McGrath, J. (2011). The Game XNA-powered serious game. In: Pan, Z.,

Development Lifecycle: A Theory for the Cheok, A., Müller, W., Liarokapis, F.

Extension of Agile Project Methodology. (eds.), Transactions on Edutainment IX,

Available at: Springer, Berlin, Heidelberg, Vol. 7544, pp.

http://blog.dopplerinteractive.com/ 1–28.

2011/04/game-development-lifecycle-theor [25] Musil, J., Schweda, A., Winkler, D., Biffl,

y-for.html. Accessed on May 1, 2014. S. (2010). Improving video game

- 4 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

development: Facilitating heterogeneous [30] Viana, R., Ponte, N, Trinta, F., Viana, W.,

team collaboration through flexible (2014). A systematic review on software

software processes. In: Riel, A., O’Connor, engineering in pervasive games

R., Tichkiewitch, S., Messnarz, R. (eds.), development, SBC - Proceedings of the

Systems, Software and Services Process SBGames 2014 | ISSN: 2179-2259

Improvement, Springer, Berlin, Heidelberg, [31] Claypool, K., & Claypool, M., (2005).

Vol. 99, pp. 83–94. Software engineering design: teaching

[26] Kasurinen, J., Laine, R., Smolander, K, software engineering through game design,

(2013). How applicable is ISO/IEC 29110 Proceedings of 10th Annual SIGCSE

in Game Software Development? Lecture conference on Innovation and Technology

Notes in Computer Science, Springer Berlin in Computer Science Education (Caprical,

Heidelberg, Vol. 7983, pp. 5-19. Portugal), June, ACM Press, New York,

[27] Kaitilla, C. (2014). How to learn Ouya NY, 123–127.

Gamdev. Available at: [32] Eric, S., De Meuse, K. P., Futrell, D.

http://gamedevelopment.tutsplus.com/articl (1990). Work teams: applications and

es/how-to-learn-ouya-gamedev--gamedev- effectiveness, American Psychologist, 45,

9197. Accessed on December 20, 2014. 120–133.

[28] Penzenstadler, B., Bauer, V., Calero, C., [33] Muchinsky, P., (2003). Psychology

Franch, X. (2012). Sustainability in Applied to Work, Seventh Edition.

Software Engineering: A Systematic Toronto, Canada: Thompson, Wadsworth.

Literature Review. München: La Mancha, [34] Rossen, E. K., Bartlett, R., & Herrick, C.

Catalunya. A., (2008). Interdisciplinary collaboration:

[29] Ampatzoglou, A. & Stamelos, I. (2010). the need to revisit. Issues in Mental Health

Software engineering research for Nursing, 29, 387–396.

computer games: A systematic review, [35] Tran, M. Q., & Biddle, R., (2008).

Information and Software Technology, Collaboration in serious game

52(9), 888-901. development: a case study, Proceedings of Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Future Play, November 3–5, ACM Press, IEEE International Conference on

Toronto, Canada, 49–56. Requirement Engineering (RE’05), August

[36] Stacey, P., Brown, A., & Nandhakumar, 29–September 2, 240–250.

J., (2007). Making sense of stories: the [41] Callele, D., Neufeld, E., & Schneider, K.,

development of new mobile computer (2010). A proposal for cognitive gameplay

game, in Proceedings of the 40th Hawaii requirement, 2010 Fifth International

International Conference on System Workshop on Requirements Engineering

Sciences, January, Waikoloa, HI, 1–10. Visualization (REV), 28th September,

[37] de Barros, R. L.B., Alves, C. F., & Sydney, 43-52.

Ramalho, G. L., (2009). Investigating the [42] Reyno, E. M., Cubel, J. A. C., (2009).

communication process in Automatic prototyping in model-driven

multidisciplinary game development game development, ACM Computers in

teams, 2009 Simposio Brasileiro de Entertainment, 7 (2), Article 29, 1–9.

Sistemas Colaborativos (SBSC), October [43] Almeida, M. S. O., & da Silva, F. S. C.,

5–7, Fortaleza, Ceara, 61–69. (2013). A systematic review of game

[38] Kasurinen, J., Maglyas, A., Smolander, design methods and tools, in Proceedings

K., (2014). Is requirement engineering of ICEC 2013, LNCS 8215, 17–29.

useless in game development? Lecture [44] Ahmed, I., & Jaafar, A., (2011). Game

Notes in Computer Sciences (LNCS) 8396, design and integration with user’s emotion,

Springer, 1–16. in Proceedings of 2011 International

[39] Bosser, A., (2004). Massively multi- Conference on User Science and

player games: matching game design with Engineering (i-USEr), IEEE Computer

technical design, Proceedings of ACE’04, Society, November 29–December 1, Shah

June 3–5, ACM, Singapore, 263–268. Alam, Selangor, 60–72.

[40] Callele, D., Neufeld, E., Schneider, K., [45] Bringula, R. P., Alcid, A. S., Bandril, L.

(2005). Requirements engineering and the B. P., Guzman, A. E. D., & Lopez, L. J. C.,

creative process in the video game (2014). Development of game design

industry, in Proceedings of the 2005 13th guidelines, Proceedings of 2nd International

- 6 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Conference on Technology, Informatics, platform development, in Proceedings of

Management, Engineering, & International Computer Science and

Environment, August 19–21, Bandung, Engineering Conference, September 4–6,

Indonesia, IEEE Computer Society, 234– Nakorn Pathom, Thailand, 313–318.

239. [51] Vanhutupa, J. (2011). On the

[46] Sherrod, A., (2007). Ultimate 3D Game development of browser game

Engine Design and Architecture, Charles technologies of an emerging genre, in

River Media, Boston MA. Proceedings of 7th International

[47] Cowan, B., & Kapralos, B., (2014). A Conference on Next Generation Web

survey of frameworks and game engine for Services Practices (NWeSP), October 19–

serious game development, in Proceedings 21, Salamanca, Spain, 363–368.

of 14th International Conference on [52] Sousa, J. A. P., & Garlan, D., (2002).

Advanced Learning Technologies, July 7– Aura: an architectural framework for user

10, Athens, 662–664. mobility in ubiquitous computing

[48] Hudlicka, E., (2009). Affective game environments, In Proceedings of the IFIP

engine: motivation and requirements, in 17th World Computer Congress, TC2

Proceedings of 4th International Stream / 3rd IEEE/IFIP Conference on

Conference of Future of Digital Games, Software Architecture: System Design,

April 26–30, Orlando, FL, USA, 299–306. Development and Maintenance, ser.

[49] Yan-Hui, W., Xia-Xia, Y., & Jin, H. WICSA 3. Deventer, The Netherlands,

(2011). Design and implementation of a Kluwer, B.V., 2002, 29–43.

game engine based on Android platform, in [53] Aitenbichler, E., Kangasharju, J., &

Proceedings of 2011 International Mühlhäuser, M., (2007). MundoCore: a

Conference on Internet Technology and light-weight infrastructure for pervasive

Applications (iTAP), August 16–18, computing, Pervasive Mobile Computing,

Wuhan, P.R. China, 1–3. 3( 4), 332–361.

[50] Rodkaew, Y., (2013). The last eternity: [54] Pimenta, M. S. C. S., Buzeto, F. N.,

3D role-playing game with a cross- Santos, L. H. O., Castanho, C. D., & Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Jacobi, R. P., (2014). A game engine for [59] Carli, D. M. De., Bevilacqua, F., Pozzer,

building Ubigames, in Proceedings of 13th C. T., & d’Ornellas, M. C., (2011). A

Annual Workshop on Network and survey of procedural content generation

Systems Support for Games (NetGames), technique suitable to game development, in

December 4–5, Nagoya, Japan, 1–3. Proceedings of 2011 Brazilian Symposium

[55] Neto, B., Fernandes, L., Werener, C., de on Games and Digital Entertainment

Souza, J. M., (2009). Reuse in digital game (SBGAMES), November 7–9, Salvador,

development, in Proceedings of the 4th 26–35.

International Conference on Ubiquitous [60] Phelps, A., (2005). Graphics don’t matter

Information Technologies & Applications, (and other assertions), available at:

2009 (ICUT ‘09), December 20–22, http://gotgame.cornate.com/archives/2005/

Fukuoka, Japan, 1–6. 06/13/graphics_dont_matter_and_other_as

[56] Peker, A. G., & Can, T., (2011). A design sertion.php, Accessed on October 24,

goal and design pattern based approach for 2015.

development of game engine for mobile [61] Pranatio, G., & Kosala, R., (2010), A

platform, In Proceedings of 16th comparative study of skeletal and

International Conference on Computer keyframe animations in a multiplayer

Games, July 27–30, Louisville, KY, 114– online game. In Proceedings of 2010

120. Second International Conference on

[57] Llopis, N., (2004). Optimizing the Advances in Computing, Control and

content pipeline, Game Developer Telecommunication Technologies (ACT),

Magazine. December 2-3, Jakarta, 143-145.

[58] Hendrikx, M., Meijer, S., Velden, J. V. [62] Lasseter, J., (1987). Principals of

D., & Iosup, A., (2011), Procedural content traditional animation applied to 3D

generation for games: a survey. ACM computer animation, Computer Graphics,

Transactions on Multimedia Computing, ACM Press, 21(4), 35–44.

Communications, and Applications, 1, 1–

24.

- 8 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

[63] Maestri, G., (2006). Digital Character [68] Migneco, R., Doll, T. M., Scott, J. J.,

Animation 3, Berkeley: New Riders Hahn, C., Diefenbach, P. J. &Kim, Y. E.,

Publishing. (2009). An audio processing library for

[64] Xu, H., & CuiPing, W. (2009). A review game development in Flash, in

and development of 3D accelerator Proceedings of International IEEE

technology for games, in Proceedings of Consumer Electronics Society Games

2nd International Symposium on Intelligent Innovations Conference, August 25–28,

Information Technology and Security London, 201–209.

Informatics, June 23–25, Moscow, 59–63. [69] Wang, A. I. & Norum, M. S. (2006).

[65] Chehimi, F., Coulton, P., Edwards, R. Issues related to development of wireless

(2006), Advances in 3D graphics for peer-to-peer games in J2ME, in

Smartphones. In Proceedings of Proceedings of the Advanced International

International Conference on Information Conference on Telecommunications and

and Communication Technologies: from International Conference on Internet Web

Theory to Applications. April 2006, Applications and Services (AICT/ICIW),

Damascus, Syria, 24–28. February 19–25, Guadelope, French

[66] Manocha, D., Calamia, P., Lin, M., Caribbean , 1–5.

Manocha, D., Savqia, L., & Tsingos, N., [70] Amendola, F., Fernandez, M., Favre, L.

(2009). Interactive sound rendering. In & Aires, B. (2015). GLIESE, a framework

SIGGRAPH ’09: ACM SIGGRAPH 2009 for experimental game development, In

Courses. ACM, New York, NY, USA, 1– Proceedings of 12th International

338. Conference on Information Technology:

[67] Pichlmair, M., (2007). Levels of sound: New Generations, April 13–15, Las Vegas,

on the principle of interactivity in music 528–533.

video games, in Proceedings of Digital [71] Gamma, E., Helm, R., Johnson, R. &

Game Research Association Conference, Vilssides, J. (1995). Design Patterns:

Simulated play, 424–430. Elements of Reusable Object-Oriented

Software, Addison-Wesley Professional. Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

[72] Lukashev, D., Puresev, A., & Communication (PIMRC’07), September

Makhlushev, I., (2006). 3D applications for 3–7, Athens, 1–5.

3G mobile phones: design, development, [76] Al-Azawi, R., Ayesh, A. & Al-Obaidy,

resource utilization, IEEE Tenth M. (2014). Towards agent-based agile

International Symposium on Consumer approach for game development

Electronics, 2006 (ISCE ‘06), St. methodology, in Proceedings of World

Petersburg, 1–4. Congress on Computer Applications and

[73] Rhalibi, A. E., Merabti, M., Carter, C., Information Systems (WCCAIS), January

Dennett, C., Cooper, S., Sabri, M. A. & 17-–9, Hammamet, Tunisia, 1–6.

Fergus, P., (2009). 3D Java Web-based [77] Segundo, R. M., da Silva, J. C. F. &

games development and deployment, in Tavares, T. A., (2010). ATHUS: A generic

Proceedings of International Conference framework for game development on

on Multimedia Computing and Systems, Ginga middleware, in Proceedings of

2009 (ICMCS ‘09), April 2–4, Ouarzazate, Brazilian Symposium on Games and

Morocoo, 553–559. Digital Entertainment, November 8–10,

[74] Jhingut, M. Z., Ghoorun, I. M., Nagowah, Florianopolis, Brazil, 89–96.

S. D., Moloo, R. & Nagowah, L. (2010). [78] Redavid, C. & Farid, A. (2011). An

Design and development of 3D mobile overview of game testing techniques,

games, in Proceedings of 3rd International Västerås: s.n. Available

Conference on Advances in Computer- at:http://www.idt.mdh.se/kurser/ct3340/ht1

Human Interaction, February 10–15, Saint 1/MINICONFERENCE/FinalPapers/ircse1

Maarten, 119–124. 1_submission_15.pdf. Accessed on:

[75] Kosmopoulos, A., Karamichali, I., October 27, 2015.

Kemerlis, V. P. & Polyzos, G. C., (2007). [79] Helppi, V. V. (2015). The agile process

Fueling game development in a mobile for mobile game development and testing,

P2P environment, In Proceedings of 18th Testdroid Blog, June 14, 2015, Available

Annual IEEE International Symposium on at: http://testdroid.com/tech/the-agile-

Personal, Indoor, and Mobile Radio process-for-mobile-game-development-

- 10 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

and-testing, Accessed on 27th October [84] Al-Azawi, R. Ayesh, A. & Obaidy, M. A.

2015. (2013). Generic evaluation framework for

[80] Charles, R. B., Schultz, P. & Langdell, T. game development methodology, in

(2005). Testing All in One, Thomson Proceedings of 3rd International

Course Technology, Boston. Conference on Communications and

[81] Wilson, D. (2009). Quality assurance: A Information Technology (ICCIT-2013):

methodology for wide-spectrum game Digital Information Management, and

testing, Gamasutra Blog, April 28, 2009, Security, June 19–21, Beirut, 55–60.

Available at: [85] Omar, H. M. & Jaafar, A. (2011).

http://www.gamasutra.com/view/feature/1 AHP_HeGES: tools to evaluate usability of

32398/quality_quality_assurance_a_.php. educational computer games (UsaECG), in

Accessed on: 27th October, 2015. Proceedings of International Conference

[82] Marri, K. K. & Sundaresasubramanian, on User Sciences and Engineering (i-

G. (2015). ExPLORE: testing the game. USEr), November 29–December 1, Shah

Available at: http://www.infosys.com/IT- Alam, Selangor, 73–76.

services/independent-validation-testing- [86] Straat, B. & Warpefelt, H. (2015),

services/Documents/test-games-users- Applying the two-factor theory to play

perspective.pdf. Accessed on October 27, heuristics, in Proceedings of DiGRA 2015:

2015. Diversity of Play: Games-Culture-

[83] Kasurinen, J. & Smolander, K. (2014). Identities, 1–12.

What do game developers test in their [87] Sarinho, V. T. & Apolinario, A. L.

products? In Proceedings of the 8th (2009). A generative programming

ACM/IEEE International Symposium on approach for game development, in

Empirical Software Engineering and Proceedings of VIII Brazilian Symposium

Measurement (ESEM’14), September 18– on Games and Digital Entertainment,

19, Tornio, Italy, ACM Digital Library, 1– October 8–10, Rio de Janeiro, 83–92.

10. [88] Czarnecki, K. & Kim, C. (2005).

Cardinality-based feature modeling and Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

constraints: a progress report, in 31st Annual International Computer

Proceedings of International Workshop on Software and Applications Conference,

Software Factories, October 17, San July 24–27, Beijing, 601–608.

Diego, California, USA, ACM Digital [93] Wang, A. I., & Nordmark, N., (2015).

Library, 1–9. Software architecture and the creative

[89] Chen, Y., Keivanloo, I., & Roy, C. K. processes in game development,

(2014). Near-miss software clones in open Entertainment Computing (ICEC 2015),

source games: an empirical study, in Vol. 9353, Lecture Notes in Computer

Proceedings, IEEE 27th Canadian Science (LNCS), 272–285.

Conference on Electrical and Computer [94] Meng, L. S., Kelvin, E. C., Parakash, P.

Engineering (CCECE), May 4–7, Toronto, & Loh, K. K. (2004). Design and

Canada, 1–7. development of a peer-to-peer online

[90] Anderson, E. F. (2011). A classification multiplayer game using DirectX and C#, in

of scripting systems for entertainment and Proceedings of IEEE Region 10

serious computer games, in Proceedings of Conference (TENCON), November 21–24,

3rd International Conference on Games and Chaing Mai, 278–281.

Virtual Worlds for Serious Applications, [95] Cronbach, L. J. (1951). Coefficient alpha

May 4–6, Athens, 47–54. and internal consistency of tests.

[91] Xu, S. & Rajlich, V. (2006). Empirical Psychometrika, 16, 297–334.

validation of test-driven pair programming [96] Osterhof, A. (2001). Classroom

in game development, in Proceedings of Applications of Educational Measurement.

the 5th IEEE/ACIS International Prentice-Hall: New Jersey.

Conference on Computer and Information [97] Nunnally, J. C. & Bernste, I. A. (1994).

Sciences, July 10–12, Honolulu, HI, 500– Psychometric Theory. McGraw-Hill.

505. [98] Van de Ven, A. H. & Ferry, D. L. (1980).

[92] Zhang, W., Han, D., Kunz, T. & Hansen, Measuring and Assessing Organizations.

K. M. (2007). Mobile game development: Wiley: New York.

object-orientation or not, in Proceedings of

- 12 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

[99] Comrey, A. L. & Lee, H. B. (1992). A [106] Martin-Rodriguez, L. S., Beaulieu,

First Course on Factor Analysis, 2nd Ed., M. D., D’Amour, D. & Ferrada-Videla, M.

Hillsdale, NJ: Lawrence Erlbaum. (2005). The determinants of successful

[100] Campbell, D. T. & Fiske, D. W. collaboration: a review of theoretical and

1959. Convergent and discriminant empirical studies. Journal of

validation by the multi-trait multi-method Interprofessional Care, 19, 121–147.

matrix. Psychological Bulletin, 56(2), 81– [107] Pietroburgo, J. & Bush, B. (2008).

105. Coming to terms: a case study of hospice

[101] Kaiser, H. F. (1960). The application collaboration challenges. American Journal

of electronic computers to factor analysis. of Hospice & Palliative Medicine, 24(6),

Educational and Psychological 487–492.

Measurement, 20, 141–151. [108] Huh, J., Ackerman, M.S., Erickson,

[102] Kaiser, H. F. (1970). A second- T., Harrison, S. & Phoebe, S., (2007).

generation little jiffy. Psychometrika, 35, Beyond usability: taking social, situational,

401–417. cultural, and other contextual factors into

[103] Fornell, C. & Bookstein, F. L. account, in Proceedings of CHI, San Jose,

(1982). Two structural equation models: California, 2007, 2113–2116.

LISREL and PLS applied to consumer exit [109] Iuppa, N. & Borst, T. (2010). End-to-

voice theory. Journal of Marketing End Game Development, Focal Press,

Research, 19, 440–452. 225–229.

[104] Joreskog, K. & Wold, H. (1982). [110] Anderson, E. F., Engel, S.,

Systems under Indirect Observation: McLoughlin, L. & Comnions, P. (2008).

Causality, Structure, and Prediction. North The case for research in game engine

Holland. architecture, in Proceedings of Future Play

[105] Westfall PH & Young SS. Resampling 2008, November 3–5, Toronto, Ontario,

based multiple testing. New York: Wiley, Canada, 228–231.

1993. [111] Aleem, S., Capretz, L.F. & Ahmed,

F. (2016). Empirical investigation of key Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

business factors for digital game outbound travel: A machine learning

performance, Entertainment Computing, Approach. Current Issues in Tourism.

Elsevier, Computing, 13, 25-36, Dr. Saiqa Aleem received her MS in Com- http://dx.doi.org/10.1016/j.entcom.2015.09 puter Science (2004) from University of Central .001. Punjab, Pakistan, MS in Information Technology [112] Easterbrooks, S., Singer, J., Storey, (2013) from UAEU, United Arab Emirates and

M. A. & Damian, D. (2007). Selecting PhD. in Software Engineering (2016) from Uni- versity of Western Ontario, Canada. She had many empirical methods for software years of academic and industrial experience hold- engineering research, in Proceedings of ing various technical positions. She is Microsoft, International Conference on Automated CompTIA, and CISCO certified professional with Software Engineering, Atlanta, Georgia, MCSE, MCDBA, A+ and CCNA certifications.

USA. Dr. Luiz Fernando Capretz has vast experi-

[113] Wohlin, P., Runeson, M. Host, M. ence in the software engineering field as practi- tioner, manager and educator. Before joining the C., Ohlsson, B., Regnell & Wesslen, A., University of Western Ontario (Canada), he (2000). Experimentation in Software worked at both technical and managerial levels, Engineering. Kluwer, Norwell. taught and did research on the engineering of

[114] Chow, S., Shao, J., Wang, H., (2008). software in Brazil, Argentina, England, Japan and the United Arab Emirates since 1981. He is cur- Sample Size Calculations in Clinical rently a professor of Software Engineering and Research, 2nd Edition, Chapman & Hall Assistant Dean (IT and e-Learning), and former CRC, Tylaor & Francis. Director of the Software Engineering Program at

[115] Bryman, A., 2008. Social research Western. His current research interests are soft- ware engineering, human aspects of software en- methods. Oxford: Oxford University Press. gineering, software analytics, and software engi- [116] Heckshaw, A. (2008). Small Studies: neering education. Dr. Capretz received his Ph.D. Strength and limitations, European from the University of Newcastle upon Tyne respiratory journal, Vol. 32, Issue 5, pp. (U.K.), M.Sc. from the National Institute for Space Research (INPE-Brazil), and B.Sc. from 1141-1143. UNICAMP (Brazil). He is a senior member of [117] Zhang ， C., Zhang, J. (2013). IEEE, a distinguished member of the ACM, a Analyzing Chinese citizens' intentions of MBTI Certified Practitioner, and a Certified Pro-

- 14 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016. fessional Engineer in Canada (P.Eng.). He can be conference proceedings in the area of software en- contacted at [email protected]; further information gineering. He is a senior member of IEEE. can be found at:

http://www.eng.uwo.ca/people/lcapretz/

Dr. Faheem Ahmed received his MS (2004) and Ph.D. (2006) in Software Engineering from the Western University, London, Canada. Current- ly he is Associate Professor and Chair at Thomp- son Rivers University, Canada. Ahmed had many years of industrial experience holding various technical positions in software development or- ganizations. During his professional career he has been actively involved in the life cycle of software development process including requirements man- agement, system analysis and design, software de- velopment, testing, delivery and maintenance.

Ahmed has authored and coauthored many peer- reviewed research articles in leading journals and Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

- 16 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

- 18 - Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

Journal of Computer Science and Technology, 31(5):925-950, DOI: 10.007/s11390-016-1673-z, Springer, September 2016.

- 20 -