An Assessment of the Ergonomic Quality of a Mobile Cinema App with Novice and Expert Users

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179

Azham Hussain1*, Emmanuel O.C. Mkpojiogu1,2, Yuneswari Muniandy1, Nur Fairuza Syahira Roslan1, Marrium Aslam1

1School of Computing, Universiti Utara Malaysia, 06010 UUM, Sintok, Malaysia 2Department of Computer and Information Technology, Veritas University, Abuja, Nigeria 1*[email protected]

Abstract

Golden Screen Cinemas Sdn Bhd formerly known as GSC or GSC Cinemas is one of the major cinema chains in Malaysia. It provides an online ticketing system where customers are able to purchase movie tickets through mobile app. This study was geared towards testing the ergonomics of the GSC Cinemas app. Twenty respondents were chosen to participate in the evaluation. The results show poor ergonomic quality of the app. The app was inefficient and users were not satisfied with the layout of information presented. Furthermore, users found some difficulty when using the app for the first time. The participants were also not satisfied with some of the features and suggested numerous improvements. Recommendations were made in order to improve on the ergonomics of the GSC Cinemas app. The findings reveal several findings and feedbacks that require improvements on the design of the app.

Keywords: Ergonomic evaluation, mobile cinema app, users’ perception

1. Introduction

Golden Screen Cinemas Sdn Bhd formerly known as GSC or GSC Cinemas is the largest cinema holdings in Malaysia. The biggest cinema is located at Mid Valley Megamall. It houses 21 screen cinemas and has 2763 seats which is the largest in Malaysia. GSC was the first cinema operator in Malaysia to feature ground-breaking audio surround technology, Dolby Atmos, which provides movie buffs a great sound and movie experience. The first ever Dolby Atmos hall was launched in 2013 and is currently available in more than 20 GSC locations. GSC launched the country’s first D-BOX motion seats, offering perfectly synchronized motions with movie scenes at GSC 1 Utama in 2014. The feature has since been extended to over 12 cinemas nationwide. In 2017, GSC also introduced the world’s leading 4D cinematic technology, 4DX in GSC Paradigm JB. Developed by South Korea’s CJ 4DPLEX, 4DX brings the movies to life with over 20 signature effects delivered through vibration and motion-activated seats, as well as environmental effects like wind, fog, rain, lightning, and scents, all of which work together cohesively to deliver a multi-sensory experience. Today, 4DX is available in GSC 1 Utama Shopping Centre and GSC IOI City Mall, Putrajaya, and will soon be available in GSC Southkey, Johor Bahru. Recently, GSC launched GSC Mobile App on two main platforms. That is on Android and iOS. The GSC Mobile App is also now available on the App Store and Google Play Store. The purpose of the Apps is to provide convenience at user’s fingertips as it brings the GSC movie listings with real-time show-times and the option to select seats and purchase movie tickets through the user’s mobile phone. UUM students were used to conduct an onsite evaluation using smartphones on either of the 2 platforms with the latest version of GSC App. The evaluation procedures include the following: introducing the group members and their background, explaining the system and what it is used for, giving the users the smartphone to execute the list of tasks with emphasis on the fact that the testing is on the application and not on the user, giving the users the questionnaire to complete and then collect back. While the evaluation lasts, the study was able to capture each participant’s navigational choices, task completion rates, comments, overall

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179 satisfaction ratings, questions and feedback. Ergonomics defines the comfort with which users of a product interact with the product in such a way that such interaction is without pain [19][22]. . 2. Methodology

Twenty participants were recruited for the study. All participants were students from the Universiti Utara Malaysia (UUM) Sintok Campus. The participants were engaged at the Versity Mall, UUM Campus, to interact with the app following a set of tasks. After interacting with the app, they were given a questionnaire to fill. The questionnaire assesses their perception about the ergonomics of the mobile app’s design. The evaluation was done between the 22nd to the 28th of July, 2019. Of the 20 participants, 80% were female while the remaining 20% were male. Their age ranges are as follows: 75% were aged 28 and below, 20% were aged 29 to 35 while 5% were aged 36 and above. Ten (10) novice and 10 expert users were used for the evaluation. Six tasks were given to participants to carry out. The participants were advised to choose without registration (as a guest) and select desired cinema location as Alor Setar– Aman Central. The tasks are as follows: i. select cinema, ii. select date, movie, time and payment, iii. Set profile, iv. select ticket and e-combo, v. select seat, and vi. make payment.

Task 1: Select cinema: Participants were required to choose from a list the cinema location for the movie they want. The steps include: start application> click the toggle left > Cinemas> choose the cinema> click to buy. Task 2: Select date, movie, time, and payment: Participants were required to select the date and time payment method for their desired movie. Task 3: Set profile: Participants were required to fill out their name, email, mobile numbers and I/C or passport numbers for profile setting. There are three available ways to setup profile. It could be as Login for participants with existing login credential, or by creating a new account or by setup as a guest. Step begins after clicking ‘Next’ in task 2. Task 4: Select ticket and e-combo: Participants were required to select ticket based on the class of the customer: adult, children, OKU, senior citizen, student, and twin and also select e-combo (for snacks). Step begins after clicking ‘Next’ in task 3. Task 5: Select seat: Participants were required to choose their desired seat location for the movie. Step begins after clicking ‘Next’ in task 4. Task 6: Make payment: Participants were required to make payment for the chosen movies according to seat and e-combo selection [1][4][5][8][11][12][17][18][21].

3. Results

Task Completion Success Rate: all participants successfully completed task 1 (select cinema), task 2 (select date, movie, time, and payment) and task 4 (select ticket and e-combo). Besides, task 3 (profile setting) was successfully achieved by 65% of participants. In task 5 (select seat) 83% of participants were successful. However, in task 6 (make payment) only 25% of participants achieved success. The overall GSC Cinemas app’s effectiveness was 50%. This shows that from the users’ perspective, the application enables them to achieve their goals successfully (by 50% rate on the average). In task 3, both novice and experts where faced with some difficulties in editing email addresses in the profile setting and this issue had impact on task 5 and task 6. Some participants proceeded without profile setting. Not all participants completed task 5 (select seat) due to the issues they had with task 3 (profile setting). However, some participants managed to complete and proceed to task 6. Task 6 had the lowest achievement rate because the inconsistency in the app made feel insecure and uncomfortable making payment [2][3][6][7][15][16] [24].

Time on Task: Some tasks were more complicated than others, thus causing delay in carrying out tasks. This reflected on the average time on task. In task 1, participants were to select date, movie, time, and payment. The mean time on task was 153 seconds. Overall, completion times ranged from 60 (approximately 1 minute) to 300 seconds (approximately 5 minutes). Task 2 required participants to select cinema. The average time on task was 336 seconds. Overall, completion times ranged from 60

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179

(approximately 1 minute) to 780 seconds (approximately 13 minutes). Task 3 required participants to set profile and the average time on task was 225 seconds. Overall, completion times ranged from 120 (approximately 2 minutes) to 360 seconds (approximately 6 minutes). In task 4 it is compulsory for participants to select ticket and e-combo. The average time on task was 144 seconds. Overall, the completion times ranged from 60 (approximately 1 minutes) to 300 seconds (approximately 5 minutes). Task 5 was also compulsory for participants as each must select seat. The mean time on this task was 207 seconds. Overall, the completion times ranged from 120 (approximately 2 minutes) to 420 seconds (more than 5 minutes). Task 6 required participants to make payment. The average time on this task was 402 seconds. Overall, the completion times ranged from 180 (approximately 3 minutes) to 1140 seconds (more than 15 minutes) [9][10][13][14][23]. Table 1 shows the time on tasks.

Task (Seconds) Participants Experience 1 2 3 4 5 6 User 1 Novice 120 540 300 180 120 420 User 2 Expert 60 180 120 120 240 180 User 3 Expert 60 300 180 120 120 300 User 4 Expert 120 120 180 60 240 180 User 5 Novice 180 60 300 180 60 480 User 6 Novice 120 660 180 240 300 300 User 7 Expert 120 540 120 120 120 360 User 8 Novice 240 480 180 60 120 540 User 9 Novice 180 780 360 300 420 1140 User 10 Expert 60 300 120 60 300 180 User 11 Expert 180 120 120 60 180 180 User 12 Novice 240 60 360 120 240 420 User 13 Novice 300 240 360 60 300 360 User 14 Novice 180 540 360 240 180 480 User 15 Expert 180 300 120 60 120 360 User 16 Novice 120 300 300 180 300 540 User 17 Expert 180 480 300 180 300 540 User 18 Expert 180 540 120 120 180 420 User 19 Expert 120 120 120 240 180 360 User 20 Novice 120 60 300 180 120 300

TABLE 1. Time on task

The table below (Table 2) displays a summary of the data on task completion, errors and time on task.

Task Task completion Errors while achieving task Time on task

1 20 17 2.05 2 20 9 3.01 3 13 7 2.16 4 20 2 1.23 5 17 4 3.51 6 5 14 4.04

TABLE 2. Summary of task completion, errors and time on task

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179

Perception Rating: After interacting with the app through the respective tasks, participants then rated the perceived ergonomics of the app based on the following three items: i) it was easy to find the information I need from the application; ii) this application has all the function and capabilities I expected it to have; iii) the information was effective in helping me to complete the task. A 5-point Likert rating scale that ranged from 1 (Strongly disagree) to 5 (Strongly Agree) was utilized for the assessment. After task interactive session, participants rated the app with a 15-item questionnaire that is divided into three parts. These items are: 1) this application is simple to be used; 2) I was able to complete the tasks and scenarios quickly using this application; 3) I feel comfortable using this application; 4) It was easy to learn to use this application; 4) this application gave error messages that clearly told me how to fix problems; 5) whenever I made mistake using this application, I could recover easily and quickly; 6) It was easy to find the information I needed; 7) the interface of this application was pleasant; 8) this application has all the functions and capabilities I expected it to have; 8) the organization of information on this application screens was clear; 9) the information was effective in helping me to complete the tasks; 10) this application need to be improved; 11) I would like to use this application in the future; 12) I would like to recommend this application to my friend/other people; and 13) overall, I’m satisfied with this application.

The majority of the participants (85%) agreed (i.e., agreed or strongly agree) that the app need to be improved. On the other hand, 25% agreed that they would like to use the app in the future. However, only 15% agreed that they were able to complete the tasks and scenarios quickly using the app, that whenever they made mistake using the app, they could recover easily and quickly, that it was easy for them to find the information they needed and that they would like to recommend the app to their friends/other people. Similarly, only 10% of the respondents agreed that the application is simple to be used, and that they feel comfortable using the app, that the interface of the application was pleasant, and that the information on it was effective in helping them to complete the tasks. Overall, 10% of participants agreed that they were satisfied with the app. On the other hand, only 5% of participants agreed that it was easy to learn to use the app, that the app gave error messages that clearly told them how to fix problems and that the organization of information on the app’s screens was clear. Lastly, no participants agree that the app has all the functions and capabilities they expected it to have. It thenceforth clear that the interface of the app has serious ergonomic challenges [20]. Details are in Table 3.

Strongly Strongly Mean Percent General Disagree Neutral Agree disagree agree Rating Agree This application is simple 7 8 3 2 0 2.7 10% to be used I was able to complete the tasks and scenarios 0 2 15 3 0 4.1 15% quickly using this application. I feel comfortable using 4 6 8 1 1 3.3 10% this application. It was easy to learn to use 0 1 18 1 0 4 5% this application This application gave error messages that 1 14 4 1 0 3 5% clearly told me how to fix problems Whenever I made mistake using this 0 9 8 3 0 3.6 15% application, I could recover easily and

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179 quickly. It was easy to find the 1 1 15 2 1 4.1 15% information I needed The interface of this 0 1 17 1 1 4.1 10% application was pleasant. This application has all the function and 2 10 8 0 0 3.1 0% capabilities I expected it to have. The organization of information on this 1 1 17 1 0 3.9 5% application screens was clear. The information was effective in helping me to 1 3 14 1 1 3.9 10% complete the tasks This application need to 0 0 1 2 17 6.2 85% be improved I would like to use this 2 1 12 3 2 4.1 25% application in the future I would like to recommend this 1 2 14 3 0 3.9 15% application to my friend/other people Overall, I’m satisfied 3 3 12 2 0 3.5 10% with this application

TABLE 3. Item based Evaluation

This evaluation provides useful feedbacks on the ergonomic quality of the Cinema app. With these feedbacks a number of recommendations were made with justification for each of them in addition to their severity rate. Table 4 displays this information.

Issues observed/experienced by users Justification Severity Recommendation Task 3 – Emails cannot be edited in the Many users had problem High Fix profile page and hence I cannot proceed. managing the profile function and this took the second highest time to complete. Task 3 - Managed to fix the issue with not Many users find High Fix been able to key in email at the profile setup problem to manage page. I need to go to the GSC website to profile function and it update my profile, and then log off from the took more time to app and login back again. Then my profile complete it due to the is auto filled in the app. use of the GSC Website to resolve the issue Task 6 - While purchasing tickets, I was On reaching Task 6, High Fix prompted to re-login. Login failed numerous suddenly prompt issue time. Already requested for password reset leads to user being

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179 via email. Waiting for more than one hour, unable to complete the with no email received from GSC. Request task completely and it for 2nd, 3rd, 4th, 5th, and 6th time, and still took time and made the there is nil response. Please assist. app not user friendly. Task 3 - Profile is gone. Unable to type The most critical issue High Fix email. Annoying Hong Leong Bank banner in task 3 made users on screen while trying to type in my details. uncomfortable while I cannot even see what I typed in the text rating the app due to fields! emotional reflections of the app. Task 3 - The ugliest UI/UX interface. No Got a low percentage of High Fix auto filled information details. Not able to agreement. filled out details in fast ticket option. Only able to filled out name. Advertisement is the biggest display on the screen. Task 6 - Not reliable. I got two vouchers Got a low percentage of High Fix that I was trying to use while purchasing my agreement. ticket on GSC app. However, it was unsuccessful until the last page with the error note unable to lock seat. When I tried again to buy the ticket using the same voucher, the voucher cannot be used anymore. Task 3 - I also understand the needs of The advertisement leads High Fix advertisement but with it covering half of to the most critical the screen when you request for our impact for user while information, it is really annoying. using task 3 which makes them not satisfied.

TABLE 4. Recommendations

As can be seen, the issues in tasks 3 and 6 were very critical and they require urgent attention from the designers of the product. These issues cause a lot of pain and discomfort on users and may likely be responsible to low patronage, conversion rate, and usage of the app.

4. Conclusion

Golden Screen Cinemas Sdn Bhd formerly known as GSC or GSC Cinemas is one of the major cinema chains in Malaysia. It offers an online ticketing system where customers will purchase movie tickets via a mobile app. This study attempts to evaluate the perceived ergonomics of the GSC Cinemas app. The evaluation was conducted with twenty users. The outcome from the study reveals a poor ergonomic experience indicating a poor design of the app that requires improvement. Users viewed the app as difficult, inefficient and unsatisfactory. They were not satisfied with the presentation, organization, and layout of information. In addition, first time users had difficulty using the app. Also, users perceived as unsatisfactory some of the features of the app and aptly recommended some improvements on them. There were serious issues with profile setting (Task 3) and payment making (task 6). Task 3 took so much time to accomplish and was unsatisfactory to users. The same goes for Task 6. Users were unable to finish tasks. Experiences from these two tasks negatively affected the ergonomic experience of the app. Therefore, it is recommended that these issues be fixed.

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179

References

[1] Balagtas-Fernandez, F., & Hussmann, H. (2009). A methodology and framework to simplify usability analysis of mobile applications. ASE2009 - 24th IEEE/ACM International Conference on Automated Software Engineering, 520–524. https://doi.org/10.1109/ASE.2009.12 [2] Hussain, A., Mkpojiogu, E.O.C., Nabeel, N., & Alathwari, A. (2019). Users perception of the mobile usability of a global bicycle sharing platform. International Journal of Interactive Mobile Technologies (IJIM), 13 (1), 125-136. [3] Hussain, A., Mkpojiogu, E.O.C., Ishak, N., & Mokhtar, N. (2019). A study on the perceived mobile experience of MyEg users. International Journal of Interactive Mobile Technologies (IJIM), 13 (1), 4-23. [4] Dray, S., & Siegel, D. (March, 2004). "Remote possibilities?: international usability testing at a distance". Interactions. 11 (2): 10–17. doi:10.1145/971258.971264. [5] Dumas, J. S., & Redish, J. (1999). A practical guide to usability testing. Intellect books. [6] Hussain, A., Mkpojiogu, E.O.C., Ishak, N., Mokhtar, N., & Ani, Z.C. (2019). An interview report on users’ perception about the usability performance of a mobile e-government application. International Journal of Interactive Mobile Technologies (IJIM) 13 (10), 169-178 [7] Hussain, A., Mkpojiogu, E.O.C., & Suleiman, K. (2019). A mobile usability testing of Achik.biz application. Journal of Advanced Research in Dynamical and Control Systems (JARDCS), 11(08- SI), 1024-1028. [8] Gough, D., & Phillips, H. (2003). "Remote Online Usability Testing: Why, How, and When to Use It". Retrived on Aug 18, 2019. [9] Hussain, A., Mkpojiogu, E.O.C., Abubukar, H., & Hassan, H.M. (2019). A mobile usability test assessment of an online shopping application. Journal of Computational and Theoretical Nanoscience.16(5-6), 2511-2516. [10] Jerz, D.G. (2000). "Usability Testing: What Is It?". Jerz's Literacy Weblog. Retrieved Aug 16, 2019. [11] Kaikkonen, A., Kekäläinen, A., Cankar, M., Kallio, T., & Kankainen, A. (2009). Evaluation of user interface design and input methods for applications on mobile touch screen devices. Lecture Notes in Computer Science, 243–246. https://doi.org/10.1007/978-3-642-03655-2_30 [12] Hussain, A., Mkpojiogu, E.O.C., Faromiki, J.O., Abdusalam, E.M.A., & Shamala, P. (2019). Determining the comfort of use of a mobile navigation app. Journal of Advanced Research in Dynamical and Control Systems (JARDCS), 11(08-SI), 1144-1146.. [13] Hussain, A., Mkpojiogu, E.O.C., & Suleiman, K. (2019). A heuristic evaluation of Achik.biz mobile shopping app. International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 123-126. [14] Nadzir, M., Hussain, A., Mkpojiogu, E.O.C., Faromiki, J.O., & Abdusalam, E.M.A. (2019). The effectiveness and efficiency of a GPS route and voice navigation app, International Journal of Innovative Technology and Exploring Engineering, 8 (8S), 425-427. [15] Nadzir, M., Hussain, A., Mkpojiogu, E.O.C., Adetunmbi, S., & Hassooni, M.N. (2019). Assessing the comfort in use of a flipboard mobile news magazine app. International Journal of Innovative Technology and Exploring Engineering, 8(8S), 428-430. [16] Nielsen, J. (1994). Usability Engineering, Academic Press Inc, p 165 [17] Ryu, Y. S. (2005). Development of Usability Questionnaires, (July), 1–206. [18] Hussain, A., Mkpojiogu, E.O.C., Tam, K.L., Hilaluddin, K., & Jamaludin, S. (2019). A measure of the effectiveness and efficiency of an m-cinema app. International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 127-130. [19] Hussain, A., Mkpojiogu, E.O.C., Jamaludin, S., Hilaluddin, K., & Nathan, S. (2019). Users’ perception of their satisfaction and experience on a mobile cinema application, International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 131-134.

International Journal of Advanced Science and Technology Vol. 29, No. 8, (2020), pp. 172-179

[20] Zhang, D., Adipat, B., & Zhang, D. (2005). Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications. International Journal of Human-Computer Interaction, 18(3), 293–308. https://doi.org/10.1207/s15327590ijhc1803 [21] Hussain, A., Mkpojiogu, E.O.C., Karmal, F.M., & Lateef, H.M. (2019). Ascertaining the UX of the word mania mobile app for children using fun toolkit v3. International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 202-205. [22] Hussain, A., Mkpojiogu, E.O.C., Adetunmbi, S., Hassooni, M.N., Basri, S. (2019). An evaluation of the effectiveness in use and efficiency in use of a mobile news aggregator magazine application. International Journal of Innovative Technology and Exploring Engineering, 8(8S), 527-531. [23] Islam, R., Ghani, A.B.A., Kusuma, B., Theseira, B.B. (2016). Education And Human Capital Effect On Malaysian Economic Growth. International Journal of Economics and Financial Issues, 6 (4), pp. 1722-1728. [24] Jauhar, J., Ghani, A.B.A., Joarder, M.H.R., Subhan, M., Islam, R. (2015). Brain Drain To Singapore: A Conceptual Framework Of Malaysians' Diaspora. Social Sciences (Pakistan), 10 (6), pp. 702-711.