Second International Computer Programming Education Conference
Second International Computer Programming Education Conference
ICPEC 2021, May 27–28, 2021, University of Minho, Braga, Portugal
Edited by Pedro Rangel Henriques Filipe Portela Ricardo Queirós Alberto Simões
OA S I c s – Vo l . 91 – ICPEC 2021 www.dagstuhl.de/oasics Editors Pedro Rangel Henriques Universidade do Minho, Portugal [email protected] Filipe Portela Universidade do Minho, Portugal [email protected] Ricardo Queirós Politécnico do Porto, Portugal [email protected] Alberto Simões Politécnico do Cávado e Ave, Portugal [email protected]
Published online and open access by Schloss Dagstuhl – Leibniz-Zentrum für Informatik GmbH, Dagstuhl Publishing, Saarbrücken/Wadern, Germany. Online available at https://www.dagstuhl.de/dagpub/978-3-95977-194-8.
Publication date July, 2021
Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografe; detailed bibliographic data are available in the Internet at https://portal.dnb.de.
License This work is licensed under a Creative Commons Attribution 4.0 International license (CC-BY 4.0): https://creativecommons.org/licenses/by/4.0/legalcode. In brief, this license authorizes each and everybody to share (to copy, distribute and transmit) the work under the following conditions, without impairing or restricting the authors’ moral rights: Attribution: The work must be attributed to its authors.
The copyright is retained by the corresponding authors.
Digital Object Identifer: 10.4230/OASIcs.ICPEC.2021.0
ISBN 978-3-95977-194-8 ISSN 1868-8969 https://www.dagstuhl.de/oasics 0:iii
OASIcs – OpenAccess Series in Informatics
OASIcs is a series of high-quality conference proceedings across all felds in informatics. OASIcs volumes are published according to the principle of Open Access, i.e., they are available online and free of charge.
Editorial Board Daniel Cremers (TU München, Germany) Barbara Hammer (Universität Bielefeld, Germany) Marc Langheinrich (Università della Svizzera Italiana – Lugano, Switzerland) Dorothea Wagner (Editor-in-Chief, Karlsruher Institut für Technologie, Germany)
ISSN 1868-8969 https://www.dagstuhl.de/oasics
I C P E C 2 0 2 1
Alfred Aho and Jefrey Ullman Turing Award Winners for 2021
for their work on the theory of compilers, allowing us to use programming languages.
Contents
Preface Pedro Rangel Henriques, Filipe Portela, Ricardo Queirós, and Alberto Simões . . . . 0:ix–0:x Committees ...... 0:xi–0:xii Authors ...... 0:xiii–0:xv Online-Teaching Environment with Gamifcation – A real Case Study Filipe Portela ...... 1:1–1:13 Moving Classes in a Large Programming Course Online: An Experience Report Hrafn Loftsson and Ásrún Matthíasdóttir ...... 2:1–2:13 Programmers’ Afnity to Languages Alvaro Costa Neto, Cristiana Araújo, Maria João Varanda Pereira, and Pedro Rangel Henriques ...... 3:1–3:7 Melodic – Teaching Computational Thinking to Visually Impaired Kids Rui Costa, Cristiana Araújo, and Pedro Rangel Henriques ...... 4:1–4:14 An Open-Source Gamifed Programming Learning Environment José Carlos Paiva, Ricardo Queirós, José Paulo Leal, Jakub Swacha, and Filip Miernik ...... 5:1–5:8 Integrating a Graph Builder into Python Tutor Diogo Soares, Maria João Varanda Pereira, and Pedro Rangel Henriques ...... 6:1–6:15 Matching User Interfaces to Assess Simple Web Applications Marco Primo and José Paulo Leal ...... 7:1–7:6 Active Methodologies in Incoming Programming Classes João Paulo Aires, Simone Bello Kaminski Aires, Maria João Varanda Pereira, and Luís M. Alves ...... 8:1–8:9 Moopec: A Tool for Creating Programming Problems Rui C. Mendes ...... 9:1–9:7 Automated Java Challenges’ Security Assessment for Training in Industry – Preliminary Results Luís Afonso Casqueiro, Tiago Espinha Gasiba, Maria Pinto-Albuquerque, and Ulrike Lechner ...... 10:1–10:11 Exploring a Board Game to Improve Cloud Security Training in Industry Tiange Zhao, Tiago Gasiba, Ulrike Lechner, and Maria Pinto-Albuquerque ...... 11:1–11:8 A System Architecture to Detect and Block Unwanted Wireless Signals in a Classroom Daniel Barros, Paulo Barros, Emanuel Lomba, Vítor Ferreira, and Pedro Pinto . . 12:1–12:7 A Teaching Assistant for the C Language Rui C. Mendes and José João Almeida ...... 13:1–13:8
Second International Computer Programming Education Conference (ICPEC 2021). Editors: Pedro Rangel Henriques, Filipe Portela, Ricardo Queirós, and Alberto Simões OpenAccess Series in Informatics Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany 0:viii Contents
SHREWS: A Game with Augmented Reality for Training Computational Thinking Francisco Saraiva, Lázaro V. O. Lima, Cristiana Araújo, Luis Gonzaga Magalhães, and Pedro Rangel Henriques ...... 14:1–14:10 Understanding Efects of the Algorithm Visualized with AR Techniques Lázaro V. O. Lima, Manuel Sousa, Luis Gonzaga Magalhães, and Pedro Rangel Henriques ...... 15:1–15:10 Experiments on PR-Based Gamifcation Alberto Simões and Ricardo Queirós ...... 16:1–16:10 User Experience Evaluation in a Code Playground Ricardo Queirós, Mário Pinto, and Teresa Terroso ...... 17:1–17:9 Can I Code? User Experience of an Assessment Platform for Programming Assignments Anne Münzner, Nadja Bruckmoser, and Alexander Meschtscherjakov ...... 18:1–18:12 Preface
The success of the 2020 edition of the ’International Computer Programming Education Conference (ICPEC)’ impelled us to organize a new edition under the general topic of Computing at School. We actually believe that children shall be introduced to the computing world at primary school, and even younger people shall start earlier to train their skills to solve problems acquiring what is nowadays called Computational Thinking. Four speakers from UK, Brasil, and Portugal were invited to talk about their motivations and experiences leading in their countries the referred international movement Computing at School. Following the frst edition of ICPEC, the second edition was once again online. We all know that a remote conference is always restrictive in terms of networking that is usually built at events of this kind. However, the typology of the event did not limit the participation among researchers and teachers on the main topic of the ICEPC, which is, the discussion on methodologies, trends and tools to improve the teaching-learning process of computer programming. Inevitably, the papers in this second edition cover diferent approaches to overcome not only the complexity inherent to the feld of computer programming, but also to mitigate the disadvantages of this new teaching paradigm caused by the pandemic. Many approaches are discussed in this conference, ranging from psychological studies to computer-mediated teaching tools including resources to aid children with special needs. This book compiles 18 papers, accepted and revised for the 2.nd edition of ICPEC’2021, held virtually at Minho University, Braga, Portugal, from 27th to 28th of May. The introduction of specialized services to automate tasks traditionally done manually by teachers or the inclusion of visualization mechanisms, playful design and gamifcation to involve students are the most discussed topics. In the frst case, the tendency is to integrate digital assistants or services in order to alleviate all (or part) of the manual phases of the teaching-learning process of computer programming. In this context, works related to tasks that typically are naturally time- consuming and error-prone are presented, such as the creation of programming exercises, program evaluation and feedback generation. It should be noted that in these topics there is a common point that concerns researchers and that relates to interoperability, not only in terms of data representation but also in the way the data is communicated between systems. In the second case, several works related to game-based solutions are presented. To involve and motivate students in the computer programming domain, there are papers describing the use of visual feedback during the execution of programs, or the injection of gamifcation elements such as the use of leaderboards, achievements, badges and levels. To foster immersion, some authors propose the inclusion of virtual reality or the resort to serious games. Researchers consider that the use of approaches that inherit many concepts from games, should be applied sparingly so systems that use them do not transform into environments that are demotivating, unfair or that foster too many competitive facets which will hinder healthy and cooperative learning process. Regardless of the approach proposed, the main objective of all these works is similar: to motivate students to learn programming by promoting the practice supported by rich and immediate feedback. As ICPEC’2021 Chairs, we want to thank the many people without whom this event would never have been possible.
Second International Computer Programming Education Conference (ICPEC 2021). Editors: Pedro Rangel Henriques, Filipe Portela, Ricardo Queirós, and Alberto Simões OpenAccess Series in Informatics Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany 0:x Preface
The invited Speakers (Simon Peyton-Jones, Sue Sentance, Christian Puhlmann Brack- mann, and Anabela Jesus Gomes) that let us learn with their researches and experiences; all the Members of the Scientifc Committee for their valuable efort reviewing the submissions to support us in deciding the fnal list of accepted papers; all the Members of the Organizing Committee for looking carefully after all the details concerned with the logistics necessary to put up the event. Last but not the least, we express our acknowledgments to: the Authors that communicate their fully implemented ideas or projects, or their fresh proposals that are intended to be realize in the near future; and the Participants that actually made the conference happen and be a fruitful forum for the exchange of experiences and know-how.
Pedro Rangel Henriques Filipe Portela Ricardo Queirós Alberto Simões Committees
Conference Chairs Alexandre Cardoso Federal University of Uberlandia, Brazil Pedro Henriques Universidade do Minho, Portugal Ana Azevedo Instituto Politécnico do Porto, Portugal Ricardo Queirós Politécnico do Porto, Portugal Anabela Gomes Instituto Politécnico de Coimbra, Portugal Alberto Simões Politécnico do Cávado e Ave, Portugal Antonio Manso Instituto Politécnico de Tomar, Portugal Filipe Portela Universidade do Minho, Portugal António Mendes Universidade de Coimbra, Portugal
Steering Committee Antonios Andreatos Hellenic Air Force Academy, Greece Ricardo Queirós Politécnico do Porto, Portugal Bertil Marques Instituto Politécnico do Porto, Portugal Alberto Simões Politécnico do Cávado e Ave, Portugal Cristiana Araújo Universidade do Minho, Portugal Mário Pinto Politécnico do Porto, Portugal Daniela Pedrosa Universidade de Aveiro, Portugal Filipe Portela Universidade do Minho, Portugal Dimitrios Koutsomitropoulos University of Patras, Greece
Organizing Committee Fernando Moreira Universidade Portucalense, Portugal Cristiana Araújo Universidade do Minho, Portugal Filipe Portela Universidade do MInho, Portugal Diana Barbosa Universidade do Minho, Portugal Inna Skarga-Bandurova Dahl East Ukrainian National, Ukrain Goretti Pereira Universidade do Minho, Portugal J. Ángel Velázquez-Iturbide Universidad Rey Juan Carlos, Spain Lázaro Lima Universidade do Minho, Portugal Jakub Swacha University of Szczecin, Poland Paula Tavares Instituto Politécnico do Porto, Portugal José Carlos Paiva Universidade do Porto, Portugal
Program Committee Kostas Kolomvatsos Universidade do Porto, Portugal Alberto Simões Instituto Politécnico do Cávado e Ave, Leonel Morgado Portugal Universidade Aberta, Portugal
Second International Computer Programming Education Conference (ICPEC 2021). Editors: Pedro Rangel Henriques, Filipe Portela, Ricardo Queirós, and Alberto Simões OpenAccess Series in Informatics Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany 0:xii Committees
Manuele Kirsch-Pinheiro Teresa Terroso University of Paris 1, France Instituto Politécnico do Porto, Portugal
Marco Temperini Vitor Sa Sapienza Universita’ di Roma, Italy Universidade Católica Portuguesa, Portugal
Maria João Varanda Pereira Zuzana Kubincová Instituto Politécnico de Bragança, Portugal Comenius University of Bratislava, Slovakia
Mário Pinto Instituto Politécnico do Porto, Portugal
Míriam Antón-Rodríguez Universidad Valladolid, Spain
Martinha Piteira Instituto Politécnico de Setúbal, Portugal
Nikolaos Matsatsinis Technical University of Crete, Greece
Paula Morais Universidade Portucalense , Portugal
Paula Tavares Instituto Politécnico do Porto, Portugal
Pedro Guerreiro Universidade do Algarve, Portugal
Pedro Rangel Henriques Universidade do Minho, Portugal
Pedro Ribeiro Universidade do Porto, Portugal
Pedro Vasconcelos Universidade do Porto, Portugal
Ricardo Queirós Instituto Politécnico do Porto, Portugal
Rita P. Ribeiro Universidade do Porto, Portugal
Roberto Hirata Jr. University of São Paulo, Brazil
Rui Mendes Universidade do Minho, Portugal
Sergio Ilarri University of Zaragoza, Spain
Štefan Korečko Technical University of Košice, Slovak Republic Authors
Alberto Simões Filipe Portela 2Ai, School of Technology, Algoritmi Research Centre IPCA, Barcelos, Portugal University of Minho, Portugal [email protected] IOTech - Innovation on Technology, Portugal [email protected] Alexander Meschtscherjakov Center for Human-Computer Interaction, Francisco Saraiva University of Salzburg, Austria Centro Algoritmi [email protected] Departamento de Informática Universidade do Minho, Portugal Alvaro Costa Neto [email protected] Instituto Federal de Educação, Ciência e Tecnologia de São Paulo Hrafn Loftsson Barretos, Brazil Department of Computer Science, [email protected] Reykjavik University, Iceland [email protected] Anne Münzner Center for Human-Computer Interaction, Jakub Swacha University of Salzburg, Austria University of Szczecin, Szczecin, Poland [email protected][email protected]
Ásrún Matthíasdóttir João Paulo Aires Department of Sport Science, Departamento Acadêmico de Computação Reykjavik University, Iceland Universidade Tecnológica Federal do Paraná [email protected] Brasil Cristiana Araújo [email protected] Centro Algoritmi José Carlos Paiva Departamento de Informática CRACS - INESC-Porto LA, Porto, Portugal Universidade do Minho, Portugal DCC - FCUP, Porto, Portugal [email protected][email protected] Daniel Barros José João Almeida Instituto Politécnico de Viana do Castelo, Centro Algoritmi Portugal Departamento de Informática [email protected] Universidade do Minho, Portugal Diogo Soares [email protected] University of Minho, Portugal José Paulo Leal [email protected] CRACS - INESC-Porto LA, Porto, Portugal Emanuel Lomba DCC - FCUP, Porto, Portugal Instituto Politécnico de Viana do Castelo, [email protected] Portugal [email protected] Lázaro Vinícius de Oliveira Lima Centro Algoritmi Filip Miernik Departamento de Informática University of Szczecin, Szczecin, Poland Universidade do Minho, Portugal [email protected][email protected]
Second International Computer Programming Education Conference (ICPEC 2021). Editors: Pedro Rangel Henriques, Filipe Portela, Ricardo Queirós, and Alberto Simões OpenAccess Series in Informatics Schloss Dagstuhl – Leibniz-Zentrum für Informatik, Dagstuhl Publishing, Germany 0:xiv Authors
Luís Afonso Casqueiro Pedro Pinto Instituto Universitário de Lisboa Instituto Politécnico de Viana do Castelo, (ISCTE-IUL), ISTAR, Lisboa, Portugal Portugal [email protected] ISMAI and INESC TEC, Porto [email protected] Luís Gonzaga Magalhães Centro Algoritmi Pedro Rangel Henriques Universidade do Minho, Portugal Centro Algoritmi [email protected] Departamento de Informática Universidade do Minho, Portugal Luís M. Alves [email protected] Research Centre in Digitalization and Ricardo Queirós Intelligent Robotics (CeDRI) CRACS - INESC-Porto LA, Porto, Portugal Instituto Politécnico de Bragança, Portugal uniMAD - ESMAD, Polytechnic of Porto, [email protected] Porto, Portugal Manuel Sousa [email protected] Universidade do Minho Rui Costa Braga, Portugal Centro Algoritmi [email protected] Departamento de Informática Universidade do Minho, Portugal Marco Primo [email protected] Faculty of Sciences, University of Porto, Portugal Rui Mendes [email protected] Centro Algoritmi, Departamento de Informática, Maria João Varanda Pereira Universidade do Minho, Portugal Research Centre in Digitalization and [email protected] Intelligent Robotics (CeDRI) Instituto Politécnico de Bragança, Portugal Simone Bello Kaminski Aires [email protected] Departamento Acadêmico de Computação Universidade Tecnológica Federal do Paraná Maria Pinto-Albuquerque Brasil Instituto Universitário de Lisboa [email protected] (ISCTE-IUL), ISTAR, Portugal Teresa Terroso [email protected] uniMAD - ESMAD, Polytechnic of Porto, Porto, Portugal Mário Pinto [email protected] uniMAD - ESMAD, Polytechnic of Porto, Porto, Portugal Tiange Zhao [email protected] Siemens AG, Munich, Germany Universität der Bundeswehr München, Nadja Bruckmoser Germany University of Salzburg, Austria [email protected][email protected] Tiago Gasiba Paulo Barros Siemens AG, Munich, Germany Instituto Politécnico de Viana do Castelo, Universität der Bundeswehr München, Portugal Germany [email protected][email protected] Authors 0:xv
Ulrike Lechner Universität der Bundeswehr München, Germany [email protected] Vítor Ferreira Instituto Politécnico de Viana do Castelo, Portugal [email protected]
I C P E C 2 0 2 1
Online-Teaching Environment with Gamifcation – A real Case Study
Filipe Portela # Algoritmi Research Centre, University of Minho, Braga, Portugal IOTech – Innovation on Technology, Trofa, Portugal
Abstract Teaching processes are changing, and Higher Education is not an exception. Professors are adapting their teaching methods to b-learning classes. They are looking for innovative approaches and tools that allow engaging students in the classrooms. This paper presents the results of a teaching and challenging experience. The TeachTeach paradigm was used in a fully online environment. Professors explored several methods/approaches during the semester, and the students were faced-of with a new reality of learning. They attested students skills like programming, resilience, innovation and entrepreneur capabilities during the development of a project with an actual web application. The results show that the eforts were tremendous but benefcial for all the stakeholders. At the end of the case study, a few numbers should be highlighted: 11,173 Downloads, 15,224 Messages, 200,000 Sessions and 208 online hour classes. Comparing this approach with other curricular units (CUnit) online strategies, 96,53% of the students considered it equal or better.
Keywords and phrases TechTeach, Case Study, Online-Learning, ioEduc, Computer Programming, Gamifcation, Innovation
Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.1
Acknowledgements I want to thank IOTECH for supporting the project.
1 Introduction
Nowadays, Education is facing a signifcant digital transformation process. Universities are experiencing a set of essential changes induced by technological and social trends towards digitisation. These changes were aggravated by Coronavirus disease (COVID-19) [1]. COVID- 19 brought new challenges to Education where professors and schools had to take the lead of this unexpected digital transformation without being prepared for it [12]. Higher Education is not an exception and, unfortunately, most professors migrated their presential classes to an online schema without them being transformed/adapted. Several techniques and approaches are being explored, but the teaching and assessment methods are not accompanying the changes, and professors limited their action to transpose their teaching mechanisms to distance classes. Conscientious of this fact, a new and innovative teaching approach was proposed. TechTeach is a disruptive approach in which the focus is the students and their motiv- ation. This approach was designed for several years and showed good results in a presential contact environment; however, two critical questions arise now: “Is TeachTeach ready or adapted to classes 100% online?”. “What is the efort needed? Is it worth it?” This paper will answer both questions after dissecting the experience performed during a semester. All the classes and assessment methods were conducted online, using sync and async activities. This experience was a challenge for everyone, and the results were promising. This article also shows some ideas about converting presential classes to B- Learning approaches with sync and async tasks. In fact, this paper presents a proof of concept of a case study on a Programming Curricular Unit (CUnit) with ten (10) European
Credit Transfer and Accumulation System (ECTS) with almost one hundred and seventy (170) students. Innovation and entrepreneur skills are applied through the development of a real-life case project and exploration of soft-skills. The paper is divided into six sections: after a brief introduction, the paper background is explained. Then, section three presents all the methods and tools used in this case study. The case study is described in section four, and the achieved results are analysed in the discussion section. Finally, a conclusion of the work is provided.
2 Background
This section helps to understand the case study theoretical context better by reviewing some concepts.
2.1 Online Classes Online learning has several characteristics that impact faculty implementation and CUnit progress [9]. One of the most signifcant changes in Education is the classroom converting from presential to online. According to a study [2], online and remote learning in higher education institutes are a necessity in times of COVID. Wahab Ali [2] also mentioned that the massive technology advance represents a shift in educational goals and aspirations. Online classes can have better results with bigger classes [16]; however, the typical conversion of teaching environments is not the best solution.
2.2 Digital Transformation at Classrooms Digital transformation can bring new knowledge and practices of teaching, learning, com- municating, and organising schoolwork [18]. A study [14] showed that the ability to have a digital competence is identifed by someone who can use laptops and various digital learning resources in a positive way. In this aspect, the same study highlighted that the context/en- vironment determine professors’ digital competences. Their decisions are based on their own value frameworks and approaches.
2.3 Related Works This work’s main goal is to do a proof concept of TeachTeach and not address or compare it to other teaching methods. Even though there are other similar works in this area, none was considered.
3 Material and Methods
This case study explored TechTeach and all the related approaches.
3.1 TechTeach TechTeach uses Blended-Learning, Gamifcation, Soft-skills, Quiz and Surveys, Flipped classrooms and Project Base Learning to proportionate the best learning environment for the students [20]. According to the author [21], TechTeach can be adapted to diferent lessons and environments. The case study explores this method in a non presential space and proves its adequacy for online teaching environments. F. Portela 1:3
3.1.1 Project Based Learning Project-based learning (PBL) is an approach to learning [4] and explores the use of exercises to proportionate practical learning. PBL is a student-centred pedagogy [10] and involves a dynamic classroom approach. Professors use the active exploration of real-world challenges and problems to stimulate students to achieve new knowledge [10].
3.1.2 Gamifcation According to Kapp et al. [13], Gamifcation consists of “using game-based mechanics, aes- thetics and game thinking to engage people, motivate action, promote learning, and solve problems”. Learning environments with gamifcation activities can have a positive impact on the learning outcome [7]. In Education, Gamifcation is being used to encourage students to perform specifc tasks; however, the lack of a narrative makes it difcult to measure the engagement infuence [15].
3.1.3 Flipped Classes Flipped learning is an instructional approach used to support teaching [7]. This type of learning allows students to have their own learning environments outside school [22]. The classes using this method are diferent and more practical because professors can use the contact time to discuss the most relevant topics and clarify students doubts instead of having monotonous exposing classes.
3.1.4 Skills Skills achievement is a natural process of human being development [3]. The constant changes in the modern world require more fexibility and more vital adapting skills [17]. According to Araujo [3], this concept is also developed in the educational scope and elevates the cognitive and constructionist pedagogical perspectives to the detriment of behaviourism. In the case study, the two types of skills were trained: technical (web programming and derivates) and soft (resilience, innovation, argumentation, communication, leadership, entrepreneurship, among others).
3.2 Tools This subsection does a brief overview of the tools used in this case study used to apply TechTeach.
3.2.1 Teach Supporting ioEduc is an owner tool of IOTech and arises due to the increasing use of mobile devices at the classroom [23]. This platform was designed to support teachers and students in their tasks during classes [23]. Its web-based features (e.g. attendances, chat, slides, quizzes, surveys, drive, FAQs, peer-assessment) make it an excellent tool to support online classes. ioChat arised as an internal communication platform of IOTech. Its success near the costumers motivated the Team to explore it in a diferent context such as Education. ioChat is based on an open-source software – RocketChat. It is fully customised and has many features like messages/conversations (text, audio and video), rooms (grouped, private, public or discussion), notifcations (online, email, push), plugins (drive, calendar, polls), among others.
I C P E C 2 0 2 1 1:4 Online-Teaching Environment with Gamifcation
Zoom is currently one of the most used video-conferencing platforms. Their robustness allows people to maintain high-quality meetings with a vast number of participants. Zoom allows creating an online teaching environment due to its functionalities. It can connect long-distance participants across rooms systems, desktops and mobile devices to seamlessly bring together from various places [25].
3.2.2 Collaborative and Competitive Kahoot! is characterised as a response system that engages participants through game- like pre-made or impromptu quizzes, discussions, and surveys [8]. Its game-based learning allows professors to support their education activities [5]. According to Carolina et al. [19], “Instructional games are gaining acceptance in the classroom as the eLearning merits of student engagement, and immediate feedback is recognised.” HackerRank [11] can be classifed as a classic competitive programming platform [6] with several diferent types of exercises (e.g. problem-solving or challenges).
3.2.3 Data Analytics Google Analytics is a web analytical tool that helps to analyse websites’ trafc [24]. This tool is essential to understand the tools/websites’ impact/usage.
4 Case Study
The following subsections characterise the case study – Application of TeachTeach on a computer programming unit at the University of Minho to create an entire online learning environment.
4.1 Context To better understand the case study, it is essential to analyse the context/environment of this Curricular Unit (CUnit): Integrated Master degree; 2nd year of study plan; Ten European Credit Transfer and Accumulation System (ECTS); Online from 5 of October 2020 and 29 of January 2021; One hundred and sixty-eight students registered (90% participants); Five professors; Three types of classes: Theoretical (T), Theoretical-practice (TP), laboratory practices (LP); Fifteen weeks of work; Thirteen weeks of contact classes; Fourteen hours of work by week/student; Six hours of contact by week; Two Hours of Tutorial Working Time (OT).
Before the CUnit starts, the coordinator professor defned two mains goals according to the TechTeach paradigm: 1. To Innovate and guarantee the success of a 100% Online CUnit; 2. To use a set of concepts/trends: Gamifcation, PBL, Quizzes, and Flipped Classes and emerging tools to provide students’ best online learning environment. F. Portela 1:5
The working/teaching plan of Web Programming CUnit was based on the following assumptions: Full online classes (100%); Gamifcation mechanisms to assess students’ performance; Online communication platform that allows simulating the classroom environment; Tests and assessments – dynamic, inclusive and online; Preparation of learning content/tutorials to be carried out outside of classes (OT); Online repository (drive) with complementary information (books, articles, videos or tutorials); FAQs with the most common issues; Recording of theoretical classes; Project-based learning with the application of a real case; Room for doubts and online chat; Continuous assessment of the CUnit – direct contact with the class delegates; Exploration of soft-skills (e.g. entrepreneurship, cross-learning, communication, adaptab- ility, resilience and leadership).
4.2 Approach TechTeach was designed to be used by any teacher who wants to innovate and have a diferent teaching view. This CUnit has a strong focus on the project component. So, to ensure the correct implementation of this mechanism, professors used several approaches.
4.2.1 Classes All the classes ran online, and three tools were used: Zoom – At the beginning of each theoretical class, the professor started the recording and shared the ioEduc live system (containing the slides and other features) in Zoom. After a brief explanation of the week’s main topics, the professor created several breakout rooms, and the students were randomly distributed. Students were faced with solving some challenges in a group according to the topics addressed in this phase. They can ask for support here or in ioChat, and professors were jumping between each room. Each room was an average of 5-6 students, i.e., each class had 20 rooms on average. Before classes ending, all the students returned to the main room, and the professor performed a rafe among the attendees to determine which students have a bonus in the weekly quiz. ioChat – ioChat was the main communication tool. The professor created several conversation rooms according to classes, teams and groups. All the students and professors were registered in ioEduc and had access to this communication tool. At the beginning of each class (TP and LP), the professor started a video-call with all of the students. The Professor used this frst video call to talk a little about the class goals. Then, all the students went to their particular room and initiated another video-call. Professor visited each room to explain possible doubts and to give support in the project development. This tool allowed professors to have parallel classes on the same platform. In the TP classes, students were grouped by Team according to their role (front-end, back-end and full-stack). In the LP classes, the students were organised by group, following the allocation of fgure 2 1. ioChat also had a set of creation rules (rooms name, username, photo, among other details) to quickly identify students, groups, projects, and professors. This tool was also used to promote and facilitate the communication inside and outside of the classes between all the stakeholders.
I C P E C 2 0 2 1 1:6 Online-Teaching Environment with Gamifcation
ioEduc – ioEduc was the centre of everything. All the slides were available in this platform; students can, for example, ask questions, answer the quizzes, consult FAQs or perform a peer-assessment. In all classes the attendances are registered at ioEduc through the attendances system. ioEduc’ drive had T recorded classes and many books, tutorials, and videos to support the OT work. Regarding the professors, the attribution of cards was done here as well. ioEduc also has a connection to many complementary tools useful to help the classes (e.g. Cloud 9 AWS, GitHub or Heroku).
4.2.2 Project
The project developed in this CUnit tried to bring some innovation and entrepreneurship to the entire process and asked students to propose a solution to a real case. In a brief overview of the project statement, it is important to mention the following points: 1. The company “Secure4All” is responsible for the coordination of operations. It needs a web application that helps managing each of the occurrences received, and a web / mobile application that allows the interaction of operations on the feld with their occurrences. 2. The intention was to develop a web/mobile application/system (PWA) that supports a public intervention unit’s management. 3. In an initial phase, Secure4All defned a strategy to digitise processes for some of its intervention units, namely: Firefghters (Fire), PJ (Cyber-Attack), GNR (Events), Municipal Police (Assaults), ASAE (Food Inspection), INEM (Accident), Maritime Police (Drowning), and PSP (Disturbances). 4. The teams selected the project themes, and each Team was composed by three groups. Each group was responsible for developing a set of features/components. 5. The Team gathered and defned the components for each group. Then, each group was responsible for developing the entire Front-end (visual) and Back-end (Server) layer of the assigned component solution. 6. ioChat was the communication platform between all the stakeholders.
Regarding the team, the distribution was the following: Group I – Operations Center; Group II – Operations manager / Administrators; Group III – Field Operations Ofcer; Group IV – Audit (external entity).
To better comprehend students distributions by each class type, fgure 1 presents how the teams and groups were organised. The teams were constituted according to the TPs shifts. Each TP shift had two teams with a specifc theme, and each Team had three or four groups. The groups were composed by 6 six elements (+/- 1 depending on the number of students in the shift).Each group was composed by at least 2 Front-End students, 2 Back-End students, and 1 Full-Stack student. Each element was responsible for the development of a set of functionalities/interactions. Each group’ functionalities was divided between front-ofce (without login) and back-ofce (with login). Each Team created the terms of reference plan at the begin of the project. This document included project requirements, distribution of roles and tasks by the group and members, and a projected project cost (i.e., the grade that they want to achieve if they meet all the requirements).The terms of reference represents their commitment to the client (teaching team) and must show their innovative and entrepreneurial vision to the project. F. Portela 1:7
Figure 1 Teams and Groups distribution retrieved from [21].
4.2.3 Assessment This CUnit has a transversal and continuous assessment method that evaluates students, professors, and the respective unit. The students’ assessment was divided into three types: A weekly quiz with a bonus to award motivation, work, and participation; Project with the cards system to value skills; Mini-tests with three synchronised methods to assess diferent parts of knowledge (Front- end, Back-end and Full-stack). The CUnit and professors were also evaluated in two moments in the middle and the end of the semester. This assessment was used to assess several factors like professors knowledge, teaching skills, CUnit plan, and classes.
4.2.4 Gamifcation Gamifcation was applied in several moments of the CUnit, and professors attributed positive and negative points during the classes. A narrative was designed for each method, and students knew from the beginning what they had to do in order to achieve specifc goals. Gamifcation was applied in the following exercises/contexts: MT1 and MT3 – Grades were calculated using a metric based on the student knowledge, questions weight, and answering time according to the average time of respondents (3rd quartile). MT2 – Exercise 1 had code gaps, and students must complete it with their personal data. Then students had to use their skills/knowledge to unlock exercise 2.2. In the end, students achieved one of three possible grades: -1 Student does not know the basics and should reprove to this component – 0%; 0 Student knows the basic and can advance – 100%; 1 Student knows the basics and overcomes all the challenges (e.g. unlock the 2nd part of the exercise) and must advance – 110%. The fnal grade of MT was achieved by the formula avg(MT1, MT3) * MT2. Quiz with bonus – The students had the opportunity to duplicate their weekly quizzes results. The Professor performed every T classes a selection of 10 to 15 students. The rules were simple: only the attendants of that T class were eligible, and no one could be selected more than two times without all the students being selected at least one time.
I C P E C 2 0 2 1 1:8 Online-Teaching Environment with Gamifcation
In the end, the student with the most points without a bonus (K) had a grade of 20. All the students with points with a bonus higher than the K had also 20; then all the other students had a relative grade regarding K. The goal of this exercise was to motivate and award students who could hit more questions. Cards System – The card system was applied during the project. Team leaders and professors attributed yellow (negative) or white cards (positive) to the students. Rescue System – This system was only available in MT2 to the students who had -1 grade and considered it unfair. In this situation, professors analysed the student case and, in case of acceptance, they allowed him to continue with a penalty of ffteen per cent (15%) in the fnal MT grade (i.e., MT2 equal to 85%).
4.3 Numbers
The following numbers help to understand the impact and the efort needed to have a CUnit with a complete online-learning environment: 13 shifts (1T, 4TP, 8LP); 1 Real project with three evaluation phases; 8 teams (+/- 20 students) and 27 groups (+/- 6 students); 3 synchronous Mini-Tests (2 Kahoot, 1 HackerRank); 2 CUnit and professors evaluations (middle and end); 115 was the average number of responses in CUnit fnal evaluation; 85% of students approved the CUnit gamifcation system; 80% was the minimum attendance of T Classes; 144 was the number of bonus attributed; 21 cards (Blue, White, Yellow and Red) were attributed; 3 students activated the rescue system. In this analysis, it is essential to highlight some of the achieved online results: 208 (two hundred and eight) hours online classes with parallel sessions, in a total of 338 (three hundred and thirty eight) hours; 80 (eighty) chat rooms; ∼ 220k page views; ∼ 25k access to the CUnit support platform (sessions); ∼ 16k access to the chat (sessions); ∼ 15k messages exchanged in the chat; ∼ 11k fle downloads; ∼ 106 uninterrupted days online (sum of all active sessions time).
A visual analysis of the CUnit impact on the students’ online usage and assessments is presented in fgures 2 to 8. Figures 2 to 5 resume some of the numbers that are possible to achieve from ioEduc and ioChat (such as page views, events, sessions, among others). Regarding Figures 6 to 8, students evaluated TechTeach approach and their Gamifcation and online component. These fgures prove the results mentioned in the before list and sustain the assumptions taken with this study. F. Portela 1:9
I C P E C 2 0 2 1 1:10 Online-Teaching Environment with Gamifcation
Figure 6 How was the gamifcation approach? Figure 7 Did you like having online classes?
Figure 8 How do you classify this approach (TechTeach) when comparing it with other CUnit?
4.4 Word Cloud
The bag of words is an excellent technique to present students’ opinions. After receiving the students’ comments, the most positive words fgure (9) – dynamic classes and professors – and negative aspects fgure (10) – much mater and materials – were identifed, as can be observed in the fgures. F. Portela 1:11
Figure 9 Positive aspects of CUnit. Figure 10 Negative aspects of CUnit.
5 Discussion
After concluding this case study, the achieved numbers can give a detailed overview of the students and professors interactions and engagement. There were more than two hundred thousand (232,433) page views, and the average session time was approximately ffteen minutes (15m25s). Users exchanged more than ffteen thousand messages (15,224), and eleven thousand (11,173) downloads were made. A particular fact to mention is the working time: only two periods of time did not have anyone online – Monday from 4 am to 5 pm, and Tuesday from 5 am to 6 am. In all the other hours/day, at least one user was online. Both platforms have at least ffteen thousand accesses during the semester. In terms of the mini-tests, the strategy used was to have simple exercises under pressure (without much time to think), and questions containing personal attributes that contributed to making each exercise unique. Generally, students liked the MT1 & MT3 (62.39 % approved the model) but disliked MT2. Both results showed the success of the strategy. MT1 and MT3 were easy and allowed to understand the basics, and MT2 was challenging to do copy or thinking; because of that, the students did not liked it. Finally, and regarding the CUnit, students’ opinion was very positive and motivating. More than ninety-fve per cent of students (96,53%) considered this CUnit on the average or higher than average compared with other CUnits in this time of COVID-19. In terms of the online class question, it is essential to mention that, although most of the students (60%) liked it, 40% answered “no” The negative answer was essentially due to working conditions (e.g. one computer at home to many users/students or lousy internet). On the positive side is the Teaching Team and CUnit and TechTeach as one whole, focusing on the professors’ availability and dynamism. On the negative side, many answers were “nothing” to mention; however, students indicate some negative aspects related to the quantity of matter and materials taught in the Theoretical classes. Unfortunately, this issue has years and happens because this CUnit is the only one that addresses this thematic in the entire course. So the time is not enough to address all topics, but it is not possible to reduce more because it is crucial to give the students a global overview of Web Programming.
I C P E C 2 0 2 1 1:12 Online-Teaching Environment with Gamifcation
6 Conclusion
This work was used as proof of concept and the case study explored allowed professors to obtain an answer to the research question: “Is TeachTeach ready or adapted to classes 100% online?”. The answer is Yes. Both stakeholders (students and professors) considered this approach a success (>80% positive answers). Although good results achieved – more than 220k page views, 40k sessions, 11k downloads of fles and 15k messages – it is vital to highlight the efort need to put into practice, for example, the coordinator’ efort was ∼ 250% multiplied by their teaching hours (8) in an average of 20h a week. With this study, it was also possible to conclude that it is essential to have some “free” time to apply this methodology and keep students engaged. It is necessary to make an extra efort, a good teaching team and, as the word cloud shows, many availabilities to help the students. TechTeach is adequate to the constantly changing teaching system; it brings fresh air to this type of classes and has a favourable acceptance. Unfortunately, this study also showed a considerable gap. In some cases, and although the results are motivating, students fear and distrust online classes. On the other side, the limitations are notorious and, from the 40% of students who did not liked to have online classes, 80% considered that they do not have the correct conditions to do so. Making a balance and after analysing students opinion (in a time of COVID-19, >95% of students considered this CUnit equal or better than the others online CUnit) and their grades (95% of the active students were approved), the results are much rewarding and ofset the efort. As the main contributions of this work are: A guide of strategies/tools needed to have an entire online class; An idea of the efort needed to have this type of classes; Case study results and the receptivity of students to the TechTeach approach. With this case study, it was possible to prove that the TechTeach approach can motivate students skills and promote innovation and entrepreneurship of them at a distance (online). As future work, all the data will be dissected, and a Web Mining process will be applied to try to correlate the students’ performance with their activity.
References 1 Emilio Abad-Segura, Mariana-Daniela González-Zamar, Juan C. Infante-Moro, and Germán Ruipérez García. Sustainable management of digital transformation in higher education: Global research trends. Sustainability, 12(5), 2020. doi:10.3390/su12052107. 2 Wahab Ali. Online and remote learning in higher education institutes: A necessity in light of covid-19 pandemic. Higher Education Studies, 10(3):16–25, 2020. 3 C.V. Araújo. The bologna process and curricular changes at higher education: what are skills for? Educação e Pesquisa, 44, 2018. 4 Stephanie Bell. Project-based learning for the 21st century: Skills for the future. The clearing house, 83(2):39–43, 2010. 5 Marisa Correia and Raquel Santos. Game-based learning: The use of kahoot in teacher education. In 2017 International Symposium on Computers in Education (SIIE), pages 1–4. IEEE, 2017. 6 Alexandru Dan and Ass Pr PhD Iftene Adrian. Computer science platform, 2018. BACHELOR THESIS. 7 C. Davis. Flipped or inverted learning: Strategies for course design. In Enhancing Instruction with Visual Media: Utilizing Video and Lecture Capture, page 25. Information Science Reference, 2013. 8 Ryan Dellos. Kahoot! a digital game resource for learning. International Journal of Instruc- tional technology and distance learning, 12(4):49–52, 2015. F. Portela 1:13
9 Amber D Dumford and Angie L Miller. Online learning in higher education: exploring advantages and disadvantages for engagement. Journal of Computing in Higher Education, 30(3):452–465, 2018. 10 Glavin. Examples of project-based learning | k12 academics. https:// www.k12academics.com/Educational%20Practices/Project-Based%20Learning/ examples-project-based-learning, 2019. (Accessed on 03/09/2021). 11 HackerRank. Hackerrank. https://www.hackerrank.com/. (Accessed on 03/15/2021). 12 Netta Iivari, Sumita Sharma, and Leena Ventä-Olkkonen. Digital transformation of everyday life – how covid-19 pandemic transformed the basic education of the young generation and why information management research should care? International Journal of Information Management, 55:102183, 2020. Impact of COVID-19 Pandemic on Information Management Research and Practice: Editorial Perspectives. doi:10.1016/j.ijinfomgt.2020.102183. 13 K.M. Kapp. . The gamifcation of learning and instruction: Game-based methods and strategies for training and education. John Wiley & Sons, 2012. 14 Anders D Olofsson, Göran Fransson, and J Ola Lindberg. A study of the use of digital techno- logy and its conditions with a view to understanding what ‘adequate digital competence’may mean in a national policy initiative. Educational studies, 46(6):727–743, 2020. 15 Paula Toledo Palomino, Armando M Toda, Wilk Oliveira, Alexandra I Cristea, and Seiji Isotani. Narrative for gamifcation in education: why should you care? In 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), volume 2161, pages 97–99. IEEE, 2019. 16 Marc Parry. Online, bigger classes may be better classes. Chronicle of Higher Education, 57(2):A1–A22, 2010. 17 C. Partenie. The bologna process: Between past reforms and the innovative future. In Conference: International Multidisciplinary Scientifc Conferences on Social Sciences and Arts, Bulgaria, 2014. Albena. 18 Fanny Pettersson. Understanding digitalization and educational change in school by means of activity theory and the levels of learning concept. Education and Information Technologies, 26(1):187–204, 2021. 19 Carolyn M Plump and Julia LaRosa. Using kahoot! in the classroom to create engagement and active learning: A game-based technology solution for elearning novices. Management Teaching Review, 2(2):151–158, 2017. 20 Filipe Portela. A new and interactive teaching approach with gamifcation for motivating students in computer science classrooms. In First International Computer Programming Education Conference (ICPEC 2020). Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2020. 21 Filipe Portela. Techteach—an innovative method to increase the students engagement at classrooms. Information, 11(10):483, 2020. 22 M.S. Ramírez-Montoya. Inverted learning environments with technology, innovation and fexibility: Student experiences and meanings. Journal of Information Technology Research (JITR, 9(1):18–33, 2016. 23 Miguel Silva, Diogo Ferreira, and Filipe Portela. Ioeduc - bring your own device to the classroom. In Ricardo Queirós, Filipe Portela, Mário Pinto, and Alberto Simões, editors, First International Computer Programming Education Conference, ICPEC 2020, June 25-26, 2020, ESMAD, Vila do Conde, Portugal (Virtual Conference), volume 81 of OASIcs, pages 23:1–23:9. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020. doi:10.4230/OASIcs.ICPEC.2020. 23. 24 Bill Su. What is google analytics, and why is it important to my business? | by bill su | analytics for humans | medium. https://medium.com/analytics-for-humans/ what-is-google-analytics-and-why-is-it-important-to-my-business-8c083a9f81be#, 2017. (Accessed on 03/16/2021). 25 Jane Sutterlin. Learning is social with zoom video conferencing in your classroom. ELearn, 2018(12), 2018.
I C P E C 2 0 2 1
Moving Classes in a Large Programming Course Online: An Experience Report
Hrafn Loftsson # Ñ Department of Computer Science, Reykjavik University, Iceland Ásrún Matthíasdóttir # Ñ Department of Sport Science, Reykjavik University, Iceland Abstract We present an experience report on moving face-to-face classes in a large CS1 course to an online format, during the COVID-19 pandemic. The course is based on the fipped classroom approach and team-based learning. Students prepare for classes by reading specifc chapters of the textbook and/or by watching pre-recorded videos. The classes are synchronous, in which students take quizzes and work on programming assignments in teams, with the guidance of tutors. To evaluate the implementation, we compared the results from surveys and exams between 2019 and 2020. The results show that students were at least as satisfed with the online classes in 2020 in comparison with face-to-face classes from the previous year, and generally satisfed with the organization of the course and the learning experience. Moreover, we found no discernible change in the grades on the midterm exams and the fnal exam between the two years. In the future, we might allow the students to choose the class format that best fts their individual needs. 2012 ACM Subject Classifcation Applied computing → Education Keywords and phrases online learning, fipped classroom, team-based learning, introductory pro- gramming, CS1, COVID-19
Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.2
Our surveys show that students were at least as satisfed with the online classes in comparison with face-to-face classes from the previous year. Moreover, we found no discernible change in the grades on the midterms and the fnal exam between the two years. We believe that the work described in this paper can be benefcial to teachers of CS1 courses who want to apply FC and TBL in their online courses. This paper is organized as follows. We discuss related work in Section 2, present background for our work in Section 3, and the moving of our classes online in Section 4. The results and discussion regarding student surveys and exams are presented in Section 5 and 6, respectively. Finally, a conclusion is presented in Section 7.
2 Related work
2.1 The Flipped Classroom Learning programming can be difcult for many students, which often leads to high failure and drop-out rates from programming courses [15]. Consequently, several CS departments have experimented with using the FC teaching method in introductory programming courses. In the FC, the “traditional” lecture is replaced with in-class activities and students are supposed to come prepared for class by watching pre-recorded videos and/or reading given text material. In what follows, we briefy review a few of the recent papers that have shown benefts of using an active learning approache like FC in introductory programming courses. Elmaleh and Shankararaman [4] report on the impact of implementing a course, with 280 students, using FC. They observed that in comparison to a previous “traditional” running of the course, the FC increased pass rates in the fnal exam and also enhanced competency acquisition. Wang et al. [22] present an experience report about using FC and PBL in a course with 132 students. They show that the adoption of FC and PBL brought signifcant enhancement to students’ performance in the fnal exam, compared to previous “traditional” running of the course. However, they also experienced inadequate preparation before class by some of the students and lack of enthusiasm in classroom interaction. Mohamed [12] evaluates the extent to which FC, combined with pair programming, enhances students learning in a relatively small (90 students), mixed-ability CS1 course. The study showed that the use of FC increased the average class grade, pass rate, and course ratings compared to a previous corresponding non-fipped course. Battestilli et al. [1] present preliminary results, from a course of 219 students, in which signifcant diferences in students’ performance is identifed based on how active they are in the online activities ofered as part of the FC approach. Sprint and Fox [18] present the implementation of a FC, gamifed CS1 course, designed to motivate students to improve their study habits. The study showed that students in the FC course submitted programming assignments and online quizzes earlier and with fewer late submissions compared to students in a “traditional” course. Nevertheless, these improved study choices did not lead to higher fnal exam scores.
2.2 Online Courses Students, teachers, and authorities had to adapt quickly to new pedagogical models and organisation of teaching delivery in 2020. Higher education institutions were informed by government mandates, executive orders, or recommended best practices to convert the modality of instruction to an online learning platform [5, 16]. H. Loftsson and Á. Matthíasdóttir 2:3
Despite the fact that online courses have become increasingly popular at universities in the last few years [7, 19], the literature only contains (to our best of knowledge) a few recent papers about converting face-to-face classes to an online format for specifc CS courses. Irani and Denaro [7] describe their experience in creating an online course, with active learning strategies, in Discrete Mathematics and compare it to a corresponding “traditional” face-to-face course. They found no discernible diference in student performance between the two class versions. Subramanian and Budhrani [20] redesigned an object-oriented systems face-to-face course as an online course. The content of the course was retained, while its structure was signifcantly modifed to use a PBL approach. They found that various factors are critical for a successful online course, e.g. course structure, content scope, project design, assessment design, instructional resources and tools. Related descriptions and studies can be found in other felds. Wang and Goryll [21] discuss the experience of changing a face-to-face 15-week lecture-lab Digital Design course to online format. Student retention and performance was similar between the online and the face-to-face course, and students reported overall satisfaction with the online course. Suggested improvements are related to development of course materials, better student support and integrating teamwork in the course. Gottipatti and Shankaraman [5] present how a Master’s degree course in Text Analytic and Applications was rapidly moved over to an online format during the COVID-19 crisis. The lesson learnt was that changing a course from face-to-face to an online format is not an easy task, and that students’ feedback is valuable in the design process. The results suggested that it might be necessary to reduce the course content when using an online form, that the instructor needs time to learn how to master advanced features of the technology tools, and the student also need time to adjust to new methods of studying. Student evaluation showed that a large majority of the students were satisfed with the online course. Roy and Covelli [16] describe moving courses in a liberal arts institution from a face-to-face format to an online format when half of the semester had passed. They found that the move, for both faculty and staf, was easier for those with prior experience with an online format and/or for those who felt comfortable with an online format. Moreover, a majority of the students expressed less interest than before in taking online classes. Grimmer et al. [6] investigated the transition to online teaching (when the semester has already started) in an academic literacies course, and how to use the experience both for future online and face-to-face courses. They recognised online methods that could be used in face-to-face environment e.g., online classroom chat in lectures. The retention rates of their face-to-face students that were forced into the online learning environment was in line with the last fve years in face-to-face learning in the course.
3 Background
The CS1 course at Reykjavik University is a large course. In fall 2020, 502 students, mainly from the departments of CS, Engineering, and Business, registered for the course. The students from CS take the course during their frst semester, while the students from Engineering and Business pursue the course in their third semester. Our CS1 course thus “attracts a diverse crowd of students who bring mixed abilities and backgrounds to the classroom” [12]. The course does not assume any prior programming knowledge and uses Python to introduce fundamental CS1 programming concepts, e.g. variables, types, control structures, and functions, as well as built-in data structures like strings, lists, and dictionaries. The
I C P E C 2 0 2 1 2:4 Moving Classes Online
concept of a class is introduced and how it supports encapsulation and information hiding in the context of object-oriented programming. Students learn to use both an Integrated Development Environment (IDE) and command prompt mechanisms for the development and execution of programs. In [10], we described our experience and the results of using a FC and a TBL approach in an introductory programming course during fall 2018. In fall 2019, we presented several improvements to the course [11], e.g. we reduced the pressure of submission of assignments during classes, emphasized textbook reading to a greater extent, showed short videos at the beginning of each class, and provided the students with several videos (for the student to watch at home) that demonstrated how to apply functional decomposition. A comparison of the results from student surveys given in both years showed that students were in general satisfed with the changes made and the performance on the exams was better in 2019 compared to 2018. The implementation in 2019 can be summarized as follows:
One faculty member, the main instructor, was responsible for the overall organization of the course (syllabus, assessment, quizzes, projects, exams, etc.). The students were divided into several sections, with 50–70 students in most sections. Inside each section, the students were divided into teams of 5–6 students. Each section met two days a week in class, for four lecture hours each day. Each section/class had one teacher and one teaching assistant (TA) as facilitators and tutors. Whenever the students had a question they could ask the tutors for help. Students were supposed to come prepared for classes by reading specifc chapters of the textbook [13] and, in some cases, by watching a video (made by the main instructor). At the beginning of each class, a 15–25 minutes video (made by the main instructor) was shown in each of the sections. Each video gave an overview of the concepts to be discussed/worked on in the class. A short individual quiz, containing ten multiple-choice questions, which were directly linked to the given textbook material and the video, was given in most of the classes after the video had been played. Thereafter, students discussed the same quiz in their teams, which turned in a single collective answer for each of the questions. For the remainder of the class, the students were given several short programming assignments to work on in the teams. Students were encouraged to work together on the solutions, but each student needed to submit his/her solution before the class fnished. Students’ solutions to each assignment were graded using automatic tests. In addition to the short programming assignments given during class, the students were given larger programming projects each week to be worked on at home, optionally in a group of two students. Mimir Classroom1 was used for quizzes and programming assignments/projects. The quizzes and programming assignments in class were automatically graded by Mimir, whereas the weekly programming assignments were graded by a group of TAs. Piazza2 was used as the question-answering platform, and Canvas3 as the Learning Management System. The results from the student surveys, presented by the authors in [11] show that, overall, the students in the programming course liked to work in teams with fellow students, that the discussion with fellow students in class helped them to learn, and that they felt that communication with teachers in class helped them to study.
1 https://www.mimirhq.com/ 2 https://www.piazza.com/ 3 https://www.instructure.com/canvas/ H. Loftsson and Á. Matthíasdóttir 2:5
4 Moving Classes Online
In light of the successful implementation described in Section 3, we intended to use the same format for the course in fall 2020. However, due to the COVID-19 pandemic, we needed to move the classes online. The main challenge we faced was how to exercise both TBL and FC when face-to-face classes were not allowed. We were indeed quite worried that applying these teaching methods would not turn out to be successful, when running the classes online. In order to emulate a face-to-face class with student teams in each class, we used Zoom4 and its breakout room functionality. One recurring meeting (twice a week) was created in Zoom for each of the student sections. In addition, one (Live Q/A) thread was created in Piazza for each section for each meeting. Our course had nine sections, with about 55 students, on average, in each section. Each typical synchronous meeting/class was run in Zoom in the following manner:
At the beginning of each class, the teacher and the TA “met” all students in the main room. The teacher started the class with an overview of the schedule for the day and then played a pre-recorded, short video (see Section 3), in which an overview was given of the concepts to be discussed/worked on in the class. After the video had been played, students were able to ask questions before the quiz started in Mimir. In addition to the individual quizzes, in 2018 and 2019 we had used team quizzes as well, implemented with Immediate Feedback Assessment Technique (IF-AT) scratch cards [10, 11]. Because these cards are “physical” cards, we were not able to include the team quizzes in our online classes. Once the quiz was fnished, the programming assignments were opened in Mimir, to be worked on in the teams. Each team was given a separate breakout room in Zoom. At the beginning of the semester, students were randomly allocated into teams, but after about three weeks the students were allowed to form their own pre-defned teams. If a pre-defned team consisted of less than fve students, the teacher moved students, not belonging to any team, into the pre-defned ones by using the corresponding functionality in Zoom. When students in a breakout room needed help, they posted a help request on the corresponding Piazza thread, monitored by the teacher and the TA. The teams were not given any special instructions on how to work collaboratively on the programming assignments in class. In some cases, each team member wrote his/her own code and mainly shared ideas in the breakout rooms. In other cases, the team members applied pair/tri programming. As mentioned in Section 3, before the end of the class each team member needed to submit his/her solution in Mimir. Visual Studio Code5 was used as the main programming development environment.
Apart from the use of Zoom, the organization and implementation of our course closely follows the one presented in Section 3. However, there is one other diference. In [10], we specifcally noted that there were two “problems” with students with previous programming knowledge, who are, in most cases, male students. First, some of the female students felt intimidated by these male students in the team work. Second, many of the students with previous programming knowledge felt bored during the frst weeks of the course, because the material was too elementary for them.
4 Zoom is a video-conferencing solution which supports high quality point-to-point and multi-party video conferencing, content sharing, and group and individual chat [14]. 5 https://code.visualstudio.com/
I C P E C 2 0 2 1 2:6 Moving Classes Online
Table 1 Student assessment.
Item Weight Quizzes in class 10% Programming assignments in class 10% Weekly programming projects 20% Two midterms 0–20% Final exam 40–60%
Table 2 Answers to the question “Are you generally pleased or displeased with the course?”.
Rating Answer Count Ratio 5 Very pleased 116 36.3% 4 Rather pleased 121 37.8% 3 Moderate 53 16.6% 2 Rather displeased 22 6.9% 1 Very displeased 8 2.5%
We decided to implement our own suggestion put forth in [10], i.e. by making a special section for students with previous programming knowledge and present additional, more challenging, programming assignments to them. The establishment of this special section was announced a week before the course started and participation in the section was optional. About 60 students, of the 502 starting the course, signed up for this special section. As we pointed out in [10], a FC/TBL version of a large CS1 course demands considerable efort and manpower. In our course, in addition to the main instructor, we had 16 teachers and TAs in the nine sections, eight TAs graded the weekly homework assignments, two TAs took care of a special helping session once a week, and two TAs were hired to specifcally help the main instructor to answer questions on Piazza. Thus, about 30 persons contributed to the running of the course in some way or another. The students’ assessment consists of the fve items shown in Table 1. To obtain full points for the programming assignments in class, the students needed only to pass 50% of the automatic tests, on average, during each class. This rule was introduced in 2019 to reduce the pressure many of the students felt when needing to submit solutions to all assignments during each class [10]. Each of the two midterm exams weighted 10%. However, if the student obtained a higher grade on the fnal exam than in one or both of the midterm exams, the weight of the fnal exam increased to either 50% or 60% and the corresponding midterm weight dropped to 0%.
5 Surveys
In this section, we present and discuss the results of two surveys carried out among students registered in the course.
5.1 First survey The frst survey (student evaluation) was administered by the Ofce of Teaching Afairs and was given to students in the ffth week of the course. At that time, 485 students were still registered in the course, of which 320 students (66%) participated in the survey. Students were asked to answer a single question rated on a fve point Likert scale. The question and the results are presented in Table 2. H. Loftsson and Á. Matthíasdóttir 2:7
According to the results, 74.1% of the students were pleased (either very pleased or rather pleased) with the course, and only 9.4% displeased (rather displeased or very displeased). The weighed average is 3.98. These results are very similar to the ones obtained for the same question in the course in fall 2019, where 76% of the students were pleased with the course, 7% displeased, and for which the weighted average was 4.0 [11]. According to this comparison, moving the classes online did not afect students’ satisfaction in the frst weeks of the course.
5.2 Second survey In order to evaluate students’ attitudes, experience and learning, we constructed a detailed survey, consisting of 28 questions, which students answered during weeks 9–10 of the course. At that time, 473 students were still registered, and 305 students (64.5%) provided answers to the questions. We used the questions from the 2019 course [11], but, in addition, we added a few questions connected to the fact that our classes had been moved online. We divide the questions into four main categories: Course organization and teaching (nine questions), Study materials and midterms (six questions), Cooperation and communication (four questions), and Use of systems (four questions)6. Where applicable, in the tables we provide the results from the same questions in fall 2019, in parentheses, i.e. “(19)”. In Tables 3–6, the columns “Totally agree”, “Agree”, “Neutral’, “Disagree’, and “Totally disagree’ correspond to points 5–1, respectively, on the Likert scale. By considering the results presented in Table 3, we see that the responses are very similar between 2020 and 2019 (a t-test between the mean scores of the two years revealed no signifcant diference in any of the nine questions). Students are generally satisfed with the organization and the learning experience in the course: 66% of the students agree (sum of columns “Totally agree” and “Agree”) that the organization of the course is good (question one), and 69% agree that the course is overall a good learning experience (question two). The results from the frst survey, presented in Table 2, show that about 74% of the students were pleased with the course after fve weeks. It is understandable that satisfaction drops a bit when the course progresses, because the material gradually becomes harder. When considering the results from questions one and two it can be deduced that by moving the classes online has not reduced students’ satisfaction between the years 2019 and 2020. It is interesting that, despite the general satisfaction among students, 42% agree that the course lacks traditional lectures (question four) and that only about half of the students agree that using the FC is suitable for the course (question fve). On the other hand, note that only 14% disagree that using the FC is suitable. According to the answers to question nine, 54% of the students agree that moving the classes online is suitable, whereas 31% disagree. Due to the general satisfaction with the organization and the learning experience, one might be inclined to keep this online format of the classes in the future, but one may have to take into account that about one-third of the students disagree of the suitability of the online move. According to Table 4, 55% of the students in 2020, compared to 48% in 2019, feel that the textbook helped in their studies (question 10). At the beginning of our course, we specifcally emphasized the importance of the textbook and that students should come prepared for classes by reading given chapters of the text. Nevertheless, according to question 11, only 47% of the students usually read the textbook before class and this fgure is a bit lower compared
6 In the accompanying tables, we skip fve questions that mainly concern students’ background.
I C P E C 2 0 2 1 2:8 Moving Classes Online
Table 3 Course organization and teaching.
Question Totally Agree Neutral Disagree Totally agree disagree 20 (19) 20 (19) 20 (19) 20 (19) 20 (19) 1. The organization of 26% (20%) 40% (42%) 22% (23%) 7% (12%) 5% (3%) the course is good 2. This course is overall 36% (33%) 33% (34%) 17% (18%) 10% (9%) 5% (6%) a good learning experience 3. The classes each week 28% (29%) 30% (29%) 23% (20%) 14% (16%) 5% (6%) are useful to me 4. The course lacks 24% (25%) 18% (17%) 21% (19%) 17% (19%) 20% (20%) traditional lectures 5. I feel that the fipped 24% (24%) 24% (29%) 38% (24%) 7% (12%) 7% (10%) classroom is a suitable approach in this course 6. I like the assessment 49% (43%) 30% (29%) 13% (16%) 5% (6%) 3% (6%) of the programming assignments in class 7. The programming 35% (37%) 31% (32%) 17% (14%) 9% (10%) 8% (7%) assignments in class are in accordance with the teaching material 8. The weekly progr. 30% (27%) 28% (25%) 20% (20%) 13% (16%) 9% (12%) projects are in accordance with the teaching material 9. I feel that moving the 33% 21% 16% 14% 17% classes online is suitable in this course
to 2019. Note that a substantially higher ratio of students watch the videos before/after class (question 13), compared to the ratio of students reading the textbook. For Table 4, a t-test between the mean scores of the two years revealed no signifcant diference. The fact that less than 50% of the students read the textbook before coming to classes is worrying, but not surprising. For several years, we have noticed a gradual decline in textbook reading by our CS students. According to Brown et al. [3], “one issue that plagues millennials is the lack of focused, in-depth reading to achieve understanding”, and “one of the biggest challenges instructors have dealt with for many years is getting students to read their required textbooks”. Table 5 (question 16) shows that a lower ratio of students in 2020 (63%) compared to 2019 (73%) agree that discussing with fellow students helped their studies (a t-test between the mean scores of the two years only revealed a signifcant diference for this question (t-value 3.11, p < 0.01)). This may indicate that the communication between students in the Zoom breakout rooms is not as straight-forward as it is when communicating in person. The teachers noted that in some teams the communication between students in the breakout rooms was generally active and that they worked collaboratively, while in other teams it seemed that the students worked rather as individuals. On the other hand, the responses to question 19 show that a higher ratio of students in 2020 (62%) compared to 2019 (55%) H. Loftsson and Á. Matthíasdóttir 2:9
Table 4 Study material and midterms.
Question Totally Agree Neutral Disagree Totally agree disagree 20 (19) 20 (19) 20 (19) 20 (19) 20 (19) 10. The textbook of 22% (25%) 23% (23%) 25% (28%) 17% (16%) 12% (8%) the course helped me in my studies 11. I usually read 26% (33%) 21% (19%) 18% (18%) 13% (11%) 21% (19%) the book before the class 12. The videos of 23% (24%) 33% (28%) 26% (27%) 12% (16%) 7% (5%) the course helped me in my studies 13. I usually watch 42% (42%) 25% (27%) 20% (16%) 6% (10%) 7% (5%) the videos given before/after the class 14. I like taking a quiz 18% (19%) 30% (31%) 28% (23%) 13% (14%) 12% (12%) at the beginning of class 15. I like the arrangements 53% (57%) 31% (31%) 11% (8%) 3% (0%) 1% (3%) of the midterm exams feel that communication with the instructors in class helped their studies. The diference between the mean scores of the two years for this question is not statistically signifcant, but it may be the case that it is easier for a student team to communicate with a teacher inside a breakout room as opposed to the team sharing a large physical classroom with all other students (as in 2019). Table 6 shows that students are generally happy with the support systems used in the course. However, there is a large reduction of satisfaction in Piazza usage (question 21) in 2020 (46%) compared to 2019 (63%). This is difcult to explain, but we conjecture that students did not like to ask teachers to “visit” them in Zoom breakout rooms by posting such a help message in Piazza, where it can be seen by everyone. Unfortunately, in the Zoom version we used, it was not possible to post a message to the host or co-host (teacher or TA) from within a breakout room. This is the reason why we used Piazza for this purpose. For Table 6, a t-test between the mean scores of the two years only revealed a signifcant diference for question 21 (t-value 4.11, p < 0.001).
6 Exams
In this section, we present the implementation of the two midterm exams, the fnal exam, and the retake exam given in the course along with exam results. All four exams were set up in Mimir, and students were able to receive assistance by using Piazza and Zoom. If a student needed assistance during an exam, he/she posted a request in a given Piazza thread and, consequently, the teacher or the TA moved the student into a breakout room in Zoom. The exams were “open book”, in the sense that students were allowed to use the textbook, slides, notes, and solutions to assignments in the exam. Students were, however, neither allowed to use web search nor any kind of communication software during the exam. Grades are given on a 0–10 scale, and a grade below 5 is a failing grade.
I C P E C 2 0 2 1 2:10 Moving Classes Online
Table 5 Cooperation and communication.
Question Totally Agree Neutral Disagree Totally agree disagree 20 (19) 20 (19) 20 (19) 20 (19) 20 (19) 16. Discussing with 35% (44%) 28% (29%) 19% (16%) 9% (6%) 9% (4%) fellow students in class helped me in my studies 17. Discussing with 35% (39%) 31% (31%) 22% (20%) 7% (4%) 6% (6%) fellow students outside class helped me in my studies 18. I like to work in a group 35% (38%) 27% (29%) 21% (19%) 10% (8%) 7% (5%) with fellow students 19. Communication with 28% (32%) 34% (23%) 21% (26%) 12% (12%) 5% (8%) instructors in class helped me in my studies
Table 6 Use of systems.
Question Totally Agree Neutral Disagree Totally agree disagree 20 (19) 20 (19) 20 (19) 20 (19) 20 (19) 20. I like to use Canvas 37% (39%) 33% (29%) 26% (26%) 2% (4%) 1% (2%) 21. I like to use Piazza 23% (34%) 23% (29%) 26% (24%) 16% (5%) 12% (8%) 22. I like to use Mimir 45% (52%) 36% (30%) 10% (13%) 4% (3%) 5% (2%) 23. I like to use Zoom 31% 31% 19% 11% 7%
The plagiarism software Moss7 was run on student solutions. We found some plagiarism cases in the midterm exams. These cases were reported (resulting in the grade 0) and, consequently, the seriousness of plagiarism was discussed with the whole student body on Piazza. As a result, we did not fnd any obvious cases of plagiarism in the fnal exam. We used the timing and the material for the exams described in [10]:
The frst midterm exam was given in the fourth week of the course. The material for the exam were basic programming concepts like variables, types, operators, assignment statements, expressions, if-statements, and loops. The second midterm exam was given in the eighth week of the course. In addition to the material covered in the frst exam, the second one included the following concepts: functions and top-down refnement, scope, fle I/O, exception handling, lists and tuples. At the time of the fnal exam, the following concepts had been added: dictionaries, sets, (large) program development, and classes.
The duration of the frst midterm exam, the second midterm exam, and the fnal/retake exam, was 2 hours, 2.5 hours, and 3 hours, respectively. Table 7 shows the results from the four exams – the numbers in parenthesis show the results from the 2019 course8. The number of students still registered in the course at the time of the four exams were 485, 474, 463 and 463, respectively. According to our experience,
7 https://theory.stanford.edu/~aiken/moss/ 8 The student body and the difculty of the exams in the courses in 2019 and 2020 are very similar, which justifes making a direct comparison. H. Loftsson and Á. Matthíasdóttir 2:11
Table 7 Exam results.
Exam Students Average grade Failure rate 2020 2020 (2019) 2020 (2019) Midterm 1 462, 95.3% 8.9 (8.5) 3.5% (8.1%) Midterm 2 433, 91.3% 5.8 (6.0) 36.7% (39.6%) Final 401, 86.6% 6.0 (5.9) 35.4% (33.9%) Retake 125, 27.0% 4.9 (4.0) 39.2% (52.7%) the participation in the second midterm exam is a good indicator of the dropout rate in the course. In 2019, 82.8% of the registered students (at the start of the course) showed up in the second midterm, whereas the corresponding ratio in 2020 was 86.3%. It is noteworthy how much the average grade decreases in midterm 2 in comparison to midterm 1, and, consequently, how much the failure rate increases. This is consistent with the exam results for the 2019 course. The reason is that the material covered on the frst midterm exam is relatively easy for most students, whereas the second midterm exam covers more complex concepts. As mentioned in Section 4, students with previous programming experience were given the opportunity to take part in a special section intended for this group of students. The average grade on the fnal exam for students in this special section was 8.4, compared to 5.7 for the students in the other sections. This diference in fnal exam performance in CS1 between students with and without previous programming experience is even greater than, for example, the results presented in [24]. The grades in the two midterm exams and the fnal exam in 2020 are very similar to the corresponding grades from the 2019 course. We thus deduce that moving our classes online did not signifcantly afect students’ performance. Of the 502 students that were registered at the start of the course, 353 students (70.3%) passed the course (either the fnal exam or the retake exam). The corresponding failure rate of 29.7% is a bit lower than the mean worldwide failure rate of 32.3% presented in [23].
7 Conclusion
In this paper, we presented an experience report on moving the classes in a large CS1 course, emphasising FC and TBL, to an online format in 2020. The motivation for the move was the COVID-19 pandemic. A priori, we were worried that moving the classes online would make it difcult to successfully use the FC and TBL teaching and collaborative approaches in our programming course. However, our experience shows that conducting these methods in online classes did not pose any special problems in comparison to face-to-face classes. We presented the results of two surveys and four exams. According to the surveys, students were generally satisfed with the organization of the course and the learning experience. A comparison of the results to the course from the previous year shows that moving the course online did neither have negative efects on students’ attitudes nor on students’ grades. When the on-going pandemic is over, we need to make a decision on whether to move back to face-to-face classes in our CS1 course. It might be an option to continue giving the classes online in the future. One argument against online classes is that, according to our results, a lower ratio of students in these classes feel that discussing with fellow students helped their studies, compared to the similar face-to-face class course from the previous year. On the other hand, our results also show that a higher ratio of students in the 2020 course, compared to the course in 2019, feel that communication with the teachers in class helped their studies.
I C P E C 2 0 2 1 2:12 Moving Classes Online
Finally, the best option may be to allow students to choose the class format that best fts their individual needs, i.e. allowing students to choose either online classes or face-to-face classes.
References 1 Lina Battestilli, Ignacio X. Domínguez, and Maanasa Thyagarajan. Toward Finding Online Activity Patterns in a Flipped Programming Course. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, SIGCSE ’20, page 1345, New York, NY, USA, 2020. Association for Computing Machinery. doi:10.1145/3328778.3372626. 2 Brett A. Becker and Keith Quille. 50 Years of CS1 at SIGCSE: A Review of the Evolution of Introductory Programming Education Research. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education, SIGCSE ’19, page 338–344, New York, NY, USA, 2019. Association for Computing Machinery. doi:10.1145/3287324.3287432. 3 Charles A. Brown, Kreag Danvers, and David T. Doran. Student Perceptions on Using Guided Reading Questions to Motivate Student Reading in the Flipped Classroom. Accounting Education, 25(3):256–271, 2016. doi:10.1080/09639284.2016.1165124. 4 Joelle Elmaleh and Venky Shankararaman. Improving student learning in an introductory programming course using fipped classroom and competency framework. In 2017 IEEE Global Engineering Education Conference (EDUCON), pages 49–55, 2017. doi:10.1109/EDUCON. 2017.7942823. 5 Swapna Gottipatti and Venky Shankaraman. Rapid Transition of a Technical Course from Face-to-Face to Online. Communications of the Association for Information Systems, pages 1–8, 2021. URL: https://ink.library.smu.edu.sg/sis_research/5401. 6 Raelke Grimmer, Andrew Pollard, and Nicola Rolls. COVID-19 induced change in higher education: Refections on rapidly transitioning a frst-year undergraduate academic literacies unit from face-to-face to online. Journal of Academic Language and Learning, 14(2):95–105, 2020. URL: https://journal.aall.org.au/index.php/jall/article/view/695. 7 Sandy Irani and Kameryn Denaro. Incorporating Active Learning Strategies and Instructor Presence into an Online Discrete Mathematics Class. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, SIGCSE ’20, page 1186–1192, New York, NY, USA, 2020. Association for Computing Machinery. doi:10.1145/3328778.3366904. 8 Krisztina V. Jakobsen and Megan Knetemann. Putting Structure to Flipped Classrooms Using Team-Based Learning. International Journal of Teaching and Learning in Higher Education, 29(1):175–185, 2017. 9 Patricia Lasserre. Adaptation of Team-based Learning on a First Term Programming Class. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education, ITiCSE ’09, page 186–190, New York, NY, USA, 2009. Association for Computing Machinery. doi:10.1145/1562877.1562937. 10 Hrafn Loftsson and Ásrún Matthíasdóttir. Using Flipped Classroom and Team-Based Learning in a First-Semester Programming Course: An Experience Report. In Proceedings of IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE ’19. IEEE, 2019. doi:10.1109/TALE48000.2019.9225985. 11 Ásrún Matthíasdóttir and Hrafn Loftsson. Improving the Implementation of a First- Semester Programming Course. In Proceedings of the 16th International CDIO Con- ference, Gothenburg, Sweden, 2020. CDIO. URL: http://www.cdio.org/content/ improving-implementation-first-semester-programming-course. 12 Abdallah Mohamed. Evaluating the Efectiveness of Flipped Teaching in a Mixed-Ability CS1 Course. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE ’20, page 452–458, New York, NY, USA, 2020. Association for Computing Machinery. doi:10.1145/3341525.3387395. H. Loftsson and Á. Matthíasdóttir 2:13
13 William F. Punch and Richard Enbody. The Practice of Computing Using Python. Pearson Education, New York, NY, 3rd. edition, 2017. 14 Wainhouse Research. Evaluation of the Zoom Cloud Video Conference Service, 2015. URL: https://zoom.us/whitepaper?case=wainhouse. 15 Anthony Robins, Janet Rountree, and Nathan Rountree. Learning and Teaching Programming: A Review and Discussion. Computer Science Education, 13(2):137–172, 2003. 16 Sudipta Roy and Bonnie Covelli. COVID-19 Induced Transition from Classroom to Online Mid Semester: Case Study on Faculty and Students’ Preferences and Opinions. Higher Learning Research Communications, 11:10–32, 2020. doi:10.18870/hlrc.v11i0.1197. 17 Manoj D. Souza and Paul Rodriques. Investigating the Efectiveness of the Flipped Classroom in an Introductory Programming Course. The New Educational Review, 40(1), 2015. doi: 10.15804/tner.2015.40.2.11. 18 Gina Sprint and Erik Fox. Improving Student Study Choices in CS1 with Gamifcation and Flipped Classrooms. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, SIGCSE ’20, page 773–779, New York, NY, USA, 2020. Association for Computing Machinery. doi:10.1145/3328778.3366888. 19 Christian Stöhr, Christophe Demazière, and Tom Adawi. The polarizing efect of the online fipped classroom. Computer Education, 147:103789, 2020. doi:10.1016/j.compedu.2019. 103789. 20 Kalpathi Subramanian and Kiran Budhrani. Infuence of Course Design on Student Engagement and Motivation in an Online Course. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education, SIGCSE ’20, page 303–308, New York, NY, USA, 2020. Association for Computing Machinery. doi:10.1145/3328778.3366828. 21 Chao Wang and Michael Goryll. Design and Implementation of an Online Digital Design Course. In 123rd ASEE Annual Conference and Exposition, volume 2016-June. American Society for Engineering Education, 2016. URL: https://asu.pure.elsevier.com/en/publications/ design-and-implementation-of-an-online-digital-design-course. 22 Gang Wang, Hong Zhao, Yun Guo, and Min Li. Integration of Flipped Classroom and Problem Based Learning Model and its Implementation in University Programming Course. In 2019 14th International Conference on Computer Science Education (ICCSE), pages 606–610, 2019. doi:10.1109/ICCSE.2019.8845525. 23 Christopher Watson and Frederick W.B. Li. Failure Rates in Introductory Programming Revisited. In Proceedings of the 2014 Conference on Innovation and Technology in Computer Science Education, ITiCSE ’14, page 39–44, New York, NY, USA, 2014. Association for Computing Machinery. doi:10.1145/2591708.2591749. 24 Chris Wilcox and Albert Lionelle. Quantifying the Benefts of Prior Programming Experience in an Introductory Computer Science Course. In Proceedings of the 49th ACM Technical Sym- posium on Computer Science Education, SIGCSE ’18, New York, NY, USA, 2018. Association for Computing Machinery. doi:10.1145/3159450.3159480.
I C P E C 2 0 2 1
Programmers’ Afnity to Languages
Alvaro Costa Neto # Federal Institute of Education, Science and Technology of São Paulo, Barretos, Brazil Cristiana Araújo # Ñ Centro ALGORITMI, Departamento de Informática, University of Minho, Campus Gualtar, Braga, Portugal Maria João Varanda Pereira # Ñ Research Centre in Digitalization and Intelligent Robotics, Polytechnic Institute of Bragança, Portugal Pedro Rangel Henriques # Ñ Centro ALGORITMI, Departamento de Informática, University of Minho, Campus Gualtar, Braga, Portugal
Abstract Students face several challenges when learning computer programming languages, a central topic to acquire programming skills. While those challenges that present a predominantly technical nature have been intensely studied by researchers along the years, the ones that are concerned with qualitative, and personal aspects have not. Afnity to a programming language is one of the many personal factors that may contribute to surpass these qualitative aspects that describe the difculties that students face. From this point-of-view, this paper presents a proposal for treating and studying programmers’ afnity to programming languages as an important factor for learning computer programming. It also reports a preliminary questionnaire conducted on a master’s degree class at Universidade do Minho that showed that afnity may have a broader relation to learning computer programming than anticipated. Finally, a set of relevant questions are stated to compose a future inquiry aimed at deepening the knowledge on the afnity between programmers and languages, paving the way for following research.
Keywords and phrases Computer programming, Programming Languages, Afnity, Education, Learning
Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.3
Funding This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020.
1 Introduction
Learning a complex topic such as computer programming is an intricate process, infuenced by several factors of varied nature. Research in this feld dates back several decades [5], and has kept constant academic interest throughout the years [6, 15, 12, 3], usually through a technical approach – development and evaluation of tools that are used to teach and learn. While of great value, these are not the only problems that students face when learning. In fact, when observed through a pedagogical lenses, many other characteristics of diferent nature also seem to explain how knowledge is constructed, allowing food for thought on how to facilitate this process in a more personal perspective. The widely known researcher Jean Piaget stated that context, especially in its social branch, has a decisive impact on learning by providing the necessary interactions that adapt and organize the mental schemes [11]. Lev Vygotsky, another very infuential pedagogy researcher, also highlighted the crucial importance that contextual factors play in the construction of knowledge [14], while positioning language
as the main mediator for learning. Paulo Freire went one step further by supporting the idea that teaching is a force of change to socioeconomic contexts, which is only possible by understanding and aggregating the students’ backgrounds into the process [7]. In a diferent recent project [4], focused in adults’ professional reconversion, we also realized the relevance of background and know-how in their learning process, including the impact on the choice of the best learning resources. Based on these authors’ fndings, it’s reasonably safe to assume that personal factors are inevitably involved in the process of knowledge construction, and therefore should be taken into account when teaching. As one of these personal factors, afnity may sustain an important relation to the learning process, even on very technical subjects such as computer programming, and it deserves, for sure, to be further analysed. While there have been studies on personal factors being infuential to the education process as a whole, afnity’s infuence is still largely unexplored when it comes to computer programming learning. Afnity’s qualitative nature and the fact that it may be directly related to technical aspects of the programming languages present a dual, albeit interesting conundrum to investigate. As such, many research questions emerge when afnity is considered an important feature of the learning process:
Which of the language’s characteristics stimulate students’ subconsciousness, subtly conducting them to like or dislike certain programming languages? Do these characteristics infuence students positively or negatively? Is the inverse actually true, meaning that the frst programming language learnt strongly infuences the consequent student’s afnity with that particular language?
Finding answers to these questions is the main goal of the study discussed in this paper, aiming to instigate further research on this subject. This paper is structured in fve sections: after the introduction, section 2 discusses which factors may infuence the learning process and how they may be categorized. The third section presents afnity as an important and challenging topic for discussion when considering the learning of computer programming. Moreover, in this section it is also presented the initial investigation that underlies the work here reported. Based on the previously presented results, section 4 raises relevant questions that must be answered in order to better understand the role of afnity in learning, and in the development of programming skills. Finally, section 5 concludes this paper presenting a proposal of further investigation that will be conducted.
2 Infuences on Learning Computer Programming
Many factors infuence directly or indirectly the learning process. Whenever possible, it is good practice to take into account these infuences in order to improve the student’s experience. Three main categories may be established: teaching methods, programming languages characteristics and personal background. The teaching methods category contains the array of factors based on the application of pedagogical methods, either formally stated or experience based. Tools and other mediating resources are also included in this category, such as proposed by [1, 2]. The most common factors include:
Quality of the communication process; Media diversifcation; Auxiliary tools; Clarity of the explanation etc. A. Costa Neto, C. Araújo, M. J. V. Pereira, and P. R. Henriques 3:3
Factors that are directly connected to the underlying principal and design of the pro- gramming language are its formal defnition, paradigm, syntax, semantics, and quality [10]. Also important are intrinsic characteristics such as:
Expressiveness; Viscosity; Verbosity Readability; Consistence; Error-proneness; Abstraction level etc.
Historical, contextual, and subconscious characteristics form the personal background category. Examples of such factors include:
Socioeconomic context; Previous contact with formal logic and computers; Experience with the English language [9]; Psychological issues such as being away from family for the frst time [8]; Afnity with specifc languages and tools; Lack of interpersonal skills etc.
Figuring in as a personal and complex factor, afnity may be established by the students to specifc programming languages. On one hand, considering afnity may lead to new paths to motivate students improving their learning process. On the other hand, afnity might also be seen as a result of the students’ experiences, guided by their courses’ curricula and the initial languages he or she has learnt.
3 Afnity to Programming Languages
The previous statements are supported by a preliminary indication observed by asking twenty three students of a Masters’ degree class in Computer Engineering at Universidade do Minho to answer a few questions about programming languages and their learning experiences. After a lecture on teaching computer programming, in which many of these topics were discussed, some snippets of code were shown in diferent languages – BASIC, Lisp, C, Java, Python, Ruby, Swift, and Haskell – and students were asked to point out which languages they like most, being C, Java and Haskell the three highest ranked. Then, a questionnaire was applied containing these questions:
1. In a range of very low to very high, how important is the language choice for learning computer programming? Justify your answer; 2. Out of those factors previously mentioned in Section 2 and presented in the lecture, which ones are most relevant, and infuential to learning computer programming? 3. Which languages would you choose to teach computer programming: BASIC, Lisp, C, Java, Python, Ruby, Swift, or some other language? Justify your answer. This was a multiple choice question, allowing students to choose more than one language.
If popularity and market share were the main infuences driving the students’ choices, it would be safe to assume that Python would be the highest ranked option, since it’s currently a very popular programming language [13], with many popular applications, such as artifcial intelligence and numerical computing. Nonetheless, C and Java overthrew Python as shown
I C P E C 2 0 2 1 3:4 Programmers’ Afnity to Languages
in Figure 1. Haskell also had a perceptive presence in the results, which is also unexpected from the perspectives of current market share and ease of use. These surprises become understandable if the afnity to the language was actually constructed as a consequence of their learning experiences, since these students had been formally taught Haskell, C and Java as their initial programming languages. While the current attractiveness of Python can’t be discarded – it ranked third, after all – these results show that programming language afnity is more subtle and complex than expected, and could ultimately infuence other activities beyond learning.
Haskell 3
Swift 0
Ruby 0
Python 6
Java 12
C 15
Lisp 1
BASIC 3
0 2 4 6 8 10 12 14 16 18 20
Figure 1 Language choices by students when asked which languages they prefer.
These unexpected fndings imply that many questions should arise from more exploratory observation of how students learn computer programming, creating opportunities for deeper investigation on afnity with programming languages and its relation to the learning process.
4 A Survey to Understand Afnity
Learning how to correctly program a computer involves not only understanding how to type commands that are syntactically correct. It also involves understanding problems, abstracting concepts and fnding recurrent patterns, delineating their boundaries, designing algorithms, and translating viable solutions to source code. In this process, the programming language acts simultaneously as a tool and as an interface of communication, interfering with both learning and problem solving. Having a close understanding of its rules is mandatory for obtaining good results, but solving a problem using a language enacts a more complex intellectual mechanism, specially when learning. It becomes clear that the mental schema that must be constructed in order to program an application involves many aspects of the human mind: memory, perceptiveness, and motivation, to name a few. As is safe to assume that afnity is an important part of motivation, its proper usage can be a valuable aid for education professionals. Several interesting questions arise to help understanding afnity and the role it might play in learning computer programming:
How does afnity to a programming language come to be? Which factors afect its growth, either hampering or stimulating it? Is this afnity a support to or a consequence of the learning process? How do teaching methodologies come into play to stimulate it? Which personal factors afect afnity growth? A. Costa Neto, C. Araújo, M. J. V. Pereira, and P. R. Henriques 3:5
These questions, while of central importance, are too broad to directly compose a survey. They serve to structure the objectives by instigating the search for their answers. In order to obtain tangible results though, it’s necessary to delve deeper into the problem of afnity, and pinpoint specifc characteristics through a more palpable inquiring. This process, in and of itself, proposes an interesting challenge: how to obtain meaningful answers for a learning factor that may present itself simultaneously technical and personal, such as afnity? This duality of nature may require an equivalent dual approach. While it’s important to directly ask questions such as “Is this language appealing to you? Why?”, or “Rank your favorite languages” to paint a personal picture of the respondent’s background, this strategy may not work if applied to more technical aspects of the programming languages. In this case, direct questions may lead to colored answers, either by previous personal opinions, or by misinterpretation of what is being asked. A dual approach would apply indirect observation of afnity on technical aspects – comparing snippets of source code, would be an example – while still relating these aspects to the respondent’s background, via more open, direct, and personal questions. This approach seems to be a more encompassing way to obtain meaningful answers, at least for the previously stated questions. According to the previous discussion, we propose a survey based on the following structure:
1. A collection of personal questions in order to capture a picture of the respondents’ contexts and backgrounds. It would also be interesting to ask directly which languages are their favorites and why, for later comparison with the answers given to the other questions; 2. Direct questions asking the respondents to analyze snippets of source code in various programming languages. These questions should be as direct and as clear as possible to avoid misinterpretations, and to accommodate the fact that not all respondents may be familiar with all presented languages. Several diferent languages should be applied – randomly, if possible – in order to assess their capacities of understanding source code, while simultaneously identifying direct and indirect characteristics that emerge as possible infuential factors for the respondents’ afnity to the languages that were chosen as their favorites; 3. A series of indirect questions, also applying several diferent programming languages and snippets of source code. These questions must be composed in such a way to detect indirectly which characteristics of the presented code are infuencing the respondents’ afnity to the presented languages, given their backgrounds and current knowledge; 4. Some direct questions inquiring what is his or her perception of the languages and snippets previously presented. These questions would serve as a leveling reference for the respondents’ personal – and biased – view of the programming languages and what he or she thinks is more infuential to create afnity. The answers for these questions are certain to form interesting comparisons with the ones given to the questions proposed on item 1.
This structure will support the construction of the survey. Implementation details, such as the order of the questions, which programming languages should be used, and complexity of the source code snippets, are yet to be determined. It would be ideal to apply dynamically constructed surveys, that would serve two main purposes:
To provide specifc inquiries for respondents that have diferent backgrounds, in order to minimize personal bias on more direct and technical questions, and; To design some mechanism of comparison between respondents that have experience with certain languages and those who have not.
I C P E C 2 0 2 1 3:6 Programmers’ Afnity to Languages
The technical resources to implement these dynamic surveys are yet to be investigated and developed, if required. While having education purposes in mind, the survey will be freely available, and account for answers given by people other than students and teachers. Computer programming professionals, mathematicians, applied computing users, to list a few, may have very interesting, and important, views on afnity to their favorite programming languages. Finally, the answers will be statistically correlated to the respondents’ background and technical knowledge in order to obtain a better understanding of how and why afnity to a programming language is established. The fnal goal is to identify which characteristics of the current, and past programming languages are most related to the rise and fall of programmers’ afnity. This information may be a valuable resource to aid educators to better choose – or when possible, allow the choice – of the programming language used in their classes.
5 Conclusion
The problem of more efciently teaching computer programming still poses relevant, and appealing challenges. While many studies have shown diverse characteristics as infuential to the learning experience, afnity to a programming language is still a largely unknown aspect. Preliminary results point toward further investigation, as the conducted studies show that there are at least two – probably complementary – perspectives on afnity: it may be viewed both as a propelling factor to knowledge construction, and as a result of experience while learning computer programming. The former indicates that educators must be aware of the relationship that students develop with the programming languages they taught, specially in their initial stages of education. This observation, and consequent adaptation of the teaching process should improve students’ learning experiences. The later – considering afnity as a product of the learning experience itself – may become a valuable indication to better understand how the programming languages taught in formal education infuence the industry. On one hand, this afnity may induce future professionals – and consequently, their workplaces – to prefer and use the languages they were taught. On the other hand, it may create a resistance in the workforce to adopt current established languages. Nonetheless, in both perspectives afnity should be considered an important factor of the teaching-learning process, and consequently be taken into account when deciding which languages students should learn, work and improve. In the near future deeper studies shall be conducted, investigating which specifc charac- teristics contribute to language afnity, both technical, and personal in nature.
References 1 M. V. P. Almeida, L. M. Alves, M. J. V. Pereira, and G. A. R. Barbosa. EasyCoding - Methodology to Support Programming Learning. In Ricardo Queirós, Filipe Portela, Mário Pinto, and Alberto Simões, editors, First International Computer Programming Education Conference (ICPEC 2020), volume 81 of OpenAccess Series in Informatics (OASIcs), pages 1:1–1:8, Dagstuhl, Germany, 2020. Schloss Dagstuhl–Leibniz-Zentrum für Informatik. doi: 10.4230/OASIcs.ICPEC.2020.1. 2 M.V.P. Almeida. Easycoding: Methodology to support programming learning. Master’s thesis, Instituto Politécnico de Bragança, 2020. 3 A.G. Applin. Second language acquisition and cs1. SIGCSE Bull., 33(1):174–178, February 2001. doi:10.1145/366413.364579. A. Costa Neto, C. Araújo, M. J. V. Pereira, and P. R. Henriques 3:7
4 D.R. Barbosa. CnE-Ar: Teaching of Computational Thinking to Adults in Reconversion. Master’s thesis, Minho University, Braga, Portugal, April 2021. MSc dissertation (to be discussed and published). 5 R.R. Fenichel, J. Weizenbaum, and J.C. Yochelson. A program to teach programming. Communications of the ACM, 13(3):141–146, March 1970. doi:10.1145/362052.362053. 6 J. Figueiredo and F.J. García-Peñalvo. Building skills in introductory programming. In Proceedings of the Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, TEEM’18, page 46–50, New York, NY, USA, 2018. Association for Computing Machinery. doi:10.1145/3284179.3284190. 7 P. Freire. Pedagogia da Autonomia: Saberes necessários à prática educativa. Paz e Terra, 2011. 8 A. Gomes and A.J. Mendes. Learning to program: difculties and solutions. In Proceedings of the 2007 ICEE International Conference on Engineering and Education, ICEE ’07. International Network on Engineering Education and Research, 2007. 9 P.J. Guo. Non-native english speakers learning computer programming: Barriers, desires, and design opportunities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, page 1–14, New York, NY, USA, 2018. Association for Computing Machinery. 10 P.R. Henriques. Brincando às Linguagens com Rigor: Engenharia gramatical. PhD thesis, Universidade do Minho, 2013. 11 J. Piaget, M. Piercy, and D.E. Berlyne. The Psychology of Intelligence. Routledge classics. Routledge, 2001. 12 S.A. Robertson and M.P. Lee. The application of second natural language acquisition pedagogy to the teaching of programming languages—a research agenda. SIGCSE Bulletin, 27(4):9–12, December 1995. doi:10.1145/216511.216517. 13 StatisticsTimes.com. Top computer languages, 2020. URL: http://statisticstimes.com/ tech/top-computer-languages.php. 14 L.S. Vygotsky, E. Hanfmann, G. Vakar, and A. Kozulin. Thought and Language. The MIT Press. MIT Press, 2012. 15 B.C. Wilson and S. Shrock. Contributing to success in an introductory computer science course: A study of twelve factors. In Proceedings of the Thirty-Second SIGCSE Technical Symposium on Computer Science Education, SIGCSE ’01, page 184–188, New York, NY, USA, 2001. Association for Computing Machinery. doi:10.1145/364447.364581.
I C P E C 2 0 2 1
Melodic – Teaching Computational Thinking to Visually Impaired Kids Rui Costa # Centro ALGORITMI, Departamento de Informática, University of Minho, Campus Gualtar, Braga, Portugal Cristiana Araújo # Ñ Centro ALGORITMI, Departamento de Informática, University of Minho, Campus Gualtar, Braga, Portugal Pedro Rangel Henriques # Ñ Centro ALGORITMI, Departamento de Informática, University of Minho, Campus Gualtar, Braga, Portugal Abstract This paper presents a proposal, called Melodic, to develop Computational Thinking skills in kids with special educational needs, in this case blindness. The aim of this research is to characterize the subject and identify what are the current most used and best practises to teach this diferent way of Thinking to kids under those circumstances. In this paper more technical aspects are discussed, such as the architecture for the proposed application in order to accomplish the project goals. Melodic was carefully designed to have an intuitive interface for blind people and a seamless workfow, combining tactile hardware, and QR Code technology with a sound based output. 2012 ACM Subject Classifcation Social and professional topics → Computational thinking Keywords and phrases Computational Thinking, Visual Impaired Students Education, Teaching through Music, Learning Resource Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.4 Funding This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020.
Acknowledgements We would like to thank Isabel Barciela and Marta Paço both from Íris Inclusiva for their valuable comments and suggestions; Without their collaboration this work would not be possible.
1 Introduction
In today’s world technology is everywhere we look. As a consequence employers look for people with strong skills in computational feld. This fact leads many young people to choose programming courses. Programming students normally face a great difculty due to the traditional way of thinking they are taught in conventional schools, while they should be trained in Computational Thinking that is the key to solve general or programming problems. Computational Thinking is based on some key concepts such as Logical Reasoning, Algorithm Design, Decomposition, Pattern Recognition, Abstraction and Evaluation and changes the whole way of thinking about the resolution of a given problem [10, 3, 12]. This change of the way of thinking is not an easy task so the sooner it is introduced to people the better. That said, it is of uttermost importance to research for a strategy to teach children in a captivating way to keep them motivated. Currently one technique that is very used is “Game Based Learning” [8]. This technique is characterized by being a type of game play with defned learning outcomes. Usually the game used is a digital one, but this is not always the case. There are many arguments in favour of game based learning, such as:
Motivation: Normally games for entertainment are able to motivate learners to stay engaged over longer periods of time through a series of features of motivational nature. These include incentive artifacts, such as stars, points, leader boards, badges, and trophies, as well as game mechanics and challenges that learners fnd interesting; Player Engagement: Is related to motivation and relies on design decisions that refect the specifc learning goal, learner characteristics, and setting; Adaptability: Learner engagement is facilitated by the adaptive part of the game, being this customizable by the player or personalized. This characteristic is defned by the capability of engage each learner in a way that refects his or her specifc profle; Graceful Failure: Rather than putting failure as an undesirable outcome, it is an expected and sometimes necessary step in the learning process. This is achieved by lowering the consequences on a failure situation and feedback on the wrong answer so that the player can learn from his or her mistakes.
In order to teach Computational Thinking properly it is mandatory to adapt the Learning Resources to the target learners. Resources must be adequate to the training of Computational Thinking so that the students develop new knowledge and the diferent abilities and acquisition of values that defne Computational Thinking [1, 11]. Reviewing the literature in the area, it is clear that there are some platforms to support the teaching/learning process aimed at a general audience; however, when it concerns special need kids, the available resources are very few. It is very important to realize that mos of the referred resources rely upon a visual interface based program paradigm. In order to have a system that successfully helps blind kids to learn the concepts of Computational Thinking, it is mandatory that the system does not rely on a visual interface but on an audio or tactile one [9]. With this in mind, the audio based interface combined with a tactile input device seems the best approach to create resources to teach some Computational Thinking concepts through music. That musical approach has clear advantages, as listed below [6]: Debugging a song to make it sound like desired, is similar to debugging a program in order to make it to have the desired behavior; Musical phrases correspond to program commands; A song is a combination of musical phrases just as a program is a combination of statements.
In this way, we intend to create a Learning Resource capable of, through musical based games, teach the fundamental concepts of Computational Thinking to blind kids to help them to solve programming or general problems using this approach.
The paper is organized as follows. Section 2 explains the concept of Learning Resource, characterizes blind and visually impaired people way of learning, ways to teach visual impaired students and the special characteristics that we have to take in consideration when choosing or creating learning resources for this context. Section 3 presents the principle of the Melodic. Here we can see understand the way of interacting with this application and what kind of output we are expecting to get. Section 4 talks about Melodic’s architecture and used technologies. Also in this chapter are described the implementation approaches and prototype testing. Section 5 are given some examples of teaching approaches and ways to use this system to reach its goal, train Computational Thinking. Section 6 summarizes all the work done until this point and the next steps that are predicted to be done for this project. R. Costa, C. Araújo, and P. R. Henriques 4:3
2 Adaptability of Learning Resources to Blind Students
Learning Resource is a device used for educational purposes in any format, real or virtual, that illustrates or supports one or more elements of a course of study; and may enrich the experience of the pupil or teacher. Learn resources will serve to train Computational Thinking, so they play a very important part in this process. For this reason the resources must be adequate to the training of Computational Thinking so that students can develop new knowledge and the diferent abilities and acquisition of values that defne Computational Thinking [1]. As far back as 1967, there were reported various issues related to the education of visually impaired learners. The focus was on the importance of vision and the mode of reading, and attempted to illustrate how intelligence manifests itself in blind learners as compared to deaf. Their work showed the relevance of blindness and information processing and also illustrated the maximum utilization of available sensory data during learning activities and the translation of visual stimuli [7]. In visual impaired learners conceptual development and abstract thinking seem to be delayed by the absence of visual stimulation or images. With this cognitive development occurs more slowly independently of the age group [5]. A technique to teach conceptual subjects is using tactile stimuli to overcome the difculty imposed by the lack of sight. Figure 1 illustrates how multiple tactile stimuli can compensate for the loss of sight, allowing learners to perceive size and shape of objects [4]. However, blind people are easily overwhelmed by the complexity of very “full” or “busy” diagrams, sketcher or models. A way to counter this feeling is to be spatially oriented when walking or reading documents. In the learning point of view, blind and visually impaired learners require specifc strategies, addressing their unique learning mediation needs. With loss or absence of vision, the amount of sensory data available to the learner is reduced and for this reason is very important the use of multi-sensory approaches, like the one in Figure 1 [2]. To facilitate this multi-sensory interface, Braille text, talking calculators, computer voice over and many more became standard equipment to teach blind and visually impaired people. In order to best adapt visually impaired students it is mandatory to adapt the curriculum with strategies such as [4]: setting a substitute task of similar scope and demand; replacing one impossible or unfriendly task with one of a diferent kind; allowing the learner to undertake the task at a later date; using another planned task to assess more outcomes than originally intended; giving the learner concessions to complete a task; using technology, aides or other special arrangements to undertake assessment tasks; using an estimate based on other assessments or work completed by the learner; considering the format in which the task is presented.
With this techniques we can adapt conventional teaching to a visually impaired students so that they can learn most of the subjects regarding programming skills.
3 Melodic, the principle and workfow
In this section the fundamental principle of Melodic will be discussed: helping visual impaired people train Computational Thinking. For this we use a set of tactile blocks with diferent
I C P E C 2 0 2 1 4:4 Melodic – Teaching Computational Thinking to Visually Impaired Kids
Figure 1 Combination of three-dimensional models with “Zytec” sketches labelled in Braille as a learning strategy.
shapes, textures and meanings (Figure 2). Each one of this blocks has an assigned value and are part of a class of blocks: Special Block: This block class is characterized by a smooth surface. Special blocks can be of three diferent types: Control: These blocks control the beginning and the end of an algorithm or instruction. The two blocks shown at the top of Figure 3 are used for beginning and ending sequences. The two blocks shown at the bottom of Figure 3 are used to start and stop loops. Speed: These blocks control the time interval between musical notes. As we can see in Figure 4, there are three levels of speed: slow (represented by the symbol “>”), medium (represented by the symbol “> >”) and fast (represented by the symbol “> > >”). The usage of this block is not mandatory; default value is “slow speed”; Instrument: These blocks control the instrument that shall be played. The letter G means Guitar, P means Piano, and F means Flute (Figure 5). This block can be used multiple times throughout the sequence creating a set of notes played by diferent instruments. This is also a non-mandatory block being Piano the default instrument. Number Block: This block class is characterized by a texture with lines and represent numbers from one to ten. These assign value to the number of iterations of a loop or the number of repetitions of a single note (Figure 6). Musical Note Block: This block class is characterized by a rough texture and represent musical notes. The output will be a sound representation of these musical notes (Figure 7); The musical notes played are: Dó, Ré, Mi, Fá, Sol, Lá, Sí, where Dó is represented by the block that contains only one point, the Ré is represented by the block that contains two points, and so on.
As previously mentioned, each block class is characterized by its texture. It is very important that each class of blocks has a diferent texture so that the visually impaired user can easily distinguish each class. The blind user must frst learn the texture that identifes each class of blocks. R. Costa, C. Araújo, and P. R. Henriques 4:5
Figure 2 Tactile Blocks used to interact with Melodic.
Figure 3 Control Blocks.
I C P E C 2 0 2 1 4:6 Melodic – Teaching Computational Thinking to Visually Impaired Kids
Figure 4 Speed Blocks.
Figure 5 Instrument Blocks.
Figure 6 Number Blocks.
Figure 7 Musical Note Blocks.
Figure 8 Top and Bottom faces of one Block. R. Costa, C. Araújo, and P. R. Henriques 4:7
In addition to the texture, the blocks always have the base painted in black (a dark color) and the symbol of the block in wooden color (light color). This color contrast is very important for low vision users, as it allows them to more easily identify the symbol of each block. With this tactile blocks the user can create algorithms to represent some melodies, being able to increase in complexity as he improves his/her Computational Thinking skills. Each block contains a QR code at the bottom that identifes its meaning (Figure 8). To read the meaning of each block, the user must install the Melodic application on his smartphone. After that, with using the mobile app the user must read the QR code located at the bottom of the block. In Section 4 will be presented the architecture of Melodic.
4 Melodic, system architecture
Melodic is a system aimed to be used with a mobile device, being this either a smartphone or tablet. To mitigate incompatibilities between Apple and Android devices, the React Native1 framework was used, which provides parity between the two platforms with the same JavaScript code. In Figure 9 we can see Melodic’s Architecture Diagram that depicts the system compon- ents: User: The user controls the composition of the blocks to form a musical sequence that produces the desired sound after being scanned by the mobile device; Melodic Blocks: Set of ordered blocks (algorithm) composed by the user to form the desired musical sequence. Each block has a QR code, at the bottom, so that the mobile device can read its meaning; Mobile Device: Component in charge of all the logical operations behind this app containing multiple sub-domains: Camera: The camera is in charge of reading the QR codes of the blocks; Melodic App: It is the core of the system. This module receives, through the camera, the QR code of each block and processes it, behaving like a compiler. After processing the QR codes, the Melodic App sends the information (musical melody) to the Speaker module; Speaker: The speaker receives the information, sent by the Melodic App, with the musical melody to play and converts it into a real sound to output to the user.
It is important to notice that the mobile device also provides tactile feedback to the user, in the form of vibrations, to alert the user that the QR code of a given block has been scanned. In order to facilitate the testing of this system in a easy way, it was developed with the help of Expo2. Expo creates a project capable of being accessed by anyone without the need of publishing it on the App Store or Play Store. To access Melodic it is just necessary to install the app Expo.Go and then scan the QR code exhibited Figure 10. Of course this means that the mobile device must be able to read QR codes. To suit this need it is used a library called expo-barcode-scanner that provides the necessary functions.
1 For more information about React Native consult: https://reactnative.dev/ 2 To download Expo consult: https://expo.io/client
I C P E C 2 0 2 1 4:8 Melodic – Teaching Computational Thinking to Visually Impaired Kids
Figure 9 Melodic’s System Architecture Diagram.
After the frst tests set succeedsswd ed, a new set of tests is being prepared to be experimented with members of Íris Inclusiva3.
Figure 10 QR code to access Melodic.
5 Melodic, using the system
After describing Melodic suite, components and interconnection, this section will provide some examples to illustrate how to utilize Melodic as a Learning Resource to train Computational
3 For more information about Íris Inclusiva consult: https://irisinclusiva.pt/ R. Costa, C. Araújo, and P. R. Henriques 4:9
Thinking. An incremental approach to the complexity of problems presented is advised. For this a collection of diferent exercises can be designed, each one increasing the complexity level: Simple Algorithm: This type of algorithm consists of only basic instructions. Figure 11 shows an example of a very simple algorithm composed of four instructions: starting block, two musical note blocks and the ending block. Repetition Algorithm: This type of algorithm is composed of blocks of basic instruc- tions and blocks of repetition. Figure 12 shows an example of this type of algorithm. In this case, in addition to the starting and ending blocks, we have one musical note block in second position, the block with the number three, and then, another musical note block. The block with the number three is intended to play three times the musical note block preceding the repetition number. Changing the Instrument and Speed Instruction: In this case, the user can create an algorithm with attributes: Instrument Played: With the Instrument Blocks the user can change one or more times the instrument played in the sequence (Figures 13 and 14 respectively). When the instrument block is used, the musical notes after the instrument block are played using the sound of the defned instrument; Speed between notes: With the Speed Block the user can change the speed between notes at the time of reproduction. This changes the speed for the whole sequence (Figure 15). Iterative Algorithm (Loop): In this case the user can program sequences that must be executed one or more times implementing the loop concept. Besides the start and end blocks for the whole algorithm, when the user wants to insert a loop, he must insert also special blocks as follows: start, end, number of iterations and the loop body. In this case, the order of the blocks is very important. Figure 16 shows an example of how to implement a loop sequence. Like all algorithms, the program starts with the starting block. To build the loop, it is necessary to place the Start Loop block, then the number of the iteration, then the body of the loop (composed of several other blocks) and, fnally, the End Loop block. The end block determines the end of the algorithm. This type of algorithms is the highest degree of complexity in this system.
The user must feel free to experiment with other block sequences at his will. To use the system the user must frstly create the required block sequence. With the sequence prepared the user must open the Melodic App and start scanning the blocks. For each one the user must turn it around, scan it, and turn it back over to avoid QR Code scanning conficts. At the end of the sequence, when the end sequence block is read the system plays the programmed notes. To listen the melodies resulting from the presented algorithms please access the project Web site at: https://epl.di.uminho.pt/~gepl/Melodic. In order to obtain a melody, blocks must be organized in a correct sequence. Otherwise Melodic will detect and signalize an error. Regarding wrong block sequences, Melodic copes with 2 diferent situations, namely: Start Error: This error occurs when the user forget to initialize the sequence of blocks with the Start Block. Figure 17 shows a sequence that misses a Start Block causing an error message to be thrown. Loop Error: In this case, the user composes the blocks to create the loop sequence in a wrong way leading to an incorrect syntax. Figure 18 shows a loop error caused by missing the number of iterations.
I C P E C 2 0 2 1 4:10 Melodic – Teaching Computational Thinking to Visually Impaired Kids
Figure 11 Example of a Simple Algorithm.
Figure 12 Example of a Repetition Algorithm.
Figure 13 Example of an Algorithm with a Single Instrument Change.
Figure 14 Example of an Algorithm with Multiple Instrument Changes.
To listen to examples of error messages the reader can consult the project homepage at: https://epl.di.uminho.pt/~gepl/Melodic. R. Costa, C. Araújo, and P. R. Henriques 4:11
Figure 15 Example of an Algorithm with Speed Change.
Figure 16 Example of an Iterative Algorithm (Loop).
Figure 17 Example of a Start Block Error.
6 Conclusion
This paper describes Melodic, the motivations behind it and all its features. Melodic is a mixed4 serious game intended to be used as an Learning Resource to train Computational Thinking adapted to visual impaired people.
4 In the sense that it is partially unplugged but also it uses a software application to complement its functionality.
I C P E C 2 0 2 1 4:12 Melodic – Teaching Computational Thinking to Visually Impaired Kids
Figure 18 Example of a Loop Creation Error.
The paper starts by an introduction to the Computational Thinking subject and how it is important to the 21.st Century youth. Following this is an explanation of some methods on how to adapt learning resources to the visual impaired community, focusing on young learners and multiple sensory platforms. Next is the introduction to Melodic, it’s principle and workfow, explaining how, with the help of tactile blocks specifcally designed for this system, we can train Computational Thinking concepts to visual impaired students. Then Melodic’s Architecture is presented. In this section the design and all technological details of the system implementation are described. Finally a description of how to use the system illustrated by simple examples is made. A core component of Melodic is the set of the tactile blocks. This was one of the biggest difculties of this project due to our lack of knowledge in the area of visually impaired people. It was here that Íris Inclusiva came to help. They gave us a lot of valuable opinions and advises on what should the block’s shapes and textures be. This guided us into the fnal system design described previously in this document. Melodic is also thought to be available to everybody, being an institution or an individual person, who wants to have a set of blocks for their own. For this, the person can use his creativity and build each block using materials such as: wood, styrofoam, clay, among others. After having all the blocks ready, it is necessary to paste the respective QR codes at the bottom of each block. These QR codes are all labeled in one document accessible online at https://epl.di.uminho.pt/~gepl/Melodic. Melodic is an inclusive Learning Resource because although it is thought to be used by blind or low vision students, it can also be used by students who do not have visual difculties. This is due to the fact that blocks use a general and intuitive alphabet and do not resort to a blind people specialized one. In conclusion Melodic aims to help visually impaired people to get in touch with a reality that normally relies on a visual interface, that is programming, enabling them to train Computational Thinking. In order to prove that Melodic works, a preliminary test was performed. For this Íris Inclusiva was a major help providing the space and the user to be the frst visual impaired to test the system. In this test we could prove that the blocks are easily identifable (Figure 19) and that the user could in few minutes get very profcient in reading the block’s QR codes (Figure 20). R. Costa, C. Araújo, and P. R. Henriques 4:13
To proceed with Melodic testing, a list of people to test is being developed in partnership with Íris Inclusiva. This tests is a proof of concept of the system proposed. To access the efectiveness of Melodic, two main issue have to be proven: Visual Impaired people can interact with the system; The system develops Computational Thinking.
The frst one has already been tested and for the second one, diagnostic and fnal tests are being designed. These tests have the objective to evaluate the student’s capability to solve problems before and after using Melodic, to understand if their Computational Thinking skill were improved.
Figure 19 User Finding the Tactile Blocks.
Figure 20 User Reading the Sequence Tactile Blocks QR Codes.
I C P E C 2 0 2 1 4:14 Melodic – Teaching Computational Thinking to Visually Impaired Kids
As future work, it is necessary to test the application with more blind and low vision students. For that, we will design and conduct some experiments with impaired members of Íris Inclusiva association. Then, we will make possible changes identifed in the testing phase. Finally other more sophisticated experiments will be conducted with other groups.
References 1 Cristiana Araújo, Lázaro Lima, and Pedro Rangel Henriques. An Ontology based approach to teach Computational Thinking. In Célio Gonçalo Marques, Isabel Pereira, and Diana Pérez, editors, 21st International Symposium on Computers in Education (SIIE), pages 1–6. IEEE Xplore, November 2019. doi:10.1109/SIIE48397.2019.8970131. 2 Dr. Elizabeth J. Erwin, Tifany S. Perkins, Jennifer Ayala, Michelle Fine, and Ellen Rubin. “you don’t have to be sighted to be a scientist, do you?” issues and outcomes in science education. Journal of Visual Impairment & Blindness, 95(6):338–352, 2001. doi:10.1177/ 0145482X0109500603. 3 Council for The Curriculum Examinations and Assessment. Computing at school: Northern ireland curriculum guide for post primary schools. Technical report, CCEA, 2018. 4 William Fraser and Mbulaheni Maguvhe. Teaching life sciences to blind and visually impaired learners. Journal of Biological Education - J BIOL EDUC, 42:84–89, March 2008. doi: 10.1080/00219266.2008.9656116. 5 J. Freeman. The Psychology of Gifted Children. Wiley Series in Developmental Psychology and Its Applications. Wiley, 1985. URL: https://books.google.pt/books?id=dtB9AAAAMAAJ. 6 Mark Guzdial. Teaching programming with music: An approach to teaching young students about logo. Logo Foundation, 1991. 7 N.G. Haring and R.L. Schiefelbusch. Methods in Special Education. McGraw-Hill series in education: Psychology and human development in education. McGraw-Hill, 1967. URL: https://books.google.pt/books?id=ADQ1AAAAMAAJ. 8 Jan L. Plass, Bruce D. Homer, and Charles K. Kinzer. Foundations of game-based learning. Educational Psychologist, 50(4):258–283, 2015. doi:10.1080/00461520.2015.1122533. 9 Alpay Sabuncuoglu. Tangible music programming blocks for visually impaired children. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, TEI ’20, page 423–429, New York, NY, USA, 2020. Association for Computing Machinery. doi:10.1145/3374920.3374939. 10 Victor Silva, Heleniara Moura, Suelen Paula, and Ângelo Jesus. Algo+ritmo: Uma proposta desplugada com a música para auxiliar no desenvolvimento do pensamento computacional. In Anais do XXV Workshop de Informática na Escola, pages 404–413, Porto Alegre, RS, Brasil, 2019. SBC. doi:10.5753/cbie.wie.2019.404. 11 Salete Teixeira, Diana Barbosa, Cristiana Araújo, and Pedro Rangel Henriques. Improving Game-Based Learning Experience Through Game Appropriation. In Ricardo Queirós, Filipe Portela, Mário Pinto, and Alberto Simões, editors, First International Computer Programming Education Conference (ICPEC 2020), volume 81 of OpenAccess Series in Informatics (OASIcs), pages 27:1–27:10, Dagstuhl, Germany, 2020. Schloss Dagstuhl–Leibniz-Zentrum für Informatik. doi:10.4230/OASIcs.ICPEC.2020.27. 12 Stephen Wolfram. How to teach computational thinking, 2016. URL: https://writings. stephenwolfram.com/2016/09/how-to-teach-computational-thinking/. An Open-Source Gamifed Programming Learning Environment José Carlos Paiva # CRACS - INESC-Porto LA, Portugal DCC - FCUP, Porto, Portugal Ricardo Queirós # Ñ CRACS - INESC-Porto LA, Portugal uniMAD - ESMAD, Polytechnic of Porto, Portugal José Paulo Leal # Ñ CRACS - INESC-Porto LA, Portugal DCC - FCUP, Porto, Portugal Jakub Swacha # Ñ University of Szczecin, Poland Filip Miernik # University of Szczecin, Poland Abstract The importance of e-learning tools facilitating the process of learning to program is growing, especially as the pandemic-caused lockdown enforced distance learning in many countries. The key success factor in this process is the provision of an instant and relevant feedback to students. In this paper, we describe a novel open-source programming learning environment featuring automatic assessment of students’ solutions and customized gamifcation. This environment has been developed as a part of the FGPE framework. 2012 ACM Subject Classifcation Applied computing → Computer-managed instruction; Applied computing → Interactive learning environments; Applied computing → E-learning Keywords and phrases learning environment, programming exercises, gamifcation, programming learning, automatic assessment Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.5 Category Short Paper
Funding This paper is based on the work done within the Framework for Gamifed Programming Education project supported by the European Union’s Erasmus Plus programme (agreement no. 2018-1-PL01-KA203-050803).
1 Introduction
The role of feedback in improving knowledge and skill acquisition is considered crucial ([29] and works cited therein). In programming education, particularly, an instant feedback can be generated automatically based on the submission code, providing the student with relevant information about general concepts, task constraints, mistakes made, hints on how to proceed or even meta-cognition [16]. There is also a major concern amongst educational researchers with the disengagement of the students from the learning activities, which frequently leads to academic failure and, lastly, dropout. This issue is even more noticeable in distance learning [28], which became the default way of learning in many countries in times of the current pandemic. One way to counteract this problem is through gamifcation, which has been proven as a capable tool to reduce the dropout rate [10].
In this paper, we describe a novel open-source programming learning environment address- ing both the above-mentioned needs, featuring automatic assessment of students’ solutions and customized gamifcation. As the environment has been developed within the Framework for Gamifed Programming Education (FGPE) project [9], we called it FGPE PLE. The rest of this paper is organized as follows. Section 2 reviews the current state of gamifcation in education, focusing on empirical studies describing experiments which applied game elements into a learning environment. Section 3 presents the FGPE PLE, its architecture and user interface. Finally, Section 4 summarizes the contributions of this work and indicates some future directions.
2 Related Work
Gamifcation of education has been a top trending in the last years [30], which can be explained by the positive boost in motivation and user experience it is accredited with. There are several case studies in the literature, particularly performed in high school and universities, covering diferent learning subjects, ranging from Computer Science / Information Technology (CS/IT) [34, 1, 2, 4, 11, 22, 33] to Mathematics (Maths) [8, 35, 6, 25], Foreign Languages (FL) [13, 26], Communication and Multimedia (C&M) [3, 12, 14, 15], and Medicine / Biology (M/B) [27, 20, 5]. A search for experimental studies that apply gamifcation methods in educational learning environments was conducted in Google Scholar and ScienceDirect databases, using combinations of the following keywords: gamifcation, serious games, education, and learning. From the returned results, only studies published since 2014 which have reported fndings in high school and university subjects of the previously mentioned areas were considered. The result was a set of about 50 empirical research studies, of which 20 were selected according to their relevance. Table 1 presents a comparative study of the collected research papers, regarding the applied game elements.
Table 1 Game elements reported in the selected research papers.
CS/IT Maths FL C&M M/B Game Elements [34] [1] [2] [4] [11] [22] [33] [8] [35] [6] [25] [13] [26] [3] [12] [14] [15] [27] [20] [5] Badges ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Collaboration ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Direct Competition1 ✓ Content Disclosure ✓ ✓ ✓ ✓ ✓ Exhaustible Resources ✓ ✓ ✓ ✓ ✓ ✓ Leaderboards ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Levels ✓ ✓ ✓ ✓ ✓ ✓ ✓ Points2 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Progress Indicator ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Quests ✓ ✓ ✓ ✓ ✓ Real-world Rewards ✓ Serious Games / Immersion ✓ ✓ ✓ ✓ ✓ ✓ SLP3 ✓ ✓ ✓ ✓ ✓ Status ✓ ✓ ✓ ✓ 1Only core competition, i.e., one player (or team) plays against other, if one wins, the other loses. 2Includes experience points (XP), skill points (score), infuence points (rating, reputation), and automatic grades. 3Similar Learning Path (SLP) consists of using data from other students to predict and display the current progress of a student (not leaderboards).
From this study, we can observe that badges, points, and leaderboards are the most applied game elements in general. This was expected as those are the easiest elements to apply, and reuse in distinct educational scenarios. Only a few works use, for instance, quests (5), content disclosure (5), status (4), and direct competition (1) as these are harder to implement and less reusable. Surprisingly, even though there is much more work on gamifcation of CS/IT education compared to other areas of education, no signifcant diference in game elements could be found. J. C. Paiva, R. Queirós, J. P. Leal, J. Swacha, and F. Miernik 5:3
Regarding experimental results, the efects on students are mostly slightly positive or neutral, with a few exceptions [12, 6]. Nevertheless, in the area of CS/IT the results are overall positive, i.e., they either promote engagement, participation, dedicated time, or learning outcome.
3 Programming Learning Environment
FGPE Programming Learning Environment (PLE) is an open-source progressive web ap- plication designed to engage learners while learning to program. To this end, it combines automated assessment with a meaningful composition of gamifcation elements, both provided by the Framework for Gamifed Programming Education (FGPE) ecosystem. The PLE is efortlessly reusable across distinct courses and programming languages, being independent of the gamifcation layer linking the activities. The following subsections provide an in-depth overview of the FGPE PLE. Subsection 3.1 details the architecture of the environment. Subsection 3.2 presents its user interface.
3.1 Architecture
The FGPE PLE is a web-based environment that aims to deliver a single interface to a complete ecosystem for gamifed programming education. This PLE handles two types of users: students – who register and play the games (i.e., courses) in which they are enrolled, by solving the proposed challenges – and teachers – who manage the courses, enroll students, assign them into groups, and collect/analyze data about their progress. As users must also be able to access other tools of the same ecosystem, particularly authors, and use diferent authentication mechanisms, authentication and authorization is managed by Keycloak [17]. Keycloak is an open-source identity and access management solution providing numerous features for the applications, such as centralized user management, authentication, single sign-on and identity brokering, and user federation and social login. Figure 1 presents the FGPE ecosystem, in which the FGPE PLE plays a key role. This ecosystem is composed of two formats, one for describing programming exercises – YAPExIL [24] – and another for gamifcation layers – GEdIL [31], three tools including an authoring tool, a gamifcation service, and an evaluation engine, and GitHub.
stores FGPE AuthorKit
Repository creates (Github) YAPExIL
Programming Exercises Gamified adhere to stored in Exercises Gamification Layers
GEdIL uses
uses Mooshak 2.0 FGPE Gamification Service
uses FGPE PLE
Figure 1 Architecture of the FGPE ecosystem, highlighting the FGPE PLE.
I C P E C 2 0 2 1 5:4 An Open-Source Gamifed Programming Learning Environment
The FGPE ecosystem supports seven distinct types of programming exercises, in particular: BLANK_SHEET, which asks the students to develop their solution from scratch; EXTENSION, which presents a partial solution for the student to complete; IMPROVEMENT, which provides a correct initial source code that needs some optimization; BUG_FIX, that gives a buggy solution for the student to fnd the right code; FILL_IN_GAPS, which provides code with missing parts; SPOT_BUG, that asks students to indicate the location of the bugs in a buggy solution; and SORT_BLOCKS, which asks students to sort the several blocks of code of a solution. To this end, programming exercises are defned in YAPExIL [24], a novel JSON format introduced to address the lack of support for non-traditional programming exercises in existing formats. Furthermore, YAPExIL does not target a specifc evaluation engine, heading all eforts to maximize expressiveness and facilitate the conversion from/to other programming exercise formats. Gamifcation layers are specifed separately from programming exercises using GEdIL [31], an open format for the specifcation of gamifcation layers for educational contents. Even though it does not target a specifc course, GEdIL was initially designed to cover particular requirements of gamifcation for programming courses [32], including a vast collection of rewarding mechanisms, such as points, badges, virtual items, and leaderboards to provide extrinsic motivation, as well as unlockable and secret content, diferent activity modes (e.g., speedup and duels), among others. The game dynamics and mechanics are, then, parsed and applied in the referenced collection of activities by the gamifcation service. FGPE Gamifcation Service [21] is a GraphQL service designed to fulfl this goal, con- suming a GEdIL layer and transforming it into a game, played by enrolled students. Hence, the service has complete support for various rewarding mechanisms such as points, badges, coupons, virtual items, leaderboards, locked and secret content, and diferent activity modes (e.g., time-bomb and speedup). Furthermore, the FGPE Gamifcation Service is the single point of access of the FGPE PLE to this ecosystem. Consequently, it should not only apply gamifcation rules, but also deliver the challenge and activity statements as well as handle the automated assessment of activities. To this end, the service uses plugins, i.e., consumers of the evaluation engine APIs, that leverage on the evaluation engine connected that is adequate to assess that type of activity. An automated evaluation engine guarantees accurate and timely feedback to students. Such an engine is responsible for marking and grading exercises. In this ecosystem, an evaluation engine will: (1) receive the identifcation of the student, a reference to the exercise, and the student’s attempt to solve it; (2) load the exercise by reference from the repository (if necessary); (3) compile the solution and run the tests against the student’s program, comparing the obtained output to the expected; and (4) build report of the evaluation with passed/failed test cases, the grade, and, possibly, some feedback. Therefore, Mooshak 2 has been selected. Mooshak [18] is an open-source web-based system for managing programming contests, which includes automatic judging of submitted programs. Version 2 inherits all features from its original version, improving feedback and attaching support for diferent types of exercises. Moreover, Mooshak supports custom static and dynamic analysis scripts which enables assessment for the non-traditional programming exercises previously described. Finally, having defned two new formats, this ecosystem required a tool to facilitate the au- thoring of new programming exercises adhering to and importing exercises in existing formats into such new programming exercise format as well as their connection with gamifcation rules. FGPE AuthorKit [23] is a web application designed to support the authors through the entire process of preparing gamifed programming exercises, including (1) creation of exercises with their associated metadata, (2) design of the gamifcation techniques for a J. C. Paiva, R. Queirós, J. P. Leal, J. Swacha, and F. Miernik 5:5 specifc exercise or their collection, (3) defnition of the content structure and sequencing rules, and (4) importing and exporting the content in several popular programming exercise formats, using YAPExIL and GEdIL as the base. The user can share content with other peers, internally or via a GitHub repository where all exercise data is synchronized.
3.2 User Interface The FGPE PLE is a progressive web application developed in ReactJS [7] – “a JavaScript library for building user interfaces” –, meaning that it is an application software delivered through the web intended to work on any platform that utilizes a standards-compliant browser, including both desktop and mobile devices. It uses the Apollo Client, a community-driven efort to build an easy-to-understand, fexible, and powerful GraphQL client, to manage and consume remote data from the FGPE Gamifcation Service without worrying about infrastructure code for networking and caching.
Figure 2 Exemplary screenshot of the FGPE PLE User interface.
The User Interface (UI), whose exemplary view is presented in Figure 2, follows a component-driven development approach, which consists in dissecting and splitting the interface into small repeatable patterns, develop these patterns, and join them together to form larger components and pages. The main components of the PLE are the following.
Code Editor is based on Monaco Editor [19], the editor that powers Visual Studio Code. This editor allows students to code in almost any programming language, starting from a skeleton provided by the exercise author, taking advantage of a vast set of features such as syntax highlighting, parameter hints, smart code navigation, and code completion. For challenges that can be solved in more than one programming language, a language switch is attached at the top of the editor allowing the student to choose the programming language of the editor. Console is where the results and feedback from executions and submissions is presented. The execution consists in taking the inputs provided by the student through a popup and running the program against these inputs. Submissions follow a similar process but run against the complete set of test cases provided by the exercise author. Unlike an execution, a submission is considered for statistics and can unlock new resources of the course.
I C P E C 2 0 2 1 5:6 An Open-Source Gamifed Programming Learning Environment
Statement Viewer is where the activity statement is displayed. This component can display either MarkDown, HTML, or raw text fles. Leaderboard is the component responsible for displaying the usernames and scores, sorted according to certain metrics. Leaderboards can be challenge-scoped or course-scoped, use any of the available metrics to sort users by, and optionally have group visibility. Push Notifcations are small rectangle boxes displayed based on events received from re- gistered GraphQL subscriptions. For instance, received rewards, results of both processed submissions and validations, and other updates. Profle is the student’s space to show of her achievements, including badges, virtual items, experience points, leaderboards’ top positions, and course progress.
4 Conclusion
Although gamifcation has its efectiveness demonstrated in engaging students with learning activities, to the best of the authors’ knowledge, until now there has been no programming learning environment that would be, at the same time, open-source, reusable across distinct courses on various programming languages, and enabling instructors to compose their own gamifcation layer providing a good selection of components to choose from. This paper introduces a novel programming learning environment having all the traits mentioned above. It is implemented as a progressive web application that constitutes a single interface to a complete ecosystem for gamifed programming education developed within the Framework for Gamifed Programming Education project [9]; particularly, it renders exercises in the YAPEXIL format [24] and connects with the FGPE GS [21], which processes students’ submissions and gamifcation rules defned in the GEdIL [31] format, generating a relevant instant feedback to the students. Both the programming exercises and gamifcation layers can be created with the companion FGPE AuthorKit tool [23]. The work described here constitutes the fnal artifact to complete the ecosystem proposed by the Framework for Gamifed Programming Education project [9]. A complete validation of the work developed in this project with a large group of students is the immediate next step in our agenda. Furthermore, the FGPE project will have its continuation in the FGPE Plus project, which aims to make the PLE embeddable in any LTI-compliant Learning Management System as well as enhance the PLE’s user experience on mobile devices.
References 1 Paul E. Anderson, Thomas Nash, and Renée McCauley. Facilitating Programming Success in Data Science Courses Through Gamifed Scafolding and Learn2Mine. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE ’15, pages 99–104, New York, NY, USA, 2015. ACM. doi:10.1145/2729094.2742597. 2 Tapio Auvinen, Lasse Hakulinen, and Lauri Malmi. Increasing students’ awareness of their behavior in online learning environments with visualizations and achievement badges. IEEE Transactions on Learning Technologies, 8(3):261–273, 2015. doi:10.1109/tlt.2015.2441718. 3 Gabriel Barata, Sandra Gama, Joaquim Jorge, and Daniel Gonçalves. Studying student diferentiation in gamifed education: A long-term study. Computers in Human Behavior, 71(Supplement C):550–585, 2017. doi:10.1016/j.chb.2016.08.049. 4 Andrija Bernik, Goran Bubas, and Danijel Radosevic. A Pilot Study of the Infuence of Gamifcation on the Efectiveness of an e-Learning Course. In Central European Conference on Information and Intelligent Systems, pages 73–79, Varaždin, Croatia, 2015. University of Zagreb, Faculty of Organization and Informatics. J. C. Paiva, R. Queirós, J. P. Leal, J. Swacha, and F. Miernik 5:7
5 Mads T. Bonde, Guido Makransky, Jakob Wandall, Mette V. Larsen, Mikkel Morsing, Hanne Jarmer, and Morten O. A. Sommer. Improving biotech education through gamifed laboratory simulations. Nature biotechnology, 32(7):694–697, 2014. doi:10.1038/nbt.2955. 6 Katheryn R. Christy and Jesse Fox. Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women’s math performance. Computers & Education, 78(Supplement C):66–77, 2014. doi:10.1016/j.compedu.2014.05.005. 7 Facebook. React: A JavaScript library for building user interfaces. https://reactjs.org, 2021. accessed on 16 Jan 2021. 8 Usef Faghihi, Albert Brautigam, Kris Jorgenson, David Martin, Angela Brown, Elizabeth Measures, and Sioui Maldonado-Bouchard. How gamifcation applies for educational purpose specially with college algebra. Procedia Computer Science, 41(Supplement C):182–187, 2014. 5th Annual International Conference on Biologically Inspired Cognitive Architectures, 2014 BICA. doi:10.1016/j.procs.2014.11.102. 9 FGPE Consortium. Framework for Gamifed Programming Education. https://fgpe.usz. edu.pl, 2018. accessed on 19 Mar 2021. 10 Wad Ghaban and Robert Hendley. How Diferent Personalities Beneft From Gamifcation. Interacting with Computers, 31(2):138–153, March 2019. doi:10.1093/iwc/iwz009. 11 Lasse Hakulinen, Tapio Auvinen, and Ari Korhonen. The Efect of Achievement Badges on Students’ Behavior: An Empirical Study in a University-Level Computer Science Course. International Journal of Emerging Technologies in Learning, 10(1):18–29, 2015. doi:10.3991/ ijet.v10i1.4221. 12 Michael D. Hanus and Jesse Fox. Assessing the efects of gamifcation in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, efort, and academic performance. Computers & Education, 80:152–161, 2015. doi:10.1016/j.compedu.2014.08. 019. 13 Tatsuhito Hasegawa, Makoto Koshino, and Hiromi Ban. An English vocabulary learning support system for the learner’s sustainable motivation. SpringerPlus, 4(1):99, 2015. doi: 10.1186/s40064-015-0792-2. 14 Caitlin Holman, Stephen J. Aguilar, Adam Levick, Jef Stern, Benjamin Plummer, and Barry Fishman. Planning for Success: How Students Use a Grade Prediction Tool to Win Their Classes. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge, pages 260–264, New York, NY, USA, 2015. ACM. doi:10.1145/2723576.2723632. 15 Jincheul Jang, Jason J. Y. Park, and Mun Y. Yi. Gamifcation of online learning. In Cristina Conati, Neil Hefernan, Antonija Mitrovic, and M. Felisa Verdejo, editors, Artifcial Intelligence in Education, pages 646–649, Cham, 2015. Springer International Publishing. 16 Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. Towards a Systematic Review of Automated Feedback Generation for Programming Exercises. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, pages 41–46, Arequipa Peru, 2016. ACM. doi:10.1145/2899415.2899422. 17 Keycloak. Keycloak: open source identity and access management solution. https://www. keycloak.org, 2014. accessed on 9 Jan 2021. 18 José Paulo Leal and Fernando Silva. Mooshak: A Web-based multi-site programming contest system. Software: Practice and Experience, 33(6):567–581, 2003. doi:10.1002/spe.522. 19 Microsoft. Monaco Editor. https://microsoft.github.io/monaco-editor/, 2021. accessed on 16 Jan 2021. 20 Christa R Nevin, Andrew O Westfall, J Martin Rodriguez, Donald M Dempsey, Andrea Cherrington, Brita Roy, Mukesh Patel, and James H Willig. Gamifcation as a tool for enhancing graduate medical education. Postgraduate Medical Journal, 90(1070):685–693, 2014. doi:10.1136/postgradmedj-2013-132486. 21 José Carlos Paiva, Alicja Haraszczuk, Ricardo Queirós, José Paulo Leal, Jakub Swacha, and Sokol Kosta. FGPE Gamifcation Service: A GraphQL Service to Gamify Online Education.
I C P E C 2 0 2 1 5:8 An Open-Source Gamifed Programming Learning Environment
In Proceedings of the 9th World Conference on Information Systems and Technologies, Terceira Island, Azores, Portugal, 2021. Springer. (to appear). 22 José Carlos Paiva, José Paulo Leal, and Ricardo Alexandre Queirós. Enki: A Pedagogical Services Aggregator for Learning Programming Languages, page 332–337. ACM, New York, NY, USA, 2016. 23 José Carlos Paiva, Ricardo Queirós, José Paulo Leal, and Jakub Swacha. FGPE AuthorKit – A Tool for Authoring Gamifed Programming Educational Content. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE ’20, page 564, New York, NY, USA, 2020. ACM. doi:10.1145/3341525.3393978. 24 José Carlos Paiva, Ricardo Queirós, José Paulo Leal, and Jakub Swacha. Yet Another Pro- gramming Exercises Interoperability Language (Short Paper). In Alberto Simões, Pedro Ran- gel Henriques, and Ricardo Queirós, editors, 9th Symposium on Languages, Applications and Technologies (SLATE 2020), volume 83 of OpenAccess Series in Informatics (OASIcs), pages 14:1–14:8, Dagstuhl, Germany, 2020. Schloss Dagstuhl–Leibniz-Zentrum für Informatik. doi:10.4230/OASIcs.SLATE.2020.14. 25 Mads Kock Pedersen, Anette Svenningsen, Niels Bonderup Dohn, Andreas Lieberoth, and Jacob Sherson. DifGame: Game-based Mathematics Learning for Physics. Procedia - Social and Behavioral Sciences, 228(Supplement C):316–322, 2016. 2nd International Conference on Higher Education Advances,HEAd’16, 21-23 June 2016, València, Spain. doi:10.1016/j. sbspro.2016.07.047. 26 Bernadette Perry. Gamifying French Language Learning: A Case Study Examining a Quest- based, Augmented Reality Mobile Learning-tool. Procedia - Social and Behavioral Sciences, 174(Supplement C):2308–2315, 2015. International Conference on New Horizons in Education, INTE 2014, 25-27 June 2014, Paris, France. doi:10.1016/j.sbspro.2015.01.892. 27 Robin K. Pettit, Lise McCoy, Marjorie Kinney, and Frederic N. Schwartz. Student perceptions of gamifed audience response system interactions in large group lectures and via lecture capture technology. BMC medical education, 15(1):92, 2015. doi:10.1186/s12909-015-0373-7. 28 Bardh Prenkaj, Giovanni Stilo, and Lorenzo Madeddu. Challenges and Solutions to the Student Dropout Prediction Problem in Online Courses. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, page 3513–3514, New York, NY, USA, 2020. ACM. doi:10.1145/3340531.3412172. 29 Valerie J. Shute. Focus on Formative Feedback. Review of Educational Research, 78(1):153–189, 2008. doi:10.3102/0034654307313795. 30 Jakub Swacha. State of Research on Gamifcation in Education: A Bibliometric Survey. Education Sciences, 11(2):69, 2021. doi:10.3390/educsci11020069. 31 Jakub Swacha, José Carlos Paiva, José Paulo Leal, Ricardo Queirós, Rafaele Montella, and Sokol Kosta. GEdIL–Gamifed Education Interoperability Language. Information, 11(6):287, May 2020. doi:10.3390/info11060287. 32 Jakub Swacha, Ricardo Queirós, José Carlos Paiva, and José Paulo Leal. Defning requirements for a gamifed programming exercises format. Procedia Computer Science, 159:2502–2511, 2019. Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 23rd International Conference KES2019. doi:10.1016/j.procs.2019.09.425. 33 Alexandru Topîrceanu. Gamifed learning: A role-playing approach to increase student in-class motivation. Procedia Computer Science, 112(Supplement C):41–50, 2017. Proceedings of KES 2017, Marseille, France. doi:10.1016/j.procs.2017.08.017. 34 Andika Y. Utomo, Affa Amriani, Alham F. Aji, Fatin R. N. Wahidah, and Kasiyah M. Junus. Gamifed E-learning model based on community of inquiry. In Advanced Computer Science and Information Systems (ICACSIS), 2014 International Conference on, pages 474–480, Jakarta, Indonesia, 2014. IEEE, IEEE. doi:10.1109/icacsis.2014.7065830. 35 Ibrahim Yildirim. The efects of gamifcation-based teaching practices on student achievement and students’ attitudes toward lessons. The Internet and Higher Education, 33(Supplement C):86–92, 2017. doi:10.1016/j.iheduc.2017.02.002. Integrating a Graph Builder into Python Tutor Diogo Soares # University of Minho, Braga Portugal Maria João Varanda Pereira1 # Ñ Research Centre in Digitalization and Intelligent Robotics, Polythechnic Insitute of Bragança, Portugal Pedro Rangel Henriques # Ñ University of Minho, Braga, Portugal
Abstract Analysing unknown source code to comprehend it is quite hard and expensive task. Therefore, the Program Comprehension (PC) subject has always been an area of interest as it helps to realize how a program works by identifying the code that implements each functionality. This means being able to map the problem domain with the program domain. PC is a complex area, but its importance for programmers is so high that many approaches and tools were proposed along the last two decades. Program Animation is one of those approaches requiring specialized techniques. For each programming language, there are already tools that enable us to execute a program step by step, visualize its execution path, observe the efect of each instruction on its data structures, and inspect the value of its variables at any point. In the present context, we sustain the idea that PC techniques and tools can also be of great value for students taking the frst steps in programming using a specifc language. To this end, we aim to improve Python Tutor, a well-known program visualization tool, with graph-based representations of source code such as Control Flow Graph (CFG), Data Flow Graph (DFG), Function Call Graph (FCG) and System Control Graph (SCG). This helps novice programmers to understand the source code analyzing not only the variable contents but also a set of automatically generated graph-based visualizations, that were not included in Python Tutor so far. This will allow the students to be focused on certain aspects of the program (depending on the graph), abstracting others such as details of its syntax.
2012 ACM Subject Classifcation Human-centered computing → Visualization systems and tools; Social and professional topics → Computer science education; Software and its engineering → Source code generation
Keywords and phrases Program Visualization, Python Tutor, Data Flow Graphs, Control Flow Graphs
Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.6
Supplementary Material Software (Graph Builder): https://graph-builder.di.uminho.pt/visualize.html
Funding This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the Projects Scopes: UIDB/05757/2020 and UIDB/00319/2020.
1 Introduction
There’s along the web many tutorials on programming and several learning methodologies but the majority of them uses the same formula [4]: start with the explanation of a set of examples and propose some similar exercises that train the students to write similar code to solve them. To fully understand a programming language, it is not enough to read the code
of programs written in that language, it is also necessary to comprehend how the computer behaves when executing the programs code. For each piece of code, the student shall be able to build a full map between the program domain and the problem domain. Usually it is hard to understand his own code, even to a software engineer that has to maintain third-part code he never saw before, the task is much harder. The average developer spends about 60% to 90% of his time understanding code previously developed; these fgures are not changing over time [19]. With this in mind, this paper describes and discusses the integration into Python Tutor of a new tool for automatic generation of graph-based visualizations to turn ease the task of program comprehension. As said in [7], the best way to understand programs is by giving programs another aspect than that of their source code. This new feature works as Python Tutor plugin and it analyses the input code and produces several types of complementary graphs depending on the user desires. As the program comprehension task is also very relevant for the beginners wishing to understand how a given program works and how it solves a given problem, we believe that Python Tutor improved with our plug-in will be very useful to help programming students. The paper has 5 sections after that. The state of the art on program animation appears in Section 2. Section 3 discusses how graph-based visualizations can be included in an Animator adding actual value to it. Section 4 exhibits the architecture to build the new tool and discusses how it can be developed and integrated into the host tool. Section 5 illustrates the fnal system built and describes a experiment conducted to assess the value added by the visualizations provided. Section 6 closes the paper and points out some directions for future research.
2 Program Visualization and Animation
Think-aloud protocol [22] is a method where a person verbalize his thoughts and it can be used not only to improve self−understanding and reasoning but also to explain the cognitive process to others. That’s why is so important to produce documentation when developing software. The use of this method also allows realizing that the thoughts of a developer difer according to the familiarity of a program’s domains [17]. With this in mind, it was derived two concepts of comprehension models, the top-down comprehension model and the bottom-up comprehension models. The top-down approach is used when the programmer is familiar with the program he’s trying to understand. He starts by creating a general hypotheses about the program goal and how it is achieved by using his previous knowledge about the program. This is called the expectation-based comprehension, where the programmer has pre-generated expectations of the code’s meaning [13]. After creating his hypotheses, the developer starts searching in the code for algorithms/structures that he can link to his theory and refnes his hypotheses as he does that [6]. This is the inference-based comprehension [13]. If the programmer has no previous knowledge about a program, he has to analyze line by line to create a hypothesis about the program purpose. In this method, the developer starts aggregating functions by their goals. What usually happens is that developers end by using an integrated model [11] where they use top-down approach when possible and bottom-up when necessary. Although the way of coding varies from person to person, follow a guide of good practices in programming (i.e. appropriate variable names and indentation rules [18]) can lead to an uniformed structure that makes program comprehension easier. A diferent form of coding can really slow down the comprehension of a program for the most experienced programmer [20]. D. Soares, M. J. V. Pereira, and P. R. Henriques 6:3
Nowadays, there’s even more factors that can afect program comprehension. For example, a simple color schema, available in syntax-oriented text editor/IDE, helps the programmers comprehending a program [16]. Newer studies have their focus not only on source code comprehension, but also on the comprehension of structures, hierarchies and architecture of the program as well on the relations between components [19]. Software visualization is the process of giving a visual representation of a program. It takes the information that is presented in the source code and converts it to a graphical representation with the goal of improving the comprehension of that program. Although a visual image can help to understand a program, it’s not that simple since a basic program can have multiple representations. The best one will depend on the task to be performed and what the developer wants to know. This also means that each representation has to be thought for specifc tasks and a good portray of the system is imperative for it to be helpful. Simply repackaging massive textual information into a massive graphical representation is not helpful, so a linear translation is not ideal since experts want to see software visualised in context – not just what the code does, but what it means [14]. The visual representation of a program can be focused in some aspects of the program and it can have diferent levels of abstraction. This can also solve scalability problems because the visualization of big programs at once it will not be a good help. So, the visualization tool must allow an high level of interactivity to go one step at a time, flter the information, to establish the data that is wanted, to defne the level of abstraction and zoom facilities [8]. Graphically these representations can be based on diagrams, maps, icons, graphs and specifc drawings. Some examples are the Nassi-Shneiderman diagram used to represent a control-fow of a program [12]; Space-flling (like tree maps or sunburts) allows to visualize in a very compressed way code metrics and statistics [5]; Graphs are the most common representation and consists of nodes and arcs where the nodes represent blocks of code and the arcs its fow [10]. In terms of functionality, there are two types of visualizations: static and dynamic [10]. A static representation can be seen as a simple image of the program, it doesn’t change over time and it’s based on static information. Dynamic representations shows the information that changes as the program is executed. A good example of an animated visualization tool is Python Tutor. Python Tutor is an animation web tool that intends to help new programmers to understand programs and currently supports some of the most popular languages at the moment like Python, Java, C, C++, JavaScript, TypeScript and Ruby [15]. An interactive step by step presentation that shows what is really happening behind every line of code during an execution instance (dynamic visualization) is the main feature of Python Tutor. This allows the user to keep up with the modifcations of values that each variable sufers as each line of code is executed. The user has total control of this presentation as he can choose if he wants to go to the next or previous step. Global variables are kept in a global frame and then local variables just appear during the function execution. Tools that produce static visualizations of source code are more usually found. AgileJ StructuredViews [1] is a plugin for eclipse that generates UML class diagrams from the source code. Since UML is a language used to structure software projects that is easy to understand, it can be used to explain to the developers and even to the client. Moreover it creates an easy to read and updated representation of the code. Sourcetrail [3] is a tool that can be connected to an IDE or a text editor that displays an interactive dependency graph and a search bar to search functions, classes or variables. Its interactivity allows the user to see both an overview of the program and the dependencies between classes, methods and variables. CodeSonar [2] ofers a visualization software that shows an interactive call
I C P E C 2 0 2 1 6:4 Graph-Based Visualization Builder
Figure 1 Python Tutor visualization.
graph where it is possible to see the directories, fles and functions rearranged hierarchically. It allows the user to choose which metrics he wants to be displayed in the graph as well compare functions/fles based on that metrics.
3 Improving Python Tutor with graph-based visualizations
The main idea is to improve the platform Python Tutor by incorporating new features to it. The objective is an interactive tool where the user can select what kind of graph he wants to see in order of being able to analyze visually the fow of code or data dependencies in a part of his code which execution is being animated. Three types of graphs are proposed: Control Flow Graph (CFG): It is a representation of all paths (sequences of instructions) that might be traversed during the execution of a given program; each node represents a basic block of code (a sequence of instructions without alternatives, with just one input and one output). By observing a control fow graph we can see to where the values go and from where they came. It is a very useful graph to realize the dependencies that exist in the program and which statements are infuenced by others. For instance, a statement B is control dependent on a statement A if B execution depends of the A outcome [23]. Data Flow Graph (DFG): In this graph there are two types of nodes where one represents the values/variables and other the operators [21]. This graph allows the user to observe all operations that any variable sufers in its lifetime and in this way understand the existing data dependencies. For instance, it is said that B is data dependent on A when A value is used to compute B. This class of graphs is usually more difcult to understand because they tend to represent a lot of information. Besides that, it is not so linear to read as a CFG for example. Function Call Graph (FCG): This class of graphs displays the relationships between a function and the functions it calls [9]. On one hand, FCG is useful to understand the functions necessary to execute a function, and this is crucial to local errors and change points. On the other hand, it allows the user to identify the most used functions and D. Soares, M. J. V. Pereira, and P. R. Henriques 6:5
Figure 2 System Workfow.
the ones that are not being used at all. This information is extracted from the program static analysis and knowing which functions are most called can be useful to improve the program performance, since improving these functions will afect a big part of the program. It is, usually, a very easy to read graph.
These new features were incorporated on Python Tutor adding a new feld with a scroll bar to select the graph type and a new button to generate the desired graph. Figure 2 shows a diagram of the desired fow.
The source program to be visualized through the dependency graphs is inserted (typed, or edited) at the home page of Python Tutor in order to reuse as much as possible the existing functionalities. So the input text is sent to the back-end of Python Tutor to be compiled. If the input program is correct and ready to be ran and animated, the front-end will receive the execution trace. Once the input is validated, the system will open a new web page created only to visualize the desired graphs. After the user choose the graph that he wants to see, the input is passed through a new route to the graph generator. The graph generator will create an image with the chosen graph and return it to the front-end. The front-end will display the image on that new page so the user can visualize it and then remove the image in order to avoid an overcrowding of fles.
I C P E C 2 0 2 1 6:6 Graph-Based Visualization Builder
Figure 3 System architecture.
4 Integration Architecture and Developments
The Graph Builder was developed in Python using AnTLR to generate the parser and static analyser for the source programs. It is incorporated into Python Tutor to increase its functionality. The user uses Python Tutor editor to write the input program and then it will be parsed by the syntactic analyser generated by AnTLR to identify the control and data dependencies among program elements and represent them as graphs. This analyser built by AnTLR from the Python grammar is able to recognize Python programs. Once the information about the input program is gathered the next step is to generate the graphs according to that information. Initially the graphs were built in Python with the help of some libraries like ’igraph’. In a second phase, it was used PyGraphViz to build better quality graphs with zoom features and more appealing for users. Figure 3 shows a diagram depicting the architecture of the developed system. As said above, to implement the referred parser it was required to defne a Python grammar. In this case, it was used the Python grammar available on AnTLR Github. Depending on the information needed to extract, it was required to make some minor modifcations on the original grammar. For instance, splitting one grammar rule into three diferent ones in order to treat the data more specifcally. After defning the grammar, semantic actions were specifed in order to collect information and generate graphs. The semantic actions were associated to each grammar rule and were constantly adjusted during the development of the graph builder tool. Every time some piece of input code allowed to detect an unhandled case, semantic actions were updated (added or changed) in one or more grammar rules. The way the information was processed had to be as accurate as possible to faithfully build graphs that deal with a large amount of data. So, the structures had to be constantly updated with new information. The three graphs presented use, by defnition, a level of granularity where each node corresponds to an instruction, which can be difcult to visualize for large programs. On the other hand, Python Tutor itself has a limit of lines of code for the input program and even so, for larger programs, it is possible to zoom in and zoom out of the graphs. In the next section, the fnal structures chosen to represent each graph will be presented and explained as well the information they hold.
4.1 Function Call Graph
There are two types of functions that can be called in a program: functions defned by the developer or built-in functions (i.e., functions that are available from Python libraries). A function is recognized by an identifer (its name) followed by parenthesis or, in case of being a function written by the programmer, by the keyword def followed by its defnition. All the function calls found in a program are gathered in a Python dictionary where the key D. Soares, M. J. V. Pereira, and P. R. Henriques 6:7
Figure 4 FCG construction. is the name of the caller (the function that calls) and the value is the callee (the function called). In Figure 4 it can be seen how the information required to create a Function Call Graph is stored. On the fnal graph the two types of functions are distinguished by color: blue for the functions created by the user, and green for Python built-in functions.
4.2 Control Flow Graph
The data for the construction of a CFG is stored in a similar way as the one explained for the previous graph, that is represented in a dictionary with an entry for each defned function. As the construction of this graph is not so linear as the previous one, it is required to save more information in the dictionary. So, instead of saving just the code statement, it also saves the type of that statement and the block number where it appears. Analysing each statement, one can notice that some do not afect the execution fow and others like loop (for or while) or conditional instructions (if,elif, else) have a strong infuence on the fow. Each node of the graph can have one or more edges coming out depending on which type of statement it represents. For example, a node of loop or conditional type will have two edges coming out of it, one in case of the condition being true and the other in case of being false. A loop node will also have at least one more edge coming in from the last statement inside the loop. An else node will always be preceded by one node of a non-simple type. The block number represents the statement level: the frst level corresponds to the main, other levels mean that is inside a (possibly nested) block depending on a condition. The body (group of statements that depends on one statement) of non-simple statements will always have a bigger depth level. When the depth level changes from one statement to another, it allows the software to recognize that it will start a new block or that one just ended. There are other statements that afect the fow of a program like for example the reserved words break or continue. As can be observed in Figure 6, the CFG also contains coloured arrows in order to be more intuitive for the user to identify which path is followed if the condition present in the node is true (green arrow) or false (red arrow). Besides the color diference, the true path arrowhead is flled unlike the false one that is not. These kind of features can help the user to better read the graph.
I C P E C 2 0 2 1 6:8 Graph-Based Visualization Builder
Figure 5 CFG construction.
Figure 6 CFG Example. D. Soares, M. J. V. Pereira, and P. R. Henriques 6:9
Figure 7 DFG construction.
4.3 Data Flow Graph
This sort of graph is not as linear as the previous ones because one variable depends not only on the other variables or function results but also on loops or conditional statements. There are also variables that only exist on a specifc scope. Despite these constraints, the information necessary to build DFG is gathered in the same way as the previous ones. In this case, there are three types of tuples in the construction of this graph, one that represents a change of value of a variable (variable name, block number, new value and operations that produced the new value), one that serves to control the loops and conditionals dependencies (depth level, statement, alternative path), and a third one to represent the case when a variable can have multiple values depending on the execution path (all the possible variable values). As can be seen in Figure 7, it is fundamental to use colors for the nodes in order to get more intuitive drawings. Therefore it is used the color green for the nodes that represent the variables which value changed, and in red the nodes that aggregate all possible values of that variable at a given moment. In addition to use diferent colors diferent arrow types are also generated as can be seen in fgure 6. The other nodes depict values or operations. This graph also includes boxes surrounding sub-graphs that represent loop blocks (cycles) or conditional blocks aiming at producing clearer and more informative drawings.
5 Python Tutor with Graph Builder
To illustrate the project fnal outcome, Figure 8 shows a screenshot of Python Tutor integrated with the Graph Builder. In this case it was generated a Control Flow Graph (showed in the right side window) for the input program written in the left side window. In order to assess the users’ satisfaction and the usability of the Graph Builder plugin for Python Tutor, a specifc experiment with Python Programmers as testers was designed and conducted. For that purpose, it was created a survey that contained instructions for the testers to follow in order to provide their opinion, answering some questions about the new features. The testers should be able to write Python functions and analyse the resulting graphs. This survey was completed by thirty one participants. The majority of the participants were students or ex-students of a Master’s in Informatics Engineering, thus fulflling the necessary requirements. These participants already have programming knowledge; that fact allows us to get the insights of people familiarized with the world of
I C P E C 2 0 2 1 6:10 Graph-Based Visualization Builder
Figure 8 Python Tutor with Graph Builder.
programming. The results so far obtained were very positive and increased the confdence on this Python Tutor plugin (the Graph Builder); they also provided a good insight on its relevance in the area of program comprehension. The survey also was helpful to identify some minor bugs in the construction of the graphs; after being fxed, the accuracy and the intuitiveness of the graphs were increased.
5.1 Survey Result Analysis
First, it will be presented the results related to the technical side, like the accuracy of the tool and the complexity level of the functions used to test the feature. Then, it will be discussed the opinion of the testers about the features ofered by the new Python Tutor plugin. These results, unlike the frst ones, will be analysed separately for each graph so that we can identify clearly those that can be more helpful for a deeper program comprehension.
5.1.1 Code Complexity
The frst question of the survey was related to the complexity of the code the tester wrote. In order to test a major diversity of programs each tester was free to chose the program to use and this question was included to relate the correctness of the resulting graph with its complexity. As it is observable on the pie-chart of Figure 9, the majority of the functions written to test were of low complexity. This is not a problem for the desired assessment because Python Tutor is directed to new programmers and, under these circumstances, the expected inputs are of low to medium complexity. It is important to note that the complexity of the program is subjective to whoever wrote it.
5.1.2 Graphs Correctness
The three graphics in Figure 10 allow to analyse the accuracy for each graph type. D. Soares, M. J. V. Pereira, and P. R. Henriques 6:11
Figure 9 Complexity level of functions used to test the feature.
The numbers shown in Figure 10 indicate a good percentage of accuracy for each graph. The most surprisingly result being the FCG (see 10a); it was expected to have the highest accuracy but ended up being the one with the least. This may have happened because as this is the simplest graph, the tests performed on this graph were not as intense as the others, not allowing to catch as many exceptions like happened with the other graphs. On the other hand, it was encouraging to realise how accurate was the CFG (see 10b), and that the DFG got also a good rating (see 10c) mainly keeping in mind the amount of dependencies necessary to show. These results were very satisfying mainly because the bugs reported were corrected which means that this accuracy is now increased.
5.1.3 Graphs and Program Comprehension
In this subsection we will see whether graphs available can help people on comprehending a program. The survey prepared for the experiment under discussion includes two questions for each graph. The frst question aims at evaluating the intuitiveness of the respective graph, or in other words, perceive whether the graphs are easy to read or not. The second question aims at understanding if the tester thinks the graphs can help the comprehension of the program being visualized.
I C P E C 2 0 2 1 6:12 Graph-Based Visualization Builder
According to the graphic in Figure 11b, 87,1% of the programmers who tested the tool think that FCG can improve a program comprehension what follows the Figure 11a where 83.8% fnd it intuitive. This is because it is a very simple graph providing a generic overview of the system behavior as it only shows the functions who are called by other functions.
(a) FCG Level of Intuitiveness. (b) FCG On Program Comprehension.
Figure 11 Perceptions on FCG.
The results shown in the pie graphics of Figure 12 reinforce the preconceived idea that the CFG will be the strength of the new feature. These positive results boost the confdence that this graph can really be helpful on program comprehension.
(a) CFG Level of Intuitiveness. (b) CFG On Program Comprehension.
Figure 12 Perceptions on CFG.
As expected, DFG was considered the least intuitive graph (see Figure 13a for details). As shown in Figure 13b, although the lack of intuitiveness of this graph, a considerable amount of people still thinks that it can help on the comprehension of a program, which means that working out on the improvement of this component is a priority in the near future. At the end, these results were very positive and increased the confdence on this Python Tutor plugin (Graph Builder) and how it can aid on program comprehension. This survey let us identify some minor bugs made along the construction of the graphs that after being fxed increased their accuracy and intuitiveness. D. Soares, M. J. V. Pereira, and P. R. Henriques 6:13
(a) DFG Level of Intuitiveness. (b) DFG On Program Comprehension.
Figure 13 Perceptions on DFG.
6 Conclusion
Control Flow Graph (CFG), Data Flow Graph (DFG) and Function Call Graph (FCG) are examples of formal instruments that help to visualize statically some program perspectives that depict the program behavior. Those instruments are considered important for professional programmers to better comprehend the software systems they have to deal with. Taking this statement as proved in area of Program Comprehension, we argued in this paper that they can also be valuable artifacts to help beginner programmers to learn that topic. Moreover, if we can combine the visualization of the referred graphs with program animation tools, we believe that we can motivate and engaged students. In this context, the paper reported a project aiming at building a tool that transforms a function into its corresponding control and data fow graphs and to integrate it with the program animator tool Python Tutor. At the end, it was proven that the usage of program animation features complemented with dependency graphs visualization tools is feasible and facilitates the comprehension of programs. The ambition was to make this tool a general one that could help all computer program- mers, not only beginners having to learn problem solving and deep their knowledge on a specifc programming language, but also professionals carrying out their software maintenance tasks. Even recognizing that scaling up this feature to cope with large, real size, programs is not so efective – because the corresponding graphs are big and confusing, and they require more resources than those provided Python Tutor – we think that the experience succeeded as a support to programming courses since it is covered the statements common to the most used imperative languages. The process of building this tool encountered obviously some obstacles. The biggest difculty faced was related to the Python Grammar used that does not cover all the real program situations causing many compiling exceptions that frequently arose. Other challenge was the integration with Python Tutor as this platform resorted to some outdated libraries. However, all these troubles have been overcome ending up with a functional tool to help understanding the algorithm and data structures implemented by small programs that students have to study following an easy, attractive and fast learning process. The design and conduction of new and more complete experiments involving a larger number of students with diferent ages and in diferent programming courses, is the most urgent task that needs to be performed to complete the project. It is important to show that it is an efective way to help programming students. The developed tool (Graph-Builder) is available at https://graph-builder.di.uminho.pt/visualize.html
I C P E C 2 0 2 1 6:14 Graph-Based Visualization Builder
As future work, the frst proposal is to implement the same control-fow and data-fow graphs for the other languages presented in the Python Tutor like Java and C. It is expected to be easier to implement it now as the new languages will only need the construction of a grammar for each language as the graph generator can be reused and the integration with Python Tutor would only need a few adaptations. The other direction for the future of this Python Tutor add-on is to improve the aspect of the DFG as the testers inquired said that it is not very intuitive and easy to comprehend.
References 1 Agilej structureviews. http://www.agilej.com/. 2 Codesonar. https://www.grammatech.com/products/source-code-analysis. 3 Sourcetrail. https://www.sourcetrail.com/. 4 Luís Alves, Dušan Gajic, Pedro Rangel Henriques, Vladimir Ivancevic, Vladimir Ivkovic, Mak- sim Lalic, Ivan Lukovic, Maria João Varanda Pereira, Srdan Popov, and Paula Correia Tavares. C Tutor usage in relation to student achievement and progress: A study of introductory pro- gramming courses in Portugal and Serbia. Computer Applications in Engineering Education, n/a(n/a), 2020. doi:10.1002/cae.22278. 5 Marla Baker and Stephen Eick. Space-flling software visualization. Journal of Visual Languages & Computing, 6:119–133, June 1995. doi:10.1006/jvlc.1995.1007. 6 Ruven Brooks. Towards a theory of the comprehension of computer programs. International Journal of Man-Machine Studies, 18(6):543–554, 1983. doi:10.1016/S0020-7373(83)80031-5. 7 Luis M. Gómez-Henríquez. Software visualization: An overview, 2001. 8 Martin Hadley. 3 benefts of interactive visualization, January 2018. 9 Ben Holland. Call graph construction algorithms explained, March 2016. 10 François Lemieux and Martin Salois. Visualization techniques for program comprehension - a literature review. In SoMeT, 2006. 11 A. Mayrhauser and A. Marie Vans. From program comprehension to tool requirements for an industrial environment. In [1993] IEEE Second Workshop on Program Comprehension, pages 78–86, August 1993. doi:10.1109/WPC.1993.263903. 12 Ike Nassi and B. Shneiderman. Flowchart techniques for structured programming. ACM SIGPLAN Notices, 8:12–26, August 1973. doi:10.1145/953349.953350. 13 Michael O’Brien, Jim Buckley, and Teresa Shaft. Expectation-based, inference-based, and bottom-up software comprehension. Journal of Software Maintenance, 16:427–447, November 2004. doi:10.1002/smr.307. 14 Marian Petre. Mental imagery, visualisation tools and team work. In Proceedings of the Second Program Visualisation Workshop, pages 3–14, January 2002. 15 Ben Putano. A look at 5 of the most popular programming languages of 2019. https: //stackify.com/popular-programming-languages-2018/. Accessed: 26-9-2019. 16 Gerard Rambally. The infuence of color on program readability and comprehensibility. ACM Sigcse Bulletin, 18(1):173–181, February 1986. doi:10.1145/5600.5702. 17 Teresa M. Shaft and Iris Vessey. The relevance of application domain knowledge: Characterizing the computer program comprehension process. J. Manage. Inf. Syst., 15(1):51–78, 1998. doi:10.1080/07421222.1998.11518196. 18 Ben Shneiderman, Richard Mayer, Don McKay, and Peter Heller. Experimental investigations of the utility of detailed fowcharts in programming. Commun. ACM, 20:373–381, June 1977. doi:10.1145/359605.359610. 19 J. Siegmund. Program comprehension: Past, present, and future. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), volume 5, pages 13–20, March 2016. doi:10.1109/SANER.2016.35. 20 E. Soloway and K. Ehrlich. Empirical studies of programming knowledge. IEEE Transactions on Software Engineering, SE-10(5):595–609, September 1984. doi:10.1109/TSE.1984.5010283. D. Soares, M. J. V. Pereira, and P. R. Henriques 6:15
21 Marilyn Wolf. Computers As Components, Third Edition: Principles of Embedded Computing System Design. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 3rd edition, 2012. 22 Mami Yoshida. Think-aloud protocols and type of reading task: The issue of reactivity in l2 reading research. In Selected Proceedings of the 2007 Second Language Research Forum, January 2008. 23 Andreas Zeller. Chapter 7 - deducing errors. In Andreas Zeller, editor, Why Programs Fail (Second Edition), pages 147–173. Morgan Kaufmann, Boston, second edition edition, 2009. doi:10.1016/B978-0-12-374515-6.00007-1.
I C P E C 2 0 2 1
Matching User Interfaces to Assess Simple Web Applications
Marco Primo # Faculty of Sciences, University of Porto, Portugal José Paulo Leal1 # Ñ Faculty of Sciences, University of Porto, Portugal CRACS – INESC, Portugal Abstract This paper presents ongoing research aiming at the automatic assessment of simple web applications, like those used in introductory web technologies courses. The distinctive feature of the proposed approach is a web interface matching procedure. This matching procedure verifes if the web interface being assessed corresponds to that of a reference application; otherwise, provides detailed feedback on the detected diferences. Since web interfaces are event-driven, this matching is instrumental to assess the functionality. After mapping web interface elements from two applications, these can be targeted with events and property changes can be compared. This paper details the proposed matching algorithm and the current state of its implementation. It also discusses future work to embed this approach in a web environment for solving web application exercises with automatic assessment. 2012 ACM Subject Classifcation Information systems → Web interfaces; Applied computing → Computer-managed instruction Keywords and phrases automatic assessment, web interfaces, learning environments Digital Object Identifer 10.4230/OASIcs.ICPEC.2021.7 Category Short Paper
Funding José Paulo Leal: This work is fnanced by National Funds through the Portuguese funding agency, FCT – Fundação para a Ciência e a Tecnologia, within project UIDB/50014/2020.
1 Introduction
Automatic assessment of programming exercises is nowadays a standard practice in intro- ductory programming courses [2]. It usually boils down to a black-box approach, where the assessed program is fed with test data, and the resulting output is compared with the output produced by a reference solution, for the same data. This approach is programming language agnostic and can be easily adapted to a wide range of settings. However, one of its shortcomings is the inability to deal with graphical user interfaces, in particular with web interfaces. Graphical user interfaces in general, and web interfaces in particular, rely on user interaction for input rather than on data streams. On these applications, data is received as events that are targeted to diferent elements, without a predefned order. Moreover, web interfaces rely on the articulation of more than a single language – namely HTML, CSS, and JavaScript. These two characteristics prevent the use of simple black-box techniques, typically used for automatic assessment in algorithms and data structure courses. To overcome this problem we propose a diferent approach to assess exercises in introduct- ory web technologies courses. These exercises usually involve coding a simple web interface with a small number of widgets, such as buttons, editing felds, and selectors. Although
