Supporting the Learning of Rapid Application Development in a Database Environment Silviu Risco BEng(Mech), PGradDipInfTech Thesis submitted in fulfillment of the requirements for the degree of Doctor of Philosophy Faculty of Science and Engineering Queensland University of Technology, Australia. August, 2012 ii Keywords A list of keywords for this thesis is as follows: Intelligent Tutoring Systems, Student Modelling, Individualised Feedback, Rapid Application Development, Microsoft Access iii Abstract Intelligent Tutoring Systems (ITSs) are computer systems designed to provide individualised help to students, learning in a problem solving context. The difference between an ITS and a Computer Assisted Instruction (CAI) system is that an ITS has a Student Model which allows it to provide a better edu- cational environment. The Student Model contains information on what the student knows, and does not know, about the domain being learnt, as well as other personal characteristics such as preferred learning style. This research has resulted in the design and development of a new ITS: Personal Access Tutor (PAT). PAT is an ITS that helps students to learn Rapid Application Development in a database environment. More specifically, PAT focuses on helping students to learn how to create forms and reports in Microsoft Access. To provide an augmented learning environment, PAT’s architecture is dif- ferent to most other ITSs. Instead of having a simulation, PAT uses a widely- used database development environment (Microsoft Access). This enables the students to ask for help, while developing real applications using real database software. As part of this research, I designed and created the knowledge base re- quired for PAT. This contains four models: the domain, student, tutoring and exercises models. The Instructional Expert I created for PAT provides individualised help to the students to help them correctly finish each exercise, and also proposes the next exercise that a student should work on. PAT was evaluated by students enrolled in the Databases subject at QUT, and by staff members involved in teaching the subject. The results of the evaluation were positive and are discussed in the thesis. Table of Contents 1 Introduction 1 1.1 ArtificialIntelligenceinEducation. 2 1.2 IntelligentTutoringSystems . 4 1.2.1 TheArchitectureofanITS . 5 1.2.2 ITSBehaviour........................ 6 1.2.3 ITSKnowledgeBase . 12 1.3 RapidApplicationDevelopment . 14 1.4 ResearchGoalandObjectives . 17 1.5 ResearchScope ........................... 21 1.5.1 WorkdonePriortothisResearch . 21 1.5.2 ResearchScope ....................... 21 1.6 OutlineofThesis .......................... 23 2 Literature Review 27 2.1 Introduction............................. 28 2.2 TheComponentsofanITS . 30 2.2.1 TheDomainModel. 30 2.2.2 TheStudentModel . 31 2.2.3 TheInstructionalExpert. 36 v vi TABLE OF CONTENTS 2.2.4 TheSimulationComponent . 42 2.3 KnowledgeRepresentationFormalisms . 44 2.4 DetailedDescriptionofSpecificSystems . 47 2.4.1 ITSsforLearningAboutDatabases . 47 2.4.2 ITSsforLearningAboutProgramming . 55 2.4.3 ITSswithaDetailedStudentModel . 59 2.4.4 ITSsthatDo/Don’tUseaSimulation. 64 2.5 ImplicationsforResearchinthisThesis . 72 3 SystemRequirementsandArchitecture 73 3.1 SystemRequirements. 74 3.1.1 AnalysingtheStudents’Needs. 74 3.1.2 FunctionalRequirements . 78 3.1.3 Non-functionalRequirements . 79 3.2 TeachingMSAccess ........................ 81 3.2.1 MicrosoftAccessOverview . 81 3.2.2 TeachingAccessatQUT . 83 3.3 PAT’sArchitecture . 85 3.3.1 ArchitecturalDesign . 85 3.3.2 KnowledgeRepresentation . 87 3.3.3 PAT’sFunctionality . 88 3.4 Summary .............................. 89 4 Designing new Models 91 4.1 Introduction............................. 92 4.2 TheDomainModel......................... 93 4.2.1 DomainModelConceptualSchema . 93 4.2.2 ExtendtheUseoftheDomainModel . 98 4.3 TheExercisesModel . .101 TABLE OF CONTENTS vii 4.3.1 ExerciseDescription . .101 4.3.2 ExerciseSolution . .105 4.3.3 CreatingaNewExercise . .113 4.4 TheStudentModel . .115 4.4.1 Student’s Knowledge About the Domain . 115 4.4.2 Student’s Personal Characteristics . 116 4.4.3 Student’sHistory . .117 4.5 UpdatingtheStudentModel. .120 4.6 TheTutoringModel . .123 4.7 Summary ..............................128 5 Giving Feedback to the Student 129 5.1 Introduction.............................130 5.2 AnalysisoftheStudent’sSolution . .132 5.3 Synthesis: GeneratingtheFeedback . .136 5.3.1 HintLevels .........................136 5.3.2 HighLevelPseudoCodeforSynthesis . .139 5.4 ProposeaNewExercise . .152 5.4.1 Considerations when Recommending a New Exercise . 152 5.4.2 Pseudo Code for Recommending a New Exercise . 154 5.5 Summary ..............................157 6 Building a new ITS 159 6.1 TechnologiesUsed. .160 6.2 Interfaces ..............................162 6.2.1 FirstTimeInterfaces . .162 6.2.2 OuterLoopInterface . .164 6.2.3 InnerLoopInterfaces. .167 6.2.4 AdditionalInterfaces . .178 viii TABLE OF CONTENTS 6.3 DatabasesandTables. .179 6.4 ModulesStructureandCode. .182 6.5 Summary ..............................184 7 System Evaluation 185 7.1 Evaluating an Intelligent Tutoring System . 186 7.1.1 AspectsofanITSEvaluation . .186 7.1.2 EvaluationMethods . .188 7.1.3 Choosing the Correct Evaluation Method . 191 7.1.4 HowwasPATEvaluated . .194 7.2 PAT’sSummativeEvaluation . .195 7.2.1 Evaluations’Objectives. .195 7.2.2 EvaluationswithStudents . .196 7.2.3 EvaluationswithStaff . .206 7.2.4 Conclusions for Summative Evaluation . 211 7.3 Pre-PostTestEvaluation. .214 7.3.1 InformationSources . .214 7.3.2 TheStudentPopulation . .215 7.3.3 TheTopicsConsidered . .215 7.3.4 ResultsofthePre-PostEvaluation . .216 7.4 Summary ..............................220 8 Summary of Contributions and Future Directions 221 8.1 ResearchContributions. .222 8.1.1 DesignofPAT’sArchitecture . .222 8.1.2 DesignNewModels. .223 8.1.3 Design PAT’sFeedbackMechanism . .224 8.1.4 EvaluationofPAT . .225 8.2 ResearchLimitations . .226 TABLE OF CONTENTS ix 8.3 FutureDirections . .227 AResearchMethodologyandProcess 229 B Initial Focus Group 233 C How to Install PAT 239 D How to Use PAT 245 List of Figures 1.1 Theoretical foundations for Intelligent Tutoring Systems. .... 4 1.2 ThegeneralarchitectureforanITS.. 5 1.3 ThelifecyclestagesofRAD. 14 2.1 ThegeneralarchitectureforanITS.. 28 2.2 Typesofstudentmodels.. 32 2.3 Exampleofanontology. 44 2.4 ThearchitectureofDB-suitetutors.. 48 2.5 SQL-TUTOR’sinterface.. 49 2.6 Kermit’suserinterface.. 50 2.7 Normit’suserinterface.. 52 2.8 Acharya’sarchitecture. 52 2.9 Acharya’suserinterface. 54 2.10 ThearchitectureofBITS. 56 2.11 TheuserinterfaceofBITS. 56 2.12 AnexampleofanexercisefromELP. 57 2.13 An example of customised compilation message from ELP. .. 58 2.14 TheuserinterfaceofJITS.. 59 2.15 Inferring students’ behaviour Wayang Outpost. ... 60 xi xii LIST OF FIGURES 2.16 ThelogicalviewofTILE’sstudentmodel. 61 2.17 Attributes’ specifications of TILE’s student model. ..... 62 2.18 TAO’suserinterface. 65 2.19 The simulation interface for Cardiac Tutor. ... 66 2.20 TheuseoftheCALsystem. 67 2.21 TheinterfaceofAlgeBraintutor. 68 2.22 The interface of the Advanced Geometry Tutor. 69 3.1 The configuration of components in the knowledge base. .. 79 3.2 Microsoft Office Access 2007 - graphical interface. .. 81 3.3 MicrosoftOfficeAccess2007-exampleofaform. 82 3.4 Microsoft Office Access 2007 - example of a report. 83 3.5 PAT’sarchitectureandmaincomponents. 86 3.6 PartofthesemanticnetworkofPAT. 88 4.1 Accessobjects. ........................... 93 4.2 Controls that can be created inside of forms or reports. .... 94 4.3 Example of properties for a TextBox control instance. .... 94 4.4 Domainmodelconceptualschema. 96 4.5 Semanticnetworkforthedomainmodel. 97 4.6 AccessvsOpenOfficeBaseControls. 98 4.7 Access vs OpenOffice Base - properties for objects. 99 4.8 Access vs OpenOffice Base - properties for a form. 99 4.9 Exercisemodelconceptualschema. .104 4.10 Exampleofaform,inanexercise. .108 4.11 Exampleofasolutiongraph.. .109 4.12 Studentmodelconceptualschema. .119 4.13 Solution graph and the correctness of a student solution. .121 4.14 An example of a diagram associated to an action. 125 LIST OF FIGURES xiii 4.15 Tutoringmodelconceptualschema. .127 5.1 TypesOfErrors. ..........................134 5.2 Generatingthehinttextforanadvicetype. 137 6.1 PAT’smenuinAccess. .161 6.2 FirstTimewindow.. .163 6.3 Questionswindow. .163 6.4 TheWelcomewindow. .164 6.5 MyProfilewindow. .165 6.6 ThefeedbacksiteforPAT. .166 6.7 Tooltipsforawindow. .167 6.8 ThisExercisewindow. .168 6.9 AnexampleoftheTrafficLightswindow.. .171 6.10 Advicewindow. .172 6.11 Screenshotfortherequiredform. .174 6.12 Alltherequirementsforanexercise.. 175 6.13 Morereadings. ...........................175 6.14 Another example of a diagram associated to an action. 176 6.15 ExampleofusingPAT. .177 6.16 WelcomeBackwindow.. .178 6.17 Tablesintheknowledgebase. .180 6.18 Student-systeminteractions. 183 7.1 ChoosinganevaluationmethodforanITS.. .193 7.2 The questions given to the students (first page). 197 7.3 The questions given to the students (second page). 198 7.4 Students’background. .199 7.5 Students’answersforQuestion4. .200 7.6 Students’answersforQuestions5and8. .201 xiv LIST OF FIGURES 7.7 Students’answersforQuestion6. .203 7.8 Students’answersforQuestion11. .205 7.9 Students’answersforQuestions10and11. 205 7.10 Thequestionsgiventostaffmembers.. .207 7.11 TeachingstaffanswersforQuestion4.. 209 7.12 Students and teaching staff answers for questions 11 and 7. .211 7.13 Topicsknownatpreandposttests. .217 7.14 Students confident using Access vs students new to Access. .218 7.15 Distribution of students by topics learned. .219 A.1 A multi-methodological approach to IS Research . 230 A.2 A Process for Systems Development Research . 231 B.1 The questions given to students - focus group . 237 C.1 Firstinstallwindow. ..