Time-Dependent Recommender Systems for the Prediction of Appropriate Learning Objects vorgelegt von M.Sc. Christopher Krauß geb. in Berlin von der Fakultät IV – Elektrotechnik und Informatik der Technische Universität Berlin zur Erlangung des akademischen Grades Doktor der Ingenieurwissenschaften - Dr.-Ing. - genehmigte Dissertation Promotionsausschuss: Vorsitzender: Prof. Dr. Axel Küpper Gutachter: Prof. Dr. Manfred Hauswirth Gutachterin: Prof. Dr. Agathe Merceron Gutachter: Prof. Dr. Hendrik Drachsler Tag der wissenschaftlichen Aussprache: 29. Mai 2018 Berlin 2018 i Abstract This dissertation deals with adaptive learning technologies which aim to optimize Technology Enhanced Learning (TEL) offerings to fit the individual learner’s needs. Thereby, Recommender Systems play a key role in supporting the user’s decision process for items of interest. This works very well for e-commerce and Video on Demand services. However, it is found to be the case that these traditional Recommender Systems cannot be directly transferred to TEL as the recommendation of course items follows a particular educational paradigm. The special conditions of this paradigm are first investigated and then taken into account for the realization of new algorithms. In order to allow a broad interoperability of a Recommender System with other technical components, a set of open standards and specifications results in a reference architecture for such an adaptive learning environment. Based on the realized architecture, activity data have been collected from students using course materials available online – the courses themselves comprising face-to-face lectures backed up by digital representations of the presented contents, blended learning settings as well as online-only courses. The courses provided access to the course materials via a novel Learning Companion Application. This app also presents learning recommendations to make the content selection more efficient and effective. Thereby, this work indicates that an educational Recommender System should not be evaluated using standard evaluation frameworks that utilize, for instance, a classical n-fold cross-validation. For this reason, a time-dependent evaluation framework is defined to investigate the precision of the Top-N learning recommendations at various points in time. Moreover, a new measure is introduced to determine the Mean Absolute Timeliness Deviation between an item recommendation and the time when it is actually accessed by the user. Subsequently, four major techniques for Recommender Systems are realized and applied to the collected data, evaluated with the time-dependent evaluation framework and successively optimized. As a reference implementation, a traditional Collaborative Filtering algorithm is developed and extended to incorporate time information. The results are then compared to the results of other time sensitive algorithms: an Item-based Collaborative Filtering approach which has previously been applied to TEL and a new learning path generator which incorporates a set of contextual information. Finally, a novel time-weighted Knowledge-based Filtering algorithm is presented and exhaustively analyzed. The evaluation results indicate that time-dependent filtering which incorporates multi-contextual activity data can produce the most precise recommendations. iii Zusammenfassung Die vorliegende Dissertation beschäftigt sich mit adaptiven Lerntechnologien, die sich an die individuellen Bedürfnisse der Lernenden anpassen. Dabei spielen vor allem Empfehlungssysteme eine Schlüsselrolle, da sie den Entscheidungsprozess der Benutzer unterstützen. Das funktioniert sehr gut für E-Commerce und Video on Demand-Dienste. Allerdings können diese Mechanismen nicht einfach für den Bereich des Technologie-gestützten Lernens übertragen werden, da die Empfehlungen von Kursinhalten einem sehr speziellen Paradigma folgen. Die Eigenschaften dieses Paradigmas werden in der Dissertation erst analysiert und anschließend als Basis für neue Algorithmen berücksichtigt. Um eine breite Interoperabilität des Empfehlungssystems mit anderen technischen Kompo- nenten zu gewährleisten, wurden offene Standards und Spezifikationen umgesetzt, mit deren Hilfe eine Referenzarchitektur für adaptive Lernumgebungen umgesetzt wurde. Basierend dar- auf wurden Aktivitätsdaten in Echtwelt-Kursen gesammelt – von Präsenzunterricht, welcher durch digitales Vorlesungsmaterial unterstützt wurde, über Blended Learning-Umgebungen bis hin zu ausschließlichen Online-Kursen. Alle Kursteilnehmer hatten Zugriff auf die Kursmate- rialien über die Lernbegleiter-App. Der Entscheidungsprozess der Lernenden wurde durch ein Lernempfehlungssystem unterstützt. Dabei hat sich herausgestellt, dass herkömmliche Evaluationstechniken, wie die n-Fold Cross- Validation, nicht für die Evaluation von Lernempfehlungssystemen geeignet sind. Deshalb wurde ein zeitabhängiges Evaluations-Framework definiert, mit dem die Präzision von Top-N-Lernempfehlungen zu verschiedenen Zeitpunkten analysiert werden kann. Zusätzlich wurde eine neuartige Messgrö- ße eingeführt, die „Mean Absolute Timeliness Deviation”, die den zeitlichen Abstand zwischen Empfehlungen und dem späteren Abruf der Inhalte durch den Benutzer misst. Darauf basierend konnten vier Haupttechniken für Empfehlungssysteme realisiert und auf die gewonnenen Datensätze angewandt werden. Dann wurden diese mit dem definierten Evaluations- Framework ausgewertet und sukzessive optimiert. Als Referenzimplementierung diente ein tra- ditioneller Collaborative Filtering-Algorithmus. Dieser ließ sich mit zeitabhängigen Algorith- men vergleichen: mit einer Item-based Collaborative Filtering-Methode, welche bereits für das Technologie-gestützte Lernen angewandt wurde, sowie mit einem Lernpfad-Generator, der kon- textabhängige Informationen verarbeitet. Anschließend ist ein neuartiger kontextsensitiver und zeitabhängiger Knowledge-based Filtering-Algorithmus vorgestellt und ausgewertet worden. Die Arbeit zeigt, dass die präzisesten Empfehlungen durch zeitabhängige Filter-Algorithmen produziert werden, die zusätzlich mehrere Typen von Aktivitätsdaten verarbeiten. v Acknowledgments First of all, I would like to thank my doctoral thesis supervisor, Prof. Dr. Manfred Hauswirth Exam (chair of Open Distributed Systems at the Technische Universität Berlin), for his support and Committee & guidance as well as for providing feedback regarding the completion of the dissertation. He is Supervisors not only my supervisor, since as the Executive Director of the Fraunhofer-Institute for Open Communication Systems (FOKUS), he is also the primary enabler of my research. Without him, this work would not have been possible. I am especially indebted to Prof. Dr. Agathe Merceron of the Beuth University of Applied Sciences, who supported me in writing a doctoral thesis on educational Recommender Systems. She helped with the structuring of the dissertation and shared her detailed knowledge on Data Mining and Technology Enhanced Learning. She understood the issues raised during the course of my study, structured my ideas and made practical suggestions as this work was carried out. I would like to offer my special thanks to Prof. Dr. Hendrick Drachsler of the Goethe University Frankfurt and the German Institute for International Educational Research, who agreed to examine this dissertation. As one of the most prominent researchers in the field of Recommender Systems for Technology Enhanced Learning, he laid substantial groundwork for this doctoral thesis. Moreover, I would like to thank Prof. Dr. Axel Küpper of the research group Service-centric Networking at Telekom Innovation Laboratories (An-Institut Technische Universität Berlin) for his willingness to support my doctoral application and the commitment to chair the doctoral committee. I am grateful for the encouragement given to me by Dr. Stefan Arbanowski who was not only my day-to-day supervisor but also my mentor. Regarding my research approach, he assisted me in thinking outside of the box and encouraged me to keep going. He offered me many opportunities to deepen my knowledge through engagement with the research community, attendance at scientific conferences and networking with industry. I also would like to thank the other colleagues who were involved in my various projects and Smart who thus also enabled this work. First and foremost, I would like to thank the entire team of the Learning Smart Learning project that produced some outstanding results. In particular, I would like to Team highlight the contributions of individuals from a number of organizations. From FOKUS: Miggi Zwicklbauer (who is also a patient officemate); from Beuth University under the leadership of Prof. Dr. Agathe Merceron: Sinh-Truong An and Francois Dubois; and from the IZT: Dr. Michael Scharp. In addition, many student research assistants supported my learning-related projects as employees vi of FOKUS. Among others and in chronological order from the time they started work: Igor Fritzsch, Rakesh Chandru, Berken Bayat, Dominique Jürgensen, Jessica Gomez and Tolga Karaoglu. Students & A lot of university students and participants from other institutions utilized the Learning Participants Companion Application – even though it was still in an early state. Their feedback was invaluable and the suggestions improved the project outcome significantly. I will not forget the large number of mainly anonymous test persons who invested their time and creativity in evaluating the app and taking part in the surveys. Within the last 5