IMPLEMENTATION OF THE RELATIONLOG SYSTEM A THESIS SUBMITTEDTO THE FACULTYOF GRADUATESTUDIES AND RESEARCH IN PARTIALFULFILLMENT OF THE REQUIREMENTS FOR THE DEGREEOF MASTEROF SCIENCE IN COMPUTERSCIENCE UNIVERSITYOF REGINA BY Riqiang Shan Regina, Saskatchewan September 4, 1998 @ Copyright 1998: Riqiang Shan National Library Bibliothèque nationale du Canada Acquisitions and Acquisitions et Bibliographie Services services bibliographiques 395 Wellington Street 395. me Wellington OttawaûN K1AW OüawaON KIAW canada Canada The author has g=anteda non- L'auteur a accordé une licence non exclusive licence dowing the exclusive permettant à la National Lïbrary of Canada to Bibliothèque nationale du Canada de reproduce, 10- distriiute or sell reproduire, prêter, distriiuer ou copies of this thesis in microform, vendre des copies de cette thèse sous paper or electronic formats. la forme de rnicrofiche/fiim. de reproduction sur papier ou sur format électronique. The author retains ownership of the L'auteur conseme la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts f?om it Ni la thèse ni des extraits substantiels may be priuted or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation, Abstract Advanced database applications could require the abilities to store data with com- plex structures and to perform inference. However, such capabilit ies are not directly supported in exist ing database systems. Relat ionlog, a persistent deductive database systern, has been designed and implernented to directly support the storage and infer- ence of data with a complex structure. It provides a uniform declarative language for defining, manipulating and querying a database. It also supports user-defined com- plex types and employs various evaluat ion strategies for query processing wit hout the user's intervention. The Relat ionlog system can be used in business management, information retrieval, industry, academic course teaching and research. This thesis gives an introduction to the Relationlog database language, which includes a SQL-like Data Definition Language, Data Manipulation Language, and Query Language. It discusses the design and irnplernentation of the Relationlog sys- tem as well as evaluation strategies. It also contains the experimental results, and a comparison with other related systems. Acknowledgments 1 would like to take this opportunity to thank that many people who helped me a lot. Many thanks go first to my supervisor, Dr. Mengchi Liu, for his valuable advice and support throughout this research project. 1 am grateful to Dr. Liu for providing valuable guidance during my research, for letting me use every cornputer that he has, and for arranging my financial support. He has provided several key ideas that appear in various chapters in this thesis. Secondly, many thanks go to the members of my theris cornmittee, Dr. Mingyuan Chen, Dr. Larry Saxton, and Dr. Brien Maguire, for their precious comments and suggestions. 1 am also very grateful for the financial support given by the Department of Computer Science, Faculty of Graduate Studies and Research of the University of Regina, Faculty of Science of the University of Regina, and NaturaI Sciences and Engineering Research Council of Canada. My special thanks go to Mr. Cory Butz, and Mr. John Palrnquist, for their patience in proofreading my thesis. I am also highly appreciated a lot of help from my generous friends: Binshu Shen, Huichu Mo, Guoqiang Huang, Xiaoyang Guan, Qinglai Zhang, and Xiaoyun Wang. Lastly, 1 would like to give my thanks to my beloved wife, Mingyuan Deng, for her encouragement, understanding, support and sacrifice in my studies. Contents Abstract Table of Contents iii List of Tables vi List of Figures vii Chapter 1 Introduction 1 Chapter 2 Background 5 2.1 Relational Mode1 ............................. 5 2.2 Complex Value Models .......................... 7 2.3 Deductive Databases ........................... 11 2.3.1 Datalog .............................. 11 2.3.2 Bottom-Up Evaluation ...................... 13 2.3.3 TopDown Evaluation ...................... 15 2.4 Implemented Deductive Database Systems ............... 16 2.4.1 The LDL System ......................... 16 2.4.2 The CORAL System ....................... 18 2.4.3 Aditi ................................ 19 2.4.4 Glue-Nail ............................. 20 iii Chapter 3 Relationlog Database Language 21 3.1 Data Definition Language ........................ 21 3.1.1 Domain Types .......................... 21 3.1.2 Schema Definition ......................... 24 3.1.3 Rules and Views ......................... 29 3.2 Query Language .............................. 32 3.3 Data Manipulation Canguage ...................... 36 3.4 Relationlog Programs ........................... 38 3.4.1 Well-Typed Programs ...................... 40 3.4.2 Stratification ........................... 40 Chapter 4 Design and Implementation of Relationlog 43 4.1 System Architecture ........................... 44 4.2 File Architecture of the Persistent Database .............. 45 4.2.1 Nodes ............................... 46 4.2.2 Data File ............................. 48 4.2.3 Index File ............................. 49 4.2.4 System Catalog Files ....................... 50 4.3 Storage and Update Subsystem ..................... 50 4.3.1 BufferManager .......................... 51 4.3.2 Index Manager .......................... 55 4.3.3 Update Manager ......................... 56 4.4 DDL Manager ............................... 57 4.4.1 Representation of Schernas .................... 59 4-42 Representation of Rules ..................... 59 4.4.3 Representation of Catalogs .................... 63 4.5 DML Manager ............................... 65 4.5.1 Representation of Relations ................... 65 4.5.2 Update Issues ........................... 67 4.6 Query Manager .............................. 67 4.6.1 Internai Representat ion ...................... 68 4.6.2 Query Optirnization ....................... 69 4.6.3 Grouping ............................. 70 Chapter 5 Query Evaluation Strategies 72 5.1 Matching ................................. 72 5.2 Semi-Naive Bottom-Up with Rule Orderings .............. 73 5.2.1 Stratification ........................... 74 5.2.2 Basic Semi-Naive Algorithm For Relationlog .......... 77 5.2.3 Extended Semi-Naive Algorit hm For Relat ionlog ........ 79 5.2.4 Rule Orderings .......................... 83 5.3 Topdown Pipelining ........................... 85 5.4 Summaxy ................................. 87 Chapter 6 Experirnental Results and Related Work 89 6.1 Experimental Results ........................... 89 6.2 CornparisonwithOtherSystems ..................... 94 Chapter 7 Conclusion and Future Research 97 7.1 Contribution ................................ 98 7.2 Future Research .............................. 99 Bibliography Appendix A Syntax of Relationlog List of Tables 2.1 Similar concepts in databases and Datalog ............... 12 6.1 Test results on STU ........................... 92 6.2 Test results on edgesd .......................... 93 6.3 Summary of Deductive Database Systems ............... 95 List of Figures 2.1 Nested relations defined with extended relational algebra operators . 10 2.2 Top-Down Evaluation of a Query .................... 16 3.1 Sarnple Dornain Definitions ....................... 23 3.2 Sample Schema Definitions ........................ 25 3.3 Sarnple Relations ............................. 26 3.4 ROBOT Relation Schema ........................ 27 3.5 ROBOT relation ............................. 28 3.6 Sample Relationlog Program f arnily .................. 39 3.7 Dependency Graph for the family Prograrn .............. 41 3.8 Dependency Graph for the jamily Program .............. 42 4.1 RelationlogSystemArchitecture ..................... 44 4.2 Filestructure ............................... 47 4.3 Index and Data File ........................... 49 4.4 Index File ................................. 49 4.5 Page Slot Address Format ........................ 52 4.6 Mapping Between Disk and Memory .................. 54 4.7 Components of the DDL Manager .................... 57 4.8 Interna1 Schema Representation ..................... 59 4.9 Schema Structure for ROBOT ...................... 60 4.10 Interna1 Rule Representation ....................... 62 4.11 System Catalog Files ........................... 64 4.12 Interna1 Representation of a Fact .................... 66 4.13 The Updated Persons Fact ....................... 68 vii 5.1 Dependency Graph of base-view . 76 5.2 Query Evaluation Flowchart . 88 viii Chapter Introduction Advanced database applications could require effective storage, efficient access and infercnce of large amounts of data with complex structures. However, such capabilities are not directly supported by the existing database systems. In recent decades, the nested relational and complex value models have been developed to extend the applicability of the traditional relational mode1 to more complex applications such as Computer-Aided Design (CAD), image processing and text retrieval. Extended relational algebra and calculus are provided for such kinds of models. It is shown in [3] that extended relational algebra with the powerset operator and sale extended relational calculus with the subset predicate are also equivalent and can simulate iteration and express transitive closure. However, they do so in a very