Implementation of the CORAL Deductive Database System Raghu Ramakrishnan Divesh Srivastava Univ of Wisconsin Madison Univ of Wisconsin Madison y Praveen Seshadri S Sudarshan ATT Bell Labs Murray Hill Univ of Wisconsin Madison Intro duction Abstract In this pap er we discuss the design and implementation of CORAL is a deductive database system that provides a mo du the CORAL deductive database system CORAL seeks to lar declarative query languageprogramming language CORAL combine features of a database query language such as e is a deductive system that supp orts a rich declarative language cient treatment of large relations aggregate op erations and provides a wide range of evaluation metho ds and allows a combi declarative semantics with those of a logic programming nation of declarative and imp erative programming The data can language such as more p owerful inference capabiliti es and b e p ersistent on disk or can reside in mainmemory We describ e the architecture and implementation of CORAL supp ort for incomplete and structured data Supp ort for p ersistent relations is provided by interfacing with the EX There were two imp ortant goals in the design of the CORAL ar chitecture to integrate the dierent optimization techniques ODUS storage manager A unique feature of CORAL is in a reasonable fashion and to allow users to inuence the that it provides a wide range of evaluation strategies such as evaluation strategies used so as to exploit the full p ower of the topdown b ottomup and their variants and allows users to CORAL implementation A CORAL declarative program can b e optionally tailor execution of a program through highlevel organized as a collection of interacting mo dules and this mo d annotations The language is describ ed in Applica ule structure is the key to satisfying b oth these goals The high tions in which large amounts of data must b e extensively level mo dule interface allows mo dules with dierent evaluation techniques to interact in a transparent fashion Further users analyzed are likely to b enet from this combination of fea can optionally tailor the execution of a program by selecting from tures In comparison to other deductive database systems among a wide range of control choices at the level of each mo dule such as Aditi EKSV LDL LOLA and CORAL also has an interface with C and users can pro NailGlue CORAL provides a more p owerful language gram in a combination of declarative CORAL and C extended and supp orts a much wider range of optimization techniques with CORAL primitives A high degree of extensibility is provided We highlight several design decisions that allowed us to by allowing C programmers to use the class structure of C to enhance the CORAL implementation integrate often unrelated evaluation techniques and opti mizations in a nearly seamless fashion Sp ecically we con This research was supp orted by a David and Lucile Packard sider the following issues Foundation Fellowship in Science and engineering a Presidential Young Investigator Award with matching grants from Digital Data representation eg constants lists terms Equipment Relation representation and implementation eg in Corp oration Tandem and Xerox and NSF grant IRI mainmemory and diskresident The address of the authors are Computer Sciences De partment University of Wisconsin Madison WI Index structures eg hashstructures and Btrees USA and ATT Bell Lab oratories Mountain Av enue Murray Hill NJ USA The authors email Evaluation techniques eg materialization and top addresses are fraghudiveshpraveengcswiscedu and sudar down sharesearchattcom y The work of this author was p erformed largely while he was In the CORAL implementation we divide evaluation into in the University of Wisconsin Madison a numb er of distinct subtasks and provide a clean inter face b etween the subtasks relevant optimization techniques can b e almost indep endentl y applied to each subtask Ex tending database programming languages has received much attention lately and we b elieve that the CORAL exp eri ence oers guidelines for resolving several common issues 0 that go b eyond the sp ecic extensions that are addressed in Page QUERY DATA CORAL MAIN EVALUATION MANAGER MEMORY SYSTEM One of our goals was to allow users to exploit the full answers p ower of the implementation CORAL supp orts a very rich and we b elieve that some user guidance is criti language query optimized USER program to eectively optimizing many sophisticated programs cal INTERFACE EXODUS is to provide users with the ability to cho ose The problem O.S. STORAGE supp orted by CORAL in a from the suite of optimizations program FILES MANAGER relatively orthogonal and highlevel way and to use a combi QUERY t parts of a program The nation of optimizations for dieren OPTIMIZER module structure was a key to solving this problem The in terface b etween mo dules is kept at a high level evaluation techniques can b e chosen on a p ermo dule basis through op tional annotations and mo dules with dierent evaluation Figure CORAL System Architecture techniques can interact in a nearly transparent fashion An overview of the CORAL declarative language is pre sented in The query language supp orts general Horn ODUS to olkit and thus by CORAL However within each clauses with complex terms setgrouping aggregate op er CORAL pro cess all data that is not managed by the EXO ations negation and data that contains universally quanti DUS storage manager is strictly lo cal to the pro cess Most of ed variables A numb er of user annotations can also b e the eort of design and implementation in CORAL has con sp ecied to guide the system in query evaluation and opti centrated on the singleuser client and the implementation mization The details of the language are b eyond the scop e has fo cused on op eration out of main memory While this of the pap er Many features of the implementation ranging is in keeping with Prologstyle systems there is no part of from lowlevel structures to the interactive system environ the design that is particularly biased towards a mainmemory ment have also b een omitted from the pap er due to shortage 1 system of space The implementation details are found in In Data stored in text les can b e consulted at which p oint Sections we discuss some imp ortant asp ects of the sys the data is converted into mainmemory relations with any tem implementation Section contains an overview of the sp ecied indices indices can also b e created at a later time CORAL system architecture Section explains the under Data stored using the EXODUS storage manager is paged lying representation of the data and Section provides an into EXODUS buers on demand making use of the index overview of query evaluation and optimization Section is ing and scan facilities of the storage manager The design of the main section that deals with implementation issues It the system do es not require that this data b e collected into covers the basic strategies used in evaluating a mo dule as mainmemory CORAL structures b efore b eing used as is well as several imp ortant renements It also addresses user usual in database systems the data can b e accessed purely guidance of query optimization and the interaction in the out of pages in the EXODUS buer p o ol evaluation of dierent mo dules The CORALC inter The query pro cessing system consists of two main parts face and supp ort for extensibility in CORAL including the a query optimizer and a query evaluation system Simple addition of new data typ es and op erations and new rela queries selecting facts from a base relation for instance tion and index implementations are discussed in Sections can b e typ ed in at the user interface and are not opti and We discuss related systems in Section Finally we mized Complex queries are typically dened in declarative provide a retrosp ective discussion of the CORAL design and program mo dules that exp ort predicates views with as outline future research directions in Section so ciated query forms ie sp ecications of what kinds of queries or selections are allowed on the predicate Architecture of the CORAL The query optimizer takes a program mo dule and a query System form as input and generates a rewritten program that is The architecture of the CORAL deductive system is shown optimized for the sp ecied query forms In addition to doing in Figure CORAL is designed primarily as a single user rewriting transformations the optimizer adds several control database system and can b e used in a standalone mo de annotations to those if any sp ecied by the user The Persistent data is stored either in text les or using the rewritten program is stored as a text le which is useful EXODUS storage manager which has a clientserver ar as a debugging aid for the user and is also converted chitecture Each CORAL singleuser pro cess is a client that into an internal representation that is used by the query accesses the common p ersistent data from the server Multi evaluation system ple CORAL pro cesses could interact by accessing p ersistent 1 data stored using the EXODUS storage manager Trans Certain optimizations like structuresharing have however actions and concurrency control are supp orted by the EX b een implemented that optimize mainmemory op eration Page typ es such as integers strings or other abstract datatyp es The query evaluation system takes as input annotated are sub classes of Arg The class Arg denes a set of virtual declarative programs in an internal representation and metho dssuch as equals hash and print