DES: A Deductive Database System des.sourceforge.net
FernandoFernando SSááenzenz PPéérezrez GrupoGrupo dede ProgramaciProgramacióónn DeclarativaDeclarativa (GPD)(GPD) DeptDept.. IngenierIngenierííaa deldel SoftwareSoftware ee InteligenciaInteligencia ArtificialArtificial UniversidadUniversidad ComplutenseComplutense dede MadridMadrid FacultadFacultad dede InformInformááticatica
PROLE 2010 10/9/2010 1 / 30 ContentsContents
1.1. IntroductionIntroduction 2.2. FeaturesFeatures 3.3. QueryQuery LanguagesLanguages 4.4. OuterOuter JoinsJoins 5.5. AggregatesAggregates 6.6. DESDES asas aa TestTest--BedBed forfor ResearchResearch 7.7. ImpactImpact FactorFactor 8.8. ConclusionsConclusions
PROLE 2010 10/9/2010 2 / 30 1.1. IntroductionIntroduction
Databases:Databases: FromFrom relationalrelational toto deductivedeductive (Declarative)(Declarative) QueryQuery Languages:Languages: FromFrom SQLSQL toto DatalogDatalog
PROLE 2010 10/9/2010 3 / 30 1.1. Introduction:Introduction: DatalogDatalog
AA databasedatabase queryquery languagelanguage stemmingstemming fromfrom PrologProlog
PrologProlog DatalogDatalog PredicatePredicate RelationRelation GoalGoal QueryQuery
MeaningMeaning ofof aa predicatepredicate ((Multi)setMulti)set ofof derivablederivable factsfacts Intensionally (Rules or Clauses) Extensionally (Facts)
PROLE 2010 10/9/2010 4 / 30 1.1. Introduction:Introduction: DatalogDatalog
WhatWhat aa typicaltypical databasedatabase useruser wouldwould expectexpect fromfrom aa queryquery language?language? FiniteFinite data,data, finitefinite computationscomputations (terminating(terminating queries)queries) No terms or bound depth Be aware of built-in infinite relations! AllAll answeranswer tuplestuples atat onceonce Prolog returns serveral answers upon backtracking
PROLE 2010 10/9/2010 5 / 30 1.1. Introduction:Introduction: SystemsSystems
DeductiveDeductive databasedatabase systems:systems: LDL++,LDL++, DLV,DLV, Coral,Coral, XSB,XSB, SDS,SDS, Declare,Declare, ConceptBaseConceptBase,, …… YetYet anotheranother system,system, Why?Why? WeWe neededneeded anan interactiveinteractive systemsystem targetedtargeted atat teachingteaching DatalogDatalog inin classroomsclassrooms So,So, whatwhat aa wholewhole setset ofof featuresfeatures wewe wouldwould askask forfor suchsuch aa system?system?
PROLE 2010 10/9/2010 6 / 30 2.2. FeaturesFeatures 2.1.2.1. RequiredRequired FeaturesFeatures
AA systemsystem orientedoriented atat teachingteaching UserUser--friendly:friendly: InstallationInstallation UsabilityUsability MultiplatformMultiplatform (Windows,(Windows, Linux,Linux, Mac,Mac, ……)) InteractiveInteractive DatabaseDatabase updatesupdates
PROLE 2010 10/9/2010 7 / 30 2.2.2.2. DESDES ConcreteConcrete FeaturesFeatures (1/2)(1/2)
Free, Open-source, Multiplatform, Portable Query languages sharing EDB/IDB: Datalog (Recursive) SQL Database updates: SQL DML Commands Temporary Datalog views Duplicates (v2.0) Declarative debugging of Datalog programs Test case generation for SQL views Datalog and SQL tracers (v2.0)
PROLE 2010 10/9/2010 8 / 30 2.2.2.2. DESDES ConcreteConcrete FeaturesFeatures (2/2)(2/2)
Null value support à la SQL Outer joins for both SQL and Datalog Aggregates Negation Integrity constraints: Domain Referential integrity Full-fledged arithmetics Type system for SQL tables and views Souce-to-source program transformations: Safety Performance (simplifications) Tabling-based implementation
PROLE 2010 10/9/2010 9 / 30 3.3. QueryQuery LanguagesLanguages 3.1.3.1. DatalogDatalog (1/2)(1/2) Program: Set of rules. Rule: head :- body. head. Head: Positive atom. Body: Conjunctions (,) and disjunctions (;) of literals Literal: Atom, negated atom or a built-in. Query: Literal with variables or constants in arguments Body (Conjunctive queries, …) Temporary views
PROLE 2010 10/9/2010 10 / 30
tom grace
3.1.3.1. DatalogDatalog (2/2)(2/2) jack amy tony carolI ExampleExample fred carolII father(tom,amy). father(jack,fred). carolIII father(tony,carolII). father(fred,carolIII). DES-Datalog> ancestor(tom,X) { mother(graceI,amy). ancestor(tom,amy), mother(amy,fred). ancestor(tom,carolIII), mother(carolI,carolII). ancestor(tom,fred) mother(carolII,carolIII). } parent(X,Y) :- father(X,Y). DES-Datalog> father(X,Y),mother(Y,Z) parent(X,Y) :- mother(X,Y). answer(X,Y,Z) :- father(X,Y), ancestor(X,Y) :- mother(Y,Z). parent(X,Y). { ancestor(X,Y) :- answer(tom,amy,fred), parent(X,Z), answer(tony,carolII,carolIII) ancestor(Z,Y). ancestor(Z,Y). }
PROLE 2010 10/9/2010 11 / 30 3.2.3.2. SQLSQL (1/3)(1/3)
DQL: SELECT … FROM … WHERE WITH RECURSIVE … DML: INSERT … UPDATE … DELETE … DDL: CREATE [OR REPLACE] TABLE … CREATE [OR REPLACE] VIEW … DROP …
PROLE 2010 10/9/2010 12 / 30 3.2.3.2. SQLSQL (2/3)(2/3) ExampleExample
CREATE VIEW parent(parent,child) AS SELECT * FROM father UNION SELECT * FROM mother;
CREATE OR REPLACE VIEW ancestor(ancestor,descendant) AS WITH RECURSIVE rec_ancestor(ancestor,descendant) AS SELECT * FROM parent UNION SELECT parent,descendant FROM parent,rec_ancestor WHERE parent.child=rec_ancestor.ancestor SELECT * FROM rec_ancestor;
DES-SQL> SELECT * FROM ancestor WHERE ancestor='tom';
PROLE 2010 10/9/2010 13 / 30 3.2.3.2. SQLSQL (3/3)(3/3) SimplifiedSimplified SyntaxSyntax
CREATE OR REPLACE VIEW ancestor(ancestor, descendant) AS SELECT parent, child FROM parent UNION SELECT parent, descendant FROM parent, ancestor WHERE parent.child=ancestor.ancestor;
PROLE 2010 10/9/2010 14 / 30 3.3.3.3. DatalogDatalog andand SQLSQL
Deductive engine (DE): Tabling implementation Datalog programs are solved by DE Compilation of SQL views and queries to Datalog programs SQL queries are also solved by DE Corollary: SQL and Datalog do share the deductive database! Datalog programs can refer to predicates defined as views in SQL
PROLE 2010 10/9/2010 15 / 30 3.4.3.4. ODBCODBC ConnectionsConnections
New feature in version 2.0, released on August Access to Relational DBMS MySQL MS Access Oracle … SQL statements injected to the DBMS engine Query results are cached by the Datalog engine So, interoperability is allowed!
PROLE 2010 10/9/2010 16 / 30 4.4. OuterOuter JoinsJoins (1/2)(1/2)
NullNull values:values: CteCte.:.: null Functions:Functions: is_null(Var) is_not_null(Var) OuterOuter joinjoin builtbuilt--insins:: Left:Left: lj(Left_Rel,Right_Rel,ON_Condition) Right:Right: rj(Left_Rel,Right_Rel,ON_Condition) Full:Full: fj(Left_Rel,Right_Rel,ON_Condition)
PROLE 2010 10/9/2010 17 / 30 4.4. OuterOuter JoinsJoins (2/2)(2/2)
lj(a(X), b(Y), X=Y) SELECT * FROM a LEFT JOIN b ON x=y; lj(a(X), b(X), true) SELECT * FROM a LEFT JOIN b WHERE x=y; lj(a(X), rj(b(Y), c(U,V), Y=U), X=Y) SELECT * FROM a LEFT JOIN (b RIGHT JOIN c ON y=u) ON x=y;
PROLE 2010 10/9/2010 18 / 30 5.5. AggregatesAggregates (1/5)(1/5)
AggregateAggregate functions:functions: count – COUNT(*) count(Var) – COUNT(Column) min(Var) max(Var) sum(Var) avg(Var) times(Var)
PROLE 2010 10/9/2010 19 / 30 5.5. AggregatesAggregates (2/5)(2/5)
PredicatePredicate group_by/3
group_by( Relation_A, % FROM / WHERE [Var_1,…,Var_n], % GroupingGrouping columnscolumns Relation_B) % HAVING / ProjectionProjection
PROLE 2010 10/9/2010 20 / 30 5.5. AggregatesAggregates (3/5)(3/5) ExampleExample employee Number of employees for each department: Name Department Salary
DES-Datalog> group_by(employee(N,D,S), anderson accounting 1200 [D], R=count) andrews accounting 1200 arlingon accounting 1000 Info: Processing: answer(D,R) :- nolan null null group_by(employee(N,D,S),[D],R = count). norton null null { randall resources 800 answer(accounting,3), answer(null,2), sanders sales null answer(resources,1), silver sales 1000 answer(sales,5) } smith sales 1000 Info: 4 tuples computed. Steel sales 1020
Sullivan sales null
PROLE 2010 10/9/2010 21 / 30 5.5. AggregatesAggregates (3/5)(3/5) ExampleExample ((contdcontd.).) employee
Name Department Salary Active employees (those with assigned salaries): anderson accounting 1200 DES-Datalog> group_by(employee(N,D,S), andrews accounting 1200 [D], R=count(S)) arlingon accounting 1000
nolan null null Info: Processing: answer(D,R) :- norton null null group_by(employee(N,D,S),[D],R = count(S)). randall resources 800 { answer(accounting,3), sanders sales null answer(null,0), silver sales 1000 answer(resources,1), answer(sales,3) smith sales 1000 } Steel sales 1020 Info: 4 tuples computed. Sullivan sales null
PROLE 2010 10/9/2010 22 / 30 5.5. AggregatesAggregates (3/5)(3/5) ExampleExample ((contdcontd.).) employee Active employees of departments with more than one active employee: Name Department Salary anderson accounting 1200
DES-Datalog> group_by(employee(N,D,S), andrews accounting 1200 [D], arlingon accounting 1000 count(S)>1) nolan null null
Info: Processing: norton null null answer(D) :- group_by(employee(N,D,S), randall resources 800 [D], sanders sales null (A = count(S),A > 1)). { silver sales 1000 answer(accounting), smith sales 1000 answer(sales) } Steel sales 1020 Info: 2 tuples computed. Sullivan sales null
PROLE 2010 10/9/2010 23 / 30 5.5. AggregatesAggregates (4/5)(4/5)
AggregateAggregate Predicates:Predicates: count(Rel) – COUNT(*) count(Rel,Var) – COUNT(Column) min(Rel,Var) max(Rel,Var) sum(Rel,Var) avg(Rel,Var) times(Rel,Var)
PROLE 2010 10/9/2010 24 / 30 5.5. AggregatesAggregates (5/5)(5/5) ExampleExample % SQL Program % Datalog Program
CREATE OR REPLACE VIEW path(X,Y,1) :- shortest_paths(Origin,Destination,Length) AS edge(X,Y). WITH RECURSIVE path(X,Y,L) :- path(Origin,Destination,Length) AS path(X,Z,L0), (SELECT edge.*,1 FROM edge) edge(Z,Y), UNION count(edge(A,B),Max), (SELECT L0 % SQL Query % Datalog Query: SELECT * FROM shortest_paths; shortest_paths(X,Y,L) :- min(path(X,Y,Z),Z,L). PROLE 2010 10/9/2010 25 / 30 6.6. DESDES asas aa TestTest--BedBed forfor ResearchResearch TestTest casecase generationgeneration forfor SQLSQL viewsviews DatalogDatalog declarativedeclarative (algorithmic)(algorithmic) debuggingdebugging DatalogDatalog andand SQLSQL tracerstracers NovelNovel proposalproposal forfor outerouter joinsjoins inin DatalogDatalog Theses,Theses, Papers,Papers, AcademiaAcademia…… SeeSee DESDES FactsFacts atat itsits webweb pagepage PROLE 2010 10/9/2010 26 / 30 7.7. ““ImpactImpact FactorFactor”” Up to more than 1,500 downloads a month More than 30,000 downloads since 2004 More than 10,000 entries in Google PROLE 2010 10/9/2010 27 / 30 8.8. ConclusionsConclusions SuccessfulSuccessful implementationimplementation guidedguided byby needneed WidelyWidely used,used, bothboth forfor teachingteaching andand researchresearch NotNot reallyreally novelnovel forfor eacheach featurefeature butbut asas aa wholewhole DatalogDatalog andand SQLSQL integrationintegration Interactive,Interactive, useruser--friendly,friendly, multiplatformmultiplatform systemsystem JustJust downloaddownload itit andand play!play! NeverthelessNevertheless thethe aforementionedaforementioned novelnovel featuresfeatures Still,Still, manymany thingsthings toto dodo…… PROLE 2010 10/9/2010 28 / 30 LimitationsLimitations ((FutureFuture WorkWork)) DataData areare constants,constants, nono termsterms DatalogDatalog databasedatabase updatesupdates SQLSQL coveragecoverage stillstill incompleteincomplete PrecisePrecise syntaxsyntax errorerror reportsreports SingleSingle--lineline inputsinputs ConstraintsConstraints ((àà lala CLP)CLP) PerformancePerformance …… onlyonly toto namename aa few!few! PROLE 2010 10/9/2010 29 / 30 Canada University of California, at Los Angeles Efficient Integrity Checking for Databases with CS240A: Databases and Knowledge Bases Recursive Views Davide Martinenghi and Henning Christiansen The University of Arizona In Advances in Databases and Information Systems: CsC372 9th East European Conference, ADBIS 2005, Tallinn, The State University of New York Estonia, September 12-15, 2005 : Proceedings Autor Johann Eder, Hele-Mai Haav, AhtoDESDESKalja, Jaan FactsFactsUniversity at Buffalo Penjam CSE 636: Data Integration ISBN 3540285857, 9783540285854 The University of British Columbia PhD CS304: Introduction to Relational Databases Computer Science and Engineering Department University of Nebraska - Lincoln, USA Datalog Tutorial PhD Master's of Information Technology in University of Texas at San Antonio, USA Arkansas Tech University, Russellville Industry: The University of Texas at Austin XLOG Technologies GmbH, Zürich CS2 CaseLab : Applied Operations Research Ideacube Australia: Links to DES: INFO2820: Database Systems 1 (Advanced) (2010 - ACM SIGMOD Online Publicly Available Database Semester 1) Software from Nonprofit Organizations Engineering and Information Technologies The ALP Newsletter. vol. 21 n. 1 The University of Sydney Datalog Wikipedia German Tab “Resources” Datalog Wikipedia English INFO2120/2820: Database Systems 1 (2009 - Semester Wapedia 1) SWI-Prolog. Related Web Resources School of Information Technologies SICStus Prolog. Third Party Software. Other Research Systems The University of Sydney SOFTPEDIA. Datalog Educational System 1.7.0 Tutorial 3 Famouswhy Allan Hancock College >> INFO >> 2120 Fall, DBpedia 2009 BDD-Based Deductive DataBase (bddbddb) Description: School of Information Technologies Other implementations of Datalog/Prolog INFO2120/2820: Database Systems I 1.Sem./2009 Reach Information Tutorial 3: SQL and Relational Algebra 23.03.2009 Ask a Word Acronym finder Africa: Acronym Geek PROLE 2010 10/9/2010 Faculty of Sciences and Technologies of 30 / 30 . Mohammedia (FSTM) - Morocco Temporary Views Consultas Tabla de extensión PROLE 2010 10/9/2010 31 DatalogDatalog DeclarativeDeclarative DebuggingDebugging Motivation:Motivation: AbstractAbstract thethe solvingsolving--orientedoriented debuggingdebugging procedureprocedure Roots:Roots: [Shaphiro83],[Shaphiro83], AlgorithmicAlgorithmic ProgramProgram DebuggingDebugging SemanticsSemantics--orientedoriented PROLE 2010 10/9/2010 32 DeclarativeDeclarative DebuggerDebugger between(X,Z):- br(X),br(Y),br(Z),X PROLE 2010 10/9/2010 33 DeclarativeDeclarative DebuggerDebugger between(X,Z):- br(X),br(Y),br(Z),X PROLE 2010 10/9/2010 34 DeclarativeDeclarative debugging:debugging: aa practicalpractical sessionsession DES> /debug br_is_even Debugger started ... Is br(b) = {br(b)} valid(v)/non-valid(n) [v]? v Is has_preceding(b) = {} valid(v)/non-valid(n) [v]? n Is br(X) = {br(b),br(a)} valid(v)/non-valid(n) [v]? v ! Error in relation: has_preceding/1 ! Witness query: has_preceding(b) = { } PROLE 2010 10/9/2010 35 DeclarativeDeclarative Debugging:Debugging: SemanticSemantic GraphGraph br_is_even = { } next(nil, Y) = {next(nil,b)} even(X) = {even(nil)} br(nil) = { } odd(X) = {odd(b)} Non-valid br(Y) = {br(a), br(b)} next(b, X) = { } Unknown has_preceding(a)= {has_preceding(a)} br(b) = {br(b)} br(a)= {br(a)} has_preceding(b)= {} PROLE 2010 10/9/2010 36 InstallingInstalling DESDES DistroDistro underunder GPLGPL inin SourceforgeSourceforge:: SourcesSources PortablePortable ExecutablesExecutables (Windows,(Windows, Linux)Linux) PortablePortable BundleBundle includingincluding JavaJava IDEIDE (Windows)(Windows) StartingStarting thethe system.system. Either:Either: FromFrom aa PrologProlog interpreterinterpreter (Ciao,(Ciao, GNU,GNU, SicstusSicstus,, SWI)SWI) SimplySimply executeexecute thethe binarybinary StartStart thethe JavaJava applicationapplication PROLE 2010 10/9/2010 37 DESDES runningrunning asas aa WindowsWindows applicationapplication PROLE 2010 10/9/2010 38 DESDES runningrunning inin aa LinuxLinux terminalterminal PROLE 2010 10/9/2010 39 DESDES runningrunning underunder ACIDEACIDE PROLE 2010 10/9/2010 40 DESDES runningrunning underunder EmacsEmacs PROLE 2010 10/9/2010 41 ImplementationImplementation DESDES commandcommand--lineline interpreter:interpreter: PrologProlog Tabling (Bottom-up Top-down driven) Computation by strata saturations (negation and aggregates) DatalogDatalog Debugger:Debugger: PrologProlog ++ JavaJava [CGS07] R. Caballero, Y. García-Ruiz, and F. Sáenz-Pérez, A new proposal for debugging datalog programs. WFLP’07 TestTest CaseCase Generator:Generator: PrologProlog ++ FDFD constraintsconstraints [CGS10a] R. Caballero, Y. García-Ruiz, and F. Sáenz-Pérez, Applying Constraint Logic Programming to SQL Test Case Generation, FLOPS 2010 ACIDE:ACIDE: JavaJava A Configurable IDE (LaTeX, SQL, Prolog, Datalog, …) PROLE 2010 10/9/2010 42