DES: A Deductive System des.sourceforge.net

FernandoFernando SSááenzenz PPéérezrez GrupoGrupo dede ProgramaciProgramacióónn DeclarativaDeclarativa (GPD)(GPD) DeptDept.. IngenierIngenierííaa deldel SoftwareSoftware ee InteligenciaInteligencia ArtificialArtificial UniversidadUniversidad ComplutenseComplutense dede MadridMadrid FacultadFacultad dede InformInformááticatica

PROLE 2010 10/9/2010 1 / 30 ContentsContents

1.1. IntroductionIntroduction 2.2. FeaturesFeatures 3.3. QueryQuery LanguagesLanguages 4.4. OuterOuter JoinsJoins 5.5. AggregatesAggregates 6.6. DESDES asas aa TestTest--BedBed forfor ResearchResearch 7.7. ImpactImpact FactorFactor 8.8. ConclusionsConclusions

PROLE 2010 10/9/2010 2 / 30 1.1. IntroductionIntroduction

„ :Databases: FromFrom relationalrelational toto deductivedeductive „ (Declarative)(Declarative) QueryQuery Languages:Languages: FromFrom SQLSQL toto DatalogDatalog

PROLE 2010 10/9/2010 3 / 30 1.1. Introduction:Introduction: DatalogDatalog

„ AA databasedatabase queryquery languagelanguage stemmingstemming fromfrom PrologProlog

PrologProlog DatalogDatalog PredicatePredicate RelationRelation GoalGoal QueryQuery

„ MeaningMeaning ofof aa predicatepredicate ((Multi)setMulti)set ofof derivablederivable factsfacts „ Intensionally (Rules or Clauses) „ Extensionally (Facts)

PROLE 2010 10/9/2010 4 / 30 1.1. Introduction:Introduction: DatalogDatalog

„ WhatWhat aa typicaltypical databasedatabase useruser wouldwould expectexpect fromfrom aa queryquery language?language? „ FiniteFinite data,data, finitefinite computationscomputations (terminating(terminating queries)queries) „ No terms or bound depth „ Be aware of built-in infinite relations! „ AllAll answeranswer tuplestuples atat onceonce „ returns serveral answers upon backtracking

PROLE 2010 10/9/2010 5 / 30 1.1. Introduction:Introduction: SystemsSystems

„ DeductiveDeductive databasedatabase systems:systems: LDL++,LDL++, DLV,DLV, Coral,Coral, XSB,XSB, SDS,SDS, Declare,Declare, ConceptBaseConceptBase,, …… „ YetYet anotheranother system,system, Why?Why? „ WeWe neededneeded anan interactiveinteractive systemsystem targetedtargeted atat teachingteaching DatalogDatalog inin classroomsclassrooms „ So,So, whatwhat aa wholewhole setset ofof featuresfeatures wewe wouldwould askask forfor suchsuch aa system?system?

PROLE 2010 10/9/2010 6 / 30 2.2. FeaturesFeatures 2.1.2.1. RequiredRequired FeaturesFeatures

„ AA systemsystem orientedoriented atat teachingteaching „ UserUser--friendly:friendly: „ InstallationInstallation „ UsabilityUsability „ MultiplatformMultiplatform (Windows,(Windows, Linux,Linux, Mac,Mac, ……)) „ InteractiveInteractive „ DatabaseDatabase updatesupdates

PROLE 2010 10/9/2010 7 / 30 2.2.2.2. DESDES ConcreteConcrete FeaturesFeatures (1/2)(1/2)

„ Free, Open-source, Multiplatform, Portable „ Query languages sharing EDB/IDB: „ „ (Recursive) SQL „ Database updates: „ SQL DML „ Commands „ Temporary Datalog views „ Duplicates (v2.0) „ Declarative debugging of Datalog programs „ Test case generation for SQL views „ Datalog and SQL tracers (v2.0)

PROLE 2010 10/9/2010 8 / 30 2.2.2.2. DESDES ConcreteConcrete FeaturesFeatures (2/2)(2/2)

„ Null value support à la SQL „ Outer joins for both SQL and Datalog „ Aggregates „ Negation „ Integrity constraints: „ Domain „ Referential integrity „ Full-fledged arithmetics „ Type system for SQL tables and views „ Souce-to-source program transformations: „ Safety „ Performance (simplifications) „ Tabling-based implementation

PROLE 2010 10/9/2010 9 / 30 3.3. QueryQuery LanguagesLanguages 3.1.3.1. DatalogDatalog (1/2)(1/2) „ Program: Set of rules. „ Rule: „ head :- body. „ head. „ Head: Positive atom. „ Body: Conjunctions (,) and disjunctions (;) of literals „ Literal: Atom, negated atom or a built-in. „ Query: „ Literal with variables or constants in arguments „ Body (Conjunctive queries, …) „ Temporary views

PROLE 2010 10/9/2010 10 / 30

tom grace

3.1.3.1. DatalogDatalog (2/2)(2/2) jack amy tony carolI ExampleExample fred carolII father(tom,amy). father(jack,fred). carolIII father(tony,carolII). father(fred,carolIII). DES-Datalog> ancestor(tom,X) { mother(graceI,amy). ancestor(tom,amy), mother(amy,fred). ancestor(tom,carolIII), mother(carolI,carolII). ancestor(tom,fred) mother(carolII,carolIII). } parent(X,Y) :- father(X,Y). DES-Datalog> father(X,Y),mother(Y,Z) parent(X,Y) :- mother(X,Y). answer(X,Y,Z) :- father(X,Y), ancestor(X,Y) :- mother(Y,Z). parent(X,Y). { ancestor(X,Y) :- answer(tom,amy,fred), parent(X,Z), answer(tony,carolII,carolIII) ancestor(Z,Y). ancestor(Z,Y). }

PROLE 2010 10/9/2010 11 / 30 3.2.3.2. SQLSQL (1/3)(1/3)

„ DQL: „ SELECT … FROM … WHERE „ WITH RECURSIVE … „ DML: „ INSERT … „ UPDATE … „ DELETE … „ DDL: „ CREATE [OR REPLACE] TABLE … „ CREATE [OR REPLACE] VIEW … „ DROP …

PROLE 2010 10/9/2010 12 / 30 3.2.3.2. SQLSQL (2/3)(2/3) ExampleExample

CREATE VIEW parent(parent,child) AS SELECT * FROM father UNION SELECT * FROM mother;

CREATE OR REPLACE VIEW ancestor(ancestor,descendant) AS WITH RECURSIVE rec_ancestor(ancestor,descendant) AS SELECT * FROM parent UNION SELECT parent,descendant FROM parent,rec_ancestor WHERE parent.child=rec_ancestor.ancestor SELECT * FROM rec_ancestor;

DES-SQL> SELECT * FROM ancestor WHERE ancestor='tom';

PROLE 2010 10/9/2010 13 / 30 3.2.3.2. SQLSQL (3/3)(3/3) SimplifiedSimplified SyntaxSyntax

CREATE OR REPLACE VIEW ancestor(ancestor, descendant) AS SELECT parent, child FROM parent UNION SELECT parent, descendant FROM parent, ancestor WHERE parent.child=ancestor.ancestor;

PROLE 2010 10/9/2010 14 / 30 3.3.3.3. DatalogDatalog andand SQLSQL

„ Deductive engine (DE): „ Tabling implementation „ Datalog programs are solved by DE „ Compilation of SQL views and queries to Datalog programs „ SQL queries are also solved by DE „ Corollary: SQL and Datalog do share the deductive database! „ Datalog programs can refer to predicates defined as views in SQL

PROLE 2010 10/9/2010 15 / 30 3.4.3.4. ODBCODBC ConnectionsConnections

„ New feature in version 2.0, released on August „ Access to Relational DBMS „ MySQL „ MS Access „ Oracle „ … „ SQL statements injected to the DBMS engine „ Query results are cached by the Datalog engine „ So, interoperability is allowed!

PROLE 2010 10/9/2010 16 / 30 4.4. OuterOuter JoinsJoins (1/2)(1/2)

„„NullNull values:values: „„CteCte.:.: null „„Functions:Functions: is_null(Var) „ is_not_null(Var) „„OuterOuter joinjoin builtbuilt--insins:: „„Left:Left: lj(Left_Rel,Right_Rel,ON_Condition) „„Right:Right: rj(Left_Rel,Right_Rel,ON_Condition) „„Full:Full: fj(Left_Rel,Right_Rel,ON_Condition)

PROLE 2010 10/9/2010 17 / 30 4.4. OuterOuter JoinsJoins (2/2)(2/2)

lj(a(X), b(Y), X=Y) SELECT * FROM a LEFT JOIN b ON x=y; lj(a(X), b(X), true) SELECT * FROM a LEFT JOIN b WHERE x=y; lj(a(X), rj(b(Y), c(U,V), Y=U), X=Y) SELECT * FROM a LEFT JOIN (b RIGHT JOIN c ON y=u) ON x=y;

PROLE 2010 10/9/2010 18 / 30 5.5. AggregatesAggregates (1/5)(1/5)

„ AggregateAggregate functions:functions: „ count – COUNT(*) „ count(Var) – COUNT(Column) „ min(Var) „ max(Var) „ sum(Var) „ avg(Var) „ times(Var)

PROLE 2010 10/9/2010 19 / 30 5.5. AggregatesAggregates (2/5)(2/5)

„ PredicatePredicate group_by/3

group_by( Relation_A, % FROM / WHERE [Var_1,…,Var_n], % GroupingGrouping columnscolumns Relation_B) % HAVING / ProjectionProjection

PROLE 2010 10/9/2010 20 / 30 5.5. AggregatesAggregates (3/5)(3/5) ExampleExample employee „ Number of employees for each department: Name Department Salary

DES-Datalog> group_by(employee(N,D,S), anderson accounting 1200 [D], R=count) andrews accounting 1200 arlingon accounting 1000 Info: Processing: answer(D,R) :- nolan null null group_by(employee(N,D,S),[D],R = count). norton null null { randall resources 800 answer(accounting,3), answer(null,2), sanders sales null answer(resources,1), silver sales 1000 answer(sales,5) } smith sales 1000 Info: 4 tuples computed. Steel sales 1020

Sullivan sales null

PROLE 2010 10/9/2010 21 / 30 5.5. AggregatesAggregates (3/5)(3/5) ExampleExample ((contdcontd.).) employee

Name Department Salary „ Active employees (those with assigned salaries): anderson accounting 1200 DES-Datalog> group_by(employee(N,D,S), andrews accounting 1200 [D], R=count(S)) arlingon accounting 1000

nolan null null Info: Processing: answer(D,R) :- norton null null group_by(employee(N,D,S),[D],R = count(S)). randall resources 800 { answer(accounting,3), sanders sales null answer(null,0), silver sales 1000 answer(resources,1), answer(sales,3) smith sales 1000 } Steel sales 1020 Info: 4 tuples computed. Sullivan sales null

PROLE 2010 10/9/2010 22 / 30 5.5. AggregatesAggregates (3/5)(3/5) ExampleExample ((contdcontd.).) employee „ Active employees of departments with more than one active employee: Name Department Salary anderson accounting 1200

DES-Datalog> group_by(employee(N,D,S), andrews accounting 1200 [D], arlingon accounting 1000 count(S)>1) nolan null null

Info: Processing: norton null null answer(D) :- group_by(employee(N,D,S), randall resources 800 [D], sanders sales null (A = count(S),A > 1)). { silver sales 1000 answer(accounting), smith sales 1000 answer(sales) } Steel sales 1020 Info: 2 tuples computed. Sullivan sales null

PROLE 2010 10/9/2010 23 / 30 5.5. AggregatesAggregates (4/5)(4/5)

„ AggregateAggregate Predicates:Predicates: „ count(Rel) – COUNT(*) „ count(Rel,Var) – COUNT(Column) „ min(Rel,Var) „ max(Rel,Var) „ sum(Rel,Var) „ avg(Rel,Var) „ times(Rel,Var)

PROLE 2010 10/9/2010 24 / 30 5.5. AggregatesAggregates (5/5)(5/5) ExampleExample % SQL Program % Datalog Program

CREATE OR REPLACE VIEW path(X,Y,1) :- shortest_paths(Origin,Destination,Length) AS edge(X,Y). WITH RECURSIVE path(X,Y,L) :- path(Origin,Destination,Length) AS path(X,Z,L0), (SELECT edge.*,1 FROM edge) edge(Z,Y), UNION count(edge(A,B),Max), (SELECT L0

% SQL Query % Datalog Query: SELECT * FROM shortest_paths; shortest_paths(X,Y,L) :- min(path(X,Y,Z),Z,L).

PROLE 2010 10/9/2010 25 / 30 6.6. DESDES asas aa TestTest--BedBed forfor ResearchResearch

„ TestTest casecase generationgeneration forfor SQLSQL viewsviews „ DatalogDatalog declarativedeclarative (algorithmic)(algorithmic) debuggingdebugging „ DatalogDatalog andand SQLSQL tracerstracers „ NovelNovel proposalproposal forfor outerouter joinsjoins inin DatalogDatalog „ Theses,Theses, Papers,Papers, AcademiaAcademia…… SeeSee DESDES FactsFacts atat itsits webweb pagepage

PROLE 2010 10/9/2010 26 / 30 7.7. ““ImpactImpact FactorFactor””

Up to more than 1,500 downloads a month More than 30,000 downloads since 2004 More than 10,000 entries in Google PROLE 2010 10/9/2010 27 / 30 8.8. ConclusionsConclusions

„ SuccessfulSuccessful implementationimplementation guidedguided byby needneed „ WidelyWidely used,used, bothboth forfor teachingteaching andand researchresearch „ NotNot reallyreally novelnovel forfor eacheach featurefeature butbut asas aa wholewhole „ DatalogDatalog andand SQLSQL integrationintegration „ Interactive,Interactive, useruser--friendly,friendly, multiplatformmultiplatform systemsystem „ JustJust downloaddownload itit andand play!play! „ NeverthelessNevertheless thethe aforementionedaforementioned novelnovel featuresfeatures „ Still,Still, manymany thingsthings toto dodo……

PROLE 2010 10/9/2010 28 / 30 LimitationsLimitations ((FutureFuture WorkWork))

„ DataData areare constants,constants, nono termsterms „ DatalogDatalog databasedatabase updatesupdates „ SQLSQL coveragecoverage stillstill incompleteincomplete „ PrecisePrecise syntaxsyntax errorerror reportsreports „ SingleSingle--lineline inputsinputs „ ConstraintsConstraints ((àà lala CLP)CLP) „ PerformancePerformance „ …… onlyonly toto namename aa few!few!

PROLE 2010 10/9/2010 29 / 30 Canada „ University of California, at Los Angeles „ Efficient Integrity Checking for Databases with CS240A: Databases and Knowledge Bases Recursive Views Davide Martinenghi and Henning Christiansen „ The University of Arizona In Advances in Databases and Information Systems: CsC372 9th East European Conference, ADBIS 2005, Tallinn, „ The State University of New York Estonia, September 12-15, 2005 : Proceedings Autor Johann Eder, Hele-Mai Haav, AhtoDESDESKalja, Jaan FactsFactsUniversity at Buffalo Penjam CSE 636: ISBN 3540285857, 9783540285854 „ The University of British Columbia „ PhD CS304: Introduction to Relational Databases Computer Science and Engineering Department University of Nebraska - Lincoln, USA Datalog Tutorial „ PhD „ Master's of Information Technology in University of Texas at San Antonio, USA Arkansas Tech University, Russellville Industry: „ The University of Texas at Austin „ XLOG Technologies GmbH, Zürich CS2 „ CaseLab : Applied Operations Research „ Ideacube Australia: Links to DES: „ INFO2820: Database Systems 1 (Advanced) (2010 - „ ACM SIGMOD Online Publicly Available Database Semester 1) Software from Nonprofit Organizations Engineering and Information Technologies „ The ALP Newsletter. vol. 21 n. 1 The University of Sydney „ Datalog Wikipedia German Tab “Resources” „ Datalog Wikipedia English „ INFO2120/2820: Database Systems 1 (2009 - Semester „ Wapedia 1) „ SWI-Prolog. Related Web Resources School of Information Technologies „ SICStus Prolog. Third Party Software. Other Research Systems The University of Sydney „ SOFTPEDIA. Datalog Educational System 1.7.0 Tutorial 3 „ Famouswhy „ Allan Hancock College >> INFO >> 2120 Fall, „ DBpedia 2009 „ BDD-Based Deductive DataBase (bddbddb) Description: School of Information Technologies Other implementations of Datalog/Prolog INFO2120/2820: Database Systems I 1.Sem./2009 „ Reach Information Tutorial 3: SQL and Relational Algebra 23.03.2009 „ Ask a Word „ Acronym finder Africa: „ Acronym Geek PROLE 2010 10/9/2010„ Faculty of Sciences and Technologies of 30 / 30 „ . Mohammedia (FSTM) - Morocco Temporary Views

Consultas

Tabla de extensión

PROLE 2010 10/9/2010 31 DatalogDatalog DeclarativeDeclarative DebuggingDebugging

„ Motivation:Motivation: „ AbstractAbstract thethe solvingsolving--orientedoriented debuggingdebugging procedureprocedure „ Roots:Roots: „ [Shaphiro83],[Shaphiro83], AlgorithmicAlgorithmic ProgramProgram DebuggingDebugging „ SemanticsSemantics--orientedoriented

PROLE 2010 10/9/2010 32 DeclarativeDeclarative DebuggerDebugger between(X,Z):- br(X),br(Y),br(Z),X Y. Elements having preceding values in the sequence even(nil). even(X) :- odd(Z), next(Z,X). Elements in an even position+nil odd(Y) :- even(Z), next(Z,Y). Elements in an odd position br_is_even :- even(X), not(next(X,Y)). Succeeds if the cardinality is even br(a). Base relation (sequence of elements) br(b).

PROLE 2010 10/9/2010 33 DeclarativeDeclarative DebuggerDebugger between(X,Z):- br(X),br(Y),br(Z),X

PROLE 2010 10/9/2010 34 DeclarativeDeclarative debugging:debugging: aa practicalpractical sessionsession

DES> /debug br_is_even

Debugger started ... Is br(b) = {br(b)} valid(v)/non-valid(n) [v]? v Is has_preceding(b) = {} valid(v)/non-valid(n) [v]? n Is br(X) = {br(b),br(a)} valid(v)/non-valid(n) [v]? v ! Error in relation: has_preceding/1 ! Witness query: has_preceding(b) = { }

PROLE 2010 10/9/2010 35 DeclarativeDeclarative Debugging:Debugging: SemanticSemantic GraphGraph br_is_even = { }

next(nil, Y) = {next(nil,b)} even(X) = {even(nil)}

br(nil) = { } odd(X) = {odd(b)} Non-valid

br(Y) = {br(a), br(b)} next(b, X) = { } Unknown

has_preceding(a)= {has_preceding(a)} br(b) = {br(b)}

br(a)= {br(a)} has_preceding(b)= {}

PROLE 2010 10/9/2010 36 InstallingInstalling DESDES

„ DistroDistro underunder GPLGPL inin SourceforgeSourceforge:: „ SourcesSources „ PortablePortable ExecutablesExecutables (Windows,(Windows, Linux)Linux) „ PortablePortable BundleBundle includingincluding JavaJava IDEIDE (Windows)(Windows) „ StartingStarting thethe system.system. Either:Either: „ FromFrom aa PrologProlog interpreterinterpreter (Ciao,(Ciao, GNU,GNU, SicstusSicstus,, SWI)SWI) „ SimplySimply executeexecute thethe binarybinary „ StartStart thethe JavaJava applicationapplication

PROLE 2010 10/9/2010 37 DESDES runningrunning asas aa WindowsWindows applicationapplication

PROLE 2010 10/9/2010 38 DESDES runningrunning inin aa LinuxLinux terminalterminal

PROLE 2010 10/9/2010 39 DESDES runningrunning underunder ACIDEACIDE

PROLE 2010 10/9/2010 40 DESDES runningrunning underunder EmacsEmacs

PROLE 2010 10/9/2010 41 ImplementationImplementation

„ DESDES commandcommand--lineline interpreter:interpreter: PrologProlog „ Tabling (Bottom-up Top-down driven) „ Computation by strata saturations (negation and aggregates) „ DatalogDatalog Debugger:Debugger: PrologProlog ++ JavaJava „ [CGS07] R. Caballero, Y. García-Ruiz, and F. Sáenz-Pérez, A new proposal for debugging datalog programs. WFLP’07 „ TestTest CaseCase Generator:Generator: PrologProlog ++ FDFD constraintsconstraints „ [CGS10a] R. Caballero, Y. García-Ruiz, and F. Sáenz-Pérez, Applying Constraint Logic Programming to SQL Test Case Generation, FLOPS 2010 „ ACIDE:ACIDE: JavaJava „ A Configurable IDE (LaTeX, SQL, Prolog, Datalog, …)

PROLE 2010 10/9/2010 42