Expert Systems

Prof.dr.sc. Bojana Dalbelo Bašić

Faculty of Electrical Engineering and Computing Department of Electronics, Microelectronics, Computer and Intelligent systems

www.zemris.fer.hr/~bojana [email protected]

Expert systems

. In the early days of AI (50s and 60s of the 20th century), people have tried to develop sophisticated inference techniques

. These techniques did not rely on knowledge. The goal in this period was to develop software systems for general problem solving (GPS)

. The failure to develop a general problem solving system gave rise to a different approach: the developement of knowledge based systems 2. RULE-BASED SYSTEMS

How can one most easily represent human knowledge? . A vast majority of human knowledge (expert knowledge) can be represented using if – then rules

. If the temperature of the patient is greater than 38°C then medications that lower the temperature should be prescribed

. If the traffic light is red then stop

. These rules are called production rules Production systems 2. RULE BASED SYSTEMS

. An important step in successful solving of AI problems → reducing the scope or domain of the problem

Expert General knowledge knowledge

. Expert knowledge (domain knowledge) is knowledge specific for a given domain (e.g., medicine, finances, chess etc.), unlike general problem solving knowledge

Expert systems 3. BASIC SCHEMA OF RULE-BASED SYSTEMS

. Expert systems are a branch of AI that uses knowledge specialized for a certain problem domain to solve a problem at the level of a human expert . Basic schema of a rule-based system – expert system.

Facts Knowledge base

UserUser

Expert Inference engine knowledge

IMPORTANT – separating domain knowledge from Expert system the inference procedure 3. BASIC SCHEMA OF RULE-BASED SYSTEMS

Facts Knowledge base

UserUser

Expert Inference engine knowledge

Inference engine -> Expert system EXPERT SYSTEM SHELL is the expert system without the rules 4. INSPIRATION – THE COGNITION PROCESS

Beginning of production systems:

. Logician E. Post 1943. suggested production rules in formal theoretical computer science. Strength of PRs is equivalent to the Turing machine: production systems can be used for all solvable problems

. 1972. Paper «Human problem Solving», Newell and Simon, 920 pages. Human cognition can be modeled by if then rules. One rule – a chunk of knowledge. 4. INSPIRATION – THE COGNITION PROCESS

. A model of the human cognition process is imitated in the production system

. Senses stimulate the brain with stimuli. Such stimuli activate knowledge in our long term memory (in a prod. system those are if – then rules)

. Short term memory – temporary storage of knowledge for the period of when then inference procedure is executed – represents the number of «chunks of knowledge» that can be considered simultaneously - (rules which can be activated at the same time) 4. INSPIRATION – THE COGNITION PROCESS

. Cognition process – finds rules that will be activated by stimuli – (in a prod. system this is the inference engine, it finds rules and resolves conflicts)

. Newell i Simon – set up the basic model of modern production systems

. 1970 – the paradigm of knowledge based systems is widely accepted 5. PRODUCTION SYSTEMS - NOMENCLATURE

Synonyms: . Rule based systems . Production systems . Expert systems (In a classical sense these are systems that contain expert knowledge, obtained through a series of conversations between the knowledge-base engineer and a domain expert)

Knowledge Expert Human- -base system expert engineer knowledge- base

This procedure is the “bottleneck” in building an expert system 6. Application

. Production systems - an alternative to conventional algorithmic programming

Some areas of application for expert systems: . various kinds of medicial diagnoses (internal medicine, pulmology, infectious diseases etc.) . diagnoses of faults in complex electronic and electromechanical devices . many applications in legal systems (e.g., planning income and assets to minimize taxes) . applications in finances and banking (e.g., evaluation of credit standing for an individual or a company) 6. Application

. An example of application: medicine . Diagnosing and determining therapy based on symptoms

knowledge acquired Knowledge acquired through in medical school experience – heuristic knowledge

doctor inference Expert knowledge – human knowledge: . human – expert . influence of fatigue and emotions on an assesment . duration of the knowledge is limited 7. Building a system

attempt of formalising expert knowledge EXPERT using a computer SYSTEMS

. Specialized (expert) knowledge gathered and organised for specialized, domain limited use

Knowledge Expert Human- -base system expert engineer knowledge- base

. This step can be automated 8. Examples of early ES

Examples of early successful expert systems:

. DENDRAL – discovering molecular structure of substances based on spectrometry

. MYCIN – diagnosing infectuous blood diseases

. PROSPECTOR – estimating existence of ore pased on geographical characteristics

. XCON/R1 – DEC a system for choosing components when building a complex computer system 9. PRODUCTION SYSTEMS

Expert systems – advantages: . built based on the knowledge of more than one expert - expert knowledge repository for a domain . can be easily multiplied an widely used

Rule based systems are based on: . production rules (prodrules) If ……………. then …………………

Condition (state) Action premise, antecedent conclusion,consequent 9. PRODUCTION SYSTEMS

. If it’s raining then take an umbrella condition-action . If car motor doesn’t start and the lights don’t work then the problem is the battery or the cables

. If the patient is in a chronically generally bad state and gender is female and age < 30 and premise - patient has symptom A and conclusion biochemical test shows value C . then the diagnosis is autoimmune chronic hepatitis 9. PRODUCTION SYSTEMS

state- . If John is a student of FER consequence then John is a student of the University in Zagreb

RuleRule based based systems systems are are used used very very oftenoften becausebecause thethe majoritymajority ofof humanhuman knowledgeknowledge cancan bebe representedrepresented usingusing productionproduction rulesrules (if(if –– then).then). 10. PRODUCTION SYSTEMS - LOGIC

. In logic: implication A → B (If A is true then B is true)

Logical formulae

. In production rules: If A then B

actions, commands, equations, programs … . Production systems are a combination of two kinds of knowledge: [1] factual (A is true) A MODUS PONENS [2] conditional (rule exists) A → B (B is true) B 11. Definition of the PRODUCTION SYSTEM

Definition – PRODUCTION SYSTEM is defined by: 1. SET OF PRODUCTION RULES [2] (production is a pair condition - action) 1. WORKING MEMORY (list of facts Knowledge – base containing the description of the current [1] state of the world in the inference procedure) 1. MATCH – ACT CYCLE This is a control mechanism consisting of a sequence of actions: pattern matching – choosing the rule to apply by resolving conflicts – applying the rule. . In a more specific sense the knowledge base can refer only to the set of production rules. Knowledge base is also called the rule base. 12. SCHEMA OF PRODUCTION SYSTEM FUNCTION

. Schema of production system function

2. Set of conflicted PRODUCTION rules RULES 3. Choosing a 1. Matching WORKING rule MEMORY

4. Applying the rule

adding new facts and rules 12. SCHEMA OF PRODUCTION SYSTEM FUNCTION

Description of a production system

1. SET OF PRODUCTION RULES – 1 PR (also called a condition-action pair) -> 1 chunk of knowledge from the problem domain.

1. WORKING MEMORY contains a set of facts . Facts (samples) from WM -> current state in the procedure of solving the problem. 12. SCHEMA OF PRODUCTION SYSTEM FUNCTION

1. MATCH – ACT cycle – is the control structure ofof a production system

Samples from the working memory are compared with the conditional parts of the rules from the rule base. When a conditional part of a rule matches samples from the working memory, then that rules is enabled, and it is put into the set of conflicted rules. At the start of the cycle the set of conflicted rules is empty. 12. SCHEMA OF PRODUCTION SYSTEM FUNCTION

After the matching procedure the set of conflicted rules can contain one or more enabled rules.

If the set of conflicted rules is empty then the procedure is finished.

If there is more than one enabled rule, then ONE rule is chosen and applied (the rule fires).

Choice of the rule that fires is called conflict resolution. Firing of a rule means applying the right hand side of the rule – changing the contents of the working memory.

After a rule has fired the control cycle repeats with the modified contents of the working memory. 12. SCHEMA OF PRODUCTION SYSTEM FUNCTION

. Conflict resolution can be very simple – e.g. the first enabled rule fires, but it can also be complex, i.e. it can include a heuristic to choose which rule fires . Note: When a production rule is applied it can also be said that the rule fires. Example . A simple production system for sorting strings which consist of letters a,b and c SET OF PRODUCTION RULES: 1. ba → ab 2. ca → ac 3. cb → bc Example 1. ba → ab 2. ca → ac 3. cb → bc

Iteration WORKING M Set of The rule that is MEMORY a conflicted chosen to t rules fire 0 cbaca c 3, 1, 2 1 h 1 cabca i 2 2 2 acbca n 3, 2 2 g 3 acbac 1, 3 1 4 acabc 2 2 5 aacbc 3 3 6 aabcc ∅ Stop Example 1. ba → ab 2. ca → ac 3. cb → bc

Iteration WORKING M Set of The rule that is MEMORY a conflicting chosen to t rules fire 0 cbaca c 3, 1, 2 1 h 1 cabca i 2 2 2 acbca n 3, 2 2 g 3 acbac 1, 3 1 4 acabc 2 2 5 aacbc 3 3 6 aabcc ∅ Stop 14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

. A sequence made of several conclusions that link the starting description of the problem with the solution is called a CHAIN

Forward chaining

SOLUTION/ KNOWN GOAL/ DATA CONCLUSION

Backward chaining

. Inference procedure, i.e. automatic advancement along the chain is called chaining. 14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

There are two main ways of advancing towards conclusions: 1. FORWARD CHAINING . → starting with known data and advancing toward a conclusion (also called forchaining, data driven processing, event driven, bottom-up, antecedent, pattern directed processing ⇔ reasoning)

1. BACKWARD CHAINING . → choosing a possible conclusion (hypothesis) and trying to prove that it is valid by finding valid evidence. (also called backchaining, goal driven processing, goal driven, top- down, consequent, expectation driven processing) 14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

. Forward chaining - when there is a small amount of data and a large number of possible solutions (for problem domains including synthesis : for designing, planning, scheduling, supervision and diagnostics of real time systems)

KNOWN GOAL DATA 14. INFERENCE PROCESS IN RULE-BASED SYSTEMS

. Backward chaining a reasonable choice whene there aren’t many possible conclusions and the amount of data known is large (Probleme dijagnosing, classification)

KNOWN GOAL DATA

. The choice of inference method depends on the properties of the problem domain and the way the expert reasons . It is also possible to implement bidirectional inference. 14. INFERENCE PROCESS

. Production rule:

CONDITION → ACTION

LEFT RIGHT HAND SIDE HAND SIDE of the rule of the rule (LHS) (RHS) 15. FORWARD CHAINING

. Forward inference starts with known data, an initial problem description (e.g., a set of logical axioms, disease symptoms, data that needs to be interpreted etc.) which is stored in working memory. . For each rule it is checked if the given data satisfy the premises of the rule. If the rule is satisfied it can be applied, and new data can be derived, which can be used to satisfy other rules. The procedure of checking the rule is called rule interpretation.

Nomenclature: . rule interpretation ≡ forward inference ≡ forward chaining 15. FORWARD CHAINING

INFERENCE ENGINE is a contorol mechanism that performs the following steps when executing the forward chaining procedure.

1. Matching – the control cycle starts by matching the state in working memory with LEFT sides of production rules.

1. Conflict resolution – if during matching more than one enabled rule was found , then the rule with the higher priority is chosen. 15. FORWARD CHAINING

1. Application (firing) of a rule – applying the right hand side of a production rule results in : . a new fact added to working memory (a new current state of the world) or, . A new rule added to the rule base which can be considered in future cycles 15. FORWARD CHAINING

RULE BASE

enabled chosen MATCHING APPLYING CONFLICT RULES rules RESOLUTION rules

WORKING MEMORY 15. FORWARD CHAINING

Example . Rule base for determining type of fruit . Parametres (i.e., variables) and their values are:

Shape: elongated Diameter: > 10 cm circular < 10 cm rounded Fruit_type: vine Surface: smooth tree coarse Fruit: banana Color: green watermelon yellow melon brown-yellow cantaloupe red apple blue apricot orange cherry Seed_number > 1 peach = 1 plum Seed_type multiple orange bony 15. FORWARD CHAINING

RULES

1 IF Shape = elongated & Color = green or yellow THEN Fruit = banana

2 IF Shape= circular or rounded & Diameter > 10 cm THEN Fruit_type = vine

3 IF Shape= circular & Diameter < 10 cm THEN Fruit_type = tree

4 IF Seed_number = 1 THEN Seed_type = bony

5 IF Seed_number > 1 THEN Seed_type = multiple

6 IF Fruit_type = vine & Color = green THEN Fruit = watermelon 15. FORWARD CHAINING

7 IF Fruit_type = vine & Surface = smooth & Color= yellow THEN Fruit = melon 8 IF Fruit_type = vine & Surface = coarse & Color= brown-yellow THEN Fruit = cantaloupe 9 IF Fruit_type = tree & Color= orange & Seed_type = bony THEN Fruit = apricot 10 IF Fruit_type = tree & Color= orange & Seed_type = multiple THEN Fruit = orange 11 IF Fruit_type = tree & Color = red & Seed_type = bony THEN Fruit = cherry 12 IF Fruit_type = tree & Color = orange & Seed_type = bony THEN Fruit = peach 13 IF Fruit_type = tree & Color = yellow or green or red & Seed_type = multiple THEN Fruit = apple 14 IF Fruit_type = tree & Color = blue & Seed_type = bony THEN Fruit = plum 15. FORWARD CHAINING

. Known data: Diameter = 2 cm Shape = circular Seed_number= 1 Color = red Conflict resolution strategy: the rule that is listed first

Working memory Set of conflicting Rule that rules fires 0 Diameter = 2 cm 3,4 3 Shape = circular Seed_number = 1 Color = red 1 Fruit_type: tree 3, 4 4 2 Seed_type = bony 3, 4, 11 11 3 Fruit = cherry 3, 4, 11 STOP 18. BACKWARD CHAINING

Backward chaining . fundamentally different from forward chaining, even though both inference methods match and apply rules . starts with a desired goal (hypothesis) and determines if the existing facts support proving the goal

. The system starts with an empty list of facts. A list of goals is given for which the system tries to determine the values. 18. BACKWARD CHAINING

1. Steps: 1. Form a stack containing the goals (hypotheses) that we want to prove.

2. On top of the stack is the goal which needs to be proven. If the stack is empty STOP

3. Find all rules which can derive the current goal (find rules which have the desired goal on their right hand side) 18. BACKWARD CHAINING

4. For each of such rules do the following: 4.a. If all premises of the rule are satisfied (each premise parameter has the correct value in the working memory) then apply the rule, i.e. The RIGHT hand side of the rule i.e. add the conclusion to the working memory. Do not consider any more rules for that goal – its value has just been derived when the rule fired. If the goal was a top goal then remove the goal from the stack & go back to step 2. If the goal was an intermediate goal then remove the goal from the stack & return to the temporarily suspended goal 18. BACKWARD CHAINING

4.b. If the value of the parameter in working memory is not the same as the one in the premise then do not apply that rule 4.c. If premises of the rule are not satisfied because one of the parameter values of the premise is not in working memory then find a rule whose right hand side derives the value of that parameter If at least one such rule exists then set that parameter as an intermediate goal, i.e. set that parameter on top of the stack & go to step 2 18. BACKWARD CHAINING

4.d. If step(c) cannot find a rule which derives the required value of the parameter then ask the user for the value of the parameter & add the value to working memory. Go to step 4a and consider the next premise of the current rule

5. If all rules that could derive the current goal have been checked and if none of them could derive the goal then goal remains underived. Remove the goal from the stack and go to step 2. 18. BACKWARD CHAINING

Example of backward chaining . Let our starting hypothesis be that we are seeking a fruit.

. hypothesis = (fruit). Knowing that it is a cherry let’s follow the backward chaining procedure to see if it is possible to derive that the fruit is a cherry.

. In the second step we consider ALL rules which derive the value of fruit 18. BACKWARD CHAINING

Goal (stack) WORKING SET OF CONFLICTING Parameter which is MEMORY RULES being checked

(fruit) 1,6,7,8,9,10,11,12, shape is not in WM or 13,14 on the RHS of a rule - ? shape?

(fruit) shape = (R1) 6,7,8,9,10,11, 12,13,14 fruit_type – RHS of rules circular 2 i 3

(fruit_type 2,3 Shape – contained in Is the current goal; WM – OK the goal fruit is Diameter – not in WM or temporarily on the RHS of a rule suspended) ?diameter?

(fruit_type diameter < (R2) 3-FIRES, both premises of R3 are fruit) 10 cm 6,7,8,9,10,11, 2,13,14 in WM (fruit) fruit_type = 6,7,8,9,10,11, 12,13,14 first premise of R6 is not stablo satisfied

(fruit) (R6) 7,8,9,10,11, 12,13,14 first premise of R7 is not satisfied

(fruit) (R7) 8,9,10,11, 12,13,14 first premise of R8 is not satisfied 18. BACKWARD CHAINING

(fruit) (R8) 9,10,11, 12,13,14 fruit_type is in WM; 2nd premise is not in WM and not at RHS of any of the rules ? color ?

(fruit) color = red (R9) 10,11, 12,13,14 Second premise of R10 is not satisfied

(fruit) (R10) 11, 12,13,14

…. … .. … … (fruit) 11 - FIRES fruit = cherry

? … ? – query to the user. All rules removed from the top of the stack for which it is not specified that they fire were not satisfied. Chaining goes backward until until the right hand side of the rule appears on the left hand side of another rule. 19. EXAMPLE OF FORWARD CHAINING

Production rule set:

1. p ∧ q → goal 2. r ∧ s → p 3. w ∧ r → q 4. t ∧ u → q 5. v → s 6. start → v ∧ r ∧ q

. Conflict resolution strategy: choose the rule from the set of conflicted rules which fired first or never fired 19. EXAMPLE OF FORWARD CHAINING

. Execution

Working memory Set of conflicting Rule that rules fires

0 start 6 6 1 start, v, r, q 6,5 5 2 start, v, r, q, s 6,5,2 2 3 start,v, r, q, s ,p 6,5,2,1 1 4 start,v, r, q, s ,p cilj 6,5,2,1 STOP 19. EXAMPLE OF FORWARD CHAINING

n . io The space which is searched ct ire d ch ar Se

t ar st q

goal

s p 19. EXAMPLE OF FORWARD CHAINING

. The space which is searched

start tion Search direc

q r v

p goal 20. EXAMPLE OF BACKWARD CHAINING

Set of production rules

1. p ∧ q → goal 2. r ∧ s → p 3. w ∧ r → q 4. t ∧ u → q 5. v → s 6. start → v ∧ r ∧ q 20. EXAMPLE OF BACKWARD CHAINING

1. p ∧ q → goal 4. t ∧ u → q 2. r ∧ s → p 5. v → s 3. w ∧ r → q 6. Start → v ∧ r ∧ q

GOAL SET OF Parametar currently (STACK) CONFLICTING being checked WORKING MEMORY RULES goal 1 p–not in WM p, goal 2 r–not in WM r, p, goal start 6 – FIRES ?start? p, goal start, v, r, q 2 r is in WM, s is not s, p, goal start, v, r, q 5 - FIRES v is in WM p, goal start, v, r, q, s 2 - FIRES r, s – in WM goal start, v, r, q, s, p 1 – FIRES p, q in WM start, v, r, q, s, p, goal STOP 20. EXAMPLE OF BACKWARD CHAINING

. The space which is searched

goal Search q p direction

s t w r

start

When backward chaining the systems searches for evidence (support) for the goal and intermediate goals. This kind of search is suitable for diagnostics and classification. 21. SUMMARY backward chaining – depth first search forward chaining – breadth first search

. MYCIN (Shortliffe, 1976) – example of backward chaining

. XCON/R1 (MCDermott, DEC, 1978) – example of forward chaining Bibliography and links

Jackson, P., Introduction to Expert Systems – 3rd Ed., Addison-Wesley, 1999. Giarratano, J., Expert Systems: Principals and Programming, PSW-Kent, 1989. Examples of ES and ES shells http://www.zemris.fer.hr/predmeti/is/StudentskiRadovi. html Download Jess: http://herzberg.ca.sandia.gov/jess/index.shtml

CLIPS Expert System Shell http://www.ghg.net/clips/CLIPS.html FuzzyJ Website http://www.iit.nrc.ca/IR_public/fuzzy/fuzzyJToolkit2.html