Inference Methods

(1)

1

Methods of Inference

₁

• Introduction

• More knowledge representation methods

• Methods of inference

– Deductive logic and syllogisms

– Rules of inference (propositional logic)

– Resolution

– Shallow and deep reasoning

(2)

2

Methods of Inference

₂

• Other methods

– Induction

– Analogy

– Generate and test

– Abduction

– non-monotonic reasoning

– TMS

– Intuition

– Heuristics

– Default

– Autoepistemic

• Meta-knowledge

(3)

3

Introduction

₁

• When a problem has no adequate algorithm or

no algorithmic solution exists, expert systems

offer a possibility of solution

• Expert system provides solution by reasoning (or

inference) on knowledge

• Some classical data structures suitable for

representing knowledge in expert system is

reviewed

• Commonly used inference methods in expert

system are introduced

(4)

4

More knowledge representation

methods

₁

• Trees

– decision tree

• Graphs

– state space

– and-or

• Lattices

– and-or

(5)

5

Tree

• Every node except the root has exactly

one parent

_{Root Node}

Level1 Level2 Level3 Level4 Branch Node Leaf

(6)

6

Decision Tree

₁

Is it very big?

Does it squeak?

No _Yes

Guess squirrel Guess mouse

No

Yes

Does it have a long neck?

No No No Yes Yes Yes

Does it have a trunk? Guess giraffe

Does it like to be in water? Guess elephant

Guess rhino Guess hippo

(7)

7

Translated into Rules

• If Question = “Is it very big?”

and Respond = No”

Then Question := “Does it squeak?”

• If “it is very big” and

“it has long neck”

(8)

8

Decision Tree

₂

• Classifying objects

• Self-learning

– if the answer is wrong, leaves can be

dynamically created and added to the tree

– an automated knowledge acquisition tool

• However, can not deal with variables, as

an expert system can

(9)

9

Decision Tree

₃

• How to build a DT?

– From expert

– From data; Machine learning, e.g. ID3

• How to represent a DT?

• How to search for a solution?

(10)

10

(11)

11

Graph

• Connected Graph

• Non-connected Graph

• Cycle or Circuit

• Digraph

• Acyclic Graph

• Lattice

(12)

12

State Space

₁

• A state

– is a collection of characteristics that can be

used to define the status or state of an object

• State space

– the set of states showing the transitions

between states that the object can experience

– a transition takes an object from one state to

(13)

13

State Space

₂

• Example: Determine valid string

WHILE, WRITE, and BEGIN

Start W B H I L R I T E E G I not G not I not N Error not E Success N

(14)

14

State Space

₃

for monkey &

bananas

problem

• Monkey, banana, couch,

floor, ladder,

• Give instructions to money

how to retrieve bananas

• Find a valid path from

starting state to success

state

Monkey On Couch

Monkey on Floor

Jump off Couch Observe couch Under bananas Couch Under Bananas Couch not under Bananas Observe couch

not under bananas

Monkey at Ladder Ladder under Bananas Monkey on Ladder Monkey not at Ladder Ladder not under Bananas Success Monkey has Bananas Move couch Observe monkey Not at ladder Observe monkey at ladder Observe ladder under bananas Climb ladder Grab Bananas Move monkey Observe ladder

not under bananas

(15)

15

State Space

₄

• Example: Traveling Salesman

• Find a complete path from

node A that visits all other

nodes (ABDCA, ACDBA)

A

B C D A B C A _C D A B D B C A B D B C A _B D A C D B C A C D B C

(16)

16

State Space

₅

(17)

17

State Space

₆

• A Vacuum-cleaner world

– Problem types:

• Deterministic, but fully observable =>

single-state

problem

• Non-observable =>

conformant problem

• Nondeterministic and/or partially observable =>

contingency problem

• Unknown state space =>

exploration problem

(18)

18

(19)

19

(20)

20

(21)

21

State Space

₁₀

• Searching

– Breadth-first search

– Depth-first search

– Uniform-cost search

– Depth-limited search

(22)

22

State Space

₁₁

(23)

23

State Space

₁₂

• Uniform-cost search

– Expand least-cost unexpanded node

– Implementation

• Fringe = queue ordered by path cost

– Equivalent to breadth-first if step costs all

equal

(24)

24

State Space

₁₃

(25)

25

State Space

₁₄

• Depth-limited search

– Depth-first with depth limit l,

(26)

26

State Space

₁₅

(27)

27

State Space

₁₆

(28)

28

AND-OR Graph

• Example: Obtain a

collage degree

• Useful in representing

backward chaining

problem

GOAL GET A COLLEGE DEGREE MAIL COMPUTER AND MODEM IN PERSON SATISFY REQUIREMENTS APPLY FOR ADMITTANCE TAKE COURSES APPLY TO GRADUATE ENROLL IN COURSES PASS COURSES

(29)

29

AND-OR Graph

• Example: Get to

work

Goal Get to Work

Car Walk Train

Car in good condition Enough gas For Trip Legs are not broken Walk to Train Station Drive to Train Station Someone else repaired defects Repaired defects myself Bring gas to car Bring car to gas Container for gas

(30)

30

AND-OR Lattice

₁

• Lattice: graph with partial order

4, 6

3, 5

2

(31)

31

AND-OR Lattice

₂

• Example: sell/repair decision

Sell the Car Repair the car Bad transmission Bad carburetor Can’t go into reverse Trouble changing gears Engine stalls A lot Poor acceleration Exclusive OR AND

(32)

32

Search and Backtracking

• Example: Traveling Salesman

• ABA unsuccessfully, backtracked to B. • From B, CA,CB,CDB,CDC unsuccessfully searched • Backtracked to B A B C D A B C A _C D A B D B C A B D B C A _B D A C D B C A C D B C

(33)

33

Methods of inference

• Deductive logic and syllogisms

• Rules of inference (propositional logic)

• Resolution

• Shallow and deep reasoning

(34)

34

Deduction

₁

• Deduction

– Logical reasoning in which conclusions must

follow from their premises

• Premises

→ Conclusion

• Deductive logic is one of the most frequently

used methods for drawing inferences

(35)

35

Deduction

₂

• A syllogism

is any valid deductive argument

having 2 premises and a conclusion

• Example: (classical syllogism)

Premise:

Anyone who can program is intelligent

Premise:

John can program

(36)

36

Deduction

₃

• Syllogism has only 4 categories of

statements

• A: All S is P

• E: No S is P

• I: Some S is P

(37)

37

Rules of Inference

₁

• Wff

– True, false, propositions, any logical combination of T,

F, and propositions

• Inference rule:

– Map premise to conclusion

– Premise: one or more well formed formulas (wff)

– Conclusion: a single wff

(38)

38

Rules of Inference

₂

• expresses knowledge in a particular

mathematical notation

(e.g. logic)

– All birds have wings i.e. – ∀ x, Bird(x) -> HasWings(x) - All robins are birds i.e. - ∀ x, Robin(x) -> Bird(x)

• rules of inference

– guarantee that, given true facts or premises, the new

facts or premises derived by applying the rules are

also true

- Chain rule

• given these two facts, application of an inference

rule gives:

(39)

39

Rules of Inference

₃

• Summary of logic languages

– propositional logic

• facts

• true/false/unknown

– first-order logic

• facts, objects, relations

• true/false/unknown

– temporal logic

• facts, objects, relations, times

• true/false/unknown

(40)

40

Rules of Inference

₃

• Summary of logic languages (con’t)

– probability theory

• facts

• degree of belief [0..1]

– fuzzy logic

• degree of truth

• degree of belief [0..1]

(41)

41

Rules of Inference

₄

Law of Inference Schemata 1. Law of Detachment (modus ponens; MP) p→q

P ∴q 2. Law of the Contrapositive p→q

∴ ∼q_{→ ∼p} 3. Law of Modus Tollent p→q

∼ q ∴ ∼ p 4. Chain Rule

(Law of the Syllogism)

p→q q→r ∴ p_→r

(42)

42

Rules of Inference

₅

Law of Inference Schemata

5. Law of the Disjunctive Inference pˇq pˇq ∼ p ∼ q ∴ q ∴ p 6. Law of the Double Negation ∼(∼ p)

∴ p

7. De Morgan’s Law ∼(p q) ∼(pˇq)

∴ ∼pˇ∼ q ∴ ∼p ∼ q 8. Law of Simplification p q ∼(p ˇ q)

(43)

43

Rules of Inference

₆

Low of Inference

Schemata

9. Law of Conjunction

p

q

∴

p q

10. Law of the Disjunctive

Addition

P

∴

p

ˇ

q

11. Law of Conjunctive Argument

∼(p q) ∼(p q)

P q

(44)

44

Example

₁

• Chip prices rise only if the yen rises.

• The yen rises only if the dollar falls and if the

dollar falls then the yen rises.

• Since chip prices have risen, the dollar must

have fallen.

– C = chip prices rise

– Y = yen rises

(45)

45

Example

₂

C

Y

D /\ D

Y

C

Y

≅ D

2

equivalence

C

D

1

substitution

D

3,5

modus ponens

1.

2.

3.

4.

5.

6.

(46)

46

First Order Predicate Logic

₁

• The 4 categories syllogism represented by

predicate logic

• A: All S is P, (

∀

x)(S(x)->P(x))

• E: No S is P, (

∀

x)(S(x)->~P(x))

• I: Some S is P, (

∃

x)(S(x)∧P(x))

(47)

47

First Order Predicate Logic

₂

• All men are mortal

• Socrates is a man

• ∴ Socrates is mortal

• H=man, M=mortal, and s=Socrates

• 1. (

∀

x)(H(x)->M(x))

• 2. H(s)

/ ∴M(s)

• 3. H(s)->M(s)

1 Universal Instantiation

(48)

48

Logic Systems

₁

• A formal logic system requires:

– An alphabet of symbols

– A set of finite strings of these symbols, the wffs

– Axioms, the definition of the system

– Rules of inference, which enable a wffs to be

deduced from a set of other wfffs

• Wffs: well-formed formulas

– All S is P

(49)

49

Logic Systems

₂

• A simple example of formal logic system

– Alphabet

: the single symbol “1”

– Axioms

: The string “1” (which happens to be the

same as the symbol)

– Rules of inference

: If any string $ is a theorem,

then so is the string $11. ($ -> $11)

• The system will produce

(50)

50

Resolution

₁

• Introduced by Robinson in 1965

• Commonly implemented in

theorem-proving AI programs

• The primary rule of inference in PROLOG

– mortal(X) :- man(X).

%all men are mortal

– man(socrates).

%socrates is a man

– :- mortal(socrates).

%Query- is socrates

mortal?

(51)

51

Resolution

₂

• Each sentence must be expressed into

disjunctive form

P Q ~P \/ Q

~(P /\ Q) ~P \/ ~Q

P \/ (Q /\ R) (P \/ Q) /\ (P \/ R)

• Prove by refutation

Negate the conclusion to be proven Apply resolutions until reaching Nil

• Clauses and resolvents：

– e.g.

A \/ C

(resolvant) A \/ B

(52)

52

Resolution

₃

A B

B C A D

C D

1.Converted into disjunctive form

~A \/ B

~B \/ C ~A \/ D

~C \/ D

2.Negate the conclusion

~(~A \/ D) A /\ ~D

3.Combine

~A \/ B ~B \/ C ~C \/ D A ~D

prove

(53)

53

Resolution

₄

~A \/ B ~A \/ C ~A \/ D D nil ~D A ~C \/ D ~B \/ C

(54)

54

Resolution

₅

• Resolution on first order predicate logic

• Some programmers hate all failures

• No programmer hates any success

• ∴ No failure is a success

– P(x) = x is a programmer

– F(x) = x is a failure

– S(x) = x is a success

– H(x, y) = x hates y

(55)

55

Resolution

₆

• (1) (

∃

x)[P(x)∧(∀y) (F(y)→H(x,y))])

• (2) (∀x)[P(x) →(∀y) (S(y)→~H(x,y))])

• (3) ~(∀y)(F(y)→~S(y))

– proof in text

(56)

56

Shallow Reasoning

• A shorter chain

• Usually be experiential knowledge

• Example

If a car has a good battery a good sparkplugs gas good tires

Then the car can move

• Disadvantage

There is little or no understanding of cause and effect for explanation

• Advantage

Easier programming

Shorter development time faster

(57)

57

Causal (Deep) Reasoning

• A longer chain

• Example

– If the battery is good Then there is electricity – If there is electricity

and the sparkplugs are good Then the sparkplugs will fire – If the sparkplugs fire

and there is gas

Then the engine will run – If the engine run

and there are good tires Then the car will move

• Provide a good explanation

• Suitable for diagnosis system

(58)

58

Forward and Backward

Chaining

• Forward Chaining from facts to conclusions

• Backward Chining from hypotheses to facts

A

B

C

D

Solution

Inference

Chain

(59)

59

Forward Chaining

• Example：

Rule：

elephant (x) mamal (x) mamal (x) animal (x)

Fact：

elephant (John)

Reasoning：

• Unification

The process of finding substitutions for variables to make arguments match

elephant (John) │x = John elephant (x) mamal (x) │ mamal (x) animal (x) │ animal (John)

(60)

60

Example

• Rule1：A1 and B1 then C1 Rule2：A2 and C1 then D2 Rule3：A3 and B2 then D3 Rule4：C1 and D3 then G

• Facts：{A1,B1,A2,A3,B2} • Forward Reasoning {A1,B1,A2,A3,B2} {r1,r3} Fire r1 {A1,B1,A2,A3,B2,C1} {r1,r2,r3} Fire r2 {A1,B1,A2,A3,B2,C1,D2} {r1,r2,r3} Fire r3 {A1,B1,A2,A3,B2,C1,D2,D3} {r1,r2,r3,r4} {A1,B1,A2,A3,B2,C1,D2,D3, G } match match match match

(61)

61

Diagram

R8 R9 R5 R6 R7 R2 R1 _R3 _R4 A B C D E F G H H H J J K I -Rule N -Given Fact -Inferred Fact -Missing Fact -Applicable Rule -Inapplicable Rule RN _CONCLUSIONS INFERRED FACTS FACTS

(62)

62

Backward Chaining

• Example

Rule： elephant (x) mamal (x) mamal (x) animal (x) Fact： elephant (John) Reasoning：

• Helpful for asking the right questions

animal (John) │x = John mamal (x) animal (x) │ Elephant (x) mamaal (x) │ Elephant (John) (evidence)

(63)

63

Example

• Rule1：A1 and B1 then C1 Rule2：A2 and C1 then D2 Rule3：A3 and B2 then D3 Rule4：C1 and D3 then G Rule5：C1 and D4 then G' • Facts：{A1,B1,A2,A3,B2} • Backward Reasoning

– Assume G ' is true

│r5

verify C1 and D4 Fail

│ r1 verify A1 and B1 – Assume G ' is true │r4 Success verify C1 and D3 │r1 verify A1 and B1 verify A3 and B2 r3 x Goal

(64)

64

Diagram

H H₁ H₂ H₃ H₄ H₅ H₆ A B C D E

= AND INITIAL HYPOTHESIS(GOAL)

EVIDENCE(FACTS) INTERMEDIATE HYPOTHESES (SUBGOALS) -Elicited Evidence (Externally Supplied) -Missing Evidence -True Hypothesis -False Hypothesis

(65)

65

Comparison

Forward Chaining

Backward Chaining

Planning, monitoring, control Present to future

Antecedent to consequent

Data driven, bottom-up reasoning Work forward to find what

solutions follow from the facts Breadth-first search facilitated Antecedents determine search Explanation not facilitated

Diagnosis

Present to past

Consequent to antecedent

Goal driven, top-down reasoning Work backward to find facts that support the hypothesis

Depth-first search facilitated Consequents determine search Explanation facilitated

(66)

66

Application

• Forward Chaining

– the tree is wide and not very deep – Too many possible goals

• Backward Chaining

– the tree is narrow and deep – Few possible goals

Facts Conclusions

Broad and Not Deep

-Facts -Conclusion

Evidence

Hypothesis

Narrow and Deep

-Evidence -Hypothsis

(67)

67

Other Methods of Inference

• Other methods

– Induction

– Analogy

– Generate and test

– Abduction

– non-monotonic reasoning

– TMS

– Intuition

– Heuristics

– Default

– Autoepistemic

(68)

68

Induction

• Inference from the specific case to the general

case

• A basis of Machine Learning

• Example

Restaurant Meal Day Cost Reaction Sam’s Breakfast Friday Cheap Yes Lobdell’s Lunch Friday Expensive No

Sam’s Lunch Saturday Cheap Yes Sarah’s Breakfast Sunday Cheap No

Sam’s Breakfast Sunday Expensive No

(69)

69

Analogy

• Try and relate old situations as a guide to ones

Problem A

Problem B

Solution A

Solution B

• Case-Based Reasoning

• Example：15 Game(pick # from 1-9, no repeats)

– First reach total 15 wins – analogy to the tic-tac-toe

– extended to 15+3N Game analogy modify recall

6

1

8

7

5

3

2

9

4

(70)

70

Generate-and-Test

1. Generate a likely solution

2. Test it

3. If not satisfactory, try another solution

• Example：DENDRAL

• Plan-Generate-Test

–

Use planning to limit the likely potential solutions

–

Plan: finding chains of rules that connect a problem

with a solution

• Example：MYCIN

–

Plan: create a prioritized list of drugs

–

Generate: generate sublists of one or two drugs

–

test: against the infections, patient’s allergies, and

other consideration

(71)

71

Abduction

• Example

If x is an elephant Then x is an animal

If x is an animal Then x is an mammal

Facts:

John is a mammal

Conclusion:

John is an elephant?

Reasoning back from a true conclusion to the

premises that may have caused the conclusion

P

Q

(72)

72

• Not a valid deductive method

• Useful under

closed world assumption

– Anything that can’t be proved is assumed false in a

closed world

• Example

If x is an elephant Then x is an animal If x is an animal Then x is a mammal If x barks Then x is an animal

If x is a dog Then x barks

Facts:

John is a mammal and can bark

Conclusion:

(73)

73

Nonmonotonic Reasoning

• Previous knowledge may be incorrect when new

evidence is obtained

• Example:

Rules:

If A and B then C

If B and C then default D If A and F then delete D

Facts

A, B D

Facts

F D is deleted

(74)

74

TMS

• Truth Maintenance System

• For nonmonotonic reasoning

• Each fact is assigned as In or Out

• Each fact is attached a list of justifications

• Example

evidences that Must be true evidences that Must be false

(1) It is winter (SL ( ) ( )) In

(2) It is cold (SL (1) (3)) In

(3) It is warm

Out

Support List

(75)

75

Example

In (1) Day (Meeting)=Wednesday (SL ( ) (2)) Out (2) Day (Meeting) NEQ Wednesday

In (1) Day (M)=Wednesday (SL ( ) (2)) Out (2) Day (M) NEQ Wednesday

In (3) Day Time (M)=1400 (SL (57,103,45) ( )) In (1) Day (M)=Wednesday (SL ( ) (2))

Out (2) Day (M) NEQ Wednesday

In (3) Time (M)=1400 (SL (57,103,45) ( )) In (4) Contradiction (SL (1,3) ( ))

In (1) Day (M)=Wednesday (SL ( ) (2)) Out (2) Day (M) NEQ Wednesday

In (3) Day Time (M)=1400 (SL (57,103,45) ( )) In (4) Contradiction (SL (1,3) ( ))

In (5) NoGood (CP 4 (1,3) ( ))

Inference Meeting Time

Inference Meeting Room Find a contradiction

Automatically generate node 5

(76)

76 Out (1) Day (M)=Wednesday (SL ( ) (2))

Out (2) Day (M) NEQ Wednesday

In (3) Time (M)=1400 (SL (57,103,45) ( )) In (4) Contradiction (SL(1,3) ( ))

In (5) NoGood (CP 4 (1,3) ( ))

Out (1) Day (M)=Wednesday (SL ( ) (2)) In (2) Day (M) NEQ Wednesday (SL (5) ( ))

In (3) Time (M)=1400 (SL (57,103,45) ( )) In (4) Contradiction (SL (1,3) ( ))

In (5) NoGood (CP 4 (1,3) ( )) Try to make the

In list in CP to be Out

Try to make the

Out list in (1) to be In

Infer some other day to meet (Need not to infer time again)

(77)

77

Other methods

• Intuition

No proven theory

• Heuristics

Rules based on experience

e.g. All polar bear feel hot in desert

• Default

In the absence of specific knowledge, assume general of common knowledge (one of heuristics)

• Autoepistemic

Self-knowledge

If I have no knowledge of X Then X is false

(78)

78

Meta-Knowledge

• Knowledge about Knowledge

• Example (from MYCIN)

If

The patient is a compromised host, and

There are rules which mention in their premise pseudomonas, and

There are rules which mention in their premise klebsiellas

Then

There is suggestive evidence (.4) that the former should be done

before the latter