Tree Search Chapter4

20070322 chap4 1

Chapter4

Informed Search and Exploration

20070322 chap4

2

Tree Search

20070322 chap4

3

Search Strategies

• A search strategy is defined by picking the order of node expansion

• Uninformed Search Strategies - By systematically generating new

states and testing against the goal.

• Informed (Heuristic) Search Strategies

- By using problem-specific knowledge to find solutions more efficiently.

• An instance of the general Tree Search.

• A node is selected for expansion based on an evaluation function, f(n),

i.e. an estimate of “desirablity“

⇒

Expand most desirable unexpanded node.

• Can be implemented via a priority queue that will maintain the fringe in ascending order of f-values.

• Best-first search is venerable but inaccurate.

Best-First Search

20070322 chap4

5

Best-First Search

•

Heuristic search uses problem-specific knowledge:

evaluation function.

• Choose the seemingly-best node based on some estimate of the cost of the corresponding solution.

• Need estimate of the cost to a goal

e.g. Depth of the current node Sum of the distances so far Euclidean distance to goal etc

.

• Heuristics: rules of thumb (概測法)

• Goal: to find solutions more efficiently

20070322 chap4

6

• Heuristic function

h(n) = estimated cost of the cheapest path from node n to a goal node.

(h(n) = 0, for a goal node)

• Special cases

- Greedy Best-First Search(or Greedy Search) Minimizing estimated cost from the node to reach a goal Expanding the node that appears to be closest to goal - A* Search

Minimizing the total estimated solution cost Avoid expanding paths that are already expensive

Best-First Search

20070322 chap4

7

Heuristics

•

Heuristic is derived from heuriskein in Greek, meaning “to find” or “to discover”

• The term heuristics is often used to describe rules of thumb (經驗法則) or advice

that are generally effective, but are not guaranteed to work in every case.

• In the context of search, a heuristic is a function that takes a state as an argument

and returns a number that is an estimate of the merit of the state with respect to the goal.

Heuristics

•

A heuristic algorithm

improves the average-case performance, does not necessarily improve the worst-case performance.

• Not all heuristic functions are beneficial.

- The time spent evaluating the heuristic function in order to select a node for expansion must be recovered by a corresponding reduction in the size of the search space explored

.

- Useful heuristics should be computationally inexpensive!

20070322 chap4

9

Romania with step costs in km

Straight-line distances to Bucharest

20070322 chap4

10

Greedy Best-First Search

•

Heuristic function :

h(n) = estimated best cost to goal from n h(n) = 0 if n is a goal

e.g. h

(n) = straight-line distance for route-finding

Function GREEDY-SEARCH(problem) return a solution or failure return BEST-FIRST-SEARCH(problem, h)

20070322 chap4

11

Greedy Best-First Search Example

• Assume that we want to use greedy search

to solve the problem of travelling from Aradto Bucharest.

• The initial state=Arad

Greedy Best-First Search Example

• The first expansion step produces:

Sibiu, Timisoara and Zerind

• Greedy best-first will select Sibiu

20070322 chap4

13

Greedy Best-First Search Example

• If Sibiu is expanded we get:

Arad, Fagaras, Oradea and Rimnicu Vilcea

• Greedy best-first search will select: Fagaras

20070322 chap4

14

Greedy Best-First Search Example

• If Fagaras is expanded we get:

Sibiu and Bucharest

• Goal reached !!

• Yet not optimal (see

)

20070322 chap4

15

Analysis of Greedy Search

Complete??

No (start down an infinite path and never return to try other possibilities) (e.g. from Iasi to Fagaras)

- Susceptible to false starts Iasi→ Neamt (dead end) - No Repeated states Checking

Iasi→ Neamt → Iasi → Neamt → Iasi → ….. (oscillation)

Time??

- worst: O(b

),

Space??

O(b

):

Analysis of Greedy Search

Optimal??

(same as depth-first search)

(e.g. from Arad to Bucharest)

Arad→Sibiu→Fagaras→Bucharest

(450=140+99+211, is not shortest) Arad→Sibiu→Rim → Pitesti→Bucharest

(418=140+80+97+101)

20070322 chap4

17

A* Search

• To minimizing the total estimated solution cost

• Evaluation function:

f(n) = estimated cost of the cheapest solution through n to goal

= g(n) + h(n)

g(n) = cost so far to reach n

h(n) = estimated cost from n to the goal

function A*-SEARCH(problem) returns a solution or failure return BEST-FIRST-SEARCH(problem, g+h)

20070322 chap4

18

Admissible Heuristic

• A* search uses an admissible heuristic i.e. h(n) ≤ h

(n)

where h^*(n) is the true costfrom n (Also h(n) ≥ 0 , so h(G) = 0 for any goal G)

• Theorem:

If h(n) is admissible, A* using tree-search is optimal .

• It is both complete and optimal if h never

overestimates the cost to the goal.

20070322 chap4

19

A* Search Example

• Find Bucharest starting at Arad

f(Arad) = c(??,Arad)+h(Arad)=0+366=366

A* Search Example

•

f(Timisoara)=c(Arad,Timisoara)+h(Timisoara)=118+329=447 f(Zerind)=c(Arad,Zerind)+h(Zerind)=75+374=449

• Best choice is Sibiu

20070322 chap4

21

A* Search Example

• Expand Sibiu and determine f(n) for each node

f(Arad)=c(Sibiu,Arad)+h(Arad)=280+366=646

f(Fagaras)=c(Sibiu,Fagaras)+h(Fagaras)=239+179=415 f(Oradea)=c(Sibiu,Oradea)+h(Oradea)=291+380=671 f(Rimnicu Vilcea)=c(Sibiu,Rimnicu Vilcea)+h(Rimnicu Vilcea)

=220+192=413

• Best choice is Rimnicu Vilcea

20070322 chap4

22

A* Search Example

• Expand Rimnicu Vilcea and determine f(n) for each node

f(Craiova)=c(Rimnicu Vilcea, Craiova)+h(Craiova)=360+160=526 f(Pitesti)=c(Rimnicu Vilcea, Pitesti)+h(Pitesti)=317+100=417 f(Sibiu)=c(Rimnicu Vilcea,Sibiu)+h(Sibiu)=300+253=553

• Best choice is Fagaras

20070322 chap4

23

A* Search Example

• Expand Fagaras and determine f(n) for each node f(Sibiu)=c(Fagaras, Sibiu)+h(Sibiu)=338+253=591 f(Bucharest)=c(Fagaras,Bucharest)+h(Bucharest)=450+0

=450

• Best choice is Pitesti

A* Search Example

• Expand Pitesti and determine f(n) for each node

f(Bucharest)=c(Pitesti,Bucharest)+h(Bucharest)=418+0=418

• Best choice is Bucharest

20070322 chap4

25

Optimality of A*

Suppose some suboptimal goal G₂has been generated and is in the queue.

Let n be an unexpanded node on a shortest path to an optimal goal G.

20070322 chap4

26

Optimality of A*

• C* : cost of the optimal solution path

A* may expand some nodes before selecting a goal node.

•Assume: G is an optimal and G₂ is a suboptimal goal . f(G₂) = g(G₂) + h(G₂) = g(G₂) > C* --- (1) For some n on an optimal path to G,

if h is admissible,then

f(n) = g(n) + h(n) ≤ C* --- (2) From (1) and (2), we have

f(n) ≤ C* < f(G₂)

So, A* will never select G₂ for expansion.

20070322 chap4

27

Optimality of A*

• BUT … graph search (instead of tree-search)

• Discards new paths to repeated state.

- Previous proof breaks down

• Solution:

- Add extra bookkeeping

i.e. remove more expensive of two paths.

- Ensure that optimal path to any repeated state is always first followed.

Extra requirement on h(n): consistency (monotonicity)

Monotonicity (Consistency) of Heuristic

• A heuristic is consistent if h(n) ≤c(n, a, n’) + h(n’)

The estimated cost of reaching the goal from n

is no greater thanthe step cost of getting to successor n’

plus the estimated cost of reaching the goal from n’

• If h is consistent, we have f(n’) = g(n’) + h(n’)

= g(n) + c(n, a, n’) + h(n’)

≥ g(n) + h(n) = f(n)

⇒ f(n’) ≥ f(n)

i.e. f(n) is non-decreasingalong any path.

20070322 chap4

29

Optimality of A*

• A* expands nodes in order of increasing f value

Gradually adds “f-contours” of nodes

Contour i has all nodes with f = f_i, where f_i< f_i+1

•All nodes with f(n) > C* (optimal solution) are pruned.

20070322 chap4

30

Analysis of A* Search

Complete?? Yes

unless there are indefinitely many nodes with f ≤ f(G)

Space?? O(b

), keep all nodes in memory Optimal?? Yes, if the heuristic is admissible Optimally Efficient?? Yes

i.e. No other optimal algorithm is guaranteed to expand fewer nodes than A*

A* is not practical for many large-scale problems.

(Since A* usually runs out of space long before it runs out of time.)

20070322 chap4

31 Overcome the space problem of A*, without sacrificing

optimality or completeness

• IDA* (Iterative Deepening A*)

- a logical extension of iterative deepening search to use heuristic

- the cutoff used is the f-cost (g+h) rather than the depth

• RBFS (Recursive best-first search)

• MA* (Memory-bounded A*)

• SMA* (Simplified MA*)

- is similar to A*, but restricts the queue size to fit into the available memory

- drop the worst-leaf node when memory is full

Memory Bounded Heuristic Search

RBFS (Recursive Best-First Search)

20070322 chap4

33

RBFS Example

140

118 75

=140+253 =118+329

=75+374

140 99 151 80

=(140+140)+366

=(140+80) +193

=(140+99) +176

• Path until Rumnicu Vilcea is already expanded

• Above node; f-limit for every recursive call is shown on top.

• Below node: f(n)

• The path is followed until Pitesti which has a f-value worse than the f-limit.

20070322 chap4

34

RBFS Example

• Unwind recursion and store best f-value for current best leaf Pitesti result, f [best] ← RBFS(problem, best, min(f_limit, alternative))

• best is now Fagaras. Call RBFS for new best best value is now 450

20070322 chap4

35

RBFS Example

• Unwind recursion and store best f-value for current best leaf Fagaras result, f [best] ← RBFS(problem, best, min(f_limit, alternative))

•

Best alternative subtree is now through Timisoara

• Solution is found since because 447 > 417.

Analysis of RBFS

Complete?? Yes Time?? Exponential Space?? O(bd)

Optimal?? Yes, if the heuristic is admissible IDA* and RBFS suffer from too little memory.

IDA* retains only one single number (the current f-cost limit)

20070322 chap4

37 e.g. for the 8-puzzle

branching factor: 3 depth: 22

# of states: 3

= 3.1 * 10

9! /2= 181,440 (reachable distinct states)

for the 15-puzzle

# of states: ≅ 10

How to find a good heuristic function ?

Heuristic Functions

20070322 chap4

38

Heuristic Functions

• Two commonly-used candidates

h

(n) = number of misplaced tiles

=8

h

(n) = total Manhattan distance

(i.e. no. of squares from desired location to each tile)

= 3+1+2+2+2+3+3+2= 18

20070322 chap4

39

Effect of Heuristic Accuracy on Performance

h₂ dominates (is more informed than) h₁

• If h2(n) >= h1(n) for all n (both admissible)

then h2 dominates h1 and is better for search

• 1200 random problems with solution lengths from 2 to 24.

Heuristic Quality

• Effective branching factor b* is defined by

A well-designed heuristic would have a value of b* close to 1, allowing fairly large problems to be solved

.

•A heuristic function

is said to be more informed than

(or

dominates

) if both are admissible and

• A* using

will never expand more nodes than A*

b d

b b

N+1=1+ ^*+( ^*)²+L+( ^*)

) ( ) (

, h₂ n h₁ n

n ≥

∀

20070322 chap4

41

• Relaxed problems

A problem with fewer restrictions on the actions The cost of an optimal solution to a relaxed problem is

an admissible heuristic for the original problem.

• ABSolver

is a program that can generate heuristics automatically from problem definitions, using the “relaxed problem”

method and various other techniques.

generated a new heuristic for 8-puzzle better than preexisting heuristic

found the first useful heuristic for Rubik’s cube puzzle.

Inventing Admissible Heuristics

20070322 chap4

42

Inventing Admissible Heuristics

e.g.

A is horizontally or vertically adjacent to B and B is blank.

(P if R and S)

3 Relaxed problems:

(a) A tile can move from square A to square B if A is horizontally or vertically adjacent to B.

(P if R) --- derive h

(b) A tile can move from square A to square B if B is blank.

(P if S)

(c) A tile can move from square A to square B.

(P) --- derive h

20070322 chap4

43

Inventing Admissible Heuristics

• Composite heuristics Given h₁, h₂, h₃, … , h_m;

none dominates any others.

h(n) = max{h₁ (n), h₂ (n), … , h_m(n)}

• Subproblem

The cost of the optimal solution of the subproblem is a lower boundon the cost of the complete problem.

(To get tiles 1,2,3 and 4 into their correct positions, without worrying about what happens to the other titles.)

• Pattern databases store the exact solution to for every possible subproblem instance.

- The complete heuristic is constructed using the patterns in the DB

Inventing Admissible Heuristics

• Weighted evaluation function :

• Through learning from experience

Experience = solving lots of 8-puzzles An inductive learning algorithm can be used

to predict costs for other states that arise during search.

• Search cost

Good heuristics should be efficiently computable.

) ( ) ( ) 1 ( )

( n w g n wh n

f

= − +

20070322 chap4

45

Local Search and Optimization

• Previously: systematic exploration of search space.

- Path to goal is solution to problem

• YET, for some problems path is irrelevant.

e.g 8-queens

What quantities of quarters, nickels, and dimes add up to $17.45 and minimizes the total number of coins

• Different algorithms can be used

- Local search

20070322 chap4

46

Local Search Algorithms

• Local search does not keep track of previous solutions

- Using a single current state (rather than multiple paths) and move only to neighboring states

• Advantages

- Use a small amount of memory (usually constant amount)

- Often find reasonable (note we aren’t saying

optimal) solutions in infinite search spaces

20070322 chap4

47

• To find the best state according to an Objective Function Example

f (q, d, n) = 1,000,000, if q*0.25 + d*0.1 + n*0.05 != 17.45

= q + n + d, otherwise To minimize f

Optimization Problems

Looking for global maximum

(or minimum)

20070322 chap4

49

Hill-Climbing Search

Like climbing Everest in thick fog with amnesia

20070322 chap4

50

Hill-Climbing Search

• Only record the state and it evaluation instead of maintaining a search tree

• “is a loop that continuously moves in the direction of increasing value”

- It terminates when a peak is reached.

• Hill climbing does not look ahead of the immediate neighbors of the current state.

• Hill-climbing chooses randomly among the set of best successors, if there is more than one.

• Hill-climbing i.e. greedy local search

20070322 chap4

51

Hill-climbing example

a) shows a state of h=17 and the h-value for each possible successor. (h=3+4+2+3+2+2+1)

b) A local minimum in the 8-queens state space (h=1).

a) b)

Hill-Climbing Search

Problems:

• Local Maxima: search halts prematurely

• Plateaux: search conducts a random walk

• Ridges: search oscillates with slow progress

Creating a sequence of local maximum that are not directly connected to each other.

From each local maximum, all the available actions point downhill.

20070322 chap4

53

Hill-climbing variations

• Stochastic hill-climbing

Random selection among the uphill moves.

The selection probability can vary with the steepness of the uphill move.

• First-choice hill-climbing

Like stochastic hill climbing but by generating successors randomly until a better one is found.

• Random-restart hill-climbing

Tries to avoid getting stuck in local maxima.

20070322 chap4

54

Simulated Annealing

20070322 chap4

55

Simulated Annealing

• Idea: escape local maxima by allowing some "bad"

moves but gradually decrease their frequency

• A term borrowed from metalworking

-

-

- during cooling, molecules reduce their movement and settle into a more stable position

- annealing is process of heating metal and letting it cool slowly to lock in the stable locations of the molecules

Simulated Annealing

• Select a random move at each iteration

• Move to the selected node if it is better than the current node.

• The probability of moving to a worse node decreases exponentially with the ``badness'' of the move,

i.e.

• The temperature T changes according to a schedule.

T

e

20070322 chap4

57

Simulated Annealing

• One can prove: If T decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching 1

• Widely used in VLSI layout, airline scheduling, etc

20070322 chap4

58

Local Beam Search

• Keep track of k states rather than just one

• Start with k randomly generated states

• At each iteration, all the successors of all k states are generated

• If any one is a goal state, stop; else select the k

best successors from the complete list and

repeat.

20070322 chap4

59

Genetic algorithms

• A successor state is generated by combining two parent states

• Start with k randomly generated states (population)

• A state is represented as a string over a finite alphabet (often a string of 0s and 1s)

• Evaluation function (fitness function).

Higher values for better states.

• Produce the next generation of states by selection, crossover, and mutation

Genetic Algorithms

• Fitness function: number of non-attacking pairs of queens (min = 0, max = 8 ×7/2 = 28)

• 24/(24+23+20+11) = 31%

• 23/(24+23+20+11) = 29% etc

20070322 chap4

61

Genetic Algorithms