• 沒有找到結果。

Michael  Tsai2017/3/28 SORTING

N/A
N/A
Protected

Academic year: 2022

Share "Michael  Tsai2017/3/28 SORTING"

Copied!
31
0
0

加載中.... (立即查看全文)

全文

(1)

Michael  Tsai 2017/3/28

(2)

Sorting

Definition:

Input:   𝑎", 𝑎$, … , 𝑎& a  sequence  of  n numbers

Output:   𝑎"', 𝑎$', … , 𝑎&' is  a  permutation  (reordering)  of  the   original  sequence,  such  that  𝑎"' ≤ 𝑎$' ≤ ⋯ ≤ 𝑎&'

In  reality,  𝑎* is  the  key  of  a  record  (of  multiple  fields)   (e.g.,  student  ID)

In  a  record,  the  data  fields  other  than  the  key  is  called  satellite   data

If  satellite  data  is  large  in  size,  we  will  only  sort  the  pointers   pointing  to  the  records.  (avoiding  moving  the  data)

(3)

Applications  of  Sorting

Example 1:  Looking  for  an  item  in  a  list

Q:  How  do  we  look  for  an  item  in  an  unsorted  list?

A:  We  likely  can  only  linearly  traverse  the  list  from  the   beginning.

à𝑂(𝑛)

Q:  What  if  it  is  sorted?

A:  We  can  do  binary  search  à𝑂 log 𝑛

But,  how  much  time  do  we  need  for  sorting?  (pre-­‐processing)

(4)

Applications  of  Sorting

Example  2:

Compare  to  see  if  two  lists  are  identical  (list  all  different  items)

The  two  lists  are  n and  m in  length

Q:  What  if  they  are  unsorted?

Compare  the  1st item  in  list  1 with  (m-­‐1)  items  in  list  2

Compare  the  2nd item  in  list  1  with  (m-­‐1)  items  in  list  2

Compare  the  n-­‐th item  in  list  1  with  (m-­‐1)  items  in  list  2

𝑂 𝑛𝑚 time  is  needed

Q:  What  if  they  are  sorted?

A:  𝑂(𝑛 + 𝑚)

Again,  do  not  forget  we  also  need  time  for  sorting.  But,  how  much?

(5)

Categories  of  Sorting  Algo.

Internal  Sort:  

Place  all  data  in  the  memory

External  Sort:  

The  data  is  too  large  to  fit  it  entirely  in  the  memory.

Some  need  to  be  temporarily  placed  onto  other  (slower)  storage,  e.g.,   hard  drive,  flash  disk,  network  storage,  etc.  

In  this  lecture,  we  will  only  discuss  internal  sort.

Storage  is  cheap nowadays.  In  most  cases,  only  internal  sort  is   needed.

(6)

Some  terms  related  to  sorting

Stability:  

If  𝑎* = 𝑎5 (equal  key  value),then  they  maintain  the  same   order  before  and  after  sorting.

In-­‐place:  

Directly  sort  the  keys  at  their  current  memory   locations.  

Therefore,  only  O(1)  additional  space  is  needed  for  sorting.

Adaptability:

If  part  of  the  sequence  is  sorted,  then  the  time  complexity  of   the  sorting  algorithm  reduces.

(7)

How  fast  can  we  sort?

Assumption:  compare  and  swap

Compare:  compare  two  items  in  the  list

Swap:  Swap  the  locations  of  these  two  items

How  much  time  do  we  need  in  the  worst  case?

𝐾" ≤ 𝐾$

𝐾$ ≤ 𝐾7 𝐾" ≤ 𝐾7

𝐾" ≤ 𝐾7 𝐾$ ≤ 𝐾7

stop stop

stop

stop

stop stop

[1,2,3]

[1,3,2]

[1,2,3]

[1,2,3]

[1,3,2]

[2,3,1]

[2,1,3]

[2,1,3]

[3,1,2]

[3,1,2] [3,2,1]

Yes No

(8)

Decision  tree  for  sorting

Every  node  represents  a  comparison  &  swap

Sorting  is  completed  when  reaching  the  leaf

How  many  leaves?

𝑛!  ,  since  there  are  that  many  possible  permutations

𝐾" ≤ 𝐾$

𝐾$ ≤ 𝐾7 𝐾" ≤ 𝐾7

𝐾" ≤ 𝐾7

stop stop

stop

stop

stop stop

[1,2,3]

[1,3,2]

[1,2,3]

[1,2,3]

[1,3,2]

[2,3,1]

[2,1,3]

[2,1,3]

[3,1,2] [3,2,1]

𝐾$ ≤ 𝐾7

(9)

How  fast  can  we  sort?

所以,  worst  case所需要花的時間,  為此binary  tree的height.

如果decision  tree  height為h,  有l個leaves

𝑙 ≥ 𝑛!,  we  have  a  least  n! outcomes  (leaves)

𝑙 ≤ 2=,  a  binary  tree  (decision  tree)  of  height  h  has  at  most  2=>"

leaves

2= ≥ 𝑙 ≥ 𝑛!

ℎ ≥ log$𝑛!

𝑛! = 𝑛 𝑛 − 1 𝑛 − 2 … 3 ⋅ 2 ⋅ 1 ≥ &$

D E

log$ 𝑛! ≥ log$ &$

D

E = &$ log$&$ = Ω 𝑛 log 𝑛

Summary: Any  “comparison-­‐based”  sorting  algorithm  has  worst-­‐case   time  complexity  of  Ω 𝑛 log 𝑛 .

(10)

Review:  Selection  Sort

Select  the  smallest,  move  it  to  the  first  position.

Select  the  second  smallest,  move  it  to  the  second  position.

….

The  last  item  will  automatically  be  placed  at  the  last  position.

ㄅ ㄆ 1

1 2

1 2

(11)

Review:  Selection  Sort

Selection  sort  does  not  change  the  execution  of  the  algorithm   due  to  the  current  conditions.

Always  going  through  the  entire  array  in  each  iteration.

Therefore,  its  best-­‐case,  worst-­‐case,  average-­‐case  running   time  are  all  𝑂 𝑛$

Not  adaptive!

In-­‐place

(12)

Insertion  Sort

In  each  iteration,  add  one  item  to  a  sorted  list  of  i item.

Turning  it  into  a  sorted  list  of  (i+1) item

2 3 6 5 1 4

2 3 6 5 1 4

2 3 6 5 1 4

2 3 5 6 1 4

2 3 5 6 1 4

1 2 3 5 6 4

1 2 3 5 6 4

1 2 3 4 5 6

(13)

Pseudo  code

Insertion-Sort(A)

1 for j = 2 to A. length

2 key = A[j]

3 // Insert A[j] into the sorted sequence A[1 . . j − 1].

4 i = j − 1

5 while i > 0 and A[i] > key

6 A[i + 1] = A[i]

7 i = i − 1

8 A[i + 1] = key

1

(14)

Insertion  Sort

Q:  How  much  time  is  needed?

A:  In  the  worst  case,  the  item  needs  to  be  placed  at  the   beginning  for  each  and  every  iteration.

(Spending  time  linear  to  the  size  of  sorted  part)

&>" 𝑖

" = & &>"

$ = 𝑂(𝑛$)

Average-­‐case  complexity:𝑂(𝑛$).  (Why?)

Possible  variation:  (do  those  improve  the  time  complexity?)

1.  Use  binary  search  to  look  for  the  location  to  insert.

2.  Use  linked  list  to  store  the  items.  Then  moving  takes  only   𝑂(1)!

(15)

What’s  good  about  insertion  sort

Simple  (small  constant  in  time  complexity  representation)

Good  choice  when  sorting  a  small  list

Stable

In-­‐place

Adaptive

Example:  In   1,2,5,3,4 ,  only  two  inversions  <5,3>,  <5,4>.

The  running  time  for  insertion sort:  O(n+d),  d  is  the  number  of  inversions   Best  case:  O(n)  (No  inversion,  sorted)

Online:  

No  need  to  know  all  the  numbers  to  be  sorted.  Possible  to  sort   and  take  input  at  the  same  time.

(16)

Merge  Sort

Use  Divide-­‐and-­‐Conquer strategy

Divide-­‐and-­‐Conquer:

Divide:  Split  the  big  problem  into  small  problems

Conquer:  Solve  the  small  problems

Combine:  Combine  the  solutions  to  the  small  problems  into  the  solution   of  the  big  problems.

Merge  sort:

Divide:  Split  the  n  numbers  into  two  sub-­‐sequences  of  n/2  numbers

Conquer:  Sort  the  two  sub-­‐sequences  (use  recursive  calls  to  delegate  to   the  clones)

Combine:  Combine  the  two  sorted  sub-­‐sequences into  the  one  sorted   sequence

(17)

Merge  Sort

17

To see that the MERGE procedure runs in ‚.n/ time, where n D r ! p C 1, observe that each of lines 1–3 and 8–11 takes constant time, the for loops of lines 4–7 take ‚.n1 C n2/ D ‚.n/ time,7 and there are n iterations of the for loop of lines 12–17, each of which takes constant time.

We can now use the MERGE procedure as a subroutine in the merge sort al- gorithm. The procedure MERGE-SORT.A; p; r/ sorts the elements in the subar- ray AŒp : : r!. If p " r, the subarray has at most one element and is therefore already sorted. Otherwise, the divide step simply computes an index q that par- titions AŒp : : r! into two subarrays: AŒp : : q!, containing dn=2e elements, and AŒq C 1 : : r!, containing bn=2c elements.8

MERGE-SORT.A; p; r/

1 if p < r

2 q D b.p C r/=2c

3 MERGE-SORT.A; p; q/

4 MERGE-SORT.A; q C 1; r/

5 MERGE.A; p; q; r/

To sort the entire sequence A D hAŒ1!; AŒ2!; : : : ; AŒn!i, we make the initial call MERGE-SORT.A; 1; A:length/, where once again A:length D n. Figure 2.4 il- lustrates the operation of the procedure bottom-up when n is a power of 2. The algorithm consists of merging pairs of 1-item sequences to form sorted sequences of length 2, merging pairs of sequences of length 2 to form sorted sequences of length 4, and so on, until two sequences of length n=2 are merged to form the final sorted sequence of length n.

2.3.2 Analyzing divide-and-conquer algorithms

When an algorithm contains a recursive call to itself, we can often describe its running time by a recurrence equation or recurrence, which describes the overall running time on a problem of size n in terms of the running time on smaller inputs.

We can then use mathematical tools to solve the recurrence and provide bounds on the performance of the algorithm.

7We shall see in Chapter 3 how to formally interpret equations containing ‚-notation.

8The expression dxe denotes the least integer greater than or equal to x, and bxc denotes the greatest integer less than or equal to x. These notations are defined in Chapter 3. The easiest way to verify that setting q to b.p C r/=2c yields subarrays AŒp : : q! and AŒq C 1 : : r! of sizes dn=2e and bn=2c, respectively, is to examine the four cases that arise depending on whether each of p and r is odd or even.

Divide

Conquer  x2 Combine

(18)

Merge  Sort

void Mergesort(int A[], int temp, int left, int right) { int mid;

if (right > left) {

mid=(right+left)/2;

Mergesort(A,temp,left,mid);

Mergesort(A,temp,mid+1,right);

Merge(A,temp,left,mid+1,right);

} }

Divide

Conquer Combine

temp:  temporarily  storage

left,right:  the  left  &  right  indices  of  the  range  to   be  sorted.

(19)

Merge  Sort:  Example

(20)

How  to  combine  (merge)?

1 4 5 8 2 3 6 9

1 2 3 4 5 6 8 9

i j

Running  time:  𝑂 𝑛" + 𝑛$ = 𝑂(𝑛),  𝑛"和𝑛$ are  the  lengths  of   the  two  sub-­‐sequences.

A  temporary  storage  of  size  O(n)  is  needed  during  the  merge   process

Original  array

Temporary  storage

(21)

Implementation:  Merge

21

we just take the remaining input pile and place it face down onto the output pile.

Computationally, each basic step takes constant time, since we are comparing just the two top cards. Since we perform at most n basic steps, merging takes ‚.n/

time.

The following pseudocode implements the above idea, but with an additional twist that avoids having to check whether either pile is empty in each basic step.

We place on the bottom of each pile a sentinel card, which contains a special value that we use to simplify our code. Here, we use 1 as the sentinel value, so that whenever a card with 1 is exposed, it cannot be the smaller card unless both piles have their sentinel cards exposed. But once that happens, all the nonsentinel cards have already been placed onto the output pile. Since we know in advance that exactly r ! p C 1 cards will be placed onto the output pile, we can stop once we have performed that many basic steps.

MERGE.A; p; q; r/

1 n1 D q ! p C 1 2 n2 D r ! q

3 let LŒ1 : : n1 C 1! and RŒ1 : : n2 C 1! be new arrays 4 for i D 1 to n1

5 LŒi ! D AŒp C i ! 1!

6 for j D 1 to n2

7 RŒj ! D AŒq C j ! 8 LŒn1 C 1! D 1

9 RŒn2 C 1! D 1 10 i D 1

11 j D 1

12 for k D p to r 13 if LŒi! " RŒj !

14 AŒk! D LŒi!

15 i D i C 1

16 else AŒk! D RŒj !

17 j D j C 1

In detail, the MERGE procedure works as follows. Line 1 computes the length n1 of the subarray AŒp : : q!, and line 2 computes the length n2 of the subarray AŒq C 1 : : r!. We create arrays L and R (“left” and “right”), of lengths n1 C 1 and n2 C 1, respectively, in line 3; the extra position in each array will hold the sentinel. The for loop of lines 4–5 copies the subarray AŒp : : q! into LŒ1 : : n1!, and the for loop of lines 6–7 copies the subarray AŒq C 1 : : r! into RŒ1 : : n2!. Lines 8–9 put the sentinels at the ends of the arrays L and R. Lines 10–17, illus-

(22)

Merge  sort

Every  item  to  be  sorted  is  processed  once  per  “pass”à𝑂(𝑛)

How  many  passes  is  needed?

The  length  of  the  sub-­‐sequence  doubles every  pass,  and   finally  it  becomes  the  large  sequence  of  n numbers

Therefore,   log$ 𝑛 passes.

Total  running  time:  𝑂 𝑛 log$𝑛 = 𝑂(𝑛 log 𝑛)

Worst-­‐case,  best-­‐case,  average-­‐case:  𝑂 𝑛 log 𝑛 (Not  adaptive)

Not  in-­‐place:  need  additional  storage  for  sorted  sub-­‐

sequences

Additional  space:  O(n)

(23)

Find  a  pivot(支點),  manipulate  the  locations  of  the  items  so  that:

(1)  all  items  to  its  left  is  smaller  or  equal (unsorted),

(2)  all  items  to  its  right  is  larger

Recursively  call  itself  to  sort  the  left  and  right  sub-­‐sequences.

26 5 37 1 61 11 59 15 48 19

26 5 37 1 61 11 59 15 48 19

26 5 19 1 61 11 59 15 48 37

26 5 19 1 15 11 59 61 48 37

11 5 19 1 15 26 59 61 48 37

(24)

Pseudo  Code

Combine: Because the subarrays are already sorted, no work is needed to combine them: the entire array AŒp : : r! is now sorted.

The following procedure implements quicksort:

QUICKSORT.A; p; r/

1 if p < r

2 q D PARTITION.A; p; r/

3 QUICKSORT.A; p; q ! 1/

4 QUICKSORT.A; q C 1; r/

To sort an entire array A, the initial call is QUICKSORT.A; 1; A:length/.

Partitioning the array

The key to the algorithm is the PARTITION procedure, which rearranges the subar- ray AŒp : : r! in place.

PARTITION.A; p; r/

1 x D AŒr!

2 i D p ! 1

3 for j D p to r ! 1 4 if AŒj ! " x

5 i D i C 1

6 exchange AŒi! with AŒj ! 7 exchange AŒi C 1! with AŒr!

8 return i C 1

Figure 7.1 shows how PARTITION works on an 8-element array. PARTITION always selects an element x D AŒr! as a pivot element around which to partition the subarray AŒp : : r!. As the procedure runs, it partitions the array into four (possibly empty) regions. At the start of each iteration of the for loop in lines 3–6, the regions satisfy certain properties, shown in Figure 7.2. We state these properties as a loop invariant:

At the beginning of each iteration of the loop of lines 3–6, for any array index k,

1. If p " k " i, then AŒk! " x.

2. If i C 1 " k " j ! 1, then AŒk! > x.

3. If k D r, then AŒk! D x.

Divide

Conquer  x2

No  Combine!

(25)

Quick  Sort

11 5 19 1 15 26 59 61 48 37

1 5 11 19 15 26 59 61 48 37

1 5 11 19 15 26 59 61 48 37

1 5 11 15 19 26 59 61 48 37

1 5 11 15 19 26 59 61 48 37

1 5 11 15 19 26 48 37 59 61

1 5 11 15 19 26 37 48 59 61

1 5 11 15 19 26 37 48 59 61

(26)

Quick  Sort:  Worst  &  Best  case

But  worst  case  running  time  is  still  O 𝑛$

Q:  Give  an  example  which  produces  worst-­‐case  running  time   for  the  quick  sort  algorithm.

In  this  case:  running  time  is  O(𝑛$)

Best  case?

Pivot  can  split  the  sequence  into  two  sub-­‐sequences  of  equal   size.

Therefore,  T(n)=2T(n/2)+O 𝑛

T(n)=O(𝑛 log 𝑛)

(27)

Randomized  Quick  Sort

Avoid  worst  case  to  happen  frequently

Randomly  select  a  pivot  (not  always  the  leftmost  key)

Reduce  the  probability  of  the  worst  case

However,  worst  case  running  time  is  still  𝑂 𝑛$

26 5 37 1 61 11 59 15 48 19

Randomly  select  a  pivot

Swap  in  advance

(28)

Average  running  time

Better  if  the  selection  of  pivot  can  evenly  split  the  sequence   into  two  sub-­‐sequences  of  equal  size

Why  the  average  running  time is  close  to  the  best-­‐case  one?

假設很糟的一個狀況:  每次都分成1:9

𝑇(𝑛) = 𝑇(9𝑛/10) + 𝑇(𝑛/10) + 𝑐𝑛

= 𝑇 Q"&

"RR + 𝑇 "RRS& + ST&"R + 𝑇 "RRS& + 𝑇 "RR"& + T&"R + 𝑐𝑛

= ⋯

Time  needed  for  the  “9/10  subsequence”

Time  needed  for  the  “1/10  subsequence”

Time  needed  for  partitioning

(29)

Average  running  time

As  long  as  the  pivot  can  partition  according  to   a  particular  ratio  (even  not  close  to  50%),  we   can  still  obtain  𝑂 𝑛 log 𝑛 running  time!

(30)

Average  running  time

Case  1:

Worst  case  for   the  first  level   partition,

but  best-­‐case   for  second   level.

Case  2:

Best  case  for  the first  level  partition

Partition  time  for  the  first  level:Θ(𝑛)

Partition  time,   second  level:

Θ(𝑛 − 1)

Partition  time  for  the  first  level:Θ(𝑛)

Θ 𝑛 + Θ 𝑛 − 1 = Θ(𝑛)

Same!

(Case  1  has  larger   constant)

The  better-­‐partitioned   level  would  “absorb”  the   extra  running  time  for   worse-­‐partitioned  level.

(31)

比較四大金剛

Insertion  sort:  quick  with  small  input  size  n.  (small  constant)

Quick  sort:  Best  average  performance (fairly  small  constant)

Merge  sort:  Best  worst-­‐case  performance

Heap  sort:  Good  worst-­‐case  performance,  no  additional  space   needed.

Real-­‐world  strategy:  a  hybrid  of  insertion  sort +  others.  Use   input  size n  to  determine  the  algorithm  to  use.

Worst Average Additional  

Space?

Insertion  sort 𝑂(𝑛$) 𝑂(𝑛$) O(1)

Merge  sort 𝑂(𝑛 log𝑛) 𝑂(𝑛 log𝑛) O(n)

Quick  sort O(𝑛$) O(𝑛 log 𝑛) O(1)

Heap sort 𝑂(𝑛 log𝑛) O(1)

Not  covered   today!

參考文獻

相關文件

Table 1: Characteristics of interviewed visitors, by place of residence Table 2: Average length of stay of interviewed visitors, by place of residence Table 3: Per-capita spending

Reading Task 6: Genre Structure and Language Features. • Now let’s look at how language features (e.g. sentence patterns) are connected to the structure

Graduate Masters/mistresses will be eligible for consideration for promotion to Senior Graduate Master/Mistress provided they have obtained a Post-Graduate

Let us suppose that the source information is in the form of strings of length k, over the input alphabet I of size r and that the r-ary block code C consist of codewords of

For the primary section, the additional teaching post(s) so created is/are at the rank of Assistant Primary School Master/Mistress (APSM) and not included in calculating the

„ Indicate the type and format of information included in the message body. „ Content-Length: the length of the message

™ ™ When ready to eat a bite of your bread, place the spoon on the When ready to eat a bite of your bread, place the spoon on the under plate, then use the same hand to take the

• Given a (singly) linked list of unknown length, design an algorithm to find the n-th node from the tail of the linked list. Your algorithm is allowed to traverse the linked