Sorting Ascending Order

SortingSorting

KMITL 01076206 Data Structures & Algorithms ก�� Sorting 1

Lecturer : Kritawan Siriboon, Room no. 913

Text : Data Structures & Algorithm Analysis in C, C++,…

Mark Allen Weiss, Addison Wesley




Decending OrderDecending Order

Sorting Sorting ก��ก��Sorting Sorting ก��ก��

Ascending Order��ก�� --> ��ก


Decending Order��ก��ก --> ��• Comparison-based sorting

ใชการเปรยบเทยบในการจดลาดบ

• Comparison-based sorting

ดทสด O(n log n)

• Comparison-based sortingใชการเปรยบเทยบในการจดลาดบ

• Comparison-based sorting

ดทสด O(n log n)

SortingSortingSortingSorting

• Bubble Sort

• Selection Sort

• Insertion Sort

• Shell Sort

(Diminishing

• Bubble Sort

• Selection Sort

• Insertion Sort

• Shell Sort

(Diminishing

• List & Tree Sorts

– Sequential Search

(Linear Search)

– Binary Search Tree

– B-Tree

• List & Tree Sorts

– Sequential Search

(Linear Search)

– Binary Search Tree

– B-Tree


Shell Sort

(Diminishing

Increment Sort)

• Merge Sort

• Quick Sort

Shell Sort

(Diminishing

Increment Sort)

• Merge Sort

• Quick Sort

– B-Tree

– AVL (Hight Balanced)

Tree

– Heap

– B-Tree

– AVL (Hight Balanced)

Tree

– Heap

25 57 48 37 12 92 86 33

25 57 48 37 12 92 86 33

25 48 57 37 12 92 86 33

25 48 37 57 12 92 86 33

Pass 1 Original File

Bubble SortBubble Sort



25 48 37 57 12 92 86 33

25 48 37 12 57 92 86 33

25 48 37 12 57 92 86 33

25 48 37 12 57 86 92 33

25 48 37 12 57 86 33 92

After Pass 1,

92, the biggest float up the water to its place.

Bubble SortBubble Sort

25 57 48 37 12 92 86 33

25 48 37 12 57 86 33 92

25 37 12 48 57 33 86 92

25 12 37 48 33 57 86 92

12 25 37 33 48 57 86 92

12 25 33 37 48 57 86 92

12 25 33 37 48 57 86 92

Original File

After Pass 1

After Pass 2

After Pass 3


12 25 33 37 48 57 86 92

12 25 33 37 48 57 86 921111 2222 3333 4444 5555 6666 7777 8888

Data size = n.

How many passes would make the ordered list? _______ passes.

How may comparisions in :

pass 1 : ______, pass 2 : ______, pass 3 : ______

Last pass, pass n-1 ______Total comparisions = ________________ = __________ = O(___)

After Pass ______Sorted File


Bubble Sort CodeBubble Sort CodeBubble Sort CodeBubble Sort Code

bubble(*int x, int n) {

int hold, j, pass;

int switched = true;

for (pass = 0; pass <= n-2 && switched; pass++){

switched = false;

for (j = 0; j <= n-pass-2; j++)

if(x[j] > x[j+1]){

bubble(*int x, int n) {

int hold, j, pass;

int switched = true;

for (pass = 0; pass <= n-2 && switched; pass++){

switched = false;

for (j = 0; j <= n-pass-2; j++)

if(x[j] > x[j+1]){

i = n-p-1 -->

p = 0, i = n-1

25 57 48 37 12 92 86 33

p0p0p0p0 25 48 37 12 57 86 33 92

p1p1p1p1 25 37 12 48 57 33 86 92

p2p2p2p2 25 12 37 48 33 57 86 92

p3p3p3p3 12 25 37 33 48 57 86 92

p4p4p4p4 12 25 33 37 48 57 86 92

p5p5p5p5 12 25 33 37 48 57 86 92

p6p6p6p6 12 25 33 37 48 57 86 920000 1111 2222 3333 4444 5555 6666 7777

25 57 48 37 12 92 86 33


if(x[j] > x[j+1]){

switched = TRUE;

hold = x[j];

x[j] = x[j+1];

x[j+1] = hold;

}//if

}//for

}//bubble

if(x[j] > x[j+1]){

switched = TRUE;

hold = x[j];

x[j] = x[j+1];

x[j+1] = hold;

}//if

}//for

}//bubble

∑∑−

=

−

=

=−−1

1

2

0

)1(n

i

n

p

ipn

= 1+2+3+...+(n-1)

= n * (n-1)/2= O(n2)

p = 0, i = n-1p = n-2 , i = 1

25 57 48 37 12 92 86 33

25 57 48 37 12 92 86 33

25 48 57 37 12 92 86 33

25 48 37 57 12 92 86 33

25 48 37 12 57 92 86 33

25 48 37 12 57 92 86 33

25 48 37 12 57 86 92 33

25 48 37 12 57 86 33 920000 1111 2222 3333 4444 5555 6666 7777

Bubble SortBubble SortBubble SortBubble Sort

Best case : when? ______________– Why? ______________________

– How many comparision ? ________ times.

– O( ___ )

Average case : O( ___ ) – ______________________________

Best case : when? ______________– Why? ______________________

– How many comparision ? ________ times.

– O( ___ )

Average case : O( ___ ) – ______________________________If it has k iterations.


– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

– ______________________________

If it has k iterations.

# comparisions = (n-1)+(n-2)+ ... +(n-k+1)+(n-k)

= (2n-1-k)*[(n-1)-(n-k)+1]/2

= (2n-1-k)*(k)/2

= 2nk-k-k2/2k = O(n) --> #comparisions = O(n2)

Straight Selection Sort(pushed down sort)Straight Selection Sort(pushed down sort)Straight Selection Sort(pushed down sort)Straight Selection Sort(pushed down sort)

25 57 48 37 12

25 12 48 37 57

25 12 37 48 57

25 12 37 48 57

12 25 37 48 57

Data size = n.

Original File

After Pass 1

Sorted File, After Pass ______



Data size = n.

How many passes would make the ordered list? _______ passes.

Pass #1, # comparisions = __________



Last Pass #________ # comparisions = __________

Total #comparisions = ______________= __________ = O(___)


Insertion SortInsertion Sort

Data size = n, how many passes? _____.

38 8 64 51 32 21

8 34 64 51 32 21

8 34 64 51 32 21

8 34 51 64 32 21

8 32 34 51 64 21

8 21 32 34 51 64

Original File

After Pass 1

After Pass 2

Sorted File, After Pass ______Ascending Order��ก�� --> ��ก


Data size = n, how many passes? _____.

Pass #0 (insert the 1st element) ________________

Pass #1, # comparisions: minimum = ____ maximum =____

Pass #2, # comparisions: minimum = ____ maximum =____

Last Pass #______ # comparisions: minimum = ____ maximum =____

Total #comparisions

• Worst case = ________ = _________= O(__) When?____________

• Best case = ________ = _________= O(__) When?____________


Insertion SortInsertion SortInsertion SortInsertion Sort

template <class T>

void insertionSort( vector<T> & a){

int j;

for( int p = 1; p < a.size(); p++ ){

T tmp = a[ p ];

for(j = p; j > 0 && tmp < a[j-1];j--) //line*

a[ j ] = a[ j - 1 ];

a[ j ] = tmp;

template <class T>

void insertionSort( vector<T> & a){

int j;

for( int p = 1; p < a.size(); p++ ){

T tmp = a[ p ];

for(j = p; j > 0 && tmp < a[j-1];j--) //line*

a[ j ] = a[ j - 1 ];

a[ j ] = tmp;

38 8 64 51 32 21

8 34 64 51 32 21

8 34 64 51 32 21

8 34 51 64 32 21

8 32 34 51 64 21

8 21 32 34 51 64

8 34 51 64 32 21

tmp=32



= 1+2+3+...+(n-1) = n * (n-1)/2= O(n2)

}

}

}

}0 1 2 3 4 5

p=4j

∑−

=

1

1

n

p

p

Times of comparisions =

#times of line*

Outer loop : p = 1 to n-1

Inner loop : j = p to 1 = p

Shell Sort (Diminishing Increment Sort) (Donald Shell)Shell Sort (Diminishing Increment Sort) (Donald Shell)Shell Sort (Diminishing Increment Sort) (Donald Shell)Shell Sort (Diminishing Increment Sort) (Donald Shell)

Incremental Sequence: h1, h2, ..., ht (h1 = 1) ex: 1 3 5Incremental Sequence: h1, h2, ..., ht (h1 = 1) ex: 1 3 5

1 2 3 4 5 1 2 3 4 5 1 2 3

5 81 94 11 96 12 35 17 95 28 58 41 75 15

35 17 11 28 12 41 75 15 96 58 81 94 95

1 2 3 1 2 3 1 2 3 1 2 3 1


3 81 94 11 96 12 35 17 95 28 58 41 75 15

11 12 15 17 28 35 41 58 75 81 94 95 96

1 28 12 11 35 15 41 58 17 94 75 81 96 95

increment ��ก => file ��ก , insertion sort O(n2) ok

increment �� => � �� sorted, insertion sort �� O(n)


Shell SortShell SortShell SortShell Sorttemplate <class T>

void shellsort( vector<T> & a ){

int j;

for(int gap = a.size( )/2; gap > 0; gap /= 2 )

for(int i = gap; i < a.size( ); i++){

T tmp = a[ i ];

for(j = i; j >= gap && tmp < a[j-gap]; j -= gap)

a[j] = a[j-gap];

a[j] = tmp;

}

template <class T>

void shellsort( vector<T> & a ){

int j;

for(int gap = a.size( )/2; gap > 0; gap /= 2 )

for(int i = gap; i < a.size( ); i++){

T tmp = a[ i ];

for(j = i; j >= gap && tmp < a[j-gap]; j -= gap)

a[j] = a[j-gap];

a[j] = tmp;

}


}

}

}

}

popular (but poor) increment (suggested by Shell)

ht = n/2 hk = h k+1/2 ก��ก increment �� performance ��กSedgewick �� increment sequence �� 1, 5, 19, 41, 109, . . .

�� term �� 9 * 4i - 9 * 2i + 1 �� 4i – 3 * 2i + 1

�� !"#��" performance � $��#�� %�� n &'�� ก� $��ก��

Heap Heap








Complete Binary TreeComplete Binary Tree

• Every level is filled except the bottom level which is filled from

left to right.

• Every level is filled except the bottom level which is filled from

left to right.


PerfectPerfect Binary TreeBinary TreePerfectPerfect Binary TreeBinary Tree

Complete Binary TreeComplete Binary Tree


• n = 20+21+...+2d =

• n = 2d+1 - 1 ( 2d external + 2d - 1 internal )

• d = log2 (n+1) –1

• O( log2(n))

• log21,000,000 < 20

• binary tree height H ==> max. nodes = 2H+1 - 1

• n = 20+21+...+2d =

• n = 2d+1 - 1 ( 2d external + 2d - 1 internal )

• d = log2 (n+1) –1

• O( log2(n))

• log21,000,000 < 20

• binary tree height H ==> max. nodes = 2H+1 - 1

Binary Tree RepresentationsBinary Tree Representations

• Dynamic

• Linked Array root 4

left data right father

1

9

3

5

6

0

1

2

3

4


• Sequential Array (Implicit Array)

(Sequential) Implicit Array(Sequential) Implicit Array


left(p) =

right(p) =father(p) =

left(p) =

right(p) =father(p) =

Heap DefinitionHeap Definition

Decending heap(Max heap)

Complete Binary Tree � !" key #�" node $%&

Ascending heap(Min heap)


Not a heap.

Complete Binary Tree � !" key #�" node $%&1. <= key #�" decendents #�" �� (Min heap)

2. >= key #�" decendents #�" �� (Max heap)

Complete Binary Tree of height

h has between 2h and 2h+1 –1 nodes

=> height = log n => O(log n)

Insert HeapInsert Heap

Insert 14

14

14Perlocate(reheap) 14 up

Perlocate 14 up


Inserting 14.

• Must have one more node at n+1.

• 14 (<31) cannot be at n+1, perlocate 14 up, 31 goes down.

• Keep perlocating 14 up to find its right place.

• Adjusting the tree to be heap again is called reheap.

Inserting 14.

• Must have one more node at n+1.

• 14 (<31) cannot be at n+1, perlocate 14 up, 31 goes down.

• Keep perlocating 14 up to find its right place.

• Adjusting the tree to be heap again is called reheap.

After adding and reheap.

13 21 16 24 31 19 68 65 26 32 31

0 1 2 3 4 5 6 7 8 9 10

Insert HeapInsert HeapInsert 14

14

14

Perlocate 14 up

Perlocate 14 up

13 21 16 24 31 19 68 65 26 32

0 1 2 3 4 5 6 7 8 9 10


0 1 2 3 4 5 6 7 8 9 10

After adding and reheap.

0 1 2 3 4 5 6 7 8 9 10heap

temp 14

13 14 16 24 21 19 68 65 26 32 31

0 1 2 3 4 5 6 7 8 9 10heap

Delete HeapDelete HeapDelete HeapDelete Heap

• Deleting root gets the minimum/maximum (min/max heap).• Deleting root gets the minimum/maximum (min/max heap).

• Reheap

makes the 2nd minimum /maximum

node be the new root.

• Keep deleting the root

-> an ordered list.

• Reheap

makes the 2nd minimum /maximum

node be the new root.

• Keep deleting the root

-> an ordered list.


• Here is the min heap.

• Deleting the root will get 13, the minimum.

• Reheap.

• 14 (the 2nd minimum) will be the new root.

• Here is the min heap.

• Deleting the root will get 13, the minimum.

• Reheap.

• 14 (the 2nd minimum) will be the new root.

-> an ordered list.-> an ordered list.

Delete HeapDelete Heap31

Perlocate (reheap) 31 down

Delete 13

temp 31

13 14 16 20 21 19 68 65 26 32 31

0 1 2 3 4 5 6 7 8 9 10heap

13 14 16 20 21 19 68 65 26 32 13

0 1 2 3 4 5 6 7 8 9 10heap

1

2

3


31

31

Perlocate 31 down

Perlocate 31 down

heap heapDeleting the root makes the heap 1 slot shorter.

temp 31

Delete Heap (Delete Delete Heap (Delete 13 13 cont.)cont.)

31Perlocate 31 down

31

• After deleting and reheap,


• After deleting and reheap,

13 becomes out of the heap.

• This is called inplace sort

since we don’t use extra spacefor the sorted data.

14 20 16 26 21 19 68 65 31 32 13

0 1 2 3 4 5 6 7 8 9 10heap

Deleting the root makes the heap 1 slot shorter.

Heap SortHeap SortHeap SortHeap Sort

97 53 59 26 41 58 31

0 1 2 3 4 5 6 7 8

59 53 58 26 41 31 97

0 1 2 3 4 5 6 7 8

• ไมไมไมไมดดดดถา ถา ถา ถา n n n n นนนนอยๆ อยๆ อยๆ อยๆ ( ( ( ( ตอตอตอตองงงงเสยเวลาใเสยเวลาใเสยเวลาใเสยเวลาในนนนการปรบใหเปการปรบใหเปการปรบใหเปการปรบใหเปน น น น heap )heap )heap )heap )• maximum time = O(n logmaximum time = O(n logmaximum time = O(n logmaximum time = O(n log2222n) n) n) n) เพราะ เพราะ เพราะ เพราะ depth depth depth depth ของ ของ ของ ของ heap heap heap heap เปน เปน เปน เปน log n log n log n log n


• maximum time = O(n logmaximum time = O(n logmaximum time = O(n logmaximum time = O(n log2222n) n) n) n) เพราะ เพราะ เพราะ เพราะ depth depth depth depth ของ ของ ของ ของ heap heap heap heap เปน เปน เปน เปน log n log n log n log n ((((ดงนน ดงนน ดงนน ดงนน deleteMin ndeleteMin ndeleteMin ndeleteMin n----1 1 1 1 ครง ครง ครง ครง = O(n log n))= O(n log n))= O(n log n))= O(n log n))

• ใช ใช ใช ใช space space space space นนนนอยมาก แทบไมม อยมาก แทบไมม อยมาก แทบไมม อยมาก แทบไมม extra space extra space extra space extra space เลย เลย เลย เลย หากทา หากทา หากทา หากทา inplace sortinplace sortinplace sortinplace sort• avg case avg case avg case avg case ดดดดไมเทา ไมเทา ไมเทา ไมเทา avg case avg case avg case avg case ขอขอขอของ ง ง ง quick sort quick sort quick sort quick sort แต แต แต แต worst case worst case worst case worst case แยกวา แยกวา แยกวา แยกวา avg case avg case avg case avg case ขอขอขอของ ง ง ง quick sort quick sort quick sort quick sort นนนนดดดดเเเเดดดดยวยวยวยว( worst case ( worst case ( worst case ( worst case ขอขอขอของ ง ง ง quick sort quick sort quick sort quick sort เปเปเปเปน น น น O(nO(nO(nO(n2222) )) )) )) )

• จจจจากการศกษาพบวาโากการศกษาพบวาโากการศกษาพบวาโากการศกษาพบวาโดดดดยทวไป ยทวไป ยทวไป ยทวไป heap sort heap sort heap sort heap sort กกกกนนนนเวลาประมาเวลาประมาเวลาประมาเวลาประมาณ ณ ณ ณ 2 2 2 2 เทาขอเทาขอเทาขอเทาของ ง ง ง quick sortquick sortquick sortquick sort• ในทางปฏบตพบวาชากวา ในทางปฏบตพบวาชากวา ในทางปฏบตพบวาชากวา ในทางปฏบตพบวาชากวา Shell Sort Shell Sort Shell Sort Shell Sort ทใช ทใช ทใช ทใช incremental sequence incremental sequence incremental sequence incremental sequence ของ ของ ของ ของ SedgewickSedgewickSedgewickSedgewick

Merge SortMerge SortMerge SortMerge Sort

1 13 24 26 2 15 27 38

Merging

a b

c

a1

temp Array

a1

temp Array


c

• How many comparisons to merge a & b to c ? ______________

• Worst case : _________________

• # comparisons nearly optimal : fine example of recursion• Divided-and-conquer

Merge SortMerge SortMerge SortMerge Sort

1 13 24 26 2 15 27 38

Merging

a b

c

a1

temp Array

a1

temp Array

mergeSort( a, tmpArray, 0, a.size( ) - 1 );


template <class T>

void mergeSort(vector<T>& a,vector<T>& tmpArray,

int left,int right){

if( left < right ) {

int center=(left+right)/2;

mergeSort(a,tmpArray,left,center);

mergeSort(a,tmpArray,center+1,right);

merge(a,tmpArray,left,center+1,right);

}

}

Quick SortQuick Sort

0 1 2 3 4 5 6 7 8 9

8 1 4 9 6 3 5 2 7 0

8 1 4 9 0 3 5 2 7 6

2 1 4 9 0 3 5 8 7 6

median of three median of three

(left, center, right)(left, center, right)leftleft rightrightcentercenter

pivotpivotii jj

jjii


Ordered ListOrdered List

2 1 4 5 0 3 9 8 7 6

2 1 4 5 0 3 6 8 7 9

0 1 2 3 4 5 6 7 8 9

ii jj

jj iiRight part < pivot (6) Left part > pivot (6)

Quick SortQuick Sort0 1 2 3 4 5 6 7 8 9

8 1 4 9 6 3 5 2 7 0

0 1 4 2 7 3 5 9 6 8

median of three(left,center,right) make data in 3 pos. order

place pivot in pos. right-1

median of three(left,center,right) make data in 3 pos. order

place pivot in pos. right-1

leftleft rightrightcentercenter

ii jj

jj

0 1 4 9 6 3 5 2 7 8

0 1 4 9 7 3 5 2 6 8pivotpivot

ii


Ordered ListOrdered List

0 1 4 2 5 3 7 9 6 8

0 1 4 2 5 3 6 9 7 8

0 1 2 3 4 5 6 7 8 9

i ji j

iijjRight part < pivot (6) Left part > pivot (6)

Quick Sort (Previous Algorithm)Quick Sort (Previous Algorithm)Quick Sort (Previous Algorithm)Quick Sort (Previous Algorithm)template <class T>

void quicksort(vector<T> & a,


if( left + 10 <= right ) {

T pivot = median3(a,left,right);

//make data in 3 pos. Order

//place pivot in pos. right - 1

//Begin partitioning

template <class T>

void quicksort(vector<T> & a,


if( left + 10 <= right ) {

T pivot = median3(a,left,right);

//make data in 3 pos. Order

//place pivot in pos. right - 1

//Begin partitioning

//restore pivot

swap(a[i],a[right-1]);

quicksort(a,left,i-1);

quicksort(a,i+1,right);

} //if

else

//restore pivot

swap(a[i],a[right-1]);

quicksort(a,left,i-1);

quicksort(a,i+1,right);

} //if

else


int i = left, j = right-1;

for( ; ; ){

while( a[++i] < pivot ) { }

while( pivot < a[--j] ) { }

if( i < j )

swap(a[i],a[j]);

else

break;

} //for

int i = left, j = right-1;

for( ; ; ){

while( a[++i] < pivot ) { }

while( pivot < a[--j] ) { }

if( i < j )

swap(a[i],a[j]);

else

break;

} //for

// sort small size data

insertionSort(a,

left,right);

}

// sort small size data

insertionSort(a,

left,right);

}

Quick Sort AnalysisQuick Sort AnalysisQuick Sort AnalysisQuick Sort Analysis

• แตละ pass compare n ครง ขนกบ pivot เปนอยางไร

• Best Case แบงได 1/2 ทกครง

• แตละ pass compare n ครง ขนกบ pivot เปนอยางไร

• Best Case แบงได 1/2 ทกครง

n

n/2 n/2

n/4 n/4 n/4 n/4

10050 50

1 1 1 . . . 1 1 1

20

21

2d

100 �� => 2d = n => d = log2n

• Worst Case = n + (n-1) + (n-2) + ... + 1 = O(n2)• Worst Case = n + (n-1) + (n-2) + ... + 1 = O(n2)


• Average Case ~ 1.386 n log2n• Average Case ~ 1.386 n log2n

n

n-1

• Worst Case = n + (n-1) + (n-2) + ... + 1 = O(n )• Worst Case = n + (n-1) + (n-2) + ... + 1 = O(n )

n-2

n--3

n

n-1

n-2

n--3

Quick SortQuick SortQuick SortQuick Sort

• best case , avg. case O(n log n) ?• worst case O(n2) เกดเมอไร ?• แมวาการ partition อาจไมได equal sizes เหมอน merge sort แตในทาง

ปฏบต พบวาเรวกวาเพราะ เปน in place partition• การเลอก pivot• first element : popular แตไมดเมอเปน presorted input

• best case , avg. case O(n log n) ?• worst case O(n2) เกดเมอไร ?• แมวาการ partition อาจไมได equal sizes เหมอน merge sort แตในทาง

ปฏบต พบวาเรวกวาเพราะ เปน in place partition• การเลอก pivot• first element : popular แตไมดเมอเปน presorted input


• first element : popular แตไมดเมอเปน presorted input• ตวทใหญกวาระหวาง first two distinct elements => พอกบแบบแรก• random ดเพราะเปนไปไมไดททกทในการ random จะไมด• คา average : ด แตเสยเวลามาก• median of group of n numbers = n/2 th largest number ==> แต

เสยเวลามาก• median of 3 values

• first element : popular แตไมดเมอเปน presorted input• ตวทใหญกวาระหวาง first two distinct elements => พอกบแบบแรก• random ดเพราะเปนไปไมไดททกทในการ random จะไมด• คา average : ด แตเสยเวลามาก• median of group of n numbers = n/2 th largest number ==> แต

เสยเวลามาก• median of 3 values

Documents

Sorting Ascending Order