INTRODUCTION

TO ALGORITHMS

A Creative Approach

UDIMANBER University of Arizona

• • • ADDISON-WESLEY PUBLISHING COMPANY Reading, Massachusetts • Menlo Park, California • New York Don Mills, Ontario • Wokingham, England • Amsterdam Bonn • Sydney • Singapore • Tokyo • Madrid • San Juan

CONTENTS

Chapter 1

Chapter 2

Chapter 3

Introduction l

Mathematical Induction 9

2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14

Introduction Three Simple Examples Counting Regions in the Plane A Simple Coloring Problem A More Complicated Summation Problem A Simple Inequality Euler's Formula A Problem in Graph Theory Gray Codes Finding Edge-Disjoint Paths in a Graph Arithmetic versus Geometrie Mean Theorem Loop Invariants: Converting a Decimal Number to Binary Common Errors Summary Bibliographie Notes and Further Reading Exercises

9 11 13 14 15 16 17 18 20 23 24 26 28 29 30 31

Analysis of Algorithms 37

3.1 3.2 3.3 3.4 3.5

3.6 3.7

Introduction The O Notation Time and Space Complexity Summations Recurrence Relations 3.5.1 Intelligent Guesses 3.5.2 Divide and Conquer Relations 3.5.3 Recurrence Relations with Füll History Useful Facts Summary Bibliographie Notes and Further Reading Exercises

37 39 42 43 46 47 50 51 53 55 55 56

IX

x Contents

Chapter 4 Data Structures 61

4.1 4.2

Introduction Elementary Data Structures 4.2.1 Elements

Arrays Records Linked Lists

4.3

4.2.2 4.2.3 4.2.4 Trees 4.3.1 4.3.2 4.3.3 4.3.4

4.4 4.5 4.6 4.7

Representation of Trees Heaps Binary Search Trees AVL Trees

Hashing The Union-Find Problem Graph s Summary Bibliographie Notes and Further Reading Exercises

Chapter 5 Design of Algorithms by Induction 91

5.1 Introduction 5.2 Evaluating Polynomials 5.3 Maximal Induced Subgraph 5.4 Finding One-to-One Mappings 5.5 The Celebrity Problem 5.6 A Divide-and-Conquer Algorithm: The Skyline Problem 5.7 Computing Balance Factors in Binary Trees 5.8 Finding the Maximum Consecutive Subsequence 5.9 Strengthening the Induction Hypothesis 5.10 Dynamic Programming: The Knapsack Problem 5.11 Common Errors 5.12 Summary

Bibliographie Notes and Further Reading Exercises

Chapter 6 Algorithms Involving Sequences and Sets 119

6.1 Introduction 6.2 Binary Search and Variations 6.3 Interpolation Search 6.4 Sorting

6.4.1 Bücket Sort and Radix Sort 6.4.2 Insertion Sort and Selection Sort 6.4.3 Mergesort

61 62 62 63 63 64 66 67 68 71 75 78 80 83 84 85 86

91 92 95 96 98 102 104 106 107 108 111 112 113 114

119 120 125 127 127 130 130

Contents xi

6.4.4 Quicksort 131 6.4.5 Heapsort 137 6.4.6 A Lower Bound for Sorting 141

6.5 Order Statistics 143 6.5.1 Maximum and Minimum Elements 143 6.5.2 Finding the M)-Smallest Element 144

6.6 Data Compression 145 6.7 String Matching 148 6.8 Sequence Comparisons 155 6.9 Probabilistic Algorithms 158

6.9.1 Random Numbers 160 6.9.2 A Coloring Problem 161 6.9.3 A Technique for Transforming Probabilistic

Algorithms into Deterministic Algorithms 161 6.10 Finding a Majority 164 6.11 Three Problems Exhibiting Interesting Proof Techniques 167

6.11.1 Longest Increasing Subsequence 167 6.11.2 Finding the Two Largest Elements in a Set 169 6.11.3 Computing the Mode of a Multiset 171

6.12 Summary 173 Bibliographie Notes and Further Reading 173 Exercises 175

Chapter7 Graph Algorithms 185

7.1 7.2 7.3

7.4 7.5 7.6 7.7 7.8 7.9

7.10

7.11 7.12

Introduction Eulerian Graphs Graph Traversais 7.3.1 Depth-First Search 7.3.2 Breadth-First Search Topological Sorting Single-Source Shortest Paths Minimum-Cost Spanning Trees All Shortest Paths Transitive Closure Decompositions of Graphs 7.9.1 Biconnected Components 7.9.2 Strongly Connected Components 7.9.3 Examples of the Use of Graph Decomposition Matching 7.10.1 Perfect Matching in Very Dense Graphs 7.10.2 Bipartite Matching Network Flows Hamiltonian Tours 7.12.1 Reversed Induction

185 187 189 190 198 199 201 208 212 214 217 217 226 230 234 ,234 235 238 243 244

xii Contents

7.12.2 Finding Hamiltonian Cycles in Very Dense Graphs 244 7.13 Summary 246

Bibliographie Notes and Further Reading 247 Exercises 248

Chapter 8 Geometrie Algorithms 265

8.1 8.2 8.3 8.4

8.5 8.6 8.7

Introduction 265 Determining Whether a Point Is Inside a Polygon 266 Constructing Simple Polygons 270 Convex Hulls 273 8.4.1 A Straightforward Approach 273 8.4.2 Gift Wrapping 274 8.4.3 Graham'sScan 275 Closest Pair 278 Intersections of Horizontal and Vertical Line Segments 281 Summary 285 Bibliographie Notes and Further Reading 286 Exercises 287

Chapter 9 Algebraic and Numeric Algorithms 293

9.1 9.2 9.3 9.4 9.5

9.6 9.7

Introduction Exponentiation Euclid's Algorithm Polynomial Multiplication Matrix Multiplication 9.5.1 Winograd's Algorithm 9.5.2 Strassen's Algorithm 9.5.3 Boolean Matrices The Fast Fourier Transform Summary Bibliographie Notes and Further Reading Exercises

Chapter 10 Reductions 321

293 294 297 298 301 301 301 304 309 316 316 317

10.1 Introduction 321 10.2 Examples of Reductions 323

10.2.1 A Simple String-Matching Problem 323 10.2.2 Systems of Distinct Representatives 323 10.2.3 A Reduction Involving Sequence Comparisons 324 10.2.4 Finding a Triangle in Undirected Graphs 325

10.3 Reductions Involving Linear Programming 327 10.3.1 Introduction and Definitions 327 10.3.2 Examples of Reductions to Linear Programming 329

Contents xiii

10.4 Reductions for Lower Bounds 331 10.4.1 A Lower Bound for Finding Simple Polygons 331 10.4.2 Simple Reductions Involving Matrices 333

10.5 Common Errors 334 10.6 Summary 336

Bibliographie Notes and Further Reading 336 Exercises 337

ChapterU NP-Completeness 341

11.1 Introduction 341 11.2 Polynomial-Time Reductions 342 11.3 Nondeterminism and Cook's Theorem 344 11.4 Examples of NP-Completeness Proofs 347

11.4.1 Vertex Cover 348 11.4.2 Dominating Set 348 11.4.3 3SAT 350 11.4.4 Clique 351 11.4.5 3-Coloring 352 11.4.6 General Observations 355 11.4.7 More NP-Complete Problems 356

11.5 Techniques For Dealing with NP-Complete Problems 357 11.5.1 Backtracking and Branch-and-Bound 358 11.5.2 Approximation Algorithms with Guaranteed

Performance 363 11.6 Summary 368

Bibliographie Notes and Further Reading 368 Exercises 370

Chapter12 Parallel Algorithms 375

12.1 Introduction 375 12.2 Models of Parallel Computation 376 12.3 Algorithms for Shared-Memory Machines 378

12.3.1 Parallel Addition 379 12.3.2 Maximum-Finding Algorithms 380 12.3.3 The Parallel-Prefix Problem 382 12.3.4 Finding Ranks in Linked Lists 385 12.3.5 The Euler's Tour Technique 387

12.4 Algorithms for Interconnection Networks 389 12.4.1 Sorting on an Array 390 12.4.2 Sorting Networks 393 12.4.3 Finding the Äth-Smallest Element on a Tree 396 12.4.4 Matrix Multiplication on the Mesh 398 12.4.5 Routing in a Hypercube 401

xiv Contents

12.5

12.6

References

Index

Systolic Computation 12.5.1 Matrix-Vector Multiplication 12.5.2 The Convolution Problem 12.5.3 Sequence Comparisons Summary Bibliographie Notes and Further Reading Exercises

tions to Selected Exercises 417

445

465

404 404 405 407 409 409 411