Lecture All

Elements of Programming

Alexander Stepanov Paul McJones

March 14, 2008

Stepanov, McJones Elements of Programming March 14, 2008 1 / 880

Contents I

1 Introduction

2 Foundations

3 Transformations and their orbits

4 Algorithms on algebraic structures

5 Orderings

6 Combining concepts

7 Refining concepts of iterators


Contents II

8 Permutations and rearrangements

9 Rotations

10 Algorithms on increasing ranges

11 Coordinate structures

12 Composite objects

13 Iterative algorithms for divide-and-conquer

14 Mathematical notation


Contents III

15 C++ machinery

16 Acknowledgments

17 Index


Approach

Programs are mathematical objectsMathematics provides tools for

reasoningderivingcomposingoptimizing

Mathematics applied to programming addressessecurityreliabilityperformanceproductivity


Blending programming and mathematics

1 Start with a practical algorithm2 Define mathematical requirements on which it depends3 Implement it abstractly and efficiently4 Place it in the general taxonomy of requirements and algorithms


Intended audience

This book is for those who seek deeper understanding ofprogramming and are willing to invest substantial effort workingthrough the materialThe book is not encyclopedic and is designed to be readsequentially, in its entirety1

While this book does not have formal prerequisites, programmingmaturity and a readiness to deal with serious mathematics areassumedWe present a unified approach to programming spanning from thehighest level of abstraction down to the machine

1Perhaps more than onceStepanov, McJones Elements of Programming March 14, 2008 7 / 880

The choice of C++

We use a small subset of C++ as a good approximation to the idealteaching languagePrograms in this book deal with both memory addresses andabstract mathematical theories

C defines an abstract machine that serves as a foundation formodern processor architectures2

C++ extends Cwith mechanisms for writing abstract versions ofalgorithms and data structures without sacrificing efficiency

2C does not address the issues of memory hierarchies, instruction-level parallelism,and thread-level parallelism; research is needed to incorporate these features into thesystem programming language of the future


The relationship to C++ STL

The book and C++ STL are results of the same researchThe book departs from it significantly by providing improvedinterfaces in a number of placesThe book does not attempt to provide a full set of libraryinterfaces or deal with all the issues a real library must address


The machine model

Our machine model is abstracted from C/C++Sequential (single-threaded) stacked-based executionA linear address space, not necessarily all accessibleMemory allocation and deallocation under programmer control

Our programs and techniques can be used as building blocks forsystems with concurrent execution and automatic storagedeallocation


C++ usage conventions

We avoid implicit conversions to make our code strongly typedwith respect to concepts 3

We do not use postincrement ++ nor do we depend on the returnvalue of preincrement ++We omit inline since its use depends on compiler, targetplatform, and the code

3Therefore all the constructors defined in the book should be considered to beexplicit


Our use of mathematics

We present only enough mathematics to provide the appropriatesettings for the programs in the bookWe strive for the level of rigor typical in modern mathematicaltextbooks, while attempting to preserve mathematical andalgorithmic intuitionsAll the lemmas in this book can be proved via straightforwardderivations from the provided definitionsWe urge the reader to prove every lemma to ensure truecomprehension


Outline of the book

Part I: Algorithms on mathematical abstractionsChapter 1 - FoundationsChapter 2 - Transformations and their orbitsChapter 3 - Algorithms on algebraic structuresChapter 4 - OrderingsChapter 5 - Combining concepts

Part II: Algorithms on abstractions of memoryChapter 6 - Refining concepts of iteratorsChapter 7 - Permutations and rearrangementsChapter 8 - RotationsChapter 9 - Algorithms on increasing rangesChapter 10 - Coordinate structuresChapter 11 - Composite objectsChapter 12 - Iterative algorithms for divide-and-conquer

Appendix 1 - Mathematical notationAppendix 2 - C++ machinery


Contents I

1 Introduction

2 FoundationsMeaningObjectsRelations on objectsProceduresAssignment, destruction, and constructionRelationships between typesConcept RegularProcedures as objectsGeneralized proceduresParameter passingRegular functions


Contents II

Complexity



5 Orderings





Contents III

9 Rotations






15 C++ machinery


Contents IV

16 Acknowledgments

17 Index


Things, ideas, and descriptions

Humans observe and interact with real thingsWe observe things: our minds form ideas (mental images) ofthem; we act: our ideas affect things or create new thingsWe create descriptions of our ideas to preserve them,communicate them, or allow them to participate in manual andautomatic processesNote that a description is itself a thing


Universe of ideas

A value is an abstract idea (logical or mathematical) such as falseor 2 or a geometrical pointAn entity is an idea of a thing such as a car or an email message

An entity has attributes with values, such as job level or salary of anemployeeAn entity can be composed of other entities called parts, such as theengine of a car or the body of an email messageAn entity can be in relationships with other entities, such as thesupervisor of an employee

An activity is an idea of changes performed on entities thatchanges their attributes or creates new entities, such as promotinga given employeeAn abstraction is what is common to a class of entities

An abstraction of values, such as IntegerAn abstraction of entities, such as EmployeeAn abstraction of activities, such as Promotion


Entities

When we form an idea of an entity, we grasp:Its existenceIts natureActivities affecting itIts particulars: attributes, parts, and relationships

In computer programsExistence corresponds to object identityNature corresponds to object typeActivities correspond to proceduresParticulars correspond to object state, object components, andobject relationships


Values

Similar values (natural numbers, real numbers, polygons, etc.)belong to different sorts


The universality of mathematics

“Mathematics, in general, is the science of quantity; or, the sciencewhich investigates the means of measuring quantity” 4

Mathematical reasoning, including discrete arithmetic reasoningand continuous spatial reasoning, is a fundamental, innate abilityof the human mind

4Leonard Euler. Elements of Algebra. Third Edition, Longman, London, 1822.Stepanov, McJones Elements of Programming March 14, 2008 22 / 880

Continuity and discreteness

Ancient mathematicians (the early Pythagoreans) discoveredspace can’t be represented using discrete quantitiesThe arithmetization of geometry required the introduction of thereal numbersThe continuity of physical processes allows us to effectivelyapproximate real numbers with discrete, finite-precision quantitiesScientists and engineers have encountered no problem wherecareful approximation has failed to workThis is the key to the applicability of digital computers


Computer descriptions of entities

As we said earlier, we understand the world in terms of entitiesComputer programmers map these entities into objects containingdiscrete, bounded quantities approximating the values of theattributesThe meaning of a program is in the mind, not in the computer


Objects

DefinitionAn object is a set of memory locations and other resources togetherwith primitive procedures defined on it that provide itscomputational basisThe state of an object is the addresses of its memory locationstogether with the content of those locations at a particular point intimeThe interpretation of an object is a mapping from its state to anentity

An interpretation maps states in such a way that procedures on theobject are consistent with the corresponding activities on entities


Well-formed object states

DefinitionAn object is in a well-formed state if there is an entity corresponding to it

ExamplesEvery state of a computer word is well-formed when interpretedas an integerA pointer containing an illegal address is not well-formedAn IEEE 754 floating point NaN (Not a Number) is notwell-formed when interpreted as a real number


Partial and total representations

DefinitionA representation for an entity of a given nature is an object statecorresponding to it; the entity is representable if such a state exists

DefinitionA representation is partial if not every entity is representable by anobject state; a representation is total if it is not partial

ExampleThe type int is a partial representation of integers


Unique representations

DefinitionA representation is unique if there is at most one object statecorresponding to every set of particulars of the entity

ExamplesA representation of a truth value as a byte that interprets zero asfalse and nonzero as true is not uniqueSigned-magnitude representation does not provide a uniquerepresentation of zero


Identity

In the real world, identity is conceptually straightforward, butoften slippery to deal with in specific situations

It was a major astronomical discovery that the Morning Star andEvening Star are identicalThe question of whether the author of Hamlet and WilliamShakespeare identical is still debatedWhen does replacing parts change the identity?

A computer programmer must determine how to map the identityof real things to objectsMultiple techniques are possible for representing object identity

A memory addressAn index into a data structureA unique value of an attribute (e.g., Social Security Number)


Equality

Leibniz’s lawx = y if, and only if, everything that may be said about any one of thethings x or ymay also be said about the other a

aAlfred Tarski, Introduction to Logic and to the Methodology of Science, Oxford, 1965


Properties of equality

Formalization of Leibniz’s Law:

x = y⇔ ∀P(P(x)⇔ P(y))

The following properties follow:reflexivity x = x

symmetry (x = y)⇔ (y = x)

transitivity ((x = y) ∧ (y = z))⇒ (x = z)


Equality in mathematics

Equational reasoning is the basis of mathematics

Arithmetic: 24 = 1

2Geometry: 4ABC = 4ACB

Algebra: (a+ b)2 = a2 + 2ab+ b2


Equality in the real world

Equality generally implies equality of essential attributesNot all attributes are necessarily essentialEssentialness of attributes often depends on context

All US quarters are equal to a fare collector, but not to a coincollector

Equality is not the same as identityA thing’s attributes can change without affecting its identityDistinct things can have equal attributes and, therefore, be equalthemselves


Equality in programming

This book extends Leibniz’s law from unchanging values tomutable software objectsLeibniz’s law is the fundamental property of equality, but is not aconstructive definition

It establishes the consequences of the equality of two integersIt does not describe how to compare them for equality

Developing constructive implementations of equality is one of thetasks of this book


Genus

DefinitionTwo objects are of the same genus if they represent entities with thesame nature under their corresponding interpretations

ExamplesPolar and Cartesian representations of a complex number are ofthe same genusTwo’s-complement and signed-magnitude representations ofintegers are of the same genus16-bit and 32-bit unsigned representations of natural numbers areof the same genus


Type

DefinitionTwo objects have the same type if they share the same procedures andthe same interpretation


Intuition for type

The English word type derives from the Greek word tupoc (tupos):“the impress made by the blow, what is formed, what leaves itsimpress, the form-giving form, hence form generally as outline”5

The technical sense of tupos was “. . . the pattern in conformity towhich a thing must be made” 6

In programming, type is a fundamental idea preceding any notionof type in programming languagesA type is a design for objects, as an engineering drawing is adesign for manufactured objects

5Kittel and Friedrich. Theological Dictionary of the New Testament, Volume 8,Eerdmans, 1972, page 246

6Thayer and Smith. “Greek Lexicon entry for Tupos”. “The New Testament GreekLexicon”


Equality of objects

DefinitionTwo objects of the same genus are equal if their states represent entitieswith equal essential particulars

Like Leibniz’s Law, this is not a constructive definition; it,however, allows us to apply the mathematical understanding ofequality on abstract values to the world of programming


Nonessential objects

DefinitionA nonessential object is one whose value does not affect thecomputation

ExampleAn object used only to instrument or tune the performance isnonessential


How procedures interact with objects

A procedure interacts with the state of the computation throughfour kinds of objects:

Arguments are objects given to a procedure at its point of invocationLocal state is objects created and accessed during a single invocationof the procedureGlobal state is objects accessible to this and other procedures acrossmultiple invocationsOwn state is objects accessible only to this procedure but sharedacross multiple invocations


Domain, argument types, and definition space

In mathematics, a function typically takes a single argument, or anoperation such as addition takes two argumentsIn programming, it is common for a procedure to take manyargumentsRather than defining the domain of a procedure as the directproduct of the types of its arguments, we refer to the argumenttypes of a procedureIn the next chapter we extend the definition of domain tohomogeneous functions (all of whose arguments are of the sametypes)

DefinitionThe definition space for a procedure is that subset of the direct productof its argument types to which the procedure is intended to be applied


Codomain and result space

DefinitionThe codomain for a procedure is the type it returns

DefinitionThe result space for a procedure is the set of values from its codomainreturned by the procedure for some inputs from its definition space


Total and partial procedures

DefinitionA procedure is partial if its definition space is a subset of the directproduct of the types of its arguments; it is total if they are equala

aIn mathematical usage, “partial” includes “total”; for example, total orderingimplies partial ordering

There is a (potentially implicit) predicate, is_defined, that is truewhen applied to a procedure and set of arguments lying withinthe definition space of that procedurePreconditions are used to specify the definition spaces forproceduresBefore a partial procedure is called, either its precondition must besatisfied, or the call must be guarded by a call on is_defined7

7For example of the former, seeweak_remainder in Chapter 5, and for an exampleof the latter, see collision_point in Chapter 2


Terminating and non-terminating procedures

DefinitionA procedure is terminating, semi-terminating, or non-terminatingdepending on whether it always terminates, sometimes terminates, ornever terminates on its definition space

A partial procedure may terminate even for arguments outside itsdefinition spaceAll the procedures in this book are terminating unless explicitlystated otherwiseA terminating procedure is known as an algorithm


Stateful and indexed procedures

DefinitionA procedure is stateful, indexed, or stateless depending on whether itsown state is mutable, immutable, or empty


Proper and improper procedures

DefinitionA procedure is proper if it does not modify any essential objects in itsglobal state; otherwise it is improper

All the procedures in this book are proper


Functions and mutators

DefinitionA function is a procedure that does not modify its arguments andreturns a newly constructed object called its result; a mutator is aprocedure whose main purpose is to modify its arguments


Procedure examples

Addition is a total function on integers, but is partial on intDivision is a partial function on integers since its definition spaceomits zero as a value for its second argumentA daemon is a non-terminating procedureThe Knuth-Bendix procedure in automatic theorem proving issemi-terminatingSorting is a terminating proper mutatorA pseudo-random number generator that keeps its state in an ownvariable is a stateful proper functionMultiplication mod k for a fixed k is an indexed proper functionwith k being its indexSetting a global graphics state variable is an improper mutator


Assignment

DefinitionAn assignment is a mutator taking two objects of the same genus whoseeffect is to make the first object equal to the second without modifyingthe second

The final interpretation of the first object must equal theinterpretation of the second object


Destruction

DefinitionA destructor is a mutator causing the cessation of an object’s existence

After a destructor has been invoked on an object, no procedurecan be applied to it and its former memory locations andresources may be reused for other purposes


Partially-formed state

DefinitionAn object is in a partially-formed state if it can be assigned to ordestroyed

For an object that is partially-formed but not well-formed, theeffect of any procedure other than assignment (only on the leftside) and destruction is not defined


Construction

DefinitionA constructor is a mutator transforming memory locations into anobject

The possible behaviors range from doing nothing to establishingcomplex object state


Type constructors

DefinitionA type constructor is a mechanism for creating a new type from one ormore existing types

ExamplesIn C++, T* is the type “pointer to T”, for any type TIn C++, struct{T0, ..., Tn−1} is an n-ary type constructorIn Chapter 10 we show how to construct types corresponding todata structures such as lists and arrays


Type attributes

DefinitionA type attribute is a mapping from a type to a value describing somecharacteristic of the type

ExamplesThe size of an object in bytesThe alignment of an objectThe number of members in a struct

DefinitionIf F is a procedure type, Arity(F) returns its number of arguments


Type functions

DefinitionA type function is a mapping from a type to an affiliated type

ExamplesGiven “pointer to T”, the value type TThe result type of the difference of two pointers of a given typeThe type of the ith member of a struct type (counting from 0)

DefinitionIf F is a procedure type and i < Arity(F), ArgumentType(F, i)returns the type of the ith argument (counting from 0)If F is a procedure type, Codomain(F) returns the type of theresult

In Appendix 2 we show how to define some type functions in C++


Concepts

DefinitionA concept is a predicate on types stated in terms of properties ofprocedures, type attributes, and type functions defined on the types

DefinitionA concept is said to be modeled by specific types, or the types modelthe concept, if the properties of the concept are satisfied for thosetypesTo assert that a concept C is modeled by types t0, . . . , tn−1, wewrite C(t0, . . . , tn−1)


Refinement and weakening of concepts

DefinitionConcept C ′ refines concept C if whenever C ′ is satisfied for a set oftypes, C is also satisfied for those types a

We say C weakens C ′ if C ′ refines C

aTechnically, it is sufficient for C to hold for a subset of the types, possibly reordered


Type concepts

DefinitionA type concept is a concept defined on one type


Examples of type concepts in C++

ExamplesIntegral type

Unsigned integral typeModeled by: unsigned, unsigned long, unsigned char

Signed integral typeModeled by: int, long, char

Pointer typeModeled by: int*, char*

SequenceModeled by: std::vector<int>, std::list<int>

Bidirectional iteratorModeled by: std::list<int>::iterator, int*


Primitive type concepts

ProcedureProperProcedureFunction


Concept HomogeneousFunction

DefinitionHomogeneousFunction(F)⇒

Function(F)

Arity(F) > 0For all i, j such that 0 6 i, j < Arity(F),ArgumentType(F, i) = ArgumentType(F, j)

DefinitionThe type function Domain(F) is defined on any function type Fsatisfying HomogeneousFunction(F):

Domain(F) ≡ ArgumentType(F, 0)


Requirements for interoperability of components

We want a sort procedure that works on an array of elementsThe requirements on the element type allowing objects to besorted should be satisfied by arrays themselves, so we can sort anarray of arraysAll built-in types (e.g., int, float) should satisfy these commonrequirementsThe requirements are based on equational reasoningSince the requirements assure regularity of behavior andinteroperability, we call the type concept corresponding to themregular and the types modeling it regular types 8

8Regular types were first introduced in:

James C. Dehnert and Alexander A. Stepanov.Fundamentals of Generic Programming.Report of the Dagstuhl Seminar on Generic Programming, Schloss Dagstuhl,Germany, April 1998; also appears in Lecture Notes in Computer Science (LNCS)volume 1766, pages 1-11.


Concept Regular

A Regular type has9:EqualityAssignmentDestructorDefault constructorCopy constructorDefault total ordering (defined in Chapter 4)

9In Chapter 12 we extend Regular with the notion of underlying type and itsaffiliated types and functions


Equality for Regular

A regular type must implement equalityEquality should be defined on well-formed object states; twoobjects are equal if and only if they represent the same abstractvalueEquality is written as a = b and inequality is written as a , b


Assignment for Regular

A regular type must implement assignmentAssignment is written as a← b;


Destructor for Regular

A regular type must implement a destructor


Default constructor for Regular

A regular type must implement a default constructorThis is a constructor that takes no arguments and leaves the objectin a partially-formed stateThe default construction for a local object a of type T is written T a;


Copy constructor for Regular

A regular type must implement a copy constructorThis is a constructor that takes an additional argument of the sametype and constructs a new object equal to itThe effect of the copy constructor is equivalent to defaultconstruction followed by assignmentThe copy constructor for a local object a of type T with initialvalue b is written T a← b;


Additional functions and affiliated types for Regular

In later chapters we introduce additional functions and affiliatedtypes required for a Regular type


Procedures as objects

A procedure is itself an object, and thus can be constructed,destructed, assigned, copied, and tested for equalityThese operations on a procedure are all operations on its own stateTwo procedures of different type that perform the samecomputation belong to the same genus

For example, heapsort and quicksort


Generalized procedures

DefinitionA generalized procedure is a procedure defined in terms of typerequirements

Each type requirement has a formal parameter name, introducedvia a typename clause following the template keywordRequirements on the types are specified via the requires clause,whose argument is an expression built up from actual types, formaltypes, applications of type functions, type equality, concepts, andtheir logical connectivesa

aIn this book we use only conjunction


Syntax of type expressions and requires clauses

<type expression> ::= <type> | <formal type> |<type function> ( <type expression> )

<type predicate> ::= <type expression> == <type expression> |<type concept> ( <type expression )

<requires expression> ::= <type predicate> |<requires expression> && <type predicate>

<requires clause> ::= requires( <requires expression> )

A <type expression> can be used to declare procedurearguments and local variablesOur requires clause is implemented in Appendix 2 with a simplemacro10

10In the future C++ is expected to include support for concepts with different syntaxStepanov, McJones Elements of Programming March 14, 2008 72 / 880

Example of a generalized procedure

template <typename Op>requires(SemigroupOperation(Op))

Domain(Op) square(const Domain(Op)& x){

return op(x, x);}


Passing parameters to procedures

We distinguish three cases of how a procedure uses an objectpassed to it as a parameter:For reading The procedure depends only on the initial value of

the object, not on its identityFor writing The procedure mutates the object

For selecting The procedure selects one of several objects(without mutating any of them) and returns theselected object (not a copy)


Passing parameters for reading

To pass a parameter for reading in C++, we use one of twotechniques

If the size of the parameter is small, we pass it by value (whichmakes a local copy)Otherwise, we pass it by const referenceIf the procedure needs a local copy it can mutate, we pass it byvalue even when it is large

template <typename I>requires(Readable(I) && Iterator(I))

I find(I f, I l, const ValueType(I)& x){

while (f != l && source(f) != x)++f;

return f;}

Regular types allow us to use call by value: there is a copyconstructor, and it creates a copy equal to the original


Passing parameters for writing

To pass a parameter for writing in C++, we pass it by reference

template <typename T>requires(Regular(T))

void swap(T& x, T& y){

T tmp = x;x = y;y = tmp;

}

(In a later chapter we show a more efficient way to write swap)


Passing parameters for selecting

To pass a parameter for selecting in C++, we must provide bothby reference and by const reference versions of the procedure:

template <typename T>requires(StrictTotallyOrdered(T))

T& min(T& a, T& b){

if (b < a) return b;else return a;

}


const T& min(const T& a, const T& b){

if (b < a) return b;else return a;

}


More on selecting

A function such as min can be used on the left side of anassignment when all its parameters are writable

Increase the smaller of two variables by 100: min(a, b) += 100;

Note the return type of the two versions of the procedure mustagree with the selecting parameters with respect to constnessThe reasons why C++ requires both versions lie outside the scopeof this book(In this book we often show only the non-const version)


Concept RegularFunction

DefinitionRegularFunction(F)⇒

Function(F)

Regular(F)For all f,g ∈ F such that f = g and for all 0 6 i < n:

Regular(ArgumentType(F, i))For all xi,yi ∈ ArgumentType(F, i) such that xi = yi,

f(x0, . . . , xn−1) = g(y0, . . . ,yn−1)

where n = Arity(F)

By extension, we call a generalized function regular if all of itsinstantiations are regular


Reasons for non-regular functions

In mathematics, all functions are regular as a consequence ofLeibniz’s LawIn programming, there are situations when it’s not so

When a function returns the address of a memory locationThe “address of” operator (& in C++)

When the function returns a value determined by the state of thereal world

Input from a device or another processWhen a stateful function returns a value depending on its own state

A pseudo-random number generatorWhen a function returns a nonessential attribute of an object

Amount of preallocated memory for a data structure


Program optimization

Regular types and regular functions allow us to many standardtransformations, such as:

Common subexpression eliminationLoop hoistingCopy propagation

ExampleIf f and g are regular, we can optimize f(g(x),g(x)) by evaluating g(x)only once


Area of an object

The area of an object is the total number of bytes it occupies11

A type is of constant area if every object of that type has the sameareaA type is of fixed area if every object of that type has an an area thatdoes not change during its lifetimeA type is of dynamic area if the area of an object of the type canvary during the lifetime of the objectFor every type, there is a (potentially implicit) function, area, thatreturns the area of an object of that type

11In a binary computer we should in principle measure areas in bits, but unless weused packed representations we deal in bytes


Storage efficiency

Definition

The byte storage efficiency of a fixed-area object x is dlog256s(x)earea(x) , where

s(x) is the number of distinct states of x

ExampleThe byte storage efficiency of struct{bool} is 1The byte storage efficiency of struct{bool,bool} is 1/2The byte storage efficiency of struct{bool,bool,bool} is 1/3


Cost of a procedure call

The cost of a procedure call is the time required for its executionThe worst-case cost of a procedure call for arguments of given areasis the maximum cost across all procedure calls with arguments ofthese areasThe average cost of a procedure call for arguments of given areas isthe average cost across all procedure calls with arguments of theseareasA procedure has uniform cost if the cost of a procedure calldepends only on the areas of its argumentsA procedure has fixed cost if the cost of a procedure call does notdepend on its argumentsThere is a (possibly implicit) function, cost, that returns the costwhen applied to a procedure and set of arguments lying withinthe definition space of that procedure


Contents I

1 Introduction

2 Foundations

3 Transformations and their orbitsTransformationsOrbitsApplicationsExamplesConclusionsProject



Contents II

5 Orderings




9 Rotations




Contents III




15 C++ machinery

16 Acknowledgments

17 Index


Concept Operation

DefinitionOperation(Op)⇒

RegularFunction(Op)

HomogeneousFunction(Op)

Codomain(Op) = Domain(Op)

ExampleUnary abs : double→ double

Binary + : double× double→ double

Ternary multiply_add : double× double× double→ double


Concept Transformation

DefinitionTransformation(F)⇒

Operation(F)

Arity(F) = 1The predicate is_defined : F×Domain(F)→ bool is defined

ExampleNumbers Square root

Geometry Plane rotations; space translationsCombinatorics Reversal of a sequence


Composition and reachability

Transformations are self-composablef(x), f(f(x)), . . .

Self-composability allows us to define iterative algorithms

Definitionf0(x) = x

fn+1(x) = f(fn(x))

Definitiony is reachable from x under a transformation f if y = x or there is a zreachable from x and f(z) = y


Cyclic and terminal elements

Definitionx is cyclic under f if f(x) is defined and x is reachable from f(x)

Definitionx is terminal under f if and only if f is not defined at x


Orbits

DefinitionsAn orbit of x under a transformation f is the set of all elementsreachable from x under f

LemmaAn orbit does not contain both a cyclic and a terminal element

LemmaAn orbit contains at most one terminal element


Classification of orbits

DefinitionsAn orbit of x under f is:

infinite if it has no cyclic or terminal elementsterminating if it has a terminal element

circular if x is cyclicρ-shaped if x is not cyclic and its orbit contains a cyclic element

An orbit of x is finite if it is not infinite


Finite orbits

terminating

circular

ρ-shaped


Structure of orbits

DefinitionThe orbit cycle is the set of cyclic elements in the orbit

DefinitionThe orbit handle is the complement of the orbit cycle with respect to theorbit

DefinitionThe connection point is the first cyclic element


Sizes

DefinitionsThe orbit size o of an orbit is the number of distinct elements in itThe handle size h of an orbit is the number of elements in the orbithandleThe cycle size c of an orbit is the number of elements in the orbitcycle

Lemma

o = h+ c


Finite orbit assumption

Finiteness of a transformation is equivalent to the halting problemThus no algorithm can determine whether an orbit is finite orinfinite for an arbitrary transformationFor many transformations, finiteness is easily provableThere is an implicit precondition of orbit finiteness for all thealgorithms in this chapter


Algorithmic intuition for cycle detection

If two cars, a fast car and a slow car, start along a path, the fast carwill catch up with the slow car if and only if there is a cycle

If there is no cycle, the fast car will reach the end of the path beforethe slow carIf there is a cycle, by the time the slow car enters the cycle, the fastcar will already be there, and will catch up eventually


Collision point

DefinitionThe collision point of a transformation f and a starting point x is theunique y such that

y = fn(x) = f2n+1(x)

and n > 0 is the smallest integer satisfying this condition


collision_point

template <typename F>requires(Transformation(F))

Domain(F) collision_point(const Domain(F)& x, F f){

Domain(F) fast = x;Domain(F) slow = x; // n← 0 (completed iterations)while (true) { // slow = fn(x) ∧ fast = f2n(x)

if (!is_defined(f, fast)) break;

fast = f(fast); // slow = fn(x) ∧ fast = f2n+1(x)A: if (fast == slow) break;

if (!is_defined(f, fast)) break;

fast = f(fast); // slow = fn(x) ∧ fast = f2n+2(x)

slow = f(slow); // slow = fn+1(x) ∧ fast = f2n+2(x)// n← n+ 1

}return fast;// Postcondition: fast is either the terminal point or the collision point of f and x

}


Proof of termination of collision_point

The movement of fast is guarded by a call of is_definedThe movement of slow is unguarded, because by the regularity off, slowwill be traversing the same orbit as fastIf there is no cycle, is_definedwill eventually return false becauseof finitenessIf there is a cycle, slow will eventually reach the connection point(the first element in the cycle):

Consider the distance d from fast to slow at the point labeled Aonce slow enters the cycle

0 6 d < c

If d = 0 the procedure terminatesOtherwise the distance decreases by one on each iteration

By induction, the procedure always terminatesWhen it terminates, slow has moved a total of h+ d steps


Position of collision_pointwhen a cycle exists

The annotations show that when there is a cycle and n > 0 is thenumber of completed iterations, then fn(x) = f2n+1(x)

n is the smallest such integer since we checked the condition forevery i < nWe express n = h+ dwhere d is the distance slowmoves afterreaching the connection pointTherefore h+ d+ qc = 2h+ 2d+ 1 for some q > 0 that counts thenumber of cycles completed by fast when slow enters the cycleSimplifying gives qc = h+ d+ 1Represent h in terms of its quotient and remainder when dividedby c: h = mc+ r for 0 6 r < c

Substitution gives qc = mc+ r+ d+ 1 or d = (q−m)c− r− 10 6 d < c implies q−m = 1, so d = c− r− 1The distance from the collision point to the connection point ise = c− d = r+ 1Stepanov, McJones Elements of Programming March 14, 2008 102 / 880

Distinguishing circular from ρ-shaped case

As a corollary of e = r+ 1, it follows that in case of a circular orbith = 0, which implies r = 0 and so e = 1So the distance from the collision point to the beginning of theorbit is equal to 1Circularity therefore can be checked with the following simpleprocedure (we leave implementation of similar checks forterminating and ρ-shaped to the reader)


is_circular


bool is_circular(const Domain(F)& x, F f){

Domain(F) y = collision_point(x, f);return is_defined(f, y) && x == f(y);

}


Finding connection point

Let xw = xh+c−r−1+1 = xh+c−r be the element one past thecollision pointThe element h steps beyond xw is xw+h = xh+c−r+h

Substituting h = mc+ r gives xh+c−r+mc+r = xh+(m+1)c = xhsince it ism+ 1 times around the cycle from xh

This suggests an algorithm for determining xh, the connectionpoint


connection_point


Domain(F) convergent_point(Domain(F) x, Domain(F) y, F f){

// Precondition: the orbits of x and y convergewhile (x != y) {

x = f(x);y = f(y);

}return x;

}


Domain(F) connection_point(const Domain(F)& x, F f){

Domain(F) y = collision_point(x, f);if (!is_defined(f, y)) return y;return convergent_point(x, f(y), f);

}


Complexity

The complexity analysis of these algorithms trivially follows fromthe termination proofs and is left as an exercise for the reader


Applications

Determining whether a linked structure contains a cycleSee for example the Common Lisp function list-length

Determining the length and the period of a random numbergenerator – see for example:

Donald Knuth.Exercise 3.1.6.The Art of Computer Programming, Volume 2, 3rd edition, 1998,page 7.Credits two-pointer orbit detection to R.W. Floyd.

Our code works for both cases


Problem: representing sizes

What type to use for o, h, and c?We need to define a type function to obtain an integer type bigenough to encode the orbit size for a given type T


Definition of distance type

DefinitionThe distance type for a type T is an integer type a that allows us toencode the maximum number of transformations from oneelement of T into anotherIf a type occupies k bits, its distance type can be represented withan unsigned integer type occupying k bitsThis avoids an infinite tower of types(It is difficult to have a count type that could count the number ofelements in any collection of type T , because that would requirean extra value)The type function DistanceType(T) returns the distance type ofT ; for the implementation in C++, see Appendix 2

aWe will discuss integer types briefly in the next chapter and in depth in Chapter 4


distance


DistanceType(Domain(F)) distance(const Domain(F)& x, const Domain(F)& y, F f){

// Precondition: y is reachable from x under f

typedef DistanceType(Domain(F)) D;Domain(F) z = x;D n = D(0);while (z != y) {

z = f(z);n = n + D(1);

}return n;

}


Representing sizes

Note that o, h, and cmay not fit in the distance type

But each of these does fit:

o− 1h− 1c− 1

For non-terminating cases, h fits also


orbit_structure


triple<DistanceType(Domain(F)), DistanceType(Domain(F)), Domain(F)>orbit_structure(const Domain(F)& x, F f){

typedef DistanceType(Domain(F)) D;Domain(F) y = connection_point(x, f);D m = distance(x, y, f);D n(0);if (is_defined(f, y))

n = distance(f(y), y, f);// Terminating: m = h− 1 ∧n = 0// Otherwise: m = h∧n = c− 1return triple<D, D, Domain(F)>(m, n, y);

}


Postcondition of orbit_structure

Case first second third

terminating h− 1 0 terminalcircular 0 c− 1 x

ρ-shaped h c− 1 connection


Testing random number generators

ExerciseGiven the C++ function object

struct random_function{

int operator()(int x){

srand(x);return rand();

}};

use orbit_structure on enough different initial elements to decidewhether the cycle length generated by std::rand on your platform issatisfactory


Conclusions

Practical algorithms can be described using basic mathematicaltheoriesNomenclature helps

e.g., orbit kinds and sizes

Equational reasoning on types and functions is essentialAbstract code facilitates reasoning and complexity analysis


Project

There are other algorithms for orbit analysis:

R.T. Sedgwick, T.G. Szymanski and A.C. Yao.The complexity of finding cycles in periodic functions.Proc. 11th SIGACT Meeting, 1979, pages 376-390.

Richard P. Brent.An improved Monte Carlo factorization algorithm.BIT, Volume 20, 1980, pages 176-184.

Leon S. Levy.An improved list-searching algorithm.Information Processing Letters, Volume 15, Issue 1, August 1982,pages 43-45.

Write generic versions of these algorithms and compare theirefficiencies


Contents I

1 Introduction

2 Foundations


4 Algorithms on algebraic structuresAlgebra and abstractionPower on semigroups, monoids, and groupsSlow power algorithmStepwise derivation of fast power algorithmConcept for the exponentComplexityApplications


Contents II

OverloadingConclusionsReferenceProjects

5 Orderings




9 Rotations


Contents III






15 C++ machinery

16 Acknowledgments


Contents IV

17 Index


Elementary algebra

Since its inception by the Arabs in the ninth century, algebra hasbeen viewed as the theory of operations on numbers irrespectiveof their particular kinds (such as integers or rationals) or valuesThe common algebraic operations are addition, subtraction,multiplication, and divisionThese operations possess fundamental properties such asassociativity of addition or distributivity of multiplication overaddition


Higher algebra

Through the eighteenth and nineteenth century algebra wasextended to include aggregates of numbers (such as matrices andpolynomials) together with the common algebraic operations onthemThe operations on aggregates turned out to satisfy the sameproperties as the operations on numbers


Precursors of abstract algebra

By the late nineteenth century mathematicians (Jordan, Klein, Lie,etc.) observed that a cluster of properties called a group (namelyan associative, invertible binary operation) appears in severaldifferent domains (such as theory of equations and geometry)At a slightly later time, mathematicians (Dedekind, Weber andHilbert) observed that a cluster of properties called a ringconnecting addition and multiplication also appears in multiplecontextsBy the early twentieth century, people learned to recognizegroups, rings, vector spaces, and other similar clusters ofproperties


Abstract algebra

In the 1920s a fundamentally new point of view on algebra wasdeveloped by several German mathematicians (Noether, Artin,etc.)The new approach was to take a well-understood cluster ofproperties (such as a group) as axioms, and explore theirconsequences independently of any underlying settingAs a result, the same algebraic techniques can be used in anysetting where these axioms are satisfied, leading to effectiveapplications in domains ranging from quantum mechanics tocrystallographyComputer science has found specialized use of these techniques inareas such as automata theory and coding theory


Abstract mathematics

As with abstract algebra, from 1930 until about 1970 a largeportion of mathematics was reinterpreted putting classical resultsin their most abstract settingsThis process led to

an increase in soundnessreuse of techniques across different domains

Many of the known theories from abstract mathematics(especially algebra) provide a natural setting for practicalalgorithms, with a corresponding increase in soundness and reuse


Abstraction

By the time of Aristotle—if not even earlier—it had been recognizedthat every formal discipline rested on what might be called“the principle of voluntary denial of complete knowledge”:abstraction or generalization literally indicates a systematicdiscarding of certain aspects of the studied objects. 12

12Jean Dieudonné. Linear Algebra and Geometry. Herman, Paris, 1969, page 17Stepanov, McJones Elements of Programming March 14, 2008 127 / 880

Grounding abstraction

Concentrating only on abstractions could lead to forgetting theunderlying realities and the problems to be solvedAbstraction in programming should always start with the bestexisting and practically useful algorithms and data structures


Concept BinaryOperation

DefinitionBinaryOperation(Op)⇒

Operation(Op)

Arity(Op) = 2


Concept SemigroupOperation

DefinitionSemigroupOperation(Op)⇒

BinaryOperation(Op)

For all op ∈ Op and for all a,b, c ∈ Domain(Op):

op(a,op(b, c)) = op(op(a,b), c)

The property is associativityA common convention is to represent a semigroup operation asthe infix operator ◦; for example: a ◦ (b ◦ c) = (a ◦ b) ◦ c


Powers

a2 = a ◦ aa3 = a ◦ a ◦ a

a1 = a

an = a ◦ a ◦ · · · ◦ a︸︷︷︸n

an ◦ am = an+m


Properties of powers

Lemmaan ◦ am = am ◦ an = an+m (powers of the same element commute)

Lemma(an)m = anm


Frobenius’s theorem

DefinitionAn element x has finite order under a semigroup operation if there existintegers 0 < n < m such that xn = xm

DefinitionAn element x is an idempotent element under a semigroup operation ifx = x2

TheoremAn element of finite order has an idempotent power a

aGeorg Ferdinand Frobenius. Über endliche Gruppen. Berlin 1895. In:Sitzungesberichte der Königlich Preussischen Akademie der Wissenschaften zu Berlin.Phys.-math. Classe 1895, pages 163-194


Proof of Frobenius’s theorem


struct multiply_transformation{

Domain(Op) x;Op op;multiply_transformation(Domain(Op) x, Op op) : x(x), op(op) {}Domain(Op) operator()(const Domain(Op)& y){

return op(y, x);}

};


Domain(Op) idempotent_power(Domain(Op) x, Op op){

return collision_point(x, multiply_transformation<Op>(x, op));}


Explanation of proof of Frobenius’s theorem

Assume x is an element of finite order under a semigroupoperation opLet

g(z) = multiply_transformation(op, x) = op(z, x) = z ◦ x

Let y = idempotent_power(x,op)Since x is an element of finite order, its orbit under g has a cycleBy the postcondition of collision_point, y = gn(x) for thesmallest n > 0 such that gn(x) = g2n+1(x), whereg1(x) = g(x) = x ◦ x = x2

Thus gn(x) = xn+1 and g2n+1(x) = x2n+2 = x2(n+1) = (xn+1)2

Therefore y = gn(x) = xn+1 is the idempotent power of x


Concept MonoidOperation

DefinitionMonoidOperation(Op)⇒

SemigroupOperation(Op)

identity_element : Op→ Domain(Op) is definedFor all op ∈ Op, for all a ∈ Domain(Op), andfor e = identity_element(op):

op(a, e) = a = op(e,a)


Concept GroupOperation

DefinitionGroupOperation(Op)⇒

MonoidOperation(Op)

inverse_operation : Op→ (Domain(Op)→ Domain(Op)) isdefinedFor all op ∈ Op, for all a ∈ Domain(Op), and forg = inverse_operation(op):

op(a,g(a)) = identity_element(op) = op(g(a),a)

A common convention is to represent the inverse of a as a−1


Extending powers to non-positive exponents

a0 = e

an ◦ a0 = an+0 = an

a−n = (an)−1

an ◦ a−n = an−n = a0

Lemma(a−1)n = a−n


Concept Integer

We will study various concepts describing integers in Chapter 4In the meantime we will rely on the intuitive understanding ofwhat integers areModels include:

all C++ integral types, signed and unsignedbignums (type allowing arbitrary-precision integers)

Operations are + - * / %with their standard semanticsAll integer types contain constants 0 and 1

Initially we use the constant 2, but later in the chapter we introducespecial-case functions that allow us to avoid its use

In C++ the constants 0, 1, and 2 of an integer type I are representedas I(0), I(1), and I(2)


slow_power_positive

template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))

Domain(Op) slow_power_positive(Domain(Op) a, I n, Op op){

// Precondition: n > I(0)if (n == I(1))

return a;else

return op(a, slow_power_positive(a, n - I(1), op));}


slow_power_nonnegative

template <typename I, typename Op>requires(Integer(I) && MonoidOperation(Op))

Domain(Op) slow_power_nonnegative(Domain(Op) a, I n, Op op){


return identity_element(op);else

return slow_power_positive(a, n, op);}


Observations

When computing an, we perform n− 1 operationsWe never perform the operation with the identity element

Except when a is the identity elementIt is not worth adding an extra check for the case of a being theidentity element

This would slow down the average caseThe caller can do this if it is important


slow_power

template <typename I, typename Op>requires(Integer(I) && GroupOperation(Op))

Domain(Op) slow_power(Domain(Op) a, I n, Op op){

if (n < I(0)){

a = inverse_operation(op)(a);n = -n;

}return slow_power_nonnegative(a, n, op);

}


Ancient Egyptian discovery

Egyptian scribe Ahmes described (circa 1650 BC) a differentpower algorithm that gives an efficient way to multiply 13

It is sometimes called the Russian peasant algorithm 14

13Gay Robins and Charles Shute. The Rhind Mathematical Papyrus, British MuseumPress, 1987

14The oldest reference to Russian origin we have found appears in: Sir ThomasHeath. A History of Greek Mathematics, Volume I, Clarendon Press, 1921, page 53.Reprint: Dover, 1981. Heath writes: “I have been told that there is a method in useto-day (some say in Russia, but I have not been able to verify this) . . . ” We have notbeen able to verify this either.


Exploiting associativity

For example: aaaaaaa = ((aa)a)((aa)a)a

In general: an = (a2)n/2an mod 2

This translates into the following procedure


Initial (recursive) version


Domain(Op) power_nonnegative_0(Domain(Op) a, I n, Op op){


return identity_element(op);else if (n == I(1))

return a;else

return op(power_nonnegative_0(op(a, a), n / I(2), op),power_nonnegative_0(a, n % I(2), op));

}


Transforming a program

We will now demonstrate a sequence of transformations thatimprove the performance of this algorithm without changing itsasymptotic complexityFor the rest of the book we will typically only show final oralmost-final versions


Eliminating unnecessary multiplication by identity


Domain(Op) power_positive_0(Domain(Op) a, I n, Op op){


return a;else if (n % I(2) == I(0))

return power_positive_0(op(a, a), n / I(2), op);else

return op(power_positive_0(op(a, a), n / I(2), op), a);}


Eliminating common subexpression




return a;Domain(Op) result = power_positive_1(op(a, a), n / I(2), op);if (n % I(2) != I(0))

result = op(result, a);return result;

}


Introducing accumulation variable


Domain(Op) power_accumulate_nonnegative_0(Domain(Op) r, Domain(Op) a, I n, Op op){


return r;if (n % I(2) != I(0)) r = op(r, a);return power_accumulate_nonnegative_0(r, op(a, a), n / I(2), op);

}

Recursion invariant: ran


Almost iterative




return r;if (n % I(2) != I(0)) r = op(r, a);a = op(a, a);n = n / I(2);return power_accumulate_nonnegative_1(r, a, n, op);

}

Recursion invariant: ran


Iterative



// Precondition: n > I(0)while (true) {

if (n == I(0))return r;

if (n % I(2) != I(0)) r = op(r, a);a = op(a, a); // wasted on last iterationn = n / I(2);

}}

Loop invariant: ran


Rotating the loop: reordering





if (n % I(2) != I(0)) r = op(r, a);n = n / I(2); // reordera = op(a, a); // independent statements

}}

Loop invariant: ran


Rotating the loop: duplicating exit





if (n % I(2) != I(0)) r = op(r, a);n = n / I(2);if (n == I(0))

return r; // early exita = op(a, a);

}}

Loop invariant: ran


Rotating the loop: hoisting exit




return r; // moved first test out of loopwhile (true) {


return r;a = op(a, a);

}}

Loop invariant: ran


Dependency between conditions

nwill reach 0 only from 11 is oddThe second condition will only be true when the first condition istrue:


return r;


Utilizing dependency




return r;while (true) {

bool odd = n % I(2) != I(0);n = n / I(2);if (odd) {

r = op(r, a);if (n == I(0))

return r;}a = op(a, a);

}}


Specializing for positive n


Domain(Op) power_accumulate_positive_0(Domain(Op) r, Domain(Op) a, I n, Op op){


bool odd = n % I(2) != I(0);n = n / I(2);if (odd) {

r = op(r, a);if (n == I(0))


}}


(Nearly final) power_accumulate_nonnegative




return r;else

return power_accumulate_positive_0(r, a, n, op);}


Eliminating accumulation variable



// Precondition: n > I(0)n = n - I(1);return power_accumulate_nonnegative_7(a, a, n, op);

}


Observation

When n is 16, we do 7 operations where only 4 are neededWhen n is odd, this code is fine


Factoring out powers of 2



// Precondition: n > I(0)while (n % I(2) == I(0)) {

a = op(a, a);n = n / I(2);

}n = n - I(1);if (n == I(0))

return a;else

return power_accumulate_positive_0(a, a, n, op);}


Observation

power_accumulate_positive_0 checks if n is odd on the first passWe know n is even at that pointOne extra operation!


Unrolling one iteration ofpower_accumulate_positive_0



// Precondition: n > I(0)while (n % I(2) == I(0)) {

a = op(a, a);n = n / I(2);

}n = n / I(2);if (n == I(0))

return a;else

return power_accumulate_positive_0(a, op(a, a), n, op);}


Operations on exponent

In the final version we used these operations:n - I(1)n = n / T(2);n % T(2) == T(0)n == T(0)

General / and % are very expensiveFor all integral types, if we know n is non-negative, we can useshifts and masks


Special cases of procedures

It is frequently useful to identify commonly-occuring expressionsinvolving procedures and constants of a type by definingspecial-case proceduresOften these special cases can be implemented more efficientlythan the general case

For built-in types there may exist machine instructions for thespecial cases (e.g., shifting instead of multiplication by a power oftwo)

For complex user-defined types there are often even moresignificant opportunities for optimizing special cases

Division of two arbitrary polynomials is more difficult thandivision of a polynomial by xDivision of two Gaussian integers (numbers of the form a+ biwhere a and b are integers and i =

√−1) is more difficult than

division of a Gaussian integer by 1 + i


Special cases of Integer procedures

Integer(I) ∧ i ∈ I⇒

successor(i) ≡ i+ 1predecessor(i) ≡ i− 1

half_nonnegative(i) ≡ bi/2cbinary_scale_down_nonnegative(i,k) ≡ i/2k

binary_scale_up_nonnegative(i,k) ≡ 2kiis_positive(i) ≡ i > 0is_negative(i) ≡ i < 0

is_zero(i) ≡ i = 0is_even(i) ≡ (i mod 2) = 0is_odd(i) ≡ (i mod 2) , 0

See default C++ implementations in Appendix 2Stepanov, McJones Elements of Programming March 14, 2008 167 / 880

power_accumulate_positive


Domain(Op) power_accumulate_positive(Domain(Op) r, Domain(Op) a, I n, Op op){

// Precondition: is_positive(n)while (true) {

bool odd = is_odd(n);halve_nonnegative(n);if (odd) {

r = op(r, a);if (is_zero(n))


}}


power_accumulate_nonnegative


Domain(Op) power_accumulate_nonnegative(Domain(Op) r, Domain(Op) a, I n, Op op){

// Precondition: ¬is_negative(n)if (is_zero(n))

return r;else

return power_accumulate_positive(r, a, n, op);}


power_positive


Domain(Op) power_positive(Domain(Op) a, I n, Op op){

// Precondition: is_positive(n)while (is_even(n)) {

a = op(a, a);halve_nonnegative(n);

}halve_nonnegative(n);if (is_zero(n))

return a;else

return power_accumulate_positive(a, op(a, a), n, op);}


power_nonnegative


Domain(Op) power_nonnegative(Domain(Op) a, I n, Op op){

// Precondition: ¬is_negative(n)if (is_zero(n))

return identity_element(op);else

return power_positive(a, n, op);}


power

template <typename I, typename Op>requires(Integer(I) && GroupOperation(Op))

Domain(Op) power(Domain(Op) a, I n, Op op){

if (is_negative(n)) {a = inverse_operation(op)(a);n = -n;

}return power_nonnegative(a, n, op);

}


The number of operations performed by power

Let u = number of significant bits in n > 0Let v = number of one-bits in nLet p = number of operations performedThen p = (u− 1) + (v− 1) = u+ v− 2 6 2blog2 nc

ExerciseProve that power achieves this complexity


The operation count of power is not optimal

For n = 15, u = 4 and v = 4, so u+ v− 2 = 6But 15 = 3× 5, so a15 = (a3)5

a3 takes 2 operationsa5 takes 3 operations(a3)5 takes 5 total

There are faster ways for 23, 27, 39, 43, . . .See Project 3


Choosing algorithms based on complexity

Complexity is an important part of specificationsFor example, a stack is not a stack if the cost of push grows linearlywith the size of the stack

When only one algorithm for solving a problem is available, themain issue is whether the algorithm is feasible for a particularsituation (set of arguments determining problem size, functionalparameters, etc.)When there are more than one algorithm for solving a problem, weneed to analyze which algorithm to use for a particular situationThe analysis depends on the complexity properties of thealternative algorithms combined with the complexity propertiesof the types to which these algorithms are applied


Specifying complexity

Knuth pioneered the analysis of algorithms based on counting themachine instructions performed by a MIX programSince we write generalized procedures defined in terms of typerequirements and sometimes taking function objects asarguments, we need to count applications of the function objectsand the operations on the types


Complexity of slow_power versus power

The total cost is the sum of the cost of the operations on theexponent, the cost of the semigroup operation, and the cost of thecontrol structureFor all the models of Integer we are familiar with, the logarithmicnumber of exponent operations performed by powerwill be fasterthan the linear number of operations performed by slow_power,except for very small nFor most models of the semigroup operation, power will be faster,but there are exceptions


When is power faster than slow_power?

Suppose cost(op, xi, xj) = k(x) · (i · j)α

If α = 0, we have a fixed-cost operation and power is alwaysmuch faster than slow_power

Examples: fixed-precision multiplication; matrix multiplicationwith fixed-precision elements

If α 6 1, power is always faster15

Examples: multiplication of univariate polynomials withfixed-precision coefficients; multiplication of bignums

If α > 2, slow_power is always faster16

Examples: multiplication of bivariate polynomials withfixed-precision coefficients; multiplication of univariatepolynomials with bignum coefficients

15R.L. Graham, A.C.-C. Yao, and F.-F. Yao. Addition chains with multiplicativecosts. Discrete Mathematics, Volume 23, Number 2, 1978, pages 115-119.

16D.P. McCarthy. Effect of Improved Multiplication Efficiency on ExponentationAlgorithms Derived from Addition Chains. Mathematics of Computation, Volume 46,Number 174, April 1986, pages 603-608.


Applications

The main application of fast exponentiation is cryptography

Donald Knuth.Secret factors.The Art of Computer Programming, Vol. 2, pages 403-407.RSA.

Donald Knuth.Improved primality tests.The Art of Computer Programming, Vol. 2, pages 394-396.Primality testing.

Manindra Agrawal, Neeraj Kayal, and Nitin Saxena.PRIMES is in P.Annals of Mathematics, 160 (2004), pages 781-793.Primality testing.


Reflections on deriving useful interfaces

power_accumulate_nonnegative is useful whenever one wantsto compute axn

Even power_accumulate_positive is useful in a situation when itis known that n > 0

ReflectionWhile developing a component, it is often possible to discoveradditional useful interfaces


Reflection on code transformations

ReflectionCompilers can perform code transformations when the semanticsof the operations are knownCurrently this is only for built-in typesSomeday we will be able to tell the compiler the semantics of ouroperations


Fibonacci numbers

Definition

f0 = 0f1 = 1

fn+2 = fn+1 + fn

Example0, 1, 1, 2, 3, 5, 8, 13, . . .


Recursive way to calculate fn

template <typename I>requires(Integer(I))

I fibonacci_recursive(I n){

if (n == I(0))return I(0);

else if (n == I(1))return I(1);

elsereturn fibonacci_recursive(n - I(1))

+ fibonacci_recursive(n - I(2));}


Iterative way to calculate fn

template <typename I>requires(Integer(I))I fibonacci_iterative(I n){

if (n == I(0))return I(0);

I fib_i = I(0);I fib_j = I(1);while (n != I(1)) {

I next = fib_i + fib_j;fib_i = fib_j;fib_j = next;n = n - I(1);

}return fib_j;

}


Fibonacci matrices

F1 =

(1 11 0

)Fn =

(fn+1 fnfn fn−1

)F1Fn =

(fn+1 + fn fn + fn−1fn+1 fn

)=

(fn+2 fn+1fn+1 fn

)= Fn+1

Fn = F1F1 . . . F1︸︷︷︸n

= Fn1

FmFn = Fm1 Fn1 = Fm+n

1 = Fm+n


Fibonacci matrices

Fm =

(fm+1 fmfm fm−1

)Fn =

(fn+1 fnfn fn−1

)FmFn =

(fm+1fn+1 + fmfn fm+1fn + fmfn−1fmfn+1 + fm−1fn fmfn + fm−1fn−1

)


Observations

We can represent the matrix with a pair corresponding to thebottom row

pair::first is fn and pair::second is fn−1

The identity fn+1 = fn−1 + fn allows us to compute the missingterm from the top row of Fn needed to compute the bottom row ofFmFn


fibonacci_multiplies


struct fibonacci_multiplies{

typedef pair<I, I> D;D operator()(const D& x, const D& y) const{

return D(x.first * (y.first + y.second) + x.second * y.first,x.first * y.first + x.second * y.second);

}};


fibonacci


I fibonacci(I n){


return I(0);fibonacci_multiplies op;return power_positive(pair<I, I>(I(1), I(0)), n, op).first;

}


Overloading

DefinitionOverloading an operator symbol or function name means using it foroperations or functions on several types

ReflectionProgress in mathematics often involves extending an operator to a newdomain

Example+ on

natural numbersintegersrationalspolynomialsmatricesStepanov, McJones Elements of Programming March 14, 2008 190 / 880

Overloading should preserve semantics

In all of these examples+ is associative+ is commutative+ obeys the cancellation law

a+ c = b+ c⇒ a = b

× distributes over +


Type parameterization results in overloading

When we define template functions such as collision_point andpower, we are overloading the function nameIf the actual type parameters satisfy the requirements thesemantics is preserved since the same algorithm is used


Two ways operations are provided to power

The group operation is passed as a parameterThis allows power to be used with different operations on the sametype, and the operation can have a state such as a modulus for usein encryption

The integer operations are provided through overloading of -, /,%, and ==

Using explicit operators is convenient whenever there are severalrelated operations on the type and no other correct interpretation ofthe operations on the type


Concept CommutativeOperation

DefinitionCommutativeOperation(Op)⇒

BinaryOperation(Op)

For all op ∈ Op and for all a,b ∈ Domain(Op),

op(a,b) = op(b,a)


Concept CancellableOperation

DefinitionCancellableOperation(Op)⇒

BinaryOperation(Op)

For all op ∈ Op and for all a,b, c ∈ Domain(Op),

op(a, c) = op(b, c)⇒ a = b

op(c,a) = op(c,b)⇒ a = b


Concept AdditiveSemigroup

DefinitionAdditiveSemigroup(T)⇒

The binary operation infix + is defined on TSemigroupOperation(+)

CommutativeOperation(+) a

aIt would have been nice to require the cancellation law, but digital logic designersuse + for an operation that is not cancellable


Concept MultiplicativeSemigroup

DefinitionMultiplicativeSemigroup(T)⇒

The binary operation infix × is defined on TSemigroupOperation(×)


Concept AdditiveMonoid

DefinitionAdditiveMonoid(T)⇒

AdditiveSemigroup(T)

MonoidOperation(+)

The constant T(0) is definedidentity_element(+) = T(0)


Concept MultiplicativeMonoid

DefinitionMultiplicativeMonoid(T)⇒

MultiplicativeSemigroup(T)

MonoidOperation(×)

The constant T(1) is definedidentity_element(×) = T(1)


Concept AdditiveGroup

DefinitionAdditiveGroup(T)⇒

AdditiveMonoid(T)

GroupOperation(+)

The unary operation negation (prefix −) is defined on Tinverse_operation(+) = negation

DefinitionFor an additive group, subtraction is defined as a− b = a+ (−b)


Concept MultiplicativeGroup

DefinitionMultiplicativeGroup(T)⇒

MultiplicativeMonoid(T)

GroupOperation(×)

The unary operation reciprocal is defined on Tinverse_operation(×) = reciprocal

DefinitionFor a multiplicative group, division is defined asa/b = a× reciprocal(b)


Power for multiplicative types

template <typename G, typename I>requires(MultiplicativeSemigroup(G) && Integer(I))

G power_positive(G a, I n){

return power_positive(a, n, multiplies<G>());}

template <typename G, typename I>requires(MultiplicativeMonoid(G) && Integer(I))

G power_nonnegative(G a, I n){

return power_nonnegative(a, n, multiplies<G>());}

template <typename G, typename I>requires(MultiplicativeGroup(G) && Integer(I))

G power(G a, I n){

return power(a, n, multiplies<G>());}

See Appendix 2 for the enabling C++ definitions


Conclusions

Algorithms are generic by natureThey can be used with different models satisfying the samerequirements

Algorithms are affiliated with algebraic structures: semigroup,monoid, . . .Stepwise refinement leads from mathematical definitions toefficient codeSpecial-case procedures can make code more efficient and evenmore genericMathematics leads to surprising algorithms: fibonacci


Reference

D. Kapur, D.R. Musser, and A.A. Stepanov.Operators and Algebraic Structures.Proceedings of the 1981 conference on Functional programminglanguages and computer architecture, pages 59-63.One of the first presentations of algorithms on algebraic structures.


Project 1

ProjectOur definition of the Fibonacci sequence starts from zero and goes up;extend the definition and our code to work for negative indices a

aSuggested by Oleg Zabluda


Project 2

Charles M. Fiduccia.An efficient formula for linear recurrences.SIAM Journal of Computing, Volume 14, Number 1, February 1985,page 106-112.Generalizes the fibonacci algorithm to arbitrary linear recurrences.

ProjectCreate a library implementing Fiduccia’s algorithm


Project 3

Donald E. Knuth.Addition chains.The Art of Computer Programming, Volume 2: SeminumericalAlgorithms, 3rd edition, Addison-Wesley, San Francisco, 1998,pages 465-481.Describes a way to do minimal-operation exponentiation usingaddition chains.

ProjectImplement a useful library doing power optimally for exponentsknown at compile time


Project 4

ProjectFloating-point multiplication and addition are not associative, somay give different results when they are used as the operation forslow_power and power; establish whether slow_power or powergives a more accurate result for:

1 Raising a floating-point number to an integral power2 Multiplying a floating-point number by an integral factor


Contents I

1 Introduction

2 Foundations



5 OrderingsMotivationRelation; transitivity; orderingStrict versus reflexiveSymmetric versus asymmetric


Contents II

Equivalence; equalitySymmetric complement; total ordering; weak ordering; partialorderingMultiple orderings; natural total ordering; default ordering;overloadingIntervalsOrder selection algorithmsConclusionsReferenceProject




Contents III


9 Rotations







Contents IV

15 C++ machinery

16 Acknowledgments

17 Index


The importance of linear ordering

Equality allows comparing individual elements easilySearching a collection using only equality requires linear timeIntersecting two collections using only equality requires timeproportional to the product of the sizes of the collectionsA linear ordering allows organizing elements so that theircollections can be

matched (intersection, subset, union) in linear timesearched in logarithmic time


Concepts Predicate and Relation

DefinitionPredicate(Op)⇒

RegularFunction(Op)

Codomain(Op) = bool

DefinitionRelation(Op)⇒

Predicate(Op)HomogeneousFunction(Op)

Arity(Op) = 2(Other books use this term for more general non-unary predicates)


Concept Ordering

DefinitionOrdering(Op)⇒

Relation(Op)

For all op ∈ Op and for all a,b, c ∈ Domain(Op)

op(a,b) ∧ op(b, c)⇒ op(a, c)

This property is called transitivity


Examples of ordering

ExamplesEqualityEquality of the first characterReachability in an orbitDivisibility


Strict versus reflexive

DefinitionAn ordering r is strict if ¬r(a,a)

An ordering r is reflexive if r(a,a)

An ordering r is weakly reflexive if r(a,b)⇒ r(a,a) ∧ r(b,b)

ExamplesReflexive: all examples on the previous slideStrict: proper factor


Symmetric versus asymmetric

DefinitionA relation r is symmetric if r(a,b)⇒ r(b,a)

A relation r is asymmetric if r(a,b)⇒ ¬r(b,a)

LemmaA strict ordering is asymmetric

LemmaA symmetric ordering is weakly reflexive

ExamplesSymmetric: siblingAsymmetric: parent


Concept EquivalenceRelation

DefinitionEquivalenceRelation(R)⇒

Ordering(R)

Reflexive(R)

Symmetric(R)

ExamplesEqualityGeometric congruencea ≡ b (mod n)


Equivalence and equality

LemmaIf r is an equivalence relation, a = b⇒ r(a,b)


Key function

DefinitionIf T is a regular type and r ∈ R is an equivalence relation on T , afunction f ∈ F : T → T ′ is a key function for r ifr(x, x ′)⇔ f(x) = f(x ′)

Given the choice of a particular key function f, we say f(x) is acanonical representation of the elements equivalent to x

One way to implement an equivalence relation is to define a keyfunction for it and then apply equality to the results


symmetric_complement

template <typename R>requires(Relation(R))

struct symmetric_complement{

R r;symmetric_complement(R r) : r(r) {}

bool operator()(const Domain(R)& a, const Domain(R)& b){

return !r(a, b) && !r(b, a);}

};


Concept StrictTotalOrdering

DefinitionStrictTotalOrdering(R)⇒

Ordering(R)

For all r ∈ R and for all a,b ∈ Domain(R)

symmetric_complementr(a,b)⇔ a = b


Properties of total ordering

LemmaA total ordering is strict

LemmaA total ordering is asymmetric

LemmaThe trichotomy law holds: r(a,b) ∨ r(b,a) ∨ (a = b)

LemmaExactly one clause of the trichotomy law holds


Extending strict total orderings to a direct product

Strict total orderings r0 ∈ R0, . . . , rn ∈ Rn can be extended to astrict total ordering on the direct productDomain(R0)× . . .×Domain(Rn) via the lexicographic ordering rdefined by:

r((x0, . . . , xn), (y0, . . . ,yn)) ≡(∃k ∈ [0,n])(x0 = y0 ∧ . . . ∧ xk−1 = yk−1 ∧ rk(xk,yk))

Lemmar is a strict total ordering


Concept StrictWeakOrdering

DefinitionStrictWeakOrdering(R)⇒

Ordering(R)

EquivalenceRelation(symmetric_complementR)


Weak ordering is a weakening of total ordering

LemmaStrictTotalOrdering(R)⇒ StrictWeakOrdering(R)


Properties of weak ordering

LemmaA weak ordering is strict

LemmaA weak ordering is asymmetric

LemmaThe trichotomy law holds:r(a,b) ∨ r(b,a) ∨ symmetric_complementr(a,b)

LemmaExactly one clause of the trichotomy law holds


Extending strict weak orderings to a direct product

Strict weak orderings can be extended to a direct product vialexicographic ordering, just as with strict total orderings


Defining a strict weak ordering using a key function

A key function f on a set T and a strict total ordering r on thecodomain of f define a weak ordering r(x,y)⇔ r(f(x), f(y))

Example

Weak ordering on ith component of a direct product

template <typename F, typename R>requires(Function(F) && Arity(F) == 1 &&StrictTotalOrdering(R) && Codomain(F) == Domain(R))

struct key_ordering models(StrictWeakOrdering){

F f;R r;key_ordering(F f, R r) : f(f), r(r) { }bool operator()(const Domain(F)& x, const Domain(F)& y) const{

return r(f(x), f(y));}

};


Not every strict ordering is weak

Suppose a relation ∝ on {a,b, c,d, e} is a ∝ b ∝ c ∝ d,a ∝ e ∝ d:

a

b c

d

e

The symmetric complement � of ∝ is b � e, e � b, e � c, c � eIf �were an equivalence relation it would have to include b � cand c � b by transitivity


Mathematical conventions for usage of weak and semi

Weak refers to weakening (which includes dropping) an axiomA strict weak ordering replaces equality with equivalence

Semi refers to dropping an operationA semigroup lacks the inverse operation


Partial ordering

There are algorithms that deal with orderings that are not weak:algorithms on partially-ordered setsThe most important is topological sort:

Donald Knuth.Topological sorting.The Art of Computer Programming, Volume 1, Addison-Wesley,1997, pages 261-268.

Operations as simple asmax andmin do not make sense withouta weak order


Multiple orderings on a type

There is one equality relation on a type TThere can be many equivalence relationsSimilarly there can be many orderings


Natural total ordering

DefinitionThe total ordering on a type that is consistent with algebraic operationson it is called the natural total ordering

ExampleExamples of consistent axioms for natural ordering:

Incrementable a < successor(a)

Incrementable a < b⇒ successor(a) < successor(b)Ordered group a < b⇒ a+ c < b+ c

Ordered ring (a < b) ∧ (0 < c)⇒ ca < cb


Default ordering

Sometimes a type does not have a natural total orderingComplex numbersIterators on linked lists

We still want a total ordering to enable logarithmic searching, sowe always define the default ordering for regular types

Lexicographic ordering for complex numbersAddress-based ordering for iterators on linked lists

When the natural order exists, it coincides with the defaultordering

Definitionless<T> defines the default ordering for T


Concept StrictTotallyOrdered

DefinitionStrictTotallyOrdered(T)⇒

Regular(T)The relation < is defined on TStrictTotalOrdering(<)

< is reserved for the natural total ordering


Clusters of derived procedures

There are procedures that naturally come in clustersIf some of the cluster are defined, the definitions of the othersnaturally follow

ExampleIn an additive group, negation and subtraction constitute such a cluster


Derived relations

Given any relation, say, r(a,b), there arecomplement ¬r(a,b)

converse r(b,a)

complement of converse ¬r(b,a)

Given a symmetric relation, say r(a,b), since the converse is equalto the original relation, the only derivable relation is thecomplement, ¬r(a,b)


Properties of derived relations

LemmaGiven an ordering, its complement, its converse, and thecomplement of its converse are orderingsIf an ordering is strict, its converse is also strict and itscomplement and complement of converse are reflexiveIf an ordering is reflexive, its converse is also reflexive and itscomplement and complement of converse are strictThere exist C++ template function object classes complement,converse, and complement_of_converse which, given a relation,construct the corresponding derived relations


Equality and inequality

For every regular type, we have = and ,We assure consistency by always defining equality and relying onthe following template for inequality


bool operator!=(const T& a, const T& b){

return !(a == b);}


<, >, 6, and >

For every totally ordered type, we have <, >, 6, and >

We assure consistency by always defining < and relying on thefollowing templates for the others


bool operator>(const T& a, const T& b) { return b < a; }


bool operator<=(const T& a, const T& b) { return !(b < a); }


bool operator>=(const T& a, const T& b) { return !(a < b); }


Intervals

DefinitionA closed interval [a,b] is the set of all elements x such that a 6 x 6 b

An open interval (a,b) is the set of all elements x such thata < x < b

A half-open on right interval [a,b) is the set of all elements x suchthat a 6 x < b

A half-open on left interval (a,b] is the set of all elements x such thata < x 6 b

A half-open interval is our short-hand for half-open on rightThese definitions generalize to weak orderings


min andmax

template <typename R>requires(StrictWeakOrdering(R))

Domain(R)& min(Domain(R)& a, Domain(R)& b, R r){

if (r(b, a)) return b;else return a;

}


Domain(R)& max(Domain(R)& a, Domain(R)& b, R r){

if (r(b, a)) return a;else return b;

}


Othermin andmax


Domain(R)& other_min(Domain(R)& a, Domain(R)& b, R r){

if (r(a, b)) return a;else return b;

}


Domain(R)& other_max(Domain(R)& a, Domain(R)& b, R r){

if (r(a, b)) return b;else return a;

}

We will see shortly whymin andmax are preferable toother_min and other_max 17

17The C++ standard library usesmin and other_maxStepanov, McJones Elements of Programming March 14, 2008 245 / 880

A convention for order selection functions

For the order selection procedures in this section, there are fouruseful versions

Since the parameters are being passed for selecting, versions takingboth references and const references are neededFor convenience, we want versions for totally ordered types (with<) as well as for an explicitly-supplied ordering

From now on, we show only the non-const reference version withan explict ordering; see Appendix 2 for the other versionsThe code below shows how to obtain a version for a totallyordered type given a version for an explicitly supplied ordering


Stability

DefinitionAn algorithm is stable if it respects the original order of equivalentelements

Stability is not relevant with a total ordering since equivalentelements are equal and, therefore, indistinguishable

ExampleStable sort

Multiple passes compose naturallySorting by first name and then by last name results in expected order

Stable partitionEven/odd partition of {1, 2, 3, 4, 5} results in {1, 3, 5, 2, 4}, not{1, 5, 3, 4, 2} as produced by a fast partitioning algorithm

Stability sometimes increases the complexity


Increasing order

Definition

A sequence . . . , x, . . . , x ′, . . . is in{

increasingstrictly increasing

}order with

respect to a strict weak ordering r if{

¬r(x ′, x)r(x, x ′)

}a

aWhile some would use the terms non-decreasing and increasing, we follow NicolasBourbaki, Theory of Sets, III.1.5, page 138.

In this book we always assume increasing order for sortedsequences


sort_2


void sort_2(Domain(R)& a, Domain(R)& b, R r){

if (r(b, a)) swap(a, b);}

sort_2 is stableIt does the minimal amount of work since it does not swapequivalent elements


Stability ofmin andmax

Natural postcondition for sort_2:a0 = a; b0 = b;sort_2(a, b, r);assert(a == min(a0, b0, r) && b == max(a0, b0, r));

min andmax satisfy it, while other_min and other_max do not


Properties ofmin andmax for StrictTotalOrdering r

Lemmaassociativity minr(minr(a,b), c) = minr(a,minr(b, c))associativity maxr(maxr(a,b), c) = maxr(a,maxr(b, c))

commutativity minr(a,b) = minr(b,a)

commutativity maxr(a,b) = maxr(b,a)

absorption minr(a,maxr(a,b)) = a

absorption maxr(minr(a,b),b) = b

idempotency minr(a,a) = a

idempotency maxr(a,a) = a


Properties ofmin andmax for StrictWeakOrdering r

Lemmaassociativity minr(minr(a,b), c) = minr(a,minr(b, c))associativity maxr(maxr(a,b), c) = maxr(a,maxr(b, c))

commutativity symmetric_complementr(minr(a,b),minr(b,a))

commutativity symmetric_complementr(maxr(a,b),maxr(b,a))

ExerciseWhat are the absorption and idempotency laws forminr andmaxr?


min_3 andmax_3


Domain(R)& min_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){

return min(min(a, b, r), c, r);}


Domain(R)& max_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){

return max(max(a, b, r), c, r);}


Implementing order selection

There are more complicated cases thanmin andmax such asmedian of 3 and select 2nd of 4Writing order selections is somewhat complicated and can behelped by decomposition into simpler subproblems


median_3


Domain(R)& median_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){

if (r(b, a)) return median_3_stage1(b, a, c, r);else return median_3_stage1(a, b, c, r);

}


Domain(R)& median_3_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){

// Precondition: a, b are sortedif (!r(c, b)) return b; // a, b, c are sortedelse return max(a, c, r); // b is not the median

}


Stability ofmedian_3

ExerciseDefine the appropriate stability property formedian_3

Lemmamedian_3 is stable


Complexity ofmedian_3

median_3 does 3 comparisons in the worst caseThe function does 2 comparisons only when c ismax_3(a,b, c)and that happens in one-third of the casesThe average number of comparison is 2 2

3


Selecting second smallest

Finding second smallest implies finding smallestIf after finding second smallest, two candidates for smallest remain,then the second smallest is not second smallest!

Finding second smallest from n elements requires at leastn+ logn− 2 comparisons

Finding second out of four requires 4 comparisons


select_2nd_4

template <typename R> requires(StrictWeakOrdering(R))Domain(R)&select_2nd_4(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {

if (r(b, a)) return select_2nd_4_stage1(b, a, c, d, r);else return select_2nd_4_stage1(a, b, c, d, r);

}

template <typename R> requires(StrictWeakOrdering(R))Domain(R)&select_2nd_4_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {

// Precondition: a 6 bif (r(d, c)) return select_2nd_4_stage2(a, b, d, c, r);else return select_2nd_4_stage2(a, b, c, d, r);

}

template <typename R> requires(StrictWeakOrdering(R))Domain(R)&select_2nd_4_stage2(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {

// Precondition: a 6 b∧ c 6 dif (r(c, a)) return min(a, d, r);else return min(b, c, r);

}


Exercises

ExerciseIs select_2nd_4 stable?

ExerciseImplement select_3rd_4


median_5

template <typename R> requires(StrictWeakOrdering(R))Domain(R)& median_5(

Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {if (r(b, a)) return median_5_stage1(b, a, c, d, e, r);else return median_5_stage1(a, b, c, d, e, r);

}

template <typename R> requires(StrictWeakOrdering(R))Domain(R)& median_5_stage1(

Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {// Precondition: a 6 bif (r(d, c)) return median_5_stage2(a, b, d, c, e, r);else return median_5_stage2(a, b, c, d, e, r);

}

template <typename R> requires(StrictWeakOrdering(R))Domain(R)& median_5_stage2(

Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {// Precondition: a 6 b∧ c 6 dif (r(c, a)) return select_2nd_4_stage1(a, b, d, e, r);else return select_2nd_4_stage1(c, d, b, e, r);

}


Exercises

ExerciseShow thatmedian_5 is not stable

ExerciseDesign a stable version ofmedian_5

ExerciseFind an algorithm for median of 5 that does slightly fewercomparisons on average


Conclusions

Weak orderings allow efficient algorithms on collections ofelementsThe axioms of ordering provide the interface to connect specificorderings with general purpose algorithmsOverloaded operators should preserve their semantics inmathematics


Reference

Our treatment of ordering is based on:

N. Bourbaki.Chapter 3, Section 1: Order relations. Ordered Sets.Theory of Sets, Springer, 2004, pages 131-148.


Project

Using material from

Donald E. Knuth.Section 5.3: Optimum Sorting.The Art of Computer Programming, Volume 3: Sorting andSearching, 2nd edition, Addison-Wesley, 1998.

create a library for generic minimum-comparison networks forsorting, merging, and selectionFor sorting and merging networks, minimize not only the numberof comparisons, but the number of data movementsAssure stability of these networks


Contents I

1 Introduction

2 Foundations



5 Orderings

6 Combining conceptsCombining algebraic structures and orderingRemainder


Contents II

Greatest common divisorExtending the domain of greatest common divisorQuotientQuotient and remainder for negative quantitiesIntegersConclusionsReferencesProject



9 Rotations


Contents III






15 C++ machinery

16 Acknowledgments


Contents IV

17 Index


Axioms combine concepts

Combining different structures on the same typeCombining different types


Ordered structures

DefinitionOrderedAdditiveSemigroup(T)⇒

AdditiveSemigroup(T)

StrictTotallyOrdered(T)

For all x,y, z ∈ T ,x < y⇒ x+ z < y+ z

DefinitionOrderedAdditiveMonoid(T)⇒Monoid(T) ∧ OrderedAdditiveSemigroup(T)

DefinitionOrderedAdditiveGroup(T)⇒ Group(T) ∧ OrderedAdditiveMonoid(T)


Absolute value

template <typename T>requires(OrderedAdditiveGroup(T))

T abs(const T& x){

if (x < T(0)) return -x;else return x;

}

The correctness of abs depends on

x < T(0)⇒ x+ (−x) < T(0) + (−x)⇒ T(0) < −x

We use the notation |x| for the absolute value of x


Concept OrderedCancellableMonoid

DefinitionOrderedCancellableMonoid(T)⇒

OrderedAdditiveMonoid(T)

The partial binary infix operator − is defined on TFor all a,b ∈ T

b 6 a⇒ is_defined(−,a,b) ∧ (a− b) + b = a


Division with remainder

As repeated addition induces multiplication,repeated subtraction induces division with remainderInterestingly, it is easier to introduce the remainder computation,leaving the quotient for later


weak_remainder

template <typename T>requires(OrderedCancellableMonoid(T))

T weak_remainder(T a, T b){

// Precondition: a > T(0) ∧ b > T(0)while (b <= a) a = a - b;return a;

}

The concept OrderedCancellableMonoid is not strong enough toprove termination of weak_remainder; we need anotherproperty to ensure there are no unreachable (infinite) elements


Concept ArchimedeanMonoid

DefinitionArchimedeanMonoid(T)⇒

OrderedCancellableMonoid(T)

For all a and all b > 0 ∈ T , weak_remainder(a,b) terminates

The property is called the Axiom of Archimedes

ReflectionA concept can be defined with an axiom stating that a particularprocedure terminates


The Axiom of Archimedes

The Axiom of Archimedes is usually written as “there exists anintegerm such that a < m · b”Archimedes originally wrote: “... that the excess by which thegreater of (two) unequal areas exceeds the less can, by beingadded to itself, be made to exceed any given finite area.” 18

Archimedes attributes this lemma to Eudoxus 19

18T.L. Heath, translator. The Works of Archimedes, Preface to Quadrature of theParabola. Dover, 2002, page 234

19Sir Thomas Heath. A History of Greek Mathematics, Volume I. Dover, 1981, page 327Stepanov, McJones Elements of Programming March 14, 2008 277 / 880

Examples of Archimedean monoids

Line segments in Euclidean geometryRational numbersBinary fractions { n2k }

Ternary fractions { n3k }


Intuition for a fast algorithm for remainder

Repeated doubling leads to fast (logarithmic complexity) powerA related algorithm is possible for remainderAs with power, the Egyptians used this algorithm to do divisionwith remainder


Deriving a fast algorithm for remainder

If c is a monoid element andm is a nonnegative integer,m · cstands for c+ · · ·+ c︸︷︷︸

m times

Let a = n · b+ r, where r = weak_remainder(a,b)Let n = 2q+ p, where q = bn/2c and p = n mod 2a = q · (2 · b) + (p · b+ r) where (p · b+ r) < 2 · bremainder(a,b) = r =a if a < ba− b if a− b requires(ArchimedeanMonoid(T))

T remainder_nonnegative(T a, T b){

// Precondition: a > T(0) ∧ b > T(0)if (a < b) return a;if (a - b < b) return a - b;a = remainder_nonnegative(a, b + b);if (a < b) return a;else return a - b;

}

The first comparison is not needed in the recursive calls and couldbe eliminated by the introduction of a helper functiona− b < b serves as a guard to ensure b+ b will not overflow


Complexity of remainder_nonnegative

It is trivial to see that remainder_nonnegative is logarithmic

ExerciseDetermine the exact counts of different operations


Concept WeaklyHalvableMonoid

In many situations while the monoid does not provide generaldivision by an integer, it has a division by 2

While general k-section of an angle by ruler and compass can notbe done, bisection is trivial

Divisibility by 2 gives an iterative version of remainder

DefinitionWeaklyHalvableMonoid(T)⇒

ArchimedeanMonoid(T)

The partial unary operation half_nonnegative is defined on T+

where T+ = {x ∈ T |x > T(0)}

For all a,b ∈ T+

a = b+ b⇒ half_nonnegative(a) = b


remainder_nonnegative_iterative 20

template <typename T>requires(WeaklyHalvableMonoid(T))

T remainder_nonnegative_iterative(T a, const T& b){

// Precondition: a > T(0) ∧ b > T(0)if (a < b) return a;T c = b;while (a - c >= c) c = c + c;a = a - c;while (c != b) {

c = half_nonnegative(c);if (c <= a) a = a - c;

}return a;

}

20Edsger W. Dijkstra attributes this algorithm to N.G. de Bruijn on page 13 of Noteson Structured Programming, in: O.-J. Dahl, E. W. Dijkstra and C.A.R. Hoare.Structured Programming. Academic Press, London and New York, 1972. The algorithmalso appears in: Niklaus Wirth. Systematic Programming: An Introduction.Prentice-Hall, 1973, program 7.22, page 38.


Divisibility on an Archimedean monoid T

DefinitionFor a > T(0) and b > T(0), b divides a⇔ remainder(a,b) = T(0)


Properties of divisibility on an Archimedean monoid

LemmaIn an Archimedean monoid T with positive x,a,b

b divides a⇒ b 6 a

b > a∧ x divides a∧ x divides b⇒ x divides (b− a)

x divides a∧ x divides b⇒ x divides remainder(a,b)


Concept DiscreteArchimedeanMonoid

DefinitionDiscreteArchimedeanMonoid(T)⇒


(∃u ∈ T)(∀x ∈ T)x < u⇒ ¬(T(0) < x)

Such an element u is called a unit

LemmaEvery element of a discrete Archimedean monoid is divisible by theunit


Greatest common divisor

DefinitionThe greatest common divisor of a and b, denoted by gcd(a,b), is adivisor of a and b that is divisible by any other divisor of a and b

This definition works for Archimedean monoid, and, as we shallsee later, for other structuresNote the definition does not depend on ordering but is expressedstrictly in terms of divisibility


Properties of greatest common divisor

LemmaIn an Archimedean monoid with positive x,a,b, the following hold

x divides a∧ x divides b⇒ x 6 gcd(a,b)gcd(a,b) is uniquegcd(a,a) = a


subtractive_gcd

template <typename T>requires(ArchimedeanMonoid(T))

T subtractive_gcd(T a, T b){

// Precondition: a > T(0) ∧ b > T(0)

while (true) {if (b < a)

a = a - b;else if (a < b)

b = b - a;else

return a;}

}

This is known as Euclid’s algorithm 21

It is fairly certain that the algorithm had been known at least acentury before Euclid

21Sir Thomas Heath, translator. Euclid’s Elements. Dover, 1956: Volume 2, Book VII,Propositions 1 and 2, pages 296-300; Volume 3, Book X, Propositions 1-3, pages 14-22


Correctness of subtractive_gcd

Proof.1 Each subtraction preserves the property of being divisible by any

divisor of the original a and b2 If the variables become equal, they divide the original a and b3 Therefore it returns a divisor of a and b that is divisible by any

other divisor4 In other words, it returns the greatest common divisor

�


Termination of subtractive_gcd

LemmaIt always terminates for integers and rationals

LemmaIt does not always terminate for reals


Concept EuclideanMonoid

DefinitionEuclideanMonoid(T)⇒


For all a > T(0),b > T(0) ∈ T , subtractive_gcd(a,b) terminates

LemmaEvery discrete Archimedean monoid is Euclidean


Extending subtractive_gcd to zero

It is straightforward to extend subtractive_gcd to the case whereone of its arguments is zero

template <typename T>requires(EuclideanMonoid(T))

T subtractive_gcd_with_zero(T a, T b){

// Precondition: a > T(0) ∧ b > T(0) ∧ ¬(a = T(0) ∧ b = T(0))

while (true) {if (b == T(0)) return a;while (b < a) a = a - b;if (a == T(0)) return b;while (a < b) b = b - a;

}}


Speeding up subtractive gcd

Each of the inner while statements in subtractive_gcd_with_zerois equivalent to a call of slow_remainderBy using our (logarithmic) remainder algorithm, we can speed upthe case where a and b are very different in magnitude


fast_subtractive_gcd

template <typename T>requires(EuclideanMonoid(T))

T fast_subtractive_gcd(T a, T b){

// Precondition: a > T(0) ∧ b > T(0) ∧ ¬(a = T(0) ∧ b = T(0))

while (true) {if (b == T(0)) return a;a = remainder_nonnegative(a, b);if (a == T(0)) return b;b = remainder_nonnegative(b, a);

}}


Generalizing the greatest common divisor algorithm

On a Euclidean monoid, the computation of greatest commondivisor is based on a particular remainder algorithmIf a structure has a properly-defined remainder, it does not have tobe a Euclidean monoid

PolynomialsGaussian integers

Mathematicians refer to these other types as Euclidean domainsOur goal is to define a concept unifying Euclidean monoids andEuclidean domains, so we can use the same gcd algorithm for both


Concept CommutativeSemiring

DefinitionCommutativeSemiring(T)⇒

AdditiveMonoid(T)

MultiplicativeSemigroup(T) ∧ CommutativeOperation(×)

For any a, T(0)× a = T(0)

The constant T(1) is definedFor any a , T(0), T(1)× a = a

For all a,b, c ∈ T , a× (b+ c) = a× b+ a× c

The last property is distributivity


Examples of CommutativeSemiring

ExampleNonnegative integers constitute a commutative semiring


Concept Semimodule

DefinitionSemimodule(T)⇒

AdditiveMonoid(T)

The type function Scalar(T) is definedCommutativeSemiring(Scalar(T))

The binary infix function · : Scalar(T)× T → T is definedFor all α,β ∈ Scalar(T) and for all x,y ∈ T ,

α · (β · x) = (α× β) · x(α+ β) · x = α · x+ β · xα · (x+ y) = α · x+ α · y

Scalar(T)(1) · x = x


Examples of Semimodule

ExampleThe set {(a,b)} of two-dimensional vectors with nonnegative realcoefficients constitutes a semimodule over nonnegative integersPolynomials with nonnegative integer coefficients constitute asemimodule over nonnegative integers


Concept QRSemimodule

DefinitionQRSemimodule(T)⇒

Semimodule(T)

The partial binary operation remainder is defined on TThe partial binary infix functionquotient : T × T → Scalar(T) is definedFor all a ∈ T ,b , T(0) ∈ T ,

a = quotient(a,b) · b+ remainder(a,b)


gcd for QRSemimodule

template <typename T>requires(QRSemimodule(T))

T gcd(T a, T b){

// Precondition: ¬(a = T(0) ∧ b = T(0))

while (true) {if (b == T(0)) return a;a = remainder(a, b);if (a == T(0)) return b;b = remainder(b, a);

}}

This procedure does not always terminate


Concept EuclideanSemimodule

DefinitionEuclideanSemimodule(T)⇒

QRSemimodule(T)

For all a,b ∈ T such that ¬(a = T(0) ∧ b = T(0)), gcd(a,b)terminates


gcd

template <typename T>requires(EuclideanSemimodule(T))

T gcd(T a, T b){

// Precondition: ¬(a = T(0) ∧ b = T(0))

while (true) {if (b == T(0)) return a;a = remainder(a, b);if (a == T(0)) return b;b = remainder(b, a);

}}


Concept EuclideanSemiring

DefinitionEuclideanSemiring(T)⇒

CommutativeSemiring(T)

For all a,b ∈ T , a× b = T(0)⇒ a = T(0) ∨ b = T(0)

A Euclidean function w : T →N is definedFor all a,b ∈ T , w(a× b) > w(a)

Partial binary operations remainder and quotientare defined on TFor all a ∈ T ,b , T(0) ∈ T ,

remainder(a,b) , T(0)⇒ w(remainder(a,b)) < w(b)

The fact that w decreases with application of remainder assuresthat gcd terminatesStepanov, McJones Elements of Programming March 14, 2008 306 / 880

Euclidean semiring is Euclidean semimodule

Every commutative semiring is a semimodule over itselfThis implies every Euclidean semiring is a Euclidean semimoduleThis implies gcd can be used both for Euclidean monoids and thetraditional concept of Euclidean semirings 22, such as polynomialsover reals

22Technically, the traditional concept introduced by Noether and van der Waerdenis the Euclidean ring, but semirings suffice


Deriving a fast algorithm for quotient and remainder

Let a = n · b+ r, where r = weak_remainder(a,b)Let n = 2q+ p, where q = bn/2c and p = n mod 2a = q · (2 · b) + (p · b+ r) where (p · b+ r) < 2 · bremainder(a,b) = r =a if a < ba− b if a− b < b

remainder(a, 2 · b)) if remainder(a, 2 · b) < bremainder(a, 2 · b)) − b otherwise

quotient(a,b) =0 if a < b1 if a− b requires(Integer(I) && ArchimedeanMonoid(T))

pair<I, T> quotient_remainder_nonnegative(T a, T b){

// Precondition: a > T(0) ∧ b > T(0)if (a < b)

return pair<I, T>(I(0), a);if (a - b < b)

return pair<I, T>(I(1), a - b);pair<I, T> q = quotient_remainder_nonnegative(a, b + b);I n = q.first + q.first;a = q.second;if (a < b)

return pair<I, T>(n, a);else

return pair<I, T>(n + I(1), a - b);}


quotient_nonnegative

While we can compute remainder without quotient, we don’tknow how to compute quotient without remainderMost computer instruction sets include an instruction thatcomputes both quotient and remainderIt would be nice if programming languages provided a bindingfor this instruction

template <typename I, typename T>requires(Integer(I) && ArchimedeanMonoid(T))

I quotient_nonnegative(T a, T b){

return quotient_remainder_nonnegative(a, b).first;}


quotient_remainder_nonnegative_iterative

template <typename I, typename T>requires(Integer(I) && WeaklyHalvableMonoid(T))

pair<I, T> quotient_remainder_nonnegative_iterative(T a, const T& b){

// Precondition: a > T(0) ∧ b > T(0)if (a < b) return pair<I, T>(I(0), a);T c = b;while (a - c >= c) c = c + c;a = a - c;I n = I(1);while (c != b) {

n = n + n;c = half_nonnegative(c);if (c <= a) {

a = a - c;n = n + I(1);

}}return pair<I, T>(n, a);

}


Requirements for extending quotient and remainderto negative quantities

Quotient and remainder on signed quantities must satisfy theseproperties:

quotient(a,b) is an integera = b · quotient(a,b) + remainder(a,b)|remainder(a,b)| < |b|

For any integer n, remainder(a,b) = remainder(a+ n · b,b)


Problems with current implementations of quotientand remainder

Our requirements are not satisfied by many programminglanguages, in which quotient truncates toward zero 23

Truncation violates our fourth requirement 24

In addition, truncation is an inferior way of rounding because itsends twice as many values to zero as to any other integer, thusleading to a nonuniform distribution

23For an excellent discussion of this and other problems, see: Raymond T. Boute.The Euclidean Definition of the Functions div and mod. ACM Transactions onProgramming Languages and Systems, Volume 14, Number 2, April 1992, pages 127-144

24This requirement is equivalent to the classical mathematical definition ofcongruence: “If two numbers a and b have the same remainder r relative to the samemodulus k they will be called congruent relative to the modulus k (following Gauss).”—P. G. L. Dirichlet, Lectures on Number Theory, American Mathematical Society, 1999


Adapting nonnegative remainder and quotient

The next two functions are adaptors that convert remainder andquotient_remainder correctly defined for nonnegative inputs toproduce “correct” results for positive or negative inputs


remainder

template <typename T, typename Op>requires(OrderedAdditiveGroup(T) && BinaryOperation(Op) && Domain(Op) == T)

T remainder(T a, T b, Op rem){

// Precondition: b , 0T r;if (a < T(0))

if (b < T(0)) {r = -rem(-a, -b);

} else {r = rem(-a, b); if (r != T(0)) r = b - r;

}else

if (b < T(0)) {r = rem(a, -b); if (r != T(0)) r = b + r;

} else {r = rem(a, b);

}return r;

}


quotient_remainder

template <typename I, typename T, typename Op>requires(Integer(I) && OrderedAdditiveGroup(T) &&

HomogeneousFunction(Op) && Arity(Op) == 2 && Domain(Op) == T && Codomain(Op) == pair<I, T>)pair<I, T> quotient_remainder(T a, T b, Op quo_rem){

// Precondition: b , 0pair<I, T> q_r;if (a < 0) {

if (b < 0) { q_r = quo_rem(-a, -b); q_r.second = -q_r.second; }else {

q_r = quo_rem(-a, b);if (q_r.second != 0) { q_r.second = b - q_r.second; q_r.first = q_r.first + 1; }q_r.first = -q_r.first;

}} else {

if (b < 0) {q_r = quo_rem( a, -b);if (q_r.second != 0) { q_r.second = b + q_r.second; q_r.first = q_r.first + 1; }q_r.first = -q_r.first;

}else

q_r = quo_rem( a, b);}return q_r;

}


Concept DiscreteArchimedeanSemiring

DefinitionDiscreteArchimedeanSemiring(T)⇒

CommutativeSemiring(T)


For all a,b, c ∈ T , a < b∧ 0 < c⇒ a× c < b× cFor all a ∈ T , a < T(1)⇒ ¬(T(0) < a)

The last property is discreteness


Concept NonnegativeDiscreteArchimedeanSemiring

DefinitionNonnegativeDiscreteArchimedeanSemiring(T)⇒

NonnegativeDiscreteArchimedeanSemiring(T)

For all a ∈ T , T(0) 6 a


Concept DiscreteArchimedeanRing

DefinitionDiscreteArchimedeanRing(T)⇒

DiscreteArchimedeanSemiring(T)

AdditiveGroup(T)


Univalent concepts

DefinitionTwo types T1 and T2 are isomorphic if it is possible to write conversionfunctions from T1 to T2 and from T2 to T1 that preserve the proceduresand their semantics

DefinitionA concept is univalent if any types satisfying it are isomorphic

NonnegativeDiscreteArchimedeanSemiring is univalent; typessatisfying it are isomorphic toN, the natural numbers 25

DiscreteArchimedeanRing is univalent; types satisfying it areisomorphic to Z, the integers

25We follow Peano and include 0 in the natural numbers: Giuseppe Peano.Formulario Mathematico, Edizioni Cremonese, Roma, 1960, page 27


Computer integer types

Computer instruction sets and programming languages providepartial implementations of natural numbers and integersWhile providing a detailed formal specification for computerintegers lies outside the scope of this book, there are someimportant points we must makeFew if any programming languages provide access to the fullpower of hardware operations for addition, subtraction,multiplication, quotient, and remainderWe sketch appropriate interfaces


Bounded unsigned and signed integer types

DefinitionA bounded unsigned integer type, Un, where n = 8, 16, 32, 64, . . ., is anunsigned integer type capable of representing a value in the interval[0, 2n)

DefinitionA bounded signed integer type, Sn, where n = 8, 16, 32, 64, . . ., is asigned integer type capable of representing a value in the interval[−2n−1, 2n−1)


Operations for bounded unsigned and signed types

Programming languages typically provide operations restrictedwithin a single typeArithmetic operations such as multiplication and addition returnresults that are larger than the type of their operandsWhile computer instructions return full results, a programmer in ahigher-level language has no access to themExtended operations such as the ones below should be provided

sum_extended : Un ×Un ×U1 → U1 ×Undifference_extended : Un ×Un ×U1 → U1 ×Unproduct_extended : Un ×Un → U2n

quotient_remainder_extended : U2n ×Un → Un ×Un


Arithmetic in a programming language

A study of the instruction sets for modern computer architecturesshows the functionality that should be encompassedA good abstraction of these instruction sets is provided byKnuth’s MMIX 26

26Donald E. Knuth. The Art of Computer Programming, Volume 1, Fascicle 1, MMIX : ARISC Computer for the New Millenium, Addison-Wesley, 2005, pages 1-28


Conclusions

Our task is to combine algorithms and mathematical structuresinto a seamless whole by describing algorithms in abstract termsand adjusting theories to fit algorithmic requirementsIf this chapter seems too mathematical, it should be noted that thealgorithms and mathematics in it are modern restatements ofresults that are thousands of years old


References

Donald Knuth.The Greatest Common Divisor, and Division of Polynomials.The Art of Computer Programming, Volume 2, 3rd edition, 1998,Sections 4.5.2 and 4.6.1, pages 333-356 and 420-439.Exhaustive coverage of both Euclidean and Stein (binary) gcd.

Pierre Samuel.About Euclidean Rings.Journal of Algebra, Volume 19, 1971, pages 282-301.Exhaustive and relatively elementary treatment of Euclidean rings.


binary_gcd_nonnegative

In 1961, Josef Stein discovered the following gcd algorithm 27:

template <typename T>requires(Integer(T))

T binary_gcd_nonnegative(T a, T b){

if (is_zero(a)) return b;if (is_zero(b)) return a;int d = 0;while (is_even(a) && is_even(b)) {

halve_nonnegative(a); halve_nonnegative(b); d = d + 1;}while (is_even(a)) halve_nonnegative(a);while (is_even(b)) halve_nonnegative(b);while (true)

if (a < b) {b = b - a; do { halve_nonnegative(b); } while (is_even(b));

} else if (b < a) {a = a - b; do { halve_nonnegative(a); } while (is_even(a));

} else return binary_scale_up_nonnegative(a, d);}

27Josef Stein. Computational problems associated with Racah algebra. J. Comput.Phys., Volume 1, 1967, pages 397-405


Generalizations of binary_gcd_nonnegative

While Stein’s algorithm might at first appear to be a “hack”dependent on binary integers, it is actually much deeperThe key observation is that 2 is the smallest integer prime, and theonly nonzero remainder mod2 is 1, a unit or invertible ringelementIt is possible to generalize it to other domains by using smallestprimes in those domains, such as the monomial x for polynomialsover reals or 1 + i for Gaussian integers

Polynomials See Knuth (Exercise 4.6.1.6, page 435 andSolution, page 673)

Gaussian integers See WeilertOther algebraic integer rings See Damgård and Frandsen,

and Agarwal and Frandsen


Project

ProjectFind the correct abstract setting for binary binary_gcd_nonnegative(Stein domain)


Additional references for binary_gcd_nonnegative

Andre Weilert.(1+ i)-ary GCD Computation in Z[i] as an Analogue of the BinaryGCD Algorithm.J. Symbolic Computation (2000) 30, pages 605-617.

Ivan Bjerre Damgård and Gudmund Skovbjerg Frandsen.Efficient algorithms for GCD and cubic residuosity in the ring ofEisenstein integers.Proceedings of the 14th International Symposium on Fundamentals ofComputation Theory, Lecture Notes in Computer Science 2751,Springer-Verlag (2003), pages 109-117.

Saurabh Agarwal and Gudmund Skovbjerg Frandsen.Binary GCD Like Algorithms for Some Complex Quadratic Rings.ANTS 2004, pages 57-71.


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II

Memory and dereferencingActionsIteratorsRangesForward iteratorsBidirectional iteratorsIndexed iteratorsRandom access iteratorsConclusionsReferenceProject


9 Rotations


Contents III






15 C++ machinery

16 Acknowledgments


Contents IV

17 Index


Memory

IntuitionMemory is a set of locations, each with an address and a valueGetting or setting the value given an address is “fast”The association of a value with a location is changeable

ExamplesThe physical and virtual address spaces of a computerThe locations visited by an algorithmThe locations owned by a data structure


Concept Readable

DefinitionReadable(T)⇒

The type function ValueType(T) is definedRegular(ValueType(T))source : T → ValueType(T) is defined a

asource returns its result by value, by reference, or by constant reference


Concept Writable

DefinitionWritable(T)⇒

The type function ValueType(T) is definedRegular(ValueType(T))If x ∈ T and v ∈ ValueType(T), then sink(x)← v is defined

No other use of sink(x) can be justified by the concept Writable,although other uses may be supported by a specific type modelingWritableFor a particular state of an object x, only a single assignment tosink(x) can be justified by the concept Writable; a specific typemight provide a protocol allowing subsequent assignments tosink(x)


is_aliased

template <typename T, typename U>requires(Writable(T) && Readable(U) && ValueType(T) == ValueType(U))

bool is_aliased(const T& x, const U& y){

// return true⇔ immediately after executing sink(x)← v, source(y) = v// For many types, this works:typedef const ValueType(T)* P;return P(&sink(x)) == P(&source(y));

}


Concept Mutable

DefinitionMutable(T)⇒

Readable(T) ∧ Writable(T)For all x ∈ T , is_aliased(x, x)


Concept Dereferenceable

DefinitionDereferenceable(T)⇒

Readable(T) ∨ Writable(T) ∨ Mutable(T)

Dereferenceable is a useful concept for describing types eventhough it is not used in programs (since it’s not clear which ofsource or sink could be used)


Extending Dereferenceable functions to all regular types

ReflectionIt is useful if dereferencing an object x that doesn’t refer to anotherobject returns x itselfTherefore we assume that unless otherwise defined,ValueType(T) = T and source and sink return the object towhich they are applied


Concept Action

DefinitionAction(T)⇒

ProperProcedure(T)Arity(T) = 1T takes its argument by reference


Duality of actions and transformations

For every action there is a corresponding transformation, and viceversaThe following two functions demonstrate this equivalenceThe complexity of an action when implemented independently ofthe corresponding transformation could be smaller

Example: interchange the first and last elements of a sequence


transformation_from_action

template <typename A>requires(Action(A))

struct transformation_from_action{

A a;transformation_from_action() {}transformation_from_action(const A& a) : a(a) {}Domain(A) operator()(Domain(A) x) { a(x); return x; }

};


action_from_transformation


struct action_from_transformation{

F f;action_from_transformation() {}action_from_transformation(const F& f) : f(f) {}void operator()(Domain(F)& x) { x = f(x); }

};


Concept RegularAction

DefinitionRegularAction(T)⇒

Action(T)

RegularFunction(transformation_from_actionT )


Concept AmortizedConstantTime

DefinitionThe amortized complexity of an operation is the complexity averagedover a worst-case sequence of operations a

aFor an extensive treatment of amortized complexity, see: Robert Endre Tarjan.Amortized Computational Complexity. SIAM Journal on Algebraic and DiscreteMethods, Volume 6, Number 2, April 1985, pages 306-318

DefinitionAmortizedConstantTime(T)⇒

ProperProcedure(T)The amortized time complexity of it is constant and “small”

Formalizing complexity specifications of abstract operations is asubject for future research


Concept Iterator 28

DefinitionIterator(T)⇒

Regular(T)The type function DistanceType(T) is definedInteger(DistanceType(T))The prefix operator ++ is defined on TAction(++)

AmortizedConstantTime(++)

28Our treatment of iterators departs significantly from that in the STLStepanov, McJones Elements of Programming March 14, 2008 348 / 880

Iterator protocols

++ is not necessarily regularIf there are two copies of an iterator and one is incremented, theother may become invalidThus an iterator is too weak to be used in a multipass algorithm

Assignment to a sink is not idempotent: a call to ++must separatetwo assignments to an iterator

The asymmetry between readable and writable iterators isintentionalThe ability to read from sourcemore than once allows us to writesimple, useful functions like find_ifNo corresponding benefits seem to accrue in the case of sink


Examples of Iterator

An iterator where ++ advances an input streamAn iterator where ++ advances an output streamAn iterator on a singly-linked listAn iterator on a doubly-linked listAn iterator on a one-dimensional arrayint∗int


successor and - for Iterator

template <typename I>requires(Iterator(I))

I successor(I f){

++f;return f;

}


DistanceType(I) operator-(I l, I f){

return distance(f, l, successor);}


+= and + for Iterator


void operator+=(I& f, DistanceType(I) n){

typedef DistanceType(I) N;while (n != N(0)) {

++f;n = n - N(1);

}}


I operator+(I f, DistanceType(I) n){

f += n;return f;

}


Orbit of successor

Since successor is a transformation, it defines an orbit starting atany iteratorIn this book, we only deal with terminating iterator orbitsTherefore, if, for i > 0, successori(x) is defined, it is not equal to x


The natural ordering on an orbit under successor

There exists a natural total ordering ≺ on the iterators in the orbitof x under successor defined by

x ≺ y⇔ (∃i > 0)y = successori(x)

≺ and the corresponding nonstrict � are used in specificationssuch as preconditions and postconditions of algorithmsFor many pairs of values of an iterator type, ≺ is not defined, sothere is often no effective way to write code implementing ≺

There is no efficient way to determine if one node precedes anotherin a linked structure (the nodes might not even be linked together)


Sequences of iterators

It is useful to have standard terminology and notation forsequences and subsequences of iterators

To describe the input and/or output of an algorithmTo describe ranges in a data structure

It is useful to describe a (possibly empty) sequence of iteratorsstarting at a particular iterator

ExampleBinary search looks for the sequence of iterators whose values areequal to a given valueThis sequence is empty if there are no such values

The location of this empty sequence carries information: the valuesin the sequence approximating the given value


Ranges

Such a located sequence of iterators is fully specified by a startingiterator and a nonnegative integerIt is often convenient to specify a sequence by specifying a startingiterator together with another one that bounds the sequenceA specification of a such a located sequence is called a rangeRanges can be either counted or bounded, and either semi-open orclosed


Different kinds of ranges

DefinitionA semi-open counted range Ji,nM, where n > 0 is an integer, denotes thesequence of iterators {j|i � j ≺ successorn(i)}

DefinitionA closed counted range Ji,nK, where n > 0 is an integer, denotes thesequence of iterators {k|i � k � successorn−1(i)}

DefinitionA semi-open bounded range [i, j) denotes the sequence of iterators{k|i � k ≺ j}

DefinitionA closed bounded range [i, j] denotes the sequence of iterators{k|i � k � j}


Terminology for iterators in a range

DefinitionAn iterator f in a range [i, j) is called the first if f = i∧ f , j

An iterator l in a range [i, j) is called the limit if l = j

An iterator k in a range [i, j) is called the lasta if successor(k) = j

Otherwise an iterator in the range is called not last

aThe terminology distinguishing limit and last was suggested by John Banning


Size of a range

DefinitionThe size of a counted range Ji,nM or Ji,nK is nThe size of a semi-open bounded range [i, j) isdistance(i, j, successor)The size of a closed bounded range [i, j] isdistance(i, j, successor) + 1

The size of a range is used in preconditions and postconditions ofalgorithms even when it cannot be effectively computed becauseof an iterator too weak for a multipass algorithm


Empty ranges

DefinitionAn empty semi-open range is specified by Ji, 0M or [i, i) for someiterator iThere are no empty closed ranges


Definition space of ++ on ranges

Lemma++ and successor are defined for every iterator in a semi-open range,and for every iterator except the last in a closed range


Readable, writable, and mutable ranges

DefinitionA range r is readable, writable, or mutable if, correspondingly, source,sink, or both of them are defined on all the iterators in the range


Type requirements of copy

template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))O copy(I f, I l, O o);

Readable(I) ensures source is definedIterator(I) ensures ++ is definedWritable(O) ensures sink is definedIterator(O) ensures ++ is definedValueType(I) = ValueType(O) ensures assignment is defined


Precondition and postcondition of copy


ValueType(I) == ValueType(O))O copy(I f, I l, O o);

Precondition The input range must be readableThe output range must be writable and of at leastthe size of the input rangeIf the ith iterator in the input range is aliased to thejth iterator of the output, then i 6 j (informally,every value is used before it is overwritten)

Postcondition The sequence of values in the output range is equalto the sequence of original values in the input range


Implementation of copy


ValueType(I) == ValueType(O))O copy(I f, I l, O o){

while (f != l) {sink(o) = source(f);++f;++o;

}return o;

}

We return the end of the output range because it might not beknown to the caller, who might find it useful

It is worth the small constant time to return it


Example using copy

template <typename O>requires(Writable(O) && Iterator(O) && Integer(ValueType(O)))

O iota(ValueType(O) n, O o) // like APL ι{

return copy(ValueType(O)(0), n, o);}


copy_bounded


ValueType(I) == ValueType(O))pair<I, O> copy_bounded(I f, I l, O o_f, O o_l){

while (f != l && o_f != o_l) {sink(o_f) = source(f);++f;++o_f;

}return pair<I, O>(f, o_f);

}

While the ends of both ranges are known to the caller, returningthe pair allows determining which range is smaller and where inthe larger range copying stopped


copy_n


ValueType(I) == ValueType(O))pair<I, O> copy_n(I f, DistanceType(I) n, O o){


sink(o) = source(f);++f;++o;n = n - N(1);

}return pair<I, O>(f, o);

}


copy_k

template <int k, typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

ValueType(I) == ValueType(O))struct copy_k{

void operator()(I& f, O& r){

copy_k<k - 1, I, O>(f, r);sink(r) = source(f);++f; ++r;

}};

template <typename I, typename O>struct copy_k<0, I, O> {

void operator()(I&, O&) { }};


copy_n_unrolled


ValueType(I) == ValueType(O))pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r){

typedef DistanceType(I) N;const int k = 4; // unroll factorwhile (n >= N(k)) {

copy_k<k, I, O>()(f, r);n = n - N(k);

}return copy_n(f, n, r);

}


Concept ForwardIterator

DefinitionForwardIterator(T)⇒

Iterator(T)RegularAction(++)

Iterator and ForwardIterator differ only by an axiom; there are nonew operations


Examples of ForwardIterator

An iterator on a singly-linked listAn iterator on a doubly-linked listAn iterator on a one-dimensional arrayint∗int


Advantages of regularity of ++

ReflectionForwardIterator allows

algorithms that maintain more than one iterator into a rangemultipass algorithms


Concept BidirectionalIterator

DefinitionBidirectionalIterator(T)⇒

ForwardIterator(T)The prefix operator -- and the corresponding transformationpredecessor are definedRegularAction(--)

AmortizedConstantTime(--)For any iterator k in a semi-open range [i, j):

k , j⇒ predecessor(successor(k)) = k

k , i⇒ successor(predecessor(k)) = k


Examples of BidirectionalIterator

An iterator on a doubly-linked listAn iterator on a one-dimensional arrayint∗int


-= and - for BidirectionalIterator

template <typename I>requires(BidirectionalIterator(I))

void operator-=(I& f, DistanceType(I) n){


--f;n = n - N(1);

}}


I operator-(I f, DistanceType(I) n){

f -= n;return f;

}


Using copy with overlapping ranges

If we need to copy a range to a destination that begins within thesource range, copy cannot be used because its aliasingprecondition is violated


copy_backward

The main difference from copy is the aliasing precondition: if theith iterator in the input range is aliased to the jth iterator of theoutput, then j 6 i

template <typename I, typename O>requires(Readable(I) && BidirectionalIterator(I) &&

Writable(O) && BidirectionalIterator(O) &&ValueType(I) == ValueType(O))

O copy_backward(I f, I l, O o){

while (f != l) {--l;--o;sink(o) = source(l);

}return o;

}


copy_backward_n


Writable(O) && BidirectionalIterator(O) &&ValueType(I) == ValueType(O))

pair<I, O> copy_backward_n(I l, DistanceType(I) n, O o){


--l;--o;sink(o) = source(l);n = n - N(1);

}return pair<I, O>(l, o);

}


Concept IndexedIterator

DefinitionIndexedIterator(T)⇒

ForwardIterator(T)The mutating assignment += is defined on T ×DistanceType(T)- : T × T → DistanceType(T) is definedAmortizedConstantTime(+=)AmortizedConstantTime(-)

+= and -, which were defined for Iterator, are now required to beprimitive


+ for IndexedIterator

The default implementation of + : I×DistanceType(I)→ I interms of += for Iterator becomes constant-time for IndexedIterator


Examples of IndexedIterator

An iterator on a one-dimensional arrayint∗int


copy_k_indexed

template <int k, typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&ValueType(I) == ValueType(O))

struct basic_copy_k_indexed {void operator()(I f, O o){

basic_copy_k_indexed<k - 1, I, O>()(f, o);sink(o + (k - 1)) = source(f + (k - 1));

}};

template <typename I, typename O>struct basic_copy_k_indexed<0, I, O>{ void operator()(I, O) { } };

template <typename I, typename O, int k>requires(Readable(I) && IndexedIterator(I) &&


void copy_k_indexed(I& f, O& o) {basic_copy_k_indexed<k, I, O>()(f, o);f += k; o += k;

}Stepanov, McJones Elements of Programming March 14, 2008 383 / 880

copy_n_unrolled_indexed

template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&


pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O o){


copy_k_indexed<k, I, O>(f, o);n = n - N(k);

}return copy_n(f, n, o);

}


Out-of-order execution

Out-of-order execution of adjacent iterations speeds up theexecution of copyA compiler cannot achieve this because it cannot determine thatthe source and destination ranges do not aliasProgrammers know when this is the case, and should be able tospecify when such interleaved execution is legitimate


copy_parallel for disjoint ranges

The main difference from copy is the aliasing precondition: theinput and output ranges do not aliasA special forall executes its body for all values of its iterationvariable in arbitrary order, possibly concurrently



void copy_parallel(I f, DistanceType(I) n, O o){

typedef DistanceType(I) N;forall(N i, N(0), n) {

sink(o + i) = source(f + i);}

}


Concept RandomAccessIterator (1 of 2)

Definition (requirements, signatures)RandomAccessIterator(T)⇒

BidirectionalIterator(T) ∧ IndexedIterator(T)StrictTotallyOrdered(T)

The type function DifferenceType(T) is definedInteger(DifferenceType(T))DifferenceType(T) is large enough to contain distances and theirnegationsThe mutating assignments += and -= are defined onT ×DifferenceType(T)+ : T ×DifferenceType(T)→ T is defined- : T ×DifferenceType(T)→ T is defined- : T × T → DifferenceType(T) is defined


Concept RandomAccessIterator (2 of 2)

Definition (complexity, definition spaces)RandomAccessIterator(T)⇒

AmortizedConstantTime(<)AmortizedConstantTime(-=)AmortizedConstantTime(- : T ×DifferenceType(T)→ T)

+=, -=, +, and - : T ×DifferenceType(T)→ T accept negativevalues for their second argumentAmortizedConstantTime(- : T × T → DifferenceType(T))

- : T × T → DifferenceType(T)) accepts iterators in either order


Axioms for RandomAccessIterator

ExerciseWrite an appropriate set of axioms relating the operations to each other


Examples of RandomAccessIterator

int∗int


Equivalence ofRandomAccessIterator and IndexedIterator

TheoremFor any function defined on a range of random access iterators, there isanother function defined on indexed iterators with the sameasymptotic complexity


Proof of the equivalence

Assuming these definitions:

U ≡ DistanceType(I)W ≡ sign_extendedU

where sign_extended is a templated struct adding a sign bit andappropriate integer operations, we rewrite the function with thesesubstitutions:

Replace WithDifferenceType(I) W

i < j i - f < j - fi += nwhen n < 0 i = f + ((i - f) - U(-n))

i -= n i += -ni - j W(i - f) - W(j - f)


Reflection on RandomAccessIterator and IndexedIterator

The theorem shows the theoretical equivalence of these conceptsin any context in which the beginning of ranges are known

copy_backward does not satisfy this requirement!

In practice we have found there is no performance penalty forusing the weaker concept


copy_backward_n_indexed

copy_backward is not realizable for indexed iteratorscopy_backward_n_indexed, which is often just as useful, isrealizable



void copy_backward_n_indexed(I f, DistanceType(I) n, O o){


n = n - N(1);sink(o + n) = source(f + n);

}}

There is no useful information to return


Relationships between iterator concepts

It FI

BI

II

RI


Conclusions

Refinement generates mathematical structures such as groups,Abelian groups, totally-ordered groups, and Archimedean groupsIt also generates concepts describing the fundamental notion ofcomputer science, iterating through a data structureWe have used three types of refinement, by adding

an operationan axioma tighter complexity requirement


Reference

Alexander Stepanov and Meng Lee.The Standard Template Library.HP Laboratories Technical Report 95-11(R.1), November 14, 1995.Introduced different categories of iterators and their axioms.


Intuition for SegmentedIterator

There are data structures where ++ can be implemented fasterwithin certain ranges or segments

hash tablesadjacency list representation of graphsSTL-like deques

It is possible to optimize many algorithms for such structures bytransforming inner loops into nested loops:

a loop over segmentsa loop within a segment

This results in a new dimension of the classification of iterators:homogeneoussegmented


Concept SegmentedIterator (1 of 2)

Definition (requirements, signatures, definition spaces)SegmentedIterator(T)⇒

ForwardIterator(T)The type function SegmentIterator(T) is definedForwardIterator(SegmentIterator(T))begin : T → SegmentIterator(T) is definedend : T → SegmentIterator(T) is defined+ : T × SegmentIterator(T)⇒ T is definedAmortizedConstantTime(begin)

AmortizedConstantTime(end)AmortizedConstantTime(+)

For any i ∈ [f, l], begin, end, and + are definedFor any w ∈ [begin(i), end(i)], i+w is defined


Concept SegmentedIterator (2 of 2)

Definition (axioms)SegmentedIterator(T)⇒If

[f, l) is a range of segmented iteratorsi ∈ [f, l)w is a segment iterator in [begin(i), end(i))

then the following hold:1 is_aliased(i,begin(i))

2 begin(i+w) = w

3 i+ begin(i) = i

4 i+ (begin(i) + 1) = i+ 15 begin(i) + 1 , end(i)⇒ begin(i) + 1 = begin(i+ 1)

6 begin(i) , end(i)


copy_from_segmented

template <typename I, typename O>requires(Readable(I) && SegmentedIterator(I) &&

Writable(O) && Iterator(O) &&ValueType(I) == ValueType(O))

O copy_from_segmented(I f, I l, O o){

while (f + end(f) != l + end(l)) {// f and l are in different segmentso = copy(begin(f), end(f), o);f = f + end(f);

}// f and l are in the same segmentreturn copy(begin(f), begin(l), o);

}


Project

Implement segmented iterators for one or more data structuresSTL-like dequeSGI STL-like hashed containers

Produce segmented iterator versions of suitable (STL) algorithmsAnalyze if the axioms are independent, consistent, and complete


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II

8 Permutations and rearrangementsPermutationsPermutation groupsCycle decomposition of a permutationRearrangementsUnderlying typeRearranging arbitrary cyclesReverse permutationReverse algorithmsCategory dispatchConclusionsReading

9 Rotations


Contents III






15 C++ machinery

16 Acknowledgments


Contents IV

17 Index


Onto functions

DefinitionA (regular) function f is onto if for all y ∈ Codomain(F), there existsx ∈ Domain(R) such that y = f(x)


One-to-one functions

DefinitionA (regular) function f is one-to-one if for all x, x ′ ∈ Domain(F),f(x) = f(x ′)⇒ x = x ′


Permutation

DefinitionA permutation is a transformation on a finite domain that is one-to-oneand onto


Example of a permutation on [0, 6)

p(0) = 5p(1) = 2p(2) = 4p(3) = 3p(4) = 1p(5) = 0


Identity permutation

DefinitionA fixed point of a transformation is an element x such that f(x) = x

DefinitionThe identity permutation on a set S, identityS, maps each element of S toitself; every element in S is a fixed point of identityS


Composition of permutations

DefinitionIf p and q are two permutations on a set S, the composition q ◦ p takesx ∈ S to q(p(x))

LemmaThe composition of permutations is a permutation

LemmaComposition of permutations is associative


Inverse of a permutation

LemmaFor every permutation p on a set S, there is an inverse permutation p−1

such that p−1 ◦ p = p ◦ p−1 = identityS


Permutation group

The permutations on a set form a group under composition

LemmaEvery group is a subgroup of a permutation group of its elementswhere every permutation in the subgroup is generated by multiplyingall the elements by an individual element

Example

× 1 2 3 41 1 2 3 42 2 4 1 33 3 1 4 24 4 3 2 1

Multiplication mod 5:Every row and column of the multiplication table is a permutation


Cycles in a permutation

DefinitionA cycle is a circular orbit within a permutation

DefinitionA trivial cycle is one with a cycle size of 1

The element in a trivial cycle is a fixed point

LemmaEvery element in a permutation of a finite set belongs to a unique cycle


Example of cycle decomposition

p(0) = 5p(1) = 2p(2) = 4p(3) = 3p(4) = 1p(5) = 0

p = (0 5)(1 2 4)(3)


Structure of cycle decomposition

LemmaAny permutation of a set with n elements contains k 6 n disjoint cyclesEach cycle is itself a permutation of this set, called a cyclic permutationDisjoint cyclic permutations commuteEvery permutation can be represented as a product of its cyclesThe inverse of a permutation is the product of the inverses of its cycles


Finite sets

DefinitionA finite set S of size n is a set for which there exists a pair of functions

chooseS : [0,n)→ S

indexS : S→ [0,n)

satisfying

chooseS(indexS(x)) = x

indexS(chooseS(i)) = i


Index permutation of a permutation

DefinitionIf p is a permutation on a finite set S of size n, there is a correspondingindex permutation p ′ on [0,n) defined as

p ′(i) = indexS(p(chooseS(i)))

Lemma

p(x) = chooseS(p′(indexS(x)))

We will frequently define permutations by the correspondingindex permutations


Rearrangement

DefinitionA rearrangement is an algorithm that rearranges the elements of amutable range to satisfy a given postcondition


Classifying rearrangements

Rearrangements are classified according to the following fundamentalcharacteristics:

Characterization of the postconditionPostcondition kindStability

Implementation constraintsIterator requirementsMutative versus copying

ComplexityTimeSpace


Postcondition kind

position-based The destination of a value depends only on its originalposition and not on the value itself; for example,“reverse the range”

bin-based The destination of a value depends only on the result ofapplying a k-way bin function to that value, and notdirectly on the value itself; for example, “move badvalues before good ones”

A useful subcase is predicate-based algorithms,where k = 2

ordering-based The destination of a value depends only on the outcomeof applications of an ordering to pairs of values in therange; for example, “put the smallest value first”


Stability

DefinitionA rearrangement is stable if it respects the original order of the range tothe maximal extent possible while satisfying the postcondition

ExamplesStable index partition Move all the elements at even positions in the

sequence before the elements at odd positions,keeping the original order in both groups

Stable partition Move all the elements with even values beforethose with odd values, keeping the originalorder in both groups

Stable sort Sort pairs of integers by their first componentsso pairs with equal first components remain inthe original order


Iterator requirements

For the same problem, there are often different algorithms fordifferent iterator requirements

Examplesreverse_forwardreverse_bidirectionalreverse_indexed


Mutative versus copying

DefinitionRearrangements as we have defined them are mutative

DefinitionA rearrangement is copying if it sets a writable range to a rearrangedcopy of a readable range in way that satisfies a given postcondition

A copying rearrangement could always be obtained by composingcopywith the corresponding mutative rearrangementOften, however, there are faster algorithms that rearrange whilecopying


Space complexity

DefinitionA mutative algorithm is in place (or in situ) if it uses an amount ofadditional space that is (poly-)logarithmic in the size of the input

DefinitionA memory-adaptive algorithm uses as much additional space as it canacquire to maximize performance

A small percentage of additional space, while theoretically“linear,” can lead to a large performance improvement

DefinitionAn algorithm with buffer requires the caller to provide a buffer to beused by the algorithm


Time complexity

Rearranging n elements might take n+ k assignments for someconstant kn logn algorithms arise

Determining the final position may require collecting additionalinformation, as in sortingAbsence of random access on some iterators requires multipassalgorithms, as with in-place random shuffle for forward iterators


cycle_2

template <typename I>requires(Mutable(I) && Iterator(I))

void cycle_2(I x, I y){

ValueType(I) t = source(x);sink(x) = source(y);sink(y) = t;

}


Problems with cycle_2

It could be much slower than necessarystd::vector copies element-by-element instead of interchangingthe headers

It could throw an exception due to unnecessary resource allocationWe should exploit the fact it is needed for implementing mutativerearrangements, which should not need to construct or destroyany objects, but just move them aroundWe will address the issue in Chapters 12 with a notion of anunderlying typeFor the time being, imagine that the value types of iterators arebuilt-in types or C-style structs


cycle_left_3


void cycle_left_3(I x, I y, I z){

ValueType(I) t = source(x);sink(x) = source(y);sink(y) = source(z);sink(z) = t;

}


To-permutation and from-permutation

DefinitionEvery rearrangement corresponds to two permutations of itsrange

A to-permutation mapping an iterator i to the iterator pointing to thedestination of the element at iA from-permutation mapping an iterator i to the iterator pointing tothe origin of the element moved to i

These two permutations are inverses of each other


do_cycle_from

template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&

Transformation(P) && Domain(P) == I)void do_cycle_from(I i, P p){

// Precondition: p is a from-permutation of the range containing iValueType(I) t = source(i);I f = i;I n = p(i);while (n != i) {

sink(f) = source(n);f = n;n = p(n);

}sink(f) = t;

}


Strict lower bound for rearrangement

TheoremThe minimum number of assignments required for a rearrangement isn+ cN − cT , where n is the number of elements, cN the number ofnontrivial cycles, and cT the number of trivial cycles


Proof of strict lower bound for rearrangement

Consider a rearrangement realizing a permutation p, and let C bea cycle of p of length > 1The rearrangement must include an assignment of a value of C toeach memory location of CSince such an assignment overwrites the value at that location,there must be an additional assignment of some value of C to alocation outside CSince the cycles are disjoint, these additional assignments aredistinct, so the total number of assignments must be at leastn+ cN − cT , where n is the number of elements of p, cN thenumber of nontrivial cycles, and cT the number of trivial cycles29

do_cycle_from (applied to nontrivial cycles) make this boundstrict

29The idea for the proof was provided to us by John WilkinsonStepanov, McJones Elements of Programming March 14, 2008 434 / 880

do_cycle_to

ExerciseImplement do_cycle_to and compare the number of assignments itperforms to do_cycle_from


Reverse permutation

DefinitionThe permutation p on a finite set with n elements defined by an indexpermutation p(i) = (n− 1)− i is called the reverse permutation of the set

The number of nontrivial cycles in a reverse permutation is bn/2c;the number of trivial cycles is n mod 2bn/2c is the largest possible number of nontrivial cycles in apermutation


Reverse rearrangement

DefinitionThe reverse rearrangement of a range is the rearrangement induced bythe reverse permutation

The lower bound formula gives the number of assignments asn+ cN − cT = 3bn/2c


reverse_n_indexed

The definition of reverse directly gives this:

template <typename I>requires(Mutable(I) && IndexedIterator(I))

void reverse_n_indexed(I f, DistanceType(I) n){

typedef DistanceType(I) N;N i(0);while (i < n / N(2)) {

cycle_2(f + i, f + ((n - N(1)) - i));i = i + N(1);

};}

The code does the lower bound number of assignmentsIf the algorithm is used with forward or bidirectional iterators, itperforms a quadratic number of iterator incrementsSince all the cycles are disjoint, this code benefits from forall


Return value of reverse

It is tempting to define the reverse algorithms to return the rangeof elements that were not moved

The middle element when the size of the range is oddThe empty range between the two “middle” elements when thesize of the range is even

However, we do not know of any example when it is useful and,therefore, return void


reverse_bidirectional

template <typename I>requires(Mutable(I) && BidirectionalIterator(I))

void reverse_bidirectional(I f, I l){

while (f != l && f != predecessor(l)) {--l;cycle_2(f, l);++f;

}}


reverse_n_bidirectional


void reverse_n_bidirectional(I f, I l, DistanceType(I) n){

// Precondition: n 6 l− ftypedef DistanceType(I) N;N i(0);while (i < n / N(2)) {

--l;cycle_2(f, l);++f;i = i + N(1);

}}

Passing n < l− f effectively gives reverse_until_n


reverse_indexed


void reverse_indexed(I f, I l){

reverse_n_indexed(f, l - f);}


Intuition for divide and conquer reverse algorithm

1 Split the range into two parts2 Reverse each part3 Interchange the parts


Illustration of divide and conquer reverse algorithm

[a] b [c] d [e] f [g] [h] i [j] k [l] m [n]

[c b a] d [g f e] [j i h] k [n m l]

[g f e d c b a] [n m l k j i h]

n m l k j i h g f e d c b a


swap_ranges_n

template <typename I1, typename I2, typename N>requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&ValueType(I1) == ValueType(I2) &&Integer(N))

pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n){

while (n != N(0)) {cycle_2(f1, f2);++f1;++f2;n = n - N(1);

};return pair<I1, I2>(f1, f2);

}


reverse_n_recursive

template <typename I>requires(Mutable(I) && ForwardIterator(I))

I reverse_n_recursive(I f, DistanceType(I) n){

typedef DistanceType(I) N;const N h = n / N(2);const N r = n - N(2) * h;if (h == N(0))

return f + n;I m = reverse_n_recursive(f, h);m += r;I l = reverse_n_recursive(m, h);swap_ranges_n(f, m, h);return l;

}


Correctness of reverse_n_recursive

LemmaThe reverse permutation on [0,n) is the only permutation satisfyingi < j⇒ p(j) < p(i)

1 This condition obviously holds for ranges of size 12 The recursive calls inductively establish that the condition holds

within each half3 swap_ranges_n reestablishes the condition between the halves

(and the skipped middle element, if any)


Complexity of reverse_n_recursive

Lemma

For a range of length n =∑blognci=0 ai2i, where ai is the ith digit in the

binary representation of n, the number of assignments equals32∑blognci=0 aii2i


reverse_n_forward


void reverse_n_forward(I f, DistanceType(I) n){

reverse_n_recursive(f, n);}


reverse_forward


void reverse_forward(I f, I l){

reverse_n_forward(f, l - f);}


reverse_copy_n

template <typename B, typename I>requires(Mutable(B) && BidirectionalIterator(B) &&

Mutable(I) && Iterator(I) &&ValueType(B) == ValueType(I))

I reverse_copy_n(B l, DistanceType(I) n, I r){


--l;sink(r) = source(l);++r;n = n - N(1);

}return r;

}


reverse_n_with_buffer

template <typename I, typename B>requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&ValueType(I) == ValueType(B))

I reverse_n_with_buffer(I f, DistanceType(I) n, B b){

return reverse_copy_n(copy_n(f, n, b), n, f);}


reverse_n_adaptive


Mutable(B) && BidirectionalIterator(B) &&ValueType(I) == ValueType(B))

I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b){


return f + n;if (n <= n_b)

return reverse_n_with_buffer(f, n, b);I m = reverse_n_adaptive(f, h, b, n_b);m += r;I l = reverse_n_adaptive(m, h, b, n_b);swap_ranges_n(f, m, h);return l;

}


Complexity of reverse_n_adaptive

ExerciseDerive a formula for the number of assignments performed byreverse_n_adaptive for given range and buffer sizes


Conjecture

ConjectureThere is no algorithm that reverses a forward iterator range withpolylogarithmic additional space and in linear time


Selecting algorithms according to requirements

It is important to be able to automatically select from a family ofalgorithms based on the requirements the types satisfyIn Appendix 2 we show a mechanism called category dispatch fordoing such selection in C++Future versions of C++may provide a more natural way ofexpressing it


Conclusions

Permutations lead us back to algebra, specifically group theoryA permutation of a range induces a rearrangementA taxonomy of rearrangements was presentedWeakening iterator requirements often raises interestingalgorithmic problemsA convention known as iterator category dispatch allowsautomatic selection of the appropriate algorithmWe introduced a new class of algorithms, memory-adaptivealgorithms


Reading

Donald Knuth.Section 1.2.5: Permutations and Factorials.The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,pages 45-50.


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 RotationsRotation permutationsLeast common multipleCycle structure of rotationsInterface for rotaterotate for indexed iteratorsLoop fusionrotate for bidirectional iteratorsrotate for forward iteratorsConclusionsReadingProjects


Contents III






15 C++ machinery

16 Acknowledgments


Contents IV

17 Index


Definition of rotation

DefinitionThe permutation p on a finite set with n elements defined by an indexpermutation p(i) = (i+ k) mod n is called the k-rotation of the set

LemmaThe inverse of a k-rotation on an n-element set is an (n− k)-rotation


Least common multiple

DefinitionThe least common multiple of integers a and b, written lcm(a,b), is thesmallest integerm such that both a and b divide it

Lemmalcm(a,b) is a multiple of a and b that divides any other multiple of aand b


Connecting lcm and gcd

Theorema · b = lcm(a,b) · gcd(a,b)

Proof.If the prime decompositions of a and b are respectively

∏puii and∏

pvii , then a · b =

∏pui+vii , lcm(a,b) =

∏p

max(ui,vi)i , and

gcd(a,b) =∏p

min(ui,vi)i , and the result follows from the useful

identity x+ y = max(x,y) + min(x,y) �


(Divisibility lattice of integers)

DefinitionAn ordered set E is said to be a lattice if every subset consisting of twoelements of E has a least upper bound and a greatest lower bound in E

ExampleThe set of integers > 1, ordered by the relation “m divides n” betweenm and n, is a lattice; the least upper bound of {m,n} is lcm(m,n), andthe greatest lower bound is gcd(m,n)

The definition and example are adapted from:

N. Bourbaki.Chapter 3, Section 1.11.Theory of Sets, Springer, 2004, pages 145-146.


Cycle structure of k-rotation on n elements

An element with index i is in the cycle

{i, (i+ k) mod n, (i+ 2k) mod n, . . .} = {(i+ uk) mod n}

The length of the cycle is the smallest positive integerm such thati = (i+mk) mod nThis is equivalent tomk mod n = 0, which shows the length ofthe cycle to be independent of iSincem is the smallest positive number such thatmk mod n = 0,it is obvious that lcm(k,n) = mk

m =lcm(k,n)k = kn

gcd(k,n)k = ngcd(k,n)

The number of cycles, therefore, is gcd(k,n)


Disjoint cycles of k-rotation on n elements

Consider two different elements in a cycle, (i+ uk) mod n and(i+ vk) mod nThe distance between them is|(i+uk) mod n−(i+vk) mod n| = (u−v)k mod n = (u−v)k−pn

where p = quotient((u− v)k,n)

Since both k and n are divisible by d = gcd(k,n), so is the distanceTherefore the distance between different elements in the samecycle is > d

Elements with indices in [0,d) belong to disjoint cycles


Interface for rotate

k-rotation rearrangement of a range [f, l) is equivalent tointerchanging the relative positions of [f,m) and [m, l), wherem = f+ ((l− f) − k) = l− k

Experimentally,m is a more useful input than k (and whenforward or bidirectional iterators are involved, it avoidsperforming linear-time operations to computem from k)Experimentally, returning the iteratorm ′ = f+ k pointing to thenew position of element at f is useful for many other algorithms30

30Joseph Tighe suggests returning a pair,m andm ′, in the order constituting a validrange; while it is an interesting suggestion and preserves all the information, we donot yet know of a compelling use of such interface


From-permutation for k-rotation in terms of f,m, l

To-permutation: p(i) = (i+ k) mod nFrom-permutation: p−1(i) = (i+ (n− k)) mod nn = l− f

Sincem goes to f,m+ k = f+ n⇒ n− k = m− f

k = n− (m− f) = l−m

i < k⇒ i+ (n− k) < n⇒ p−1(i) = i+ (n− k)

i > k⇒ p−1(i) = i+ (n− k) − n = i− k

p−1(i) =

{i+ (n− k) if i < ki− k if i > k


A from-permutation for RandomAccessIterator

template <typename I>requires(Mutable(I) && RandomAccessIterator(I))

struct k_rotate_from_permutation_random_access{

DistanceType(I) k;DistanceType(I) n_minus_k;I m_prime;k_rotate_from_permutation_random_access(I f, I m, I l) :

k(l - m), n_minus_k(m - f), m_prime(f + (l - m)) {}I operator()(I x) {

if (x < m_prime)return x + n_minus_k;

elsereturn x - k;

}};


A from-permutation for IndexedIterator

The absence of < and − for indexed iterators costs us an extraoperation or two (+ and −)On modern processors this may not cost any time


struct k_rotate_from_permutation_indexed{

DistanceType(I) k;DistanceType(I) n_minus_k;I f;k_rotate_from_permutation_indexed(I f, I m, I l) :

k(l - m), n_minus_k(m - f), f(f) {}I operator()(I x) {

DistanceType(I) i = x - f;if (i < k)

return x + n_minus_k;else

return f + (i - k);}

};


rotate_indexed_helper

template <typename I, typename P>requires(Mutable(I) && IndexedIterator(I) &&

Transformation(P) && Domain(P) == I)I rotate_indexed_helper(I f, I m, I l, P p){

// Precondition: p is a from-permutation on [f, l)typedef DistanceType(I) N;N d = gcd(m - f, l - m);N i = N(0);while (i < d) {

do_cycle_from(f + i, p);++i;

}return f + (l - m);

};

This algorithm was first published by:

William Fletcher and Roland Silver.Algorithm 284: Interchange of Two Blocks of Data.CACM, Volume 9, Number 5, May 1966, page 326.


rotate_indexed and rotate_random_access

template <typename I> requires(Mutable(I) && IndexedIterator(I))I rotate(I f, I m, I l, indexed_iterator_tag){

k_rotate_from_permutation_indexed p(f, m, l);return rotate_indexed_helper(f, m, l, p);

};

template <typename I> requires(Mutable(I) && RandomAccessIterator(I))I rotate(I f, I m, I l, random_access_iterator_tag){

k_rotate_from_permutation_random_access p(f, m, l);return rotate_indexed_helper(f, m, l, p);

};

template <typename I> requires(Mutable(I) && ForwardIterator(I))I rotate(I f, I m, I l){

if (m == f) return l;if (m == l) return f;return rotate(f, m, l, IteratorCategory(I)());

}


Complexity of rotate for IndexedIterator

The number of assignments is n+ c− t = n+ gcd(n,k)On average, gcd� n


Loop fusion

Assuming uniform distribution of rotation points, the expectedvalue of k = n/2That gives very bad locality of referenceSince the cycles are disjoint, we can improve locality of referenceby operating on adjacent elements in different cyclesThis technique is called loop fusion, and was first discussed in thispaper:

A.P. Yershov.ALPHA–An Automatic Programming System of HighEfficiency.J. ACM, Volume 13, Number 1, January 1966, pages 17-24.


do_fused_cycles_from_with_buffer

template <typename I, typename P, typename B>requires(Mutable(I) && IndexedIterator(I) &&

Transformation(P) && Domain(P) == I &&Mutable(B) && IndexedIterator(B) &&ValueType(I) == ValueType(B))

void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n){

// Precondition: p is a from-permutation on the range containing icopy_n(i, b, n);I f = i;I next = p(i);while (next != i) {

copy_n(next, f, n);f = next;next = p(next);

}copy_n(b, f, n);

}


rotate_indexed_helper_fused


Transformation(P) && Domain(P) == I)I rotate_indexed_helper_fused(I f, I m, I l, P p){

// Precondition: p is a from-permutation on [f, l)typedef DistanceType(I) N;const N fusion_factor(16);ValueType(I) buffer[fusion_factor];N d = gcd(m - f, l - m);N i(0);while (i + fusion_factor < d) {

do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);i = i + fusion_factor;

}do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);return f + (l - m);

};


Issues with fused rotate

Loop fusion is an important optimization technique thatprogrammers need to knowIn this particular case, it is not that beneficial since often there arenot too many loops to fuse

gcd(n,k) = 1 with probability ≈ 60%

There are algorithms that do more more assignments but havegood locality of reference and work for weaker iteratorrequirements


Connection of rotate and reverse

LemmaThe k-rotation on [0,n) is the only permutation p such that

1 i < n− k∧ n− k 6 j < n⇒ p(j) < p(i)

2 i < j < n− k∨ n− k 6 i < j⇒ p(i) < p(j)

The reverse permutation will satisfy condition 1, but not 2Applying reverse to subranges [0,n− k) and [n− k,n) and thenapplying reverse to the entire range will satisfy both conditions


rotate_three_reverses

This insight gives us the following:


void rotate_three_reverses(I f, I m, I l){

reverse(f, m);reverse(m, l);reverse(f, l);

}

The only difficulty is finding the return value,m ′


reverse_until

The following auxiliary function allows us to find the return valuewithout doing any extra work31


pair<I, I> reverse_until(I f, I m, I l){

while (f != m && m != l) {--l;cycle_2(f, l);++f;

}return pair<I, I>(f, l);

}

31It was suggested to us by Raymond Lo and Wilson HoStepanov, McJones Elements of Programming March 14, 2008 482 / 880

rotate for BidirectionalIterator


I rotate(I f, I m, I l, bidirectional_iterator_tag){

reverse(f, m);reverse(m, l);pair<I, I> p = reverse_until(f, m, l);reverse(p.first, p.second);if (m == p.first)

return p.second;else

return p.first;}


Complexity of rotate for BidirectionalIterator

LemmaThe number of assignments is 3(bn/2c+ bk/2c+ b(n− k)/2c), whichgives us 3nwhen n and k are both even and 3(n− 2) in every othercase


swap_ranges

template <typename I1, typename I2>requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&ValueType(I1) == ValueType(I2))

pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2){

while (f1 != l1 && f2 != l2) {cycle_2(f1, f2);++f1;++f2;


}

Form ∈ [f, l) such that l−m = m− f, swap_ranges(f,m,m, l)rotates [f, l) aboutm


Intuition for ForwardIterator rotate algorithm

m = m ′ swap_ranges is sufficientm ′ < m after swap_ranges, [f,m ′) are in the final position; we

need to rotate [m ′, l) aroundmm < m ′ after swap_ranges, [f,m) are in the final position; we

need to rotate [m, l) aroundm ′

This algorithm was first published by:

David Gries and Harlan Mills.Swapping Sections.Technical Report 81-452, Department of Computer Science, CornellUniversity, January 1981.


rotate_0


void rotate_0(I f, I m, I l){

I c = m;while (true) {

pair<I, I> p = swap_ranges(f, m, m, l);if (p.first == m && p.second == l)

return;if (p.first == m) {

m = p.second;f = p.first;

} else {assert(p.second == l);f = p.first;

}}

}


Annotated rotate_0


void rotate_0_annotated(I f, I m, I l){

DistanceType(I) u = m - f;DistanceType(I) v = l - m;

I c = m;while (true) {

pair<I, I> p = swap_ranges(f, m, m, l);if (p.first == m && p.second == l) { assert(u == v);

return;} else if (p.first == m) { assert(v > u);

m = p.second;f = p.first; v = v - u;

} else {assert(p.second == l); assert(u > v);f = p.first; u = u - v;

}}

}

u and v compute subtractive gcd of the initial lengths


Complexity of rotate for ForwardIterator

LemmaThe number of assignments is 3(n− gcd(n,k))


rotate_unguarded

Inlining swap_ranges and small optimizations give:


void rotate_unguarded(I f, I m, I l){

// Precondition: f ,m∧m , lI c = m;while (c != l) {

cycle_2(f, c);++f;++c;if (f == m)

m = c;if (c == l)

c = m;}

}

The first time the second if clause is satisfied, f equalsm ′


rotate for ForwardIterator


I rotate(I f, I m, I l, forward_iterator_tag){

// Precondition: f ,m∧m , lI m_prime = l;I c = m;while (c != l) {

cycle_2(f, c);++f;++c;if (f == m)

m = c;if (c == l) {

if (m_prime == l) m_prime = f;c = m;

}}return m_prime;

}

The guards in the category dispatch rotate guarantee the assertion


Conclusions

Rotation rearrangements allow interchanging the order ofadjacent ranges of different sizesApplying elementary number theory allows us to determine thecycle structure of rotationsInteresting algorithms exist for forward, bidirectional, andindexed iterator algorithmsComplexity is often dominated by locality of reference concernsrather than minimizing operation counts


Reading

Donald Knuth.Section 1.3.3: Permutations and Factorials.The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,pages 164-185.In particular, see problems 34 and 35.


Project 1: industrial-strength rotate

Theoretically rotate for indexed iterators is much faster than theother two algorithms because it performs approximately 1

3 asmany assignments, but its locality of reference is poorDesign a benchmark comparing performance of all threealgorithms for different array sizes, element sizes, and rotationamountsBased on the results of the benchmark, design a compositealgorithm that appropriately uses one of the three algorithmsdepending on its inputs


Project 2: Research in position-based rearrangements

We have presented two kinds of position-based rearrangementalgorithms: reverse and rotateThere are, however, many other examples of such algorithms; youcan find some of them in the section of Knuth given in Reading aswell as in the Collected Algorithms of the ACM (see, for example,numbers 302, 380, and 467)Develop a taxonomy of position-based rearrangements, discoveradditional algorithms, and produce a library


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 Rotations

10 Algorithms on increasing rangesIncreasing rangesPartitionSearchingCopying merging and sortingIn-place merging and sortingAdaptive merging and sortingConclusionsProject



Contents III




15 C++ machinery

16 Acknowledgments

17 Index


Increasing ranges

DefinitionA range [f, l) is increasing if for every non-last iteratorm ∈ [f, l),¬(source(successor(m)) < source(m))

A range [f, l) is increasing with respect to a strict weak ordering r iffor every non-last iteratorm ∈ [f, l),¬r(source(successor(m)), source(m))

A range [f, l) is key-increasing a with respect to a regular functionkey and a strict weak ordering r if for every non-last iteratorm ∈ [f, l), ¬r(key(source(successor(m))),key(source(k)))

aIn database parlance, a key is a part of a record on which sorting is done


Defaulting the key function

Note that an increasing range is a key-increasing range with thekey function being the identity function on the value type of theiterator


find_out_of_order

template <typename I, typename F, typename R>requires(Readable(I) && ForwardIterator(I) &&

RegularFunction(F) && Domain(F) == ValueType(I) &&StrictWeakOrdering(R) && Domain(R) == Codomain(F))

I find_out_of_order(I f, I l, F key, R r, forward_iterator_tag){

if (f == l) return l;I n(successor(f));while (n != l) {

if (r(key(source(n)), key(source(f))))return n;

f = n;++n;

}return l;

}

To adapt the algorithm for Iterator, we need to copy the valuebefore we advance


find_out_of_order

template <typename I, typename F, typename R>requires(Readable(I) && Iterator(I) &&


I find_out_of_order(I f, I l, F key, R r, iterator_tag){

if (f == l) return l;I n(successor(f));Codomain(F) v = key(source(f));while (n != l) {

Codomain(F) u = key(source(n));if (r(u, v))

return n;v = u;++n;

}return l;

}


is_increasing

template <typename I, typename F, typename R>requires(Readable(I) && Iterator(I) &&


bool is_increasing(I f, I l, F key, R r){

return l == find_out_of_order(f, l, key, r);}


Partition

DefinitionUnaryPredicate(P)⇒

Predicate(P)Arity(P) = 1

DefinitionA range is partitioned if it is key-increasing with respect to a keyfunction that is a unary predicateMore generally, a range is k-partitioned if it is key-increasing withrespect to a key function that returns one of k distinct values forsome (compile-time) constant integer k > 0

There exist faster specialized versions of many algorithms dealingwith key-increasing ranges when the range of the key function is abounded set of integersStepanov, McJones Elements of Programming March 14, 2008 504 / 880

Partition points

DefinitionA partition point in a range [f, l) partitioned by a unary predicate p is aniterator i ∈ [f, l] such that ¬p(source(j)) for all j ∈ [f, i) andp(source(k)) for all k ∈ [i, l)

Informally, a partition point is the first position where thepredicate is satisfied, if such a position exists, or the limit of therange otherwise


Lower bound and upper bound

DefinitionIn an increasing range [f, l), every value a of the value type of therange determines two partition points, lower bound and upper bound,defined by, respectively:

Pa(x) ≡ x < a

P ′a(x) ≡ ¬(a < x)

It is trivial to provide similar definitions for key-increasingsequences and sequences with respect to a strict weak order r

Lemmalower bound � upper bound


Intuition for lower bound and upper bound

Informally, a lower bound is the first position where a value equalto a could occur in the increasing sequenceSimilarly, an upper bound is the successor of the last positionwhere a value equal to a could occurTherefore, elements equal to a appear only in the semi-open rangefrom lower bound to upper bound

A sequence with lower bound i and upper bound j for the value a(note any of the three regions may be empty):

x0, x1, . . . , xi−1︸︷︷︸<a

, xi, . . . , xj−1︸︷︷︸=a

, xj, xj+1, . . . , xn−1︸︷︷︸a<


Partition algorithms

Mathematically a partitioned range is simply a key-increasingrangeAlgorithmically, it is worth providing special-case algorithms fordealing with partitions

Testing whether a range is partitioned dereferences half as manyiterators as testing whether it is increasingPartitioning a range of size n is linear while sorting it is n log2 n

Efficient algorithms for dealing with increasing ranges can becomposed from algorithms dealing with partitioned ranges

Quicksort is implemented using a specialized partition algorithmFinding a lower or upper bound uses the algorithm for finding apartition point


Testing whether a range is partitioned

LemmaGiven any range and a unary predicate, some prefix of the range ispartitioned by the predicate

If we can determine the longest partitioned prefix of a range, thenby testing whether the prefix equals the range we can determinewhether the range is partitioned


find_if and find_if_not

We need to find the first element satisfying the predicate

template <typename I, typename P>requires(Readable(I) && Iterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))I find_if(I f, I l, P p) {

while (f != l && !p(source(f)))++f;

return f;}

Then finding the first element not satisfying the predicate givesthe longest partitioned prefix


UnaryPredicate(P) && Domain(P) == ValueType(I))I find_if_not(I f, I l, P p) {

while (f != l && p(source(f)))++f;

return f;}


is_partitioned


UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> find_partitioned_prefix(I f, I l, P p){

I i = find_if(f, l, p);I j = find_if_not(i, l, p);return pair<I, I>(i, j);

}

The first value returned is the partition point of the partitionedprefix


UnaryPredicate(P) && Domain(P) == ValueType(I))bool is_partitioned(I f, I l, P p){

return l == find_partitioned_prefix(f, l, p).second;}


Quantifier functions


UnaryPredicate(P) && Domain(P) == ValueType(I))bool all(I f, I l, P p) { return l == find_if_not(f, l, p); }


UnaryPredicate(P) && Domain(P) == ValueType(I))bool none(I f, I l, P p) { return l == find_if(f, l, p); }


UnaryPredicate(P) && Domain(P) == ValueType(I))bool not_all(I f, I l, P p) { return !all(f, l, p); }


UnaryPredicate(P) && Domain(P) == ValueType(I))bool some(I f, I l, P p) { return !none(f, l, p); }


is_partition_point


UnaryPredicate(P) && Domain(P) == ValueType(I))bool is_partition_point(I f, I m, I l, P p){

return none(f, m, p) && all(m, l, p);}


Importance of finding a partition point

Finding the partition point in an already-partitioned range byitself is not that usefulOn an increasing range, it can be combined with an appropriatepredicate to implement all the varieties of binary search –allowing us to find a value without examining every position


Intuition for finding a partition point

Suppose we evaluate a predicate p at some pointm in apartitioned range [f, l)

If p(m) is false, we know the partition point of [f, l) is greater thanm, and in fact is equal to the partition point of [successor(m), l)If p(m) is true, we know the partition point is not greater thanm, soit is equal to the partition point of [f,m)

Note the partition point of [f,m) will equalm if the predicate p isfalse for every iterator in [f,m)

LemmaChoosingm in the middle of the range assures optimal worst-caseperformance

If we have prior knowledge of the distribution, biasing the choiceofmmight give better average performance


partition_point_n

template <typename I, typename P>requires(Readable(I) && ForwardIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))I partition_point_n(I f, DistanceType(I) n, P p){


N h = half_nonnegative(n);I m = f + h;if (!p(source(m))) {

n = n - successor(h); f = successor(m);} else

n = h;}return f;

}


UnaryPredicate(P) && Domain(P) == ValueType(I))I partition_point(I f, I l, P p) { return partition_point_n(f, l - f, p); }


Complexity of partition_point

partition_point_n applies the predicate blog2 nc+ 1 times sincethe length of the range is reduced by a factor of 2 at each stepIt also performs a logarithmic number of iterator/integeradditions, which for an indexed or random access iterator areconstant-time but which for a forward or bidirectional iterator(singly- or doubly-linked list) require n iterator incrementspartition_point adds another n iterator increments for forwardand bidirectional iteratorsIt may still be worthwhile to use it on linked lists when thepredicate application is much more expensive than an iteratorincrement


Origin of the bisection technique

The bisection technique underlying partition point dates back atleast as far as the proof of the Intermediate Value Theoremdiscovered by Bernard Bolzano32 and, independently, byAugustin-Louis Cauchy about 4 years later33

The proof of the Bolzano-Cauchy theorem leads us to thefollowing interpolation algorithm34

32Bernard Bolzano. Rein analytischer Beweis des Lehrsatzes, daß zwischen je zweyWerthen, die ein entgegengesetztes Resultat gewähren, wenigstens eine reelle Wurzel derGleichung liege. 1817

33A.-L. Cauchy. Cours D’Analyse de L’Ecole Royale Polytechnique. 182134This book explores the connections between some simple algebraic concepts and

algorithms; we strongly believe it is also possible to define algorithms on analyticconcepts


bisection_interpolation

template <typename F>requires(Transformation(F) && ArchimedeanOrderedField(Domain(F)))

Domain(F) bisection_interpolation(F f, Domain(F) x0, Domain(F) x1, Domain(F) y0, Domain(F) e)

{// Precondition: f is continuous and increasing on [x0,x1]typedef Domain(F) R;R x;while (true) {

x = (x0 + x1) / R(2);R y = f(x);if (y <= y0) {

x0 = x; if (y0 < y + e) break;} else {

x1 = x; if (y < y0 + e) break;}

}return x;// Postcondition: x is in the original interval [x0,x1] and |f(x) − y0| < ε

}


count_if and count_if_not


UnaryPredicate(P) && Domain(P) == ValueType(I))I count_if(I f, I l, P p) {

DistanceType(I) n(0);while (f != l) {

if (p(source(f))) n = successor(n);++f;

}return n;

}


UnaryPredicate(P) && Domain(P) == ValueType(I))I count_if_not(I f, I l, P p) {

DistanceType(I) n(0);while (f != l) {

if (!source(f)) n = successor(n);++f;

}return n;

}


potential_partition_point


UnaryPredicate(P) && Domain(P) == ValueType(I))I potential_partition_point(I f, I l, P p){

I i = f;while (f != l) {

if (!p(source(f))) ++i;++f;

}return i;// Postcondition: i = f+ count_if_not(f, l,p)

}


Multiplicity of partitioned rearrangements

Since partition requires only that false values of the predicateprecede true values, there are u!v! ways to partition a range with ufalse values and v true valuesA partition algorithm would satisfy all our needs if it had theseproperties:

Requires only forward iteratorsStableIn placeLinear time, with minimal number of assignments and predicateapplications

We conjecture no such algorithm existsWe present some partition algorithms to demonstrate the kind oftradeoffs that must be made


partition_copy35

template <typename I, typename O0, typename O1, typename P>requires(Readable(I) && Iterator(I) &&

Writable(O0) && Iterator(O0) &&Writable(O1) && Iterator(O1) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(O0) &&ValueType(I) == ValueType(O1))

pair<O0, O1> partition_copy(I f, I l, O0 o0, O1 o1, P p){

typedef DistanceType(I) N;while (f != l) {

if (p(source(f))) {sink(o1) = source(f); ++o1;

} else {sink(o0) = source(f); ++o0;

}++f;

}return pair<O0, O1>(o0, o1);

}

35T.K. Lakshman suggested the interfaceStepanov, McJones Elements of Programming March 14, 2008 523 / 880

Properties of partition_copy 37

It is stable: the values of each of the two output ranges are in thesame relative order as in the input rangeIt is a copying rearrangement36

It is optimal in terms both of number of predicate applications andassignments (both numbers are n, the size of the range)

36Note that if the input range is defined by mutable forward iterators, it may besupplied as one of the output ranges

37A variation we use later, partition_copy_n, is given in Appendix 2Stepanov, McJones Elements of Programming March 14, 2008 524 / 880

Semistable partition rearrangements

DefinitionA partition rearrangement is semistable if the relative order of elementsnot satisfying the predicate is preserved

Semistable partition can be used for efficient removal from anextent-based sequence (such as the array of Chapter 11 or theC++ std::vector) of elements satisfying a given predicate

First partition the sequence by the predicateErase the elements from the partition point to the end of thesequenceThe sequence now contains the elements not satisfying thepredicate, in their original order 38

38In Chapter 12 we will explain why this is the right thing to doStepanov, McJones Elements of Programming March 14, 2008 525 / 880

semistable_partition39


UnaryPredicate(P) && Domain(P) == ValueType(I))I semistable_partition(I f, I l, P p){

I i = find_if(f, l, p); // none(f, i,p)I j = find_if_not(i, l, p); // none(f, i,p) ∧all(i, j,p) ∧ (j = l∨ ¬p(j))while (j != l) { // none(f, i,p) ∧all(i, j,p) ∧ p(i) ∧ ¬p(j)

cycle_2(i, j); // none(f, i,p) ∧all(i+ 1, j,p) ∧ ¬p(i) ∧ p(j)// (none(f, i,p)∧¬p(i))∧(all(i+1, j,p)∧p(j))

++i; ++j; // none(f, i,p) ∧all(i, j,p)j = find_if_not(j, l, p); // j = l∨ ¬p(j)

}return i;

}

39The algorithm is due to Nico Lomuto, according to: Jon Bentley. ProgrammingPearls. Communications of the ACM, Volume 27, Number 4, April 1984, pages 287-291


Useful lemmas

The annotations on the previous slide follow from some trivial butuseful lemmas

LemmaAfter executing i = find_if(f, l,p), none(f, i,p) ∧ (i = l∨ p(i))

After executing j = find_if_not(f, l,p), all(f, j,p)∧ (j = l∨¬p(j))

If p(source(i)) ∧ q(source(j)) before executing cycle_2(i, j), thenafterward q(source(i)) ∧ p(source(j))

none(f, l,p) ∧ ¬p(l)⇒ none(f, l+ 1,p)all(f+ 1, l,p) ∧ p(f)⇒ all(f, l,p)


Properties of semistable_partition

It is in placeLet n = l− f be the number of elements in the range and let w bethe number of elements not satisfying the predicate following thepartitioned prefix40

The predicate is applied n timescycle_2 is performed w timesThe number of iterator increments is n+w

It is totally inappropriate for use in quicksort since it does notdivide a range of equal values at the midpoint and that leads toquadratic complexity in many practical situations

40w = count_if_not(find_partitioned_prefix(f, l,p), l,p)Stepanov, McJones Elements of Programming March 14, 2008 528 / 880

semistable_partition performs too manyassignments

Consider a range t, f, f, f, f, f, f, fsemistable_partitionwill perform seven calls of cycle_2, whileone suffices


assured_find_if and assured_find_if_not

We can eliminate one of the two tests in the loop for find_if if weare assured there is an element in the range that satisfies thepredicate


UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_if(I f, P p) {

// Let l be the end of the implied range starting with f// Precondition: some(f, l,p)while (!p(source(f))) ++f;return f;

}


UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_if_not(I f, P p) {

// Let l be the end of the implied range starting with f// Precondition: not_all(f, l,p)while (p(source(f))) ++f;return f;


unstable_forward_partition


UnaryPredicate(P) && Domain(P) == ValueType(I))I unstable_forward_partition(I f, I l, P p){

I i = potential_partition_point(f, l, p);I j = find_if_not(i, l, p);while (j != l) {

f = assured_find_if(f, p);cycle_2(f, j); ++f; ++j;j = find_if_not(j, l, p);

}return i;

}

We can use assured_find_if because the number of misplacedelements before and after the partition point are equalThe code does one extra iterator comparison that could beeliminated by inlining both calls to find_if_not (the sameoptimization applies to many following algorithms)


Properties of unstable_forward_partition

It is in placeThe number of times cycle_2 is performed, v, equals the numberof misplaced elements not satisfying the predicate, that is, thenumber of elements not satisfying the predicate in the subrangefrom the potential partition point to the end of the range41

The algorithm applies the predicate 2n times, therefore it is usefulwhen the cost of predicate application is insignificant compared tothe cost of cycle_2This technique of computing the sizes of a set of buckets and thenmoving the elements appears in radix sort42

With a bidirectional iterator, we can avoid precomputing thebucket sizes

41v = count_if_not(potential_partition_point(f, l,p))42See the use of this technique for in-place k-partition Exercise 5.2-13 in Donald

Knuth. The Art of Computer Programming. Volume 3, Addison-Wesley, 1998, page 80;Solution, page 618


find_backward_if and find_backward_if_not

To indicate “not found”, we return f, which forces us to returni+ 1 if we find a satisfying element at position i

template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))I find_backward_if(I f, I l, P p) {

while (true) {if (f == l) return f;--l;if (p(source(l))) return successor(l);

}}template <typename I, typename P>

requires(Readable(I) && BidirectionalIterator(I) &&UnaryPredicate(P) && Domain(P) == ValueType(I))

I find_backward_if_not(I f, I l, P p) {while (true) {

if (f == l) return f;--l;if (!p(source(l))) return successor(l);

}}


unstable_bidirectional_partition

template <typename I, typename P>requires(Mutable(I) && BidirectionalIterator(I) &&

UnaryPredicate(P) && Domain(P) == ValueType(I))I unstable_bidirectional_partition(I f, I l, P p){

while (true) {f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;--l;cycle_2(f, l);++f;

}return f;

}


Properties of unstable_bidirectional_partition

It is in placeThe predicate is applied n timescycle_2 is called v times (where v is as previously defined forunstable_forward_partition)The algorithm moves only those elements that need to be moved,and puts two elements in their final destination with each call tocycle_2The total number of assignments, therefore, is 3v


Minimizing the number of assignments

It is possible to accomplish partition with a differentrearrangement that has only a single cycle, resulting in 2v+ 1assignmentsThe idea is to:

Save the first misplaced element, creating a “hole”Repeatedly find a misplaced element on the opposite side of thepotential partition point and move it into the hole, creating a newholeMove the saved element into the final hole


assured_find_backward_if andassured_find_backward_if_not


UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_backward_if(I l, P p) {

do --l; while (!p(source(l)));return l;

}


UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_backward_if_not(I l, P p) {

do --l; while (p(source(l)));return l;

}

We chose to return the iterator rather than its successor as infind_backward_if since we do not need to indicate failure


single_cycle_partition


UnaryPredicate(P) && Domain(P) == ValueType(I))I single_cycle_partition(I f, I l, P p){

f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;--l;ValueType(I) tmp = source(f);while (true) {

sink(f) = source(l);f = find_if(successor(f), l, p);if (f == l) {

sink(l) = tmp;return f;

}sink(l) = source(f);l = assured_find_backward_if_not(l, p);

}}


Properties of single_cycle_partition

It remains to be observed whether using one large cycle instead ofmultiple 2-cycles is faster on modern computers withinstruction-level parallelismThe technique, however, is worth careful study


assured_bidirectional_partition


UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_bidirectional_partition(I f, I l, P p){

// Precondition:// (¬all(f, l,p) ∧ some(f, l,p)) ∨ (¬p(source(f− 1)) ∧ p(source(l)))while (true) {

f = assured_find_if(f, p);l = assured_find_backward_if_not(l, p);if (successor(l) == f) return f;cycle_2(f, l);++f; // ¬p(source(f− 1)) ∧ p(source(l))

}}


Correctness of assured_bidirectional_partition

The precondition provides the guards for assured_find_if andassured_find_backward_if_notAfter these two calls, f , l, since they point to elements withdifferent predicate valuesl− f decreases by at least two each iterationf and l can cross over by only one position

Suppose they had crossed by more than one, and consider anyiteratorm between l and fIf ¬p(source(m)), then fwould have stopped there; otherwise, lwould have stopped thereTherefore, there is no suchm

If the exit is not taken, f ≺ l, and f and l point, correspondingly, tosatisfying and non-satisfying elementsThe cycle_2 call moves these elements to correct positions, andthe increment of f restores the invariant


sentinel_partition

With a little extra work we can establish the appropriateprecondition to use an assured function43


UnaryPredicate(P) && Domain(P) == ValueType(I))I sentinel_partition(I f, I l, P p){

f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;--l;cycle_2(f, l);++f;return assured_bidirectional_partition(f, l, p);

}

43It is possible to combine the sentinel and single-cycle techniques, producingsentinel_single_cycle_partition


The origin of partition algorithms

Three previous algorithms are based on a paper44 by C.A.R.Hoare:

unstable_bidirectional_partitionsingle_cycle_partitionsentinel_partition

44C.A.R. Hoare. Quicksort. The Computer Journal, Volume 5, Number 1, 1962, pages10-16. This paper, which is a serious contender for the best-ever computer sciencepaper, should not be confused with his earlier publication: C.A.R. Hoare. Algorithm63, Partition; Algorithm 64, Quicksort; Algorithm 65, Find. Communications of theACM, Volume 4, Number 7, July 1961, pages 321-322. No one should attempt toimplement or teach quicksortwithout studying this paper.


Regularity of predicate for partition algorithms

The UnaryPredicate concept requires that the predicate is a regularfunction; this assures the same boolean value is returned everytime the predicate is applied to a particular objectThis in turn assures that after the partition algorithm returns, acall to is_partitioned(f, l,p) would return trueHowever the algorithms using assured finds depend on regularityin a more fundamental way: to assure they stay within bounds ofthe rangeConsider, for example, using a nonregular predicate to perform arandom shuffle


Exercise

ExerciseDesign an algorithm implementing in-place uniform randomshuffle of a range of forward iterators (hint: recursively partitionthe range with a non-regular coin-tossing predicate a )Prove that the expected number of coin tosses is n log2 n

Prove that in-place uniform random shuffle of a range of forwarditerators cannot be done in linear time

aThe algorithm was discovered by Raymond Lo and Wilson Ho


Intuition for stable partition

An empty range is trivially partitionedA singleton range can be partitioned with one predicateapplication and possibly one iterator incrementAn arbitrary range can be partitioned with a divide-and-conqueralgorithm:

Divide the range in the middleStably partition each halfCombine the results by rotating the range bounded by the first andsecond partition points around the middle: **** simple picture *****

This easily extends to a memory adaptive version


Base cases for stable partition


UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> stable_partition_0(I f, P){

return pair<I, I>(f, f);}


UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> stable_partition_1(I f, P p){

I l = successor(f);if (p(source(f)))

return pair<I, I>(f, l);else

return pair<I, I>(l, l);}


Combining operation for stable partition


pair<I, I> stable_partition_combine(pair<I, I> r0, pair<I, I> r1){

I m = rotate(r0.first, r0.second, r1.first);return pair<I, I>(m, r1.second);

}


stable_partition_n


UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> stable_partition_n(I f, DistanceType(I) n, P p){

typedef DistanceType(I) N;if (n == N(0))

return stable_partition_0(f, p);if (n == N(1))

return stable_partition_1(f, p);N h = half_nonnegative(n);pair<I, I> r0 = stable_partition_n(f, h, p);pair<I, I> r1 = stable_partition_n(r0.second, n - h, p);return stable_partition_combine(r0, r1);

}


stable_partition_n_with_buffer

template <typename I, typename B, typename P>requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(B))

pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p){

pair<I, B> r = partition_copy_n(f, n, f, b, p);return pair<I, I>(r.first, copy(b, r.second, r.first));

}


stable_partition_n_adaptive


Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(B))

pair<I, I> stable_partition_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)

{typedef DistanceType(I) N;if (n == N(0))


return stable_partition_1(f, p);if (n <= n_b)

return stable_partition_n_with_buffer(f, n, b, p);N h = half_nonnegative(n);pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);pair<I, I> r1 = stable_partition_n_adaptive(r0.second, n - h, b, n_b, p);return stable_partition_combine(r0, r1);

}


Properties of stable partition

Exactly n predicate applications are performed at the bottom levelof recursionThe depth of the recursion for stable_partition_n is dlog2 neAt every recursive level we rotate n/2 elements on the average,requiring between n/2 and 3n/2 assignments depending on theiterator categoryThe total number of assignments is n log2 n/2 for random accessiterators and 3n log2 n/2 for forward and bidirectional iterators

ExerciseDetermine the number of assignments for the adaptive version


Interfaces for binary search

What is commonly referred to as binary search is really threerelated algorithms for determining the lower bound, the upperbound, or bothThe advantage of these as compared to the way binary search isusually specified in textbooks is that the lower and upper boundsare always defined, so there is no issue of what to return when theelement is not presentSeveral algorithms in this chapter will demonstrate the utility ofthese interfaces


Implementing lower bound and upper bound

Earlier in this chapter we defined the lower bound and upperbound for a value a as the partition points of two predicates:

Pa(x) ≡ x < a

P ′a(x) ≡ ¬(a < x)

They can be implemented by passing functions corresponding toPa and P ′a to partition_point_nSee Appendix 2 for the C++ versions

ExerciseImplement a function that returns both lower and upper bounds anddoes fewer comparisons than the sum of the comparisons that wouldbe done by calling both lower_bound_n and upper_bound_na

aA similar STL function is called equal_range


Merging

DefinitionMerge is an operation that combines two increasing ranges into a singleincreasing range that contains all the elements of both ranges and noother elements

DefinitionA merge is stable if the output range preserves the relative order ofequivalent elements both within each input range, and between thefirst and second input range

LemmaStability establishes a unique ordering of the elements in the outputrange


Merge interfaces

A mutating45 merge operation takes two adjacent increasingranges and rearranges the elements in the combined range intoincreasing order; a range [f, l) is mergeable if there is an iteratorm ∈ [f, l) such that the subranges [f,m) and [m, l) are increasingA copying merge operation sets a writable range to the merge oftwo readable increasing rangesA melding merge operation combines two ranges defined by nodeiterators by changing their relative order46

45A mutating merge may or may not be in place depending on how much extramemory it uses

46We define node iterators in the next chapterStepanov, McJones Elements of Programming March 14, 2008 556 / 880

is_mergeable

template <typename I, typename R>requires(Readable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))bool is_mergeable(I f, I m, I l, R r){

return is_increasing(f, m, r) && is_increasing(m, l, r);}


is_merged

ExerciseImplement this function that checks whether the range starting at o isequal to the merge of the ranges [f0, l0) and [f1, l1)


StrictWeakOrdering(R) && Domain(R) == ValueType(I))bool is_merged(I f0, I l0, I f1, I l1, I o, R r);


Algebraic properties of merge

Merge is a binary operation on rangesAn empty range is its identity element (both left and right)Merge is associative

Its complexity needs to be taken into consideration whenreassociating

Merge is commutative when stability is not taken into account


Complexity property of copying merge

If ◦ denotes the merge operation and C(x,y) denotes the cost (thenumber of assignments) to merge ranges x and y, then Csatisfies47:

C(x ◦ y, z) =C(x,y) + C(x, z) + C(y, z)

2

A consequence of this is:

C(x,y) < C(y, z)⇒ C(x,y) + C(x ◦ y, z) < C(y, z) + C(x,y ◦ z)

In other words, it is better to do the least-costly merge first

47In the next chapter we will define a Huffman semigroup, where the cost of thesemigroup operation satisfies this axiom


merge_copy_n

template <typename I0, typename I1, typename O, typename R>requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&

Writable(O) && Iterator(O) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))

void merge_copy_n(I0 f0, DistanceType(I0) n0,I1 f1, DistanceType(I1) n1, O o, R r) {

// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the input ranges do not overlap the output rangetypedef DistanceType(I0) N0; typedef DistanceType(I1) N1;while (true) {

if (n0 == N0(0)) { copy_n(f1, n1, o); return; }if (n1 == N1(0)) { copy_n(f0, n0, o); return; }if (r(source(f1), source(f0))) {

sink(o) = source(f1); ++f1; n1 = n1 - N1(1);} else {

sink(o) = source(f0); ++f0; n0 = n0 - N0(1);}++o;

}}

The overlap precondition is not the weakest possibleStepanov, McJones Elements of Programming March 14, 2008 561 / 880

merge_copy_backward_n

template <typename I0, typename I1, typename O, typename R>requires(Readable(I0) && BidirectionalIterator(I0) &&

Readable(I1) && BidirectionalIterator(I1) &&Writable(O) && BidirectionalIterator(O) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))

void merge_copy_backward_n(I0 l0, DistanceType(I0) n0,I1 l1, DistanceType(I1) n1, O o, R r) {

// Let f0 = l0 −n0 and f1 = l1 −n1// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;while (true) {

if (n0 == N0(0)) { copy_backward_n(l1, n1, o); return; }if (n1 == N1(0)) { copy_backward_n(l0, n0, o); return; }--o;if (r(source(predecessor(l1)), source(predecessor(l0)))) {

--l0; sink(o) = source(l0); n0 = n0 - N0(1);} else {

--l1; sink(o) = source(l1); n1 = n1 - N1(1);}

}}

Stability requires the order of the if clauses is reversed from the one inmerge_copy_nStepanov, McJones Elements of Programming March 14, 2008 562 / 880

Return value for copying merges

ExerciseReimplementmerge_copy_n andmerge_copy_backward_nwith thecorrect return values


Complexity of copying merges

LemmaThe number of assignments is always n0 + n1 and the worst-casenumber of comparisons is n0 + n1 − 1

ExerciseDetermine the average number of comparisons


merge_n_with_buffer for ForwardIterator

template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&

Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r,forward_iterator_tag)

{// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)if (n0 <= n1) {

copy_n(f0, n0, b);merge_copy_n(b, n0, f1, n1, f0, r);

} else {f1 = rotate(f0, f1, f1 + n1);copy_n(f0, n1, b);merge_copy_n(f1, n0, b, n1, f0, r);

}}


merge_n_with_buffer for BidirectionalIterator

template <typename I, typename B, typename R>requires(Mutable(I) && BidirectionalIterator(I) &&

Mutable(B) && BidirectionalIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r,bidirectional_iterator_tag)

{// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)if (n0 <= n1) {

copy_n(f0, n0, b);merge_copy_n(b, n0, f1, n1, f0, r);

} else {pair<I, B> tmp = copy_n(f1, n1, b);merge_copy_backward_n(f1, n0, tmp.second, n1, tmp.first, r);

}}


Concept Merger

DefinitionMerger(M)⇒

The type function IteratorType(M) is definedProcedure(M)

For allm ∈M, I = IteratorType(M), and N = DistanceType(I),m : I×N× I×N→ void

There is an implicit function ordering such that for allm ∈M,ordering(m) is a strict weak ordering on the value type of theiterator type ofMFor allm ∈M, a callm(f0,n0, f1,n1) satisfying the precondition

f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1,ordering(m))

will establish the postcondition

is_increasing(f0, f0 + n0 + n1,ordering(m))Stepanov, McJones Elements of Programming March 14, 2008 567 / 880

merger_with_buffer



struct merger_with_buffer{

B b;R r;merger_with_buffer(B b, R r) : b(b), r(r) { }void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1){

// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)merge_n_with_buffer(f0, n0, f1, n1, b, r);

}};


Sorting algorithms

DefinitionA key-sorting (or just sorting) algorithm is a rearrangement thosepostcondition is that the range is key-increasing (or justincreasing) with respect to a given strict weak orderA key-sorting (or sorting) algorithm is stable if it does not changethe relative order of values that are equivalent under thesymmetric complement of the given strict weak order

While the key function is a useful feature, including it in thesorting functions in this chapter is not instructive sincetransforming a function dealing with an increasing sequence intoone with a key-increasing sequence is straightforward


merge_sort_n

template <typename I, typename M>requires(Mutable(I) && ForwardIterator(I) &&

Merger(M) && IteratorType(M) == I)I merge_sort_n(I f, DistanceType(I) n, M merger){

typedef DistanceType(I) N;if (n < N(2)) return f + n;N h = half_nonnegative(n);I m = merge_sort_n(f, h, merger);I l = merge_sort_n(m, n - h, merger);merger(f, h, m, n - h);return l;

}


stable_sort_n_with_buffer



I stable_sort_n_with_buffer(I f, DistanceType(I) n, B b, R r){

// Precondition: the range beginning with b is of size at least n/2return merge_sort_n(f, n, merger_with_buffer<I, B, R>(b, r));

}

Note that h 6 n− h always holds, so the call on rotate in theForwardIterator version ofmerge_n_with_buffer will never beexecuted


Complexity of stable_sort_n_with_buffer

The algorithm has dlog2 ne recursive levelsEach level performs 3n/2 assignments, for a total number ofassignments of 3

2ndlog2 neAt ith level from the bottom, the worst-case number ofcomparisons is n− n

2i ; summing over all levels gives

ndlog2 ne−dlog2ne∑i=1

n

2i≈ ndlog2 ne− n


Intuition for stable in-place merge

TheoremIf is_mergeable(f,m, l, r), then for every iterator i ∈ [f,m), there is aunique j ∈ [m, l) such that after x← source(i) andm ′ ← rotate(i,m, j),

[f, l) is stably partitioned, with [f,m ′) containing elements lessthan or equal to x and [m ′, l) containing elements greater than orequal to xis_mergeable(f, i,m ′, r) ∧ is_mergeable(m ′, j, l, r)

Proof.Consider j = lower_bound(m, l, source(i), r) �

An analogous result holds for every j ∈ [m, l): leti = upper_bound(f,m, source(j), r), and performm ′ ← rotate(i,m, successor(j))


A sketch of an algorithm for stable in-place merge

This suggests a divide-and-conquer algorithm: choose an i in[f,m) and the corresponding j in [m, l), or a j in [m, l) and acorresponding i in [f,m), rotate appropriately, and then mergeeach of the mergeable subrangesNow since j−m ′ = m− i, choosing i halfway between f andm orj halfway betweenm and lwill cut the minimum ofm− f andl−m in halfThese considerations lead to the following algorithm


merge_n_in_place

template <typename I, typename R>requires(Mutable(I) && ForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))void merge_n_in_place(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, R r) {

// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)typedef DistanceType(I) N;if (min(n0, n1) == N(0)) return;I m0, m1;N h0, h1;if (n0 < n1) {

h0 = half_nonnegative(n0); m0 = f0 + h0;m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;I m = rotate(m0, f1, m1);merge_n_in_place(f0, h0, m0, h1, r);merge_n_in_place(successor(m), n0 - successor(h0), m1, n1 - h1, r);

} else {h1 = half_nonnegative(n1); m1 = f1 + h1;m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;I m = rotate(m0, f1, successor(m1));merge_n_in_place(f0, h0, m0, h1, r);merge_n_in_place(m, n0 - h0, successor(m1), n1 - successor(h1), r);

}}


Properties ofmerge_n_in_place

Lemma1 Each call of rotate puts one element into its final position2 The algorithm terminates with a sorted range3 Each level of recursion reducesm = min(n0,n1) to no more thanbm/2c

4 There are at most blog2(min(n0,n1))c+ 1 recursive levels

The algorithm is due to Dudzinski and Dydek, whose paper48

also contains a careful complexity analysis

48Krzysztof Dudzinski and Andrzej Dydek. On a Stable Minimum Storage MergingAlgorithm. Information Processing Letters, Volume 12, Number 1, February 1981, pages5-8.


Complexity ofmerge_n_in_place: number ofassignments

TheoremLet

kn be the maximum number of assignments required to rotate arange of length na(n0,n1) be the maximum number of assignments required for astable in-place merge of two ranges with lengths n0 and n1, wheren0 6 n1

Thena(n0,n1) 6 k(n1 + dn0/2e) log2(2n0) 6 kn log2 n

kmay be taken to be 3/2 for indexed iterators and 3 for forward(and bidirectional) iterators


Number of assignments ofmerge_n_in_place49

Proof.By induction on n0.

n0 = 1 A single rotation of a range of length at most n1 + 1 accomplishes the merge, soa(1,n1) = r(n1 + 1) 6 k(n1 + 1) = k(n1 + d1/2e) log2(2).

n0 > 1 We have a(n0,n1) 6 r(n1 +n0 −h0) +a(h0,h1) +a(n0 −h0 − 1,n1 −h1). Bythe induction hypothesis, a(h0,h1) 6 k(h1 + dh0/2e) log2(2h0) and

a(n0 −h0 − 1,n1 −h1) 6 k(n1 −h1 + d(n0 −h0 − 1)/2e) log2(2(n0 −h0 − 1)).

It is easily verified that dx/2e+ dy/2e 6 d(x+ y+ 1)/2e for any x and y. Thus

dh0/2e+ d(n0 −h0 − 1)/2e 6 dn0/2e

and

a(n0,n1) 6 k(n1 + dn0/2e)(log2(n0) + 1) = k(n1 + dn0/2e)(log2(2n0)).

�

49We are indebted to John Wilkinson for this proofStepanov, McJones Elements of Programming March 14, 2008 578 / 880

Complexity ofmerge_n_in_place: number ofcomparisons

TheoremLet c(n0,n1) be the maximum number of comparisons required for astable in-place merge of a range of length n0 with a range of length n1,where n0 6 n1; then c(n0,n1) 6 kn1

ExerciseDetermine the bound k and prove the theorem


merger_in_place

template <typename I, typename R>requires(Mutable(I) && BidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))struct merger_in_place{

R r;merger_in_place(R r) : r(r) { }void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1){

// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)merge_n_in_place(f0, n0, f1, n1, r);

}};


stable_sort_n_in_place


StrictWeakOrdering(R) && Domain(R) == ValueType(I))I stable_sort_n_in_place(I f, DistanceType(I) n, R r){

return merge_sort_n(f, n, merger_in_place<I, R>(r));}


merge_n



void merge_n(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,B b, DistanceType(I) n_b, R r) {

// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)if (min(n0, n1) <= n_b) { merge_n_with_buffer(f0, n0, f1, n1, b, r); return; }I m0, m1; DistanceType(I) h0, h1;if (n0 < n1) {

h0 = half_nonnegative(n0); m0 = f0 + h0;m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;I m = rotate(m0, f1, m1);merge_n(f0, h0, m0, h1, b, n_b, r);merge_n(successor(m), n0 - successor(h0), m1, n1 - h1, b, n_b, r);

} else {h1 = half_nonnegative(n1); m1 = f1 + h1;m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;I m = rotate(m0, f1, successor(m1));merge_n(f0, h0, m0, h1, b, n_b, r);merge_n(m, n0 - h0, successor(m1), n1 - successor(h1), b, n_b, r);

}}


merger



struct merger_adaptive{

typedef DistanceType(B) N;B b;N n_b;R r;merger_adaptive(B b, N n_b, R r) : b(b), n_b(n_b), r(r) { }void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1){

// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)merge_n(f0, n0, f1, n1, b, n_b, r);

}};


stable_sort_n



I stable_sort_n(I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r){

return merge_sort_n(f, n, merger_adaptive<I, B, R>(b, n_b, r));}


Importance of stable in-place and adaptive merge andsort

Stable in-place sort is of theoretical interest50; in practice, it shouldnever be usedIts simple extension to the memory-adaptive algorithm leads to agood practical algorithm when a buffer of as low as 1% is availableand becomes extremely competitive with 10-25%

50Donald Knuth ranked this problem as 47 out of 50 – just a notch below the LastFermat Theorem – in the first edition Volume 3 of his Art of Computer Programming


The origin of computer sorting and binary searching

The first description of binary searching, binary insertion sort, andcopying merge sort was John W. Mauchly’s 1946 lecture “Sortingand Collating”51

51Martin Campbell-Kelly and Michael R. Williams, editors. The Moore SchoolLectures: Theory and Techniques for Design of Electronic Digital Computers. TheMIT Press and Tomash Publishers, 1985.


Conclusions

Mathematicians use definitions and theorems as building blocksfor ever more powerful resultsThis chapter shows how a difficult problem of stable sorting inminimal space can be effectively solved using simple buildingblocks

Copying (forward and backward)Rotate (for various iterator categories)Lower and upper bound

It provides specific examples of why our attention to interfaces(arguments and return values) is importantIt shows that the technique of memory adaptive algorithms,which we first introduced in a rather artifical context (reverse forforward iterators), leads to a simple, efficient version of animportant algorithm (stable sort)Lower and upper bound, merge, and sort allow sequences to beused efficiently as representations of sets and multisets


Project

ProjectDesign a benchmark comparing performance ofunstable_bidirectional_partition, single_cycle_partition,sentinel_partition, and sentinel_single_cycle_partition fordifferent array sizes, element sizes, and distributions of elementssatisfying the predicateDesign a benchmark to analyze the performance of stable_sort_nfor different array sizes, element sizes, and buffer sizes


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 Rotations


11 Coordinate structuresCoordinate structuresLinkable iteratorsLinking operationsLink rearrangementReversing a linkable rangeExamples of linkable iteratorsGeneralized rearrangementsPartition


Contents III

MergingSortingTemporarily breaking invariantsBifurcate coordinatesTraversing treesSimilarity, equivalence, and orderingConclusionProjects





Contents IV

15 C++ machinery

16 Acknowledgments

17 Index


Intuition for coordinate structures

In Chapter 6 we defined a family of iterator concepts thatgeneralize pointers and describe linear traversals within anytype-homogenous data structureTo allow traversal within arbitrary data structures, we generalizeiterators to a family of concepts called coordinate structuresWhile an iterator type has only linear traversals (e.g., i 7→ i+ 1 ori 7→ i+ n), and a single static value type, a coordinate structuremay have several interrelated coordinate types, each with its ownvalue type and diverse traversal functionsCoordinate structures abstract the navigational aspects of datastructures, while containers (introduced in the next chapter)abstract storage management and ownershipThere could be multiple coordinate structures describing the sameset of objects


Definition of coordinate structure

DefinitionA concept is a coordinate structure if it consists of:

One or more coordinate typesFor each coordinate type Ti, a value type ValueType(Ti)For each coordinate type, dereferencing functions source and/orsink providing “fast” readable, writable, or mutable access

Access via a coordinate value is as fast as via any other mechanism,so there is no need to “cache” a low-level pointer or reference

For each coordinate type Ti, one or more traversal functions:

fi,j : Ti × · · · → Tk


Reflections on coordinate structure

Note that “coordinate structure” is not a concept but a family ofrelated concepts

It is not a concept because it does not have specific coordinate typesor traversal functions

No algorithms can be defined on coordinate structure as such;meta-algorithms are possible but are outside the scope of this book


Taxonomy of coordinate structures

Related types Most coordinate structures have a singlecoordinate type; a few have two or more

Dereference regularity The dereferencing functions on a coordinatetype may be regular (repeatable) or may beweak, meaning a protocol must be followedbetween dereferences

Topology The traversal functions may allow linear,bifurcating, multi-dimensional, or graphtraversal

Traversal regularity The traversal functions may be regular(supporting multi-pass algorithms), or weak(supporting only single-pass algorithms)

Complexity The traversal functions may have linear,logarithmic, or amortized constant-timecomplexity

Changeable topology The topology between coordinate values maybe fixed or mutable


Plan for the chapter

Iterators are the simplest and most useful coordinate structures;they are concepts of linear fixed-topology coordinatesThis chapter introduces coordinate structures illustrating moredimensions in our taxonomy:

Linkable iterators: changeable topology, with different algorithmsfor partition, merging, and sortingBifurcated coordinates: nonlinear topologies, with algorithms onbinary treesLinkable bifurcated coordinates: algorithms for iterative,constant-space binary tree traversal


Linkable iterators

A linkable iterator does not have a fixed successor or predecessor;these relationships between linkable iterators are mutableA linkable range is a range defined by a pair of linkable iterators[f, l)The mutability of the relationships between linkable iteratorsrequires care to preserve the invariants of ranges:

Iterators in a range are totally ordered by ≺Multiple input or output ranges of an algorithm are disjointThe iterators in the input to an algorithm are preserved in its output


Linking operations for linkable iterators

The iterator concepts require the unique traversal operationssuccessor and predecessor for a given typeWe treat the linking operation differently, passing them asparameters, because of different ways linking is performed on thesame typeSimilarly, in chapter 3 the operation was an explicit parameter topower

To define the requirements on the type of objects performinglinking operations, we define related concepts ForwardLinker,BackwardLinker, and BidirectionalLinker


Concept ForwardLinker

DefinitionForwardLinker(S)⇒

A type function IteratorType(S) is definedForwardIterator(IteratorType(S))An object of type S is a proper mutator taking two iterators of theaffiliated typeFor all set_link ∈ S, immediately after set_link(i, j) is performed,successor(i) = j


Concept BackwardLinker

DefinitionBackwardLinker(S)⇒

A type function IteratorType(S) is definedBidirectionalIterator(IteratorType(S))An object of type S is a proper mutator taking two iterators of theaffiliated typeFor all set_link ∈ S, immediately after set_link(i, j) is performed,i = predecessor(j)


Concept BidirectionalLinker

DefinitionBidirectionalLinker(S)⇒

ForwardLinker(S)BackwardLinker(S)


Link rearrangements

A link rearrangement52 is an algorithm taking a linked range andproducing a linkable range, with the properties:Precondition (The input range is well-formed, i.e., doesn’t

contain a cycle)

Postcondition (The output range is well-formed)Every iterator in the input range appears in theoutput rangeEvery iterator in the output range appeared in theinput rangeEvery iterator in the output range designates thesame object as before the rearrangement, and thisobject has the same valuesuccessor and predecessor relationships that heldin the input range may or may not hold in theoutput range

52The rearrangements introduced in Chapter 7 rearrange data; link rearrangementsrearrange links


reverse_linkable

template <typename S>requires(ForwardLinker(S))

IteratorType(S) reverse_linkable(IteratorType(S) f, IteratorType(S) l,S set_link)

{typedef IteratorType(S) I;I r = l;while (f != l) {

I tmp = successor(f);set_link(f, r);r = f;f = tmp;

}return r;

}


Concept LinkableForwardIterator

DefinitionLinkableForwardIterator(I)⇒

ForwardIterator(I)For all i ∈ I,

source(i.p).forward_link = successor(i)For all j ∈ I, immediately after sink(i.p).forward_link← j,successor(i) = j


Concept LinkableBidirectionalIterator

DefinitionLinkableBidirectionalIterator(I)⇒

BidirectionalIterator(I) ∧ LinkableForwardIterator(I)For all j ∈ I,

source(j.p).backward_link = predecessor(j)For all i ∈ I, immediately after sink(j.p).backward_link← i,i = predecessor(j)


forward_linker, backward_linker andbidirectional_linker

template <typename I> requires(LinkableForwardIterator(I))struct forward_linker{

void operator()(I x, I y){ sink(x.p).forward_link = y.p; }

};

template <typename I> requires(LinkableBidirectionalIterator(I))struct backward_linker{

void operator()(I x, I y){ sink(y.p).backward_link = x.p; }

};

template <typename I> requires(LinkableBidirectionalIterator(I))struct bidirectional_linker{

void operator()(I x, I y){ sink(x.p).forward_link = y.p; sink(y.p).backward_link = x.p; }

};


reverse_forward_linkable andreverse_bidirectional_linkable

template <typename I>requires(LinkableForwardIterator(I))

I reverse_forward_linkable(I f, I l){

return reverse_linkable(f, l, l, forward_linker());}

template <typename I>requires(LinkableBidirectionalIterator(I))

I reverse_bidirectional_linkable(I f, I l){

return reverse_linkable(f, l, l, bidirectional_linker());}


Generalized rearrangements

A generalized rearrangement is an algorithm taking k > 1 rangesand producing l > 1 ranges

Copying merge takes two input ranges and produces one outputrangeIn-place partition may be viewed as taking one input range andproducing two output rangesIn-place rotate may be viewed as taking two input ranges ([f,m)and [m, l)) and producing two output ranges ([f,m ′) and [m ′, l))


Generalized link rearrangements

A generalized link rearrangement has additional properties:Precondition The input ranges must be disjoint

Postcondition Every iterator in an input range appears in one ofthe output rangesEvery iterator in an output range appeared in oneof the input rangesEvery iterator in the output range designates thesame object as before the rearrangement, and thisobject has the same valuesuccessor and predecessor relationships that heldin the input range may or may not hold in theoutput rangeThe output ranges are disjoint


partition_linkable_unoptimized

template <typename S, typename P>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))pair<pair<IteratorType(S), IteratorType(S)>,

pair<IteratorType(S), IteratorType(S)> >partition_linkable_unoptimized(IteratorType(S) f, IteratorType(S) l, P p,

S set_link){

typedef IteratorType(S) I;I h0 = l; I t0 = l;I h1 = l; I t1 = l;while (f != l) {

if (p(source(f))) {if (h1 == l) { h1 = f; t1 = f; } else { set_link(t1, f); t1 = f; }

} else {if (h0 == l) { h0 = f; t1 = f; } else { set_link(t0, f); t0 = f; }

}++f;

}return pair<pair<I, I>, pair<I, I> >

(pair<I, I>(h0, t0), pair<I, I>(h1, t1));}

Compare with partition_copy_n in Chapter 9Stepanov, McJones Elements of Programming March 14, 2008 611 / 880

Improving on partition_linkable_unoptimized

By unrolling the loop, the inner tests on h0 and h1 can be avoidedafter the not-satisfying and satisfying objects are foundBy maintaining a state indicating whether the predicate wassatisfied for the previous element, set_link calls that do notchange the successor can be avoidedThe state can be represented by the location in the code ratherthan by an explicit state variable; using goto avoids setting andtesting a state variable53

53Undisciplined use of goto leads to code that is harder to understand and lesslikely to be correct; disciplined use frequently improves clarity and efficiency.Benchmarking is required to verify that versions with goto are faster on processorswith instruction level parallelism.


partition_linkable signature

template <typename S, typename P>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))pair<pair<IteratorType(S), IteratorType(S)>,

pair<IteratorType(S), IteratorType(S)> >partition_linkable(IteratorType(S) f, IteratorType(S) l, P p, S set_link){

typedef IteratorType(S) I;


partition_linkable body

entry: I h0 = l; I t0 = l; I h1 = l; I t1 = l;if (f == l) { goto exit; }if (p(source(f))) { h1 = f; goto s1; }else { h0 = f; goto s0; }

s0: t0 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { h1 = f; goto s3; }else { goto s0; }

s1: t1 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { goto s1; }else { h0 = f; goto s2; }

s2: t0 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { set_link(t1, f); goto s3; }else { goto s2; }

s3: t1 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { goto s3; }else { set_link(t0, f); goto s2; }

exit: return pair<pair<I, I>, pair<I, I> >(pair<I, I>(h0, t0), pair<I, I>(h1, t1));

}


merge_linkable_nonempty

template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))triple<IteratorType(S), IteratorType(S), IteratorType(S)>merge_linkable_nonempty(IteratorType(S) f0, IteratorType(S) l0,

IteratorType(S) f1, IteratorType(S) l1, R r, S set_link){

typedef IteratorType(S) I;typedef triple<I, I, I> Triple;

entry: I h, t;if (r(source(f1), source(f0))) { h = f1; goto s1; }else { h = f0; goto s0; }

s0: t = f0; ++f0;if (f0 == l0) { set_link(t, f1); return Triple(h, t, l1); }if (r(source(f1), source(f0))) { set_link(t, f1); goto s1; }else { goto s0; }

s1: t = f1; ++f1;if (f1 == l1) { set_link(t, f0); return Triple(h, t, l0); }if (r(source(f1), source(f0))) { goto s1; }else { set_link(t, f0); goto s0; }

}


Interface ofmerge_linkable_nonempty

merge_linkable_nonempty takes two linkable ranges andreturns the beginning and end of the merged linkable range as thefirst and third elements of a tripleIt returns the last-visited iterator in the output range as the secondelement of the triple; this iterator could be used with the followingprocedure to link the range to another linkable range

template <typename I>requires(LinkableForwardIterator(I))

void link_at_end(I f0, I l0, I f1){

// Precondition: f0 , l0I next = successor(f0);while (next != l0) {

f0 = next;++next;

}set_successor(f0, f1);

}


Properties of partition_linkable andmerge_linkable_nonempty

partition_linkable andmerge_linkable_nonempty are stable:every set_link call links to an iterator occurring not earlier in theoriginal list(s)


sort_linkable_n


StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))pair<IteratorType(S), IteratorType(S)>sort_linkable_n(IteratorType(S) f, DistanceType(IteratorType(S)) n,

R r, S set_link){

// Precondition: n > 0typedef IteratorType(S) I;typedef triple<I, I, I> Triple;typedef pair<I, I> Pair;if (n == 1) return Pair(f, successor(f));Pair p1 = sort_linkable_n(f, half_nonnegative(n), r, set_link);Pair p2 = sort_linkable_n(p1.second, n - half_nonnegative(n), r, set_link);Triple t = merge_linkable_nonempty(p1.first, p1.second,

p2.first, p2.second, r, set_link);link_at_end(t.second, t.third, p2.second);return Pair(t.first, p2.second);

}

While the number of operations performed is close to optimal,poor locality of reference limits its usefulness if the linkedstructure does not fit into cache memoryStepanov, McJones Elements of Programming March 14, 2008 618 / 880

Exercise

Exercisesort_linkable_n andmerge_sort_n from Chapter 9 are very similar;figure out how to unify them


sort_linkable_recursive


StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))IteratorType(S) sort_linkable_recursive(IteratorType(S) f, IteratorType(S) l,

R r, S set_link){

if (f == l) return f;else return sort_linkable_n(f, l - f, r, set_link).first;

}


sort_forward_linkable_recursive

template <typename I, typename R>requires(Readable(I) && LinkableForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_forward_linkable_recursive(I f, I l, R r){

return sort_linkable_recursive(f, l, r, forward_linker());}


Temporarily breaking invariants

A nontrivial invariant involving more than one object will betemporarily violated during an update of those variables

For example, there is a point during the execution of swap(x,y)when both x and y are copies of the same value

Most linked rearrangements do not depend on predecessor linksMaintaining the invariant i = predecessor(successor(i)) requiresextra time; for sorting this requires n log2 n extra timeParameterizing the ForwardLinker used by a linked rearrangementallows the caller to control when to restore the predecessorinvariant


restore_backward_links

template <typename S>requires(BackwardLinker(S))

void restore_backward_links(IteratorType(S) f, IteratorType(S) l, S set_link){

while (f != l) {set_link(f, successor(f));++f;

};}


sort_bidirectional_linkable_recursive

template <int k, typename I, typename R>requires(Readable(I) && LinkableBidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_bidirectional_linkable_recursive(I f, I l, R r){

f = sort_linkable_recursive(f, l, r, forward_linker());restore_backward_links(f, l, backward_linker());return f;

}


Forward versus bidirectional linkable structures

We have demonstrated efficient algorithms for reverse, partition,and merge (hence merge sort) of forward linkable structuresAdding backward links to a forward linkable structure does notappear to improve algorithms on the structure as a wholeHowever, backward links allow constant-time removal andinsertion of elements at an arbitrary locationSince it is the efficiency of insertion and deletion that is often thereason for choosing linkable structures in the first place,bidirectional linkage should be seriously considered


Concept WeakBifurcateCoordinate

DefinitionWeakBifurcateCoordinate(T)⇒

Regular(T)The type functionWeightType(T) is definedInteger(WeightType(T))The unary predicates is_empty, has_left_successor andhas_right_successor are defined on TThe actions step_left and step_right, and the correspondingtransformations left_successor and right_successor, are definedon TAmortizedConstantTime(step_left) ∧

AmortizedConstantTime(step_right)


Implication of weakness of WeakBifurcateCoordinate

Because of the lack of regularity of step_left and step_right, afteran algorithm takes one of the branches, the other branch can neverbe taken


Concept BifurcateCoordinate

DefinitionBifurcateCoordinate(T)⇒

WeakBifurcateCoordinate(T)RegularAction(step_left) ∧ RegularAction(step_right)


Concept BidirectionalBifurcateCoordinate (1 of 3)

Definition (requirements, signatures)BidirectionalBifurcateCoordinate(T)⇒

BifurcateCoordinate(T)The unary predicate has_predecessor is defined on TThe action step_up and the corresponding transformationpredecessor are defined on TRegularAction(step_up)AmortizedConstantTime(step_up)



Definition (signatures)

template <typename C>requires(BidirectionalBifurcateCoordinate(C))

bool is_left_successor(C c){

// Precondition: has_predecessor(c)C parent = predecessor(c);return has_left_successor(parent) && left_successor(parent) == c;

}


bool is_right_successor(C c){

// Precondition: has_predecessor(c)C parent = predecessor(c);return has_right_successor(parent) && right_successor(parent) == c;

}



Definition (axioms)For any coordinate x ∈ T :

has_left_successor(x)⇒ predecessor(left_successor(x)) = x

has_right_successor(x)⇒ predecessor(right_successor(x)) = x

has_predecessor(x)⇒is_left_successor(x) ∨ is_right_successor(x)


Reachability of bifurcate coordinates

template <typename C>requires(BifurcateCoordinate(C))

bool reachable(C x, C y){

if (x == y) return true;if (has_left_successor(x)) return reachable(left_successor(x), y);if (has_right_successor(x)) return reachable(right_successor(x), y);return false;

}


bool left_reachable(C x, C y){

return has_left_successor(x) && reachable(left_successor(x), y);}


bool right_reachable(C x, C y){

return has_right_successor(x) && reachable(right_successor(x), y);}


Properties of bifurcate coordinates

DefinitionThe descendants of a bifurcate coordinate x are all the coordinates ysuch that reachable(x,y)

DefinitionThe descendants of x are acyclic if for all y in the descendants of x,¬left_reachable(y,y) ∧ ¬right_reachable(y,y)

DefinitionThe descendants of x are a tree if they are acyclic and for all y, z in thedescendants of x, ¬left_reachable(y, z) ∨ ¬right_reachable(y, z)


weight_recursive


WeightType(C) weight_recursive(C f){

WeightType(C) w(1);if (has_left_successor(f)) w += weight_recursive(left_successor(f));if (has_right_successor(f)) w += weight_recursive(right_successor(f));return w;

}

LemmaThe descendants of x are a finite tree if and only if they are a tree andweight_recursive(x) terminates


height_recursive


WeightType(C) height_recursive(C f){

WeightType(C) l(0), r(0);if (has_left_successor(f)) l = height_recursive(left_successor(f));if (has_right_successor(f)) r = height_recursive(right_successor(f));return 1 + max(l, r);

}

Lemmaheight(x) 6 weight(x)


Recursive tree traversal

template <typename C, typename Pre, typename In, typename Post>requires(BifurcateCoordinate(I) &&

UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&UnaryProcedure(Post) && Domain(Pre) == ValueType(C))

void traverse_nonempty_recursive(C r, Pre f, In g, Post h){

f(source(r));if (has_left_successor(r))

traverse_nonempty_recursive(left_successor(r), v);g(source(r));if (has_right_successor(r))

traverse_nonempty_recursive(right_successor(r), v);h(source(r));

}


Problems with recursive traversal

It requires stack space proportional to the height of the tree, whichcan be as large as the weight, which is often unacceptable forlarge, unbalanced treesThe interface is not flexible, for example it cannot be used tocompare two treesWe will now develop low-cost iterator-like traversal mechanisms


traverse_step

enum visit { pre, in, post };


void traverse_step(C& c, visit& v){

// Precondition: has_predecessor(c) ∨ v , postswitch (v) {case pre:

if (has_left_successor(c)) step_left(c);else v = in;break;

case in:if (has_right_successor(c)) { v = pre; step_right(c); }else v = post;break;

case post:if (is_left_successor(c)) v = in;step_up(c);

}}


Iterative tree traversal

template <typename C, typename Pre, typename In, typename Post>requires(BidirectionalBifurcateCoordinate(C) &&

UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&UnaryProcedure(Post) && Domain(Pre) == ValueType(C))

void traverse(C c, Pre f_pre, In f_in, Post f_post){

if (is_empty(c)) return;C root = c;visit v = pre;while (true) {

switch (v) {case pre: f_pre (source(c)); break;case in: f_in (source(c)); break;case post: f_post(source(c)); if (c == root) return;}traverse_step(c, v);

}}


Binary trees and iterators

Traversal through binary trees is fundamentally different thaniterating through a range because of the three directions ofmovementtraverse_step and traverse provide full access to the structure ofa binary treeIt is possible to write iterator adaptors to traverse_step thatprovide sequential access to the nodes of a binary tree in preorder,inorder, or postorder, but such an iterator loses structuralinformationAn iterator adapter whose value type pairs a visit with acoordinate preserves the structural information(See the Projects section at the end of this chapter for more details)


Advantages of bidirectional binary trees

We recommended bidirectional linkage for constant-time insertionand deletion of elementsWith binary trees the situation is different: without predecessorlinks even the simplest traversal algorithms require linear spaceAdding predecessor links allows constant-space traversal, andalso makes possible various rebalancing algorithms

The overhead for the extra link, while 50% in the worst case, istypically much less in practice


Linkable bifurcate coordinates

For linkable iterators, we passed the linking operation as aparameter because of the need to use different linking operations,such as restoring back links after sortFor linkable bifurcate coordinates there does not appear to be aneed for alternate versions of the linking operations, so weoverload them on the concept


Concept LinkableBifurcateCoordinate

DefinitionLinkableBifurcateCoordinate(T)⇒

BifurcateCoordinate(T)Proper mutators set_left_successor and set_right_successortaking two iterators of type T are definedImmediately after set_left_successor(i, j) is performed,left_succesor(i) = j

Immediately after set_right_successor(i, j) is performed,right_succesor(i) = j


Concept LinkableBidirectionalBifurcateCoordinate

DefinitionLinkableBidirectionalBifurcateCoordinate(T)⇒

LinkableBifurcateCoordinate(T)Immediately after set_left_successor(i, j) is performed,has_left_successor(i)⇒ predecessor(j) = i

Immediately after set_right_successor(i, j) is performed,has_right_successor(i)⇒ predecessor(j) = i


Reversing links

traverse_step is an efficient way to traverse via bidirectionalbifurcating coordinates, but requires the predecessor functionWhen the predecessor function is not available and recursive(stack-based) traversal is unacceptable because of unbalancedtrees, there are interesting algorithms that depend on temporarilystoring the link to the predecessor in a link normally containing asuccessor; this assures there is a path back to the root 54

54Link reversal was introduced in: H. Schorr and W.M. Waite. An Efficient andMachine-Independent Procedure for Garbage Collection in Various List Structures.Communications of the ACM, Volume 10, Number 8, August 1967, pages 501-506 (it wasindependetly discovered by L.P. Deutsch). A version without tag bits was publishedin: J.M. Robson. An Improved Algorithm for Traversing Binary Trees WithoutAuxiliary Stack. Information Processing Letters, Volume 2, 1973, pages 12-14; and JosephM. Morris. Traversing Binary Trees Simply and Cheaply, Information Processing Letters,Volume 9, Number 5, 1979, pages 197-200. An ingenious technique of “rotating” thelinks was published in: Gary Lindstrom. Scanning List Structures Without Stack orTag Bits. Information Processing Letters, Volume 2, 1973, pages 47-51; andindependently in: Barry Dwyer. Simple Algorithms for Traversing a Tree Without anAuxiliary Stack. Information Processing Letters, Volume 2, 1974, pages 143-145.


Exclusive-or’ing links

An interesting technique for iterative traversal without extrastorage for predecessor links stores in each successor link theexclusive-or of the predecessor and the corresponding successor55

By additionally requiring that the address of a node’s leftsuccessor is less than the address of its right successor, it ispossible to design a data structure that supports a non-recursivetraversal algorithm that does not modify the tree and uses only aconstant amount of additional storage

55The exclusive-or technique was first applied to binary trees in: Laurent Siklóssy.Fast and Read-only Algorithms for Traversing Trees Without an Auxiliary Stack.Information Processing Letters, Volume 1, 1972, pages 149-152.


Similarity

DefinitionTwo sets S and S ′ of coordinates from the same coordinate type T of acoordinate structure are similar under a given one-to-onecorrespondence if any traversal function maps correspondingcoordinates to corresponding coordinates

This definition can be extended in various waysTo coordinate structures with more than one coordinate typeTo sets of coordinates from different but isomorphic coordinatestructures

Similarity does not depend on the values of the objects pointed toby the coordinates

Algorithms for testing similarity use only traversal functions butnot dereferencing functions


Examples of similarity

ExamplesTwo ranges of iterators [f, l) and [f ′, l ′) are similar if l− f = l ′ − f ′

Two trees are similar if both are empty or they have similar leftand right subtrees


Equivalence

DefinitionTwo sets S and S ′ of coordinates from the same coordinate type T of acoordinate structure represent equivalent value sets under a givenone-to-one correspondence and a given equivalence relation if they aresimilar and dereferencing corresponding coordinates gives equivalentobjects


lexicographic_equivalence

template <typename I0, typename I1, typename R>requires(Readable(I0) && Iterator(I0) &&

Readable(I1) && Iterator(I1) &&ValueType(I0) == ValueType(I1) &&EquivalenceRelation(R) && Domain(R) == ValueType(I0))

bool lexicographic_equivalence(I0 f0, I0 l0, I1 f1, I1 l1, R r){

while (true) {if (f0 == l0 && f1 == l1) return true;if (f0 == l0 || f1 == l1) return false;if (!r(source(f0), source(f1))) return false;++f0; ++f1;

}}


lexicographic_ordering

template <typename I0, typename I1, typename R>requires(Readable(I0) && Iterator(I0) &&

Readable(I1) && Iterator(I1) &&ValueType(I0) == ValueType(I1) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I0))

bool lexicographic_ordering(I0 f0, I0 l0, I1 f1, I1 l1, R r){

while (true) {if (f1 == l1) return false;if (f0 == l0) return true;if (r(source(f0), source(f1))) return true;if (r(source(f0), source(f1))) return false;++f0; ++f1;

}}

This would benefit from having a three-way comparison insteadof an ordering


bifurcate_similarity

template <typename C, typename R>requires(BidirectionalBifurcateCoordinate(C))

bool bifurcate_similarity(C c0, C c1){

if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);C root0 = c0; C root1 = c1;visit v0 = pre; visit v1 = pre;while (true) {

traverse_step(c0, v0); traverse_step(c1, v1);if (d0 != d1) return false;if (c0 == root0 && v0 == post) return true;

}}


bifurcate_equivalence

template <typename C, typename R>requires(BidirectionalBifurcateCoordinate(C) &&

EquivalenceRelation(R) && Domain(R) == ValueType(C))bool bifurcate_equivalence(C c0, C c1, R r){

if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);C root0 = c0; C root1 = c1;visit v0 = pre; visit v1 = pre;while (true) {

if (v0 == pre && !r(source(c0), source(c1))) return false;traverse_step(c0, v0); traverse_step(c1, v1);if (v0 != v1) return false;if (c0 == root0 && v0 == post) return true;

}}


bifurcate_ordering

template <typename C, typename R>requires(BidirectionalBifurcateCoordinate(C) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(C))bool bifurcate_ordering(C c0, C c1, R r){

if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && !is_empty(c1);C root0 = c0; C root1 = c1;visit v0 = pre; visit v1 = pre;while (true) {

if (v0 == pre) {if (r(source(c0), source(c1))) return true;if (r(source(c1), source(c0))) return false;

}traverse_step(c0, v0); traverse_step(c1, v1);if (v0 != v1) return v0 > v1;if (c0 == root0 && v0 == post) return false;

}}


Conclusions

While iterators are very flexible, we introduced severalgeneralizations:

Linkable iterators allow the topology between iterators to bechanged rather than changing the objects to which the iteratorspointBifurcate coordinates extend from linear to binary tree topology

There is, of course, a wide variety of different coordinate structuresNo computational device (e.g., goto) is harmful per se; it could andshould be used when appropriateAn invariant is associated with a scope, such as a loop, analgorithm, or a datatype

Outside the scope, the invariant must holdInside the scope, it is permissible to violate the invariantProper maintenance of the invariant requires attention toconcurrency and exceptions


Projects

Project 1Design an adapter that converts traverse_step into an iterator, takinginto consideration the following issues:

What should the value type be?What iterator concept should be modeled?What should the limit iterator be to allow traversal of a subtreerooted at an arbitrary coordinate?

Project 2Extend ordering to an arbitrary coordinate structure by defining anappropriate meta-algorithm


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 Rotations



12 Composite objectsComposite objects, regular types, and containersEquality on composite objectsCollocated typesDistributed typesConstruction


Contents III

Memory allocation and deallocationExplicit construction and destructionListsArraysContainers as elementsRelational conceptsCompatibilityGeneralized copy algorithmsUnderlying typeGeneralized permutation algorithmsMore generalized permutation algorithmsGeneralized partition algorithmsUnderlying iteratorMore generalized partition algorithmsunderlying_compare


Contents IV

More generalized merge and sort algorithmsComposite objects



15 C++ machinery

16 Acknowledgments

17 Index


Definition of composite object (1 of 4)

DefinitionA type T belongs to a concept that is a composite object concept if:

Parts Objects of type T are made up of other objects calledparts

Subparts Subsidiary definition: x is a subpart of y if x is a part of yor x is a subpart of a part of y



DefinitionConnected T has an affiliated coordinate structure

Every object of type T has a distinguished startingaddressIt is possible to reach every part of an object of type Tusing traversals in the coordinate structure, beginningat the starting address



DefinitionNoncircular No object is a subpart of itself

Disjoint Subsidiary definition: Two objects are disjoint if theyhave no subpart in commonIf x and y are objects, either one is a subpart of theother or they are disjoint



DefinitionComplete destruction Destroying an object of type T causes the

destruction of every part of the object (and,therefore, every subpart)


Reflections on composite objects

Like coordinate structure, “composite object” is not a concept buta family of related conceptsNo algorithms can be defined on composite objects as such;meta-algorithms are possible but are outside the scope of this book


Examples of composite objects

ExampleA tuple: traversal via member accessAn array: traversal via random access iteratorA binary tree: traversal via bifurcate coordinateA complex number: traversal via member access


Regularity and composite objects

In Chapter 1 we introduced the concept Regular and noted

The requirements on the element type allowing objects to besorted should be satisfied by arrays themselves, so we can sortan array of arrays

In this chapter we show how to implement a variety of compositeobjects in such a way that they satisfy the requirements for regulartypes

EqualityAssignment (respecting equality)Default construction (making an object assignable/destructable)Copy construction (equivalent to default construction followed byassignment)Total ordering (natural or default)Destruction


Container

DefinitionA container is a composite object whose interpretation is a collection ofthe interpretations of the parts of the container

ExampleA tupleAn arrayA binary tree


The role of containers

Mathematicians construct all of mathematics on a foundationconsisting only of setsThe computer science analog of sets is containers, and they comein many varieties


Coordinate structure of a container

***** This is subsumed by our definition of composite object

DefinitionEach container type defines a corresponding coordinate structure andfunctions giving one or more specific coordinates to initiate traversalsor access

ExampleTuple: member accessArray: random-access iterators and begin and endBinary tree: bifurcate coordinates and root


Equality on containers

DefinitionTwo containers are equal if corresponding parts are equal

ExampleArray equality: use lexicographic_equivalence with equality onthe element typeBinary tree equality: use bifircate_equivalence with equality onthe element type

As with attributes, there may be nonessential parts that are nottaken into account by equality; we will see examples later in thischapter


Structural equality

Recall from Chapter 1 that we follow Leibniz in defining equalityin terms of consistent observed behaviorThere is a related notion of structural equality

DefinitionTwo composite objects are structurally equal if their states are equal

For a container, structural equality is the same as Leibniz equalityObjects that are not structurally equal could still be equal in theLeibniz sense (such objects do not have unique representations)


Structural equality versus equality

ExamplesComplex numbers x+ iy and x ′ + iy ′ are equal if x = x ′ ∧ y = y ′

Complex numbers reiϕ and r ′eiϕ′

are equal if

r = r ′ ∧ϕ ≡ ϕ ′ (mod 2π)

Rational numbers a/b and a ′/b ′ are equal if ab ′ = ba ′

For two objects of the same type, these implications hold

They are identical (same object)⇒They are structurally equal (same state)⇒They are equal (same behavior)⇒They are equivalent (same under some equivalence relation)


Leibniz equality is difficult to implement for certaintypes

ExampleImplementing Leibniz equality for priority queues requires comparingsuccessive values from both queues

This consumes the elements compared, requiring copies to bemade of the two queuesIt requires n logn time


Collocated types

DefinitionA collocated type is one all of whose parts are adjacent in memory

ExampleGiven types T0, . . . , Tk−1, we can construct the heterogeneous typestructT0,...,Tk−1 with useful special cases

pairT0,T1

tripleT0,T1,T2

If all the Ti are the same, we can define the homogeneous typearrayk,T


Properties of collocated types

The correct semantics for assignment, default and copyconstruction, and destruction of a collocated type is elementwise;for equality and ordering it is the lexicographic techniquesdefined in the previous chapter56

Collocated types have constant area, so they can be fully allocatedat compile timeThere are several situations when a collocated type is not optimal

When the area of an object is not constant, that is, its area is fixed ordynamicWhen the object might be frequently moved (e.g., an element of anarray being sorted)

56In some situations C++ automatically generates elementwise constructors,destructor, and assignment for struct types; equality and ordering are neverautomatically generated. Constant-size arrays in C++ are not first-class citizens andone must define a type such as the array_k in Appendix 2 to make them be regular.


Distributed types

DefinitionA distributed type has parts that are not adjacent:

Data representation of the externally-visible behavior of thetype

Connectors traversal through all the dataHeader the origin for all traversal

Padding memory locations included to maintain hardwarealignment or optimize cache behavior

Reserve memory locations reserved for future growthOther locks, software caches, handles for system resources, etc.


Ownership: containers versus iterators

We distinguish pointers representing ownership within acontainer (connectors) from pointers representing intermediatepositions within an algorithm (iterators or coordinates)

Example: list

The functions begin and end, when applied to a list, each return a listiterator

The function successor, when applied to a list iterator, returns anotherlist iterator; there is no function like successor defined on list

The destructor of the listT type is responsible for iterating through allthe elements, destroying each object of type T , and freeing the storage ofits containing list node

Destroying a list iterator has no impact on any list object


list: example of container versus iterator

template <typename T> requires(Regular(T))struct list_node{

T value;list_node* forward_link;// Destructor does nothing

};

template <typename T> requires(Regular(T))struct list_iterator{

list_node<T>* p;// Destructor does nothing

};

template <typename T> requires(Regular(T))struct list{

list_iterator<T> first;// Destructor erases all elements

};


Primary objects

DefinitionA primary object is one that is not a subpart of another object; an objectthat is not primary is secondary

A primary object may have static storage durationA primary object may have automatic storage durationA distributed primary object may have parts with dynamicstorage duration


Construction

Construction of a primary object takes place in static or automaticstorageConstruction of a secondary object takes place in a subpart of aprimary object

Example***** Show code with global object, local object, and insert constructor?????


(Languages without ownership)

Many languages follow the example of Lisp57 use the same typefor both a list as a container and a list iteratorThis design decision means it is not possible for the programmerto explicitly free a list since there might be other variablescontaining list objects with overlapping storage

57John McCarthy. Recursive Functions of Symbolic Expressions and TheirComputation by Machine, Part I. Communications of the ACM, Volume 3, Number 4,1960, pages 184-195


remote


struct remote{

T* p;remote() : p(0) {}remote(const T& x) : p(new T(x)) { }remote(const remote& x) : p(new T(source(x.p))) { }~remote() { delete p; }friend bool operator==(const remote& x, const remote& y){ return source(x.p) == source(y.p); }friend void swap(remote& x, remote& y) { swap(x.p, y.p); }void operator=(remote& x) { swap(sink(this), x); }friend bool operator<(const remote& x, const remote& y){ return source(x.p) < source(y.p); }

};


Allocation and deallocation

void* raw_allocate(size_t n){

return ::operator new(n);}


T* allocate(size_t n){

return (T*)raw_allocate(n * sizeof(T));}

void deallocate(void* p){

::operator delete(p);}


construct_copy and destroy


struct construct_copy{

const T* p;construct_copy(const T& x) : p(&x) {}void operator()(T& q){

new (&q) T(source(p));}

};


void destroy(T& p){

sink(&p).~T();}


list_node and list_iterator


struct list_node{

T value;list_node* forward_link;list_node(const T& v, list_node* f) : value(v), forward_link(f) {}

};


struct list_iterator{

list_node<T>* p;list_iterator() : p(0) {}list_iterator(list_node<T>* p) : p(p) {}

};


Functions for list_iterator

template <typename T> requires(Regular(T))void operator++(list_iterator<T>& i) { i.p = source(i.p).forward_link; }

template <typename T> requires(Regular(T))void set_successor(list_iterator<T> i, list_iterator<T> j){ sink(i.p).forward_link = j.p; }

template <typename T> requires(Regular(T))bool operator==(list_iterator<T> i, list_iterator<T> j){ return i.p == j.p; }

template <typename T> requires(Regular(T))const T& source(list_iterator<T> i) { return source(i.p).value; }

template <typename T> requires(Regular(T))T& sink(list_iterator<T> i) { return sink(i.p).value; }


Type functions for list_iterator


struct value_type< list_iterator<T> >{

typedef T type;};


struct distance_type< list_iterator<T> >{

typedef DistanceType(list_node<T>*) type;};


struct iterator_category< list_iterator<T> >{

typedef forward_iterator_tag category;};


Insert functions for list_node

template <typename T, typename F>requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)

list_iterator<T> insert_construct(list_iterator<T> j, F f){

list_iterator<T> i(allocate< list_node<T> >(1));f(sink(i));set_successor(i, j);return i;

}


list_iterator<T> insert(list_iterator<T> i, const T& x){

return insert_construct(i, construct_copy<T>(x));}


More insert functions for list_node


list_iterator<T> insert_after_construct(list_iterator<T> i, F f){

list_iterator<T> j = insert_construct(successor(i), f);set_successor(i, j);return j;

}


list_iterator<T> insert_after(list_iterator<T> i, const T& x){

return insert_after_construct(i, construct_copy<T>(x));}


list_insert_after_iterator


struct list_insert_after_iterator{

list_iterator<T> h;list_iterator<T> t;list_insert_after_iterator() { }list_insert_after_iterator(list_iterator<T> h, list_iterator<T> t)

: h(h), t(t) { }void operator=(const T& x){

if (t == list_iterator<T>()) {h = insert(h, x);t = h;

} elset = insert_after(t, x);

}void operator++() { }

};


(Equality for list_insert_after_iterator)


bool operator==(const list_insert_after_iterator<T>& i,const list_insert_after_iterator<T>& j)

{return i.h == j.h && i.t == j.t;

}


Type functions for list_insert_after_iterator


struct value_type< list_insert_after_iterator<T> >{

typedef T type;};


struct distance_type< list_insert_after_iterator<T> >{

typedef DistanceType(list_iterator<T>) type;};


struct iterator_category< list_insert_after_iterator<T> >{

typedef iterator_tag category;};


Erase functions for list_node


list_iterator<T> erase_first(list_iterator<T> i){

list_iterator<T> j = successor(i);destroy(sink(i));deallocate(i.p);return j;

}


void erase_after(list_iterator<T> i){

set_successor(i, erase_first(successor(i)));}


list container


struct list{

list_iterator<T> first;list() : first(0) {}template <typename I>

requires(Readable(I) && Iterator(I) && ValueType(I) == T)list(I f, I l);list(const list& x);~list();friend void swap(list& x, list& y) { swap(x.first, y.first); }void operator=(list x) { swap(sink(this), x); }

};


Type functions for list

***** Really need a concept Container ?????

DefinitionThe type function IteratorType(C) is defined on any container type C;it returns a type such that Iterator(IteratorType(C))


Type functions for list


struct iterator_type< list<T> >{

typedef list_iterator<T> type;};


Functions for list

template <typename T> requires(Regular(T))IteratorType(list<T>) begin(const list<T>& x){

return x.first;}

template <typename T> requires(Regular(T))IteratorType(list<T>) end(const list<T>& x){

return list_iterator<T>();}

template <typename C> requires(Container(C))DistanceType(IteratorType(C)) size(const C& x){

return end(x) - begin(x);}

template <typename C>requires(Container(C))

bool is_empty(const C& x){

return begin(x) == end(x);}


Constructors for list

template <typename T, typename I>requires(Regular(T) && Iterator(I) &&

ValueType(I) == T)void insert(list<T>& x, IteratorType(list<T>) i, I f, I l){

x.first = copy(f, l, list_insert_after_iterator<T>(begin(x), i)).h;}

template <typename T>template <typename I>

requires(Regular(T) && Readable(I) && Iterator(I) &&ValueType(I) == T)

list<T>::list(I f, I l) : first(0){

insert(sink(this), end(sink(this)), f, l);}


list<T>::list(const list& x) : first(0){

insert(sink(this), end(sink(this)), begin(x), end(x));}


Destructor for list


void pop_all(list<T>& x){

while (!is_empty(x))x.first = erase_first(begin(x));

}


list<T>::~list(){

pop_all(sink(this));}


Equality and less than for list


struct equality{

bool operator()(const T& x, const T& y) { return x == y; }};


bool operator==(const list<T>& x, const list<T>& y){

return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),equality<T>());

}


bool operator<(const list<T>& x, const list<T>& y){

return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());}

These definitions work for any container *****Stepanov, McJones Elements of Programming March 14, 2008 701 / 880

partition for list

template <typename T, typename P>requires(Regular(T) && UnaryPredicate(P) && Domain(P) == T)

void partition_list(list<T>& x, list<T>& y, P p){

typedef IteratorType(list<T>) I;pair< pair<I, I>, pair<I, I> > pp =

partition_linkable(begin(x), end(x), p, forward_linker());x.first = pp.m0.m0;if (pp.m1.m0 != end(x)) {

set_successor(pp.m1.m1, begin(y));y.first = pp.m1.first;

}}


merge for list

template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)

void merge(list<T>& x, list<T>& y, R r){

if (is_empty(y)) return;if (is_empty(x)) x = y;else

x.first = merge_linkable_nonempty(begin(x), end(x), begin(y), end(y),r, forward_linker<IteratorType(list<T>)>()).m0;

y.first = end(y);}


sort for list


void sort_recursive(list<T>& x, R r){

x.first = sort_forward_linkable_recursive(begin(x), end(x), r);}


void sort(list<T>& x, R r){

x.first = sort_forward_linkable<32>(begin(x), end(x), r);}


array_header


struct array_header{

T* m;T* l;T a;

};

Invariants, where f = &a:[f,m) are constructed elements[m, l) are unconstructed (reserved) elements


allocate_array_header


array_header<T>* allocate_array(DistanceType(T*) n){

typedef array_header<T>* P;size_t bsize = size_t(predecessor(n)) * sizeof(T);P p = P(raw_allocate(sizeof(array_header<T>) + bsize));T* f = &sink(p).a;sink(p).m = f;sink(p).l = f + n;return p;

}


deallocate_array_header


void deallocate_array(array_header<T>* p){

deallocate(p);}


array container


struct array{

typedef DistanceType(IteratorType(array<T>)) N;array_header<T>* p;array() : p(0) {}array(N c); // size is 0 and capacity is carray(N s, N c, const T& x); // size is s, capacity is c, all elements equal to xtemplate <typename I>

requires(Readable(I) && Iterator(I) && ValueType(I) == T)array(I f, I l);array(const array& x);~array();friend void swap(array& x, array& y) { swap(x.p, y.p); }void operator=(array x) { swap(sink(this), x); }

};


void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n);


Type functions for array


struct iterator_type< array<T> >{

typedef T* type;};


Functions for array


IteratorType(array<T>) begin(const array<T>& x){

if (x.p == 0) return IteratorType(array<T>)(0);return IteratorType(array<T>)(&source(x.p).a);

}


IteratorType(array<T>) end(const array<T>& x){

if (x.p == 0) return IteratorType(array<T>)(0);return IteratorType(array<T>)(source(x.p).m);

}


IteratorType(array<T>) end_of_storage(const array<T>& x){

if (x.p == 0) return IteratorType(array<T>)(0);return IteratorType(array<T>)(source(x.p).l);

}


More functions for array


DistanceType(IteratorType(array<T>)) capacity(const array<T>& x){

return end_of_storage(x) - begin(x);}


bool is_full(const array<T>& x){

return end(x) == end_of_storage(x);}


push_construct


T& push_construct(array<T>& x, F f){

typedef DistanceType(IteratorType(array<T>)) N;N n = size(x);if (n == capacity(x))

reserve(x, max(N(1), n + n));f(sink(source(x.p).m));++sink(x.p).m;return sink(end(x) - 1);

}


push


T& push(array<T>& x, const T& y){

return push_construct(x, construct_copy<T>(y));}


array_push_iterator


struct array_push_iterator{

array<T>* p;array_push_iterator() {}array_push_iterator(array<T>& x) : p(&x) {}friend bool operator==(array_push_iterator x, array_push_iterator y){

return x.p == y.p;}friend void operator++(array_push_iterator& x) {}void operator=(const T& x){

push(sink(p), x);}

};


Type functions for array_push_iterator


struct value_type< array_push_iterator<T> >{

typedef T type;};


struct distance_type< array_push_iterator<T> >{

typedef DistanceType(IteratorType(array<T>)) type;};


struct iterator_category< array_push_iterator<T> >{



push for a range

template <typename T, typename I>requires(Regular(T) &&

Readable(I) && Iterator(I) && ValueType(I) == T)void push(array<T>& x, I f, I l){

copy(f, l, array_push_iterator<T>(x));}


Pop functions for array


void pop(array<T>& x){

--sink(x.p).m;destroy(sink(source(x.p).m));if (is_empty(x)) {

deallocate_array(x.p);x.p = 0;

}}


void pop_all(array<T>& x){

while (!is_empty(x)) pop(x);}


Constructors for array


array<T>::array(DistanceType(IteratorType(array<T>)) c) :p(allocate_array<T>(c))

{}


array<T>::array(DistanceType(IteratorType(array<T>)) s,DistanceType(IteratorType(array<T>)) c,const T& x) :

p(allocate_array<T>(c)){

while (s != N(0)) { push(sink(this), x); --s; }}


More constructors for array

template <typename T>template <typename I>

requires(Regular(T) && Readable(I) && Iterator(I) && ValueType(I) == T)array<T>::array(I f, I l) : p(0){

push<T, I>(sink(this), f, l);}


array<T>::array(const array& x) : p(allocate_array<T>(size(x))){

push<T>(sink(this), begin(x), end(x));}


Destructor for array


array<T>::~array(){

pop_all(sink(this));}


reserve


void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n){

if (n < size(x) || n == capacity(x)) return;array<T> tmp(n);copy(begin(x), end(x), array_push_iterator<T>(tmp));swap(tmp, x);

}


Equality and less than for array


bool operator==(const array<T>& x, const array<T>& y){

return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),equality<T>());

}


bool operator<(const array<T>& x, const array<T>& y){

return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());}


insert for array

template <typename T, typename I>requires(Regular(T) &&

Readable(I) && Iterator(I) && ValueType(I) == T)void insert(array<T>& x, IteratorType(array<T>) i, I f, I l){

DistanceType(IteratorType(array<T>)) o_f = i - begin(x);DistanceType(IteratorType(array<T>)) o_m = end(x) - begin(x);push(x, f, l);rotate(begin(x) + o_f, begin(x) + o_m, end(x));

}


sort for array


void sort(array<T>& x, R r){

typedef DistanceType(IteratorType(array<T>)) N;N n = size(x) / N(10);array<T> buffer(n, n, T());stable_sort_n(begin(x), size(x), begin(buffer), n, r);

}


Introduction

We have presented algorithms and containers that work welltogether when the elements of the containers are built-in types orC-style structsHowever performance suffers when the elements are themselveslarger containers

Sorting an array of arrays

In this section we present a solution to this problem using thenotions of underlying type and type compatibilityWe rewrite versions of various algorithms from chapters 6-10using this approach and define the underlying type for thecontainers in chapter 11


Relational concepts (from Chapter 1)

DefinitionA relational concept is a concept defined on two types


Concept SameGenus (from Chapter 1)

DefinitionSameGenus(T , T ′)⇒

The genus of T and the genus of T ′ are equal

ExampleSameGenus(int, long)


Concept PartiallyCompatible (1 of 3) (from Chapter 6)

Definition (requirements, signatures)PartiallyCompatible(T ,U)⇒

SameGenus(T ,U)

The partial equality predicates (=) on T ×U and U× T are definedThe (potentially implicit) unary predicates is_representableU(T)

and is_representableT (U) are definedThe partial assignments (←) on T ×U and U× T are definedThe morphisms U(T) and T(U) are defined



Definition (definition spaces)PartiallyCompatible(T ,U)⇒

For all t ∈ T and for all u ∈ U,is_representableU(t) ∧ is_representableT (u)⇒is_defined(=, t,u) ∧ is_defined(=,u, t)For all t ∈ T and for all u ∈ U,is_representableT (u)⇒ is_defined(←, t,u)

For all t ∈ T and for all u ∈ U,is_representableU(t) ∧ is_representableT (u)⇒is_defined(U(·), t) ∧ is_defined(T(·),u)



Definition (axioms)PartiallyCompatible(T ,U)⇒

Where = is defined, it is true if and only if its arguments have thesame interpretation← depends on the type but not the value of its first argument;where it is defined, it makes the first argument equal to the secondwithout modifying the secondWhere the morphisms U(T) and T(U) are defined, each constructsa new element of the target type that has the same interpretationas its argument


Properties of PartiallyCompatible types (from Chapter 6)

LemmaPartiallyCompatible(T ,U)⇒ PartiallyCompatible(U, T)

LemmaIf PartiallyCompatible(T ,U), PartiallyCompatible(U,V),PartiallyCompatible(T ,V), then (∀t ∈ T)(∀u ∈ U)(∀v ∈ V):

t = u⇒ u = t

(t = u∧ u = v)⇒ t = v

t← u⇒ t = u

T t(u)⇒ t = u

(Implication between a statement and a predicate means thepredicate holds immediately after execution of the statement)


Concept Compatible (from Chapter 6)

DefinitionCompatible(T ,U)⇒

PartiallyCompatible(T ,U)

For all t ∈ T and for all u ∈ U,is_representableU(t) ∧ is_representableT (u)


Type requirements of copy (from Chapter 6)

#if 0 // *****


PartiallyCompatible(ValueType(I), ValueType(O)))O copy(I f, I l, O r);

Readable(I) ensures source is definedIterator(I) ensures ++ is definedWritable(O) ensures sink is definedIterator(O) ensures ++ is definedPartiallyCompatible(ValueType(I),ValueType(O)) ensuresassignment is defined


Preconditions and postconditions of copy (fromChapter 6)


PartiallyCompatible(ValueType(I), ValueType(O)))O copy(I f, I l, O r);

Preconditions The input range must be readableThe output range must be writable and of at leastthe size of the input rangeEach value of the input range must berepresentable in the output value typeIf the ith iterator in the input range is aliased to thejth iterator of the output, then i 6 j (informally,every value is used before it is overwritten)

Postconditions The sequence of values in the output range isequal to the sequence of original values in theinput range


Implementation of copy (from Chapter 6)


PartiallyCompatible(ValueType(I), ValueType(O)))O copy(I f, I l, O r){

while (f != l) {sink(r) = source(f);++f;++r;

}return r;

}

We return the end of the output range because it might not beknown to the caller, who might find it useful

It is worth the small constant time to return it


Example using copy (from Chapter 6)

template <typename N, typename O>requires(Integer(N) && Writable(O) && Iterator(O) &&

PartiallyCompatible(N, ValueType(O)))O iota(N n, O r) // like APL ι{

return copy(N(0), n, r);}


copy_bounded (from Chapter 6)


PartiallyCompatible(ValueType(I), ValueType(O)))pair<I, O> copy_bounded(I f, I l, O r_f, O r_l){

while (f != l && r_f != r_l) {sink(r_f) = source(f);++f;++r_f;

}return pair<I, O>(f, r_f);

}

While the ends of both ranges are known to the caller, returningthe pair allows determining which range is smaller and where inthe larger range copying stopped


copy_n (from Chapter 6)


PartiallyCompatible(ValueType(I), ValueType(O)))pair<I, O> copy_n(I f, DistanceType(I) n, O r){


sink(r) = source(f);++f;++r;n = n - N(1);

}return pair<I, O>(f, r);

}


copy_k (from Chapter 6)

template <typename I, typename O, int k>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&

PartiallyCompatible(ValueType(I), ValueType(O)))void copy_k(I& f, O& r){

copy_k<k - 1>(f, r);sink(r) = source(f);++f; ++r;

}

template <typename I, typename O>void copy_k<0>(I&, O&) { }


copy_n_unrolled (from Chapter 6)


PartiallyCompatible(ValueType(I), ValueType(O)))pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r){


copy_k<k>(f, r);n = n - N(k);


}


copy_backward (from Chapter 6)

The main difference from copy is the aliasing precondition: if theith iterator in the input range is aliased to the jth iterator of theoutput, then j 6 i


Writable(O) && BidirectionalIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))

O copy_backward(I f, I l, O r){

while (f != l) {--l;--r;sink(r) = source(l);

}return r;

}


copy_k_indexed (from Chapter 6)

template <typename I, typename O, int k>requires(Readable(I) && IndexedIterator(I) &&

Writable(O) && IndexedIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))

void basic_copy_k_indexed(I f, O r){

basic_copy_k<k - 1>()(f, r);sink(r + (k - 1)) = source(f + (k - 1));

}

template <typename I, typename O>void basic_copy_k_indexed<0>(I, O) { }

template <typename I, typename O, int k>requires(Readable(I) && IndexedIterator(I) &&Writable(O) && IndexedIterator(O) && ValueType(I) == ValueType(O))

void copy_k_indexed(I& f, O& r){

basic_copy_k<k>(f, r);f += k; r += k;

}


copy_n_unrolled_indexed (from Chapter 6)



pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O r){


copy_k_indexed<k>(f, r);n = n - N(k);


}


copy_parallel for disjoint ranges (from Chapter 6)

The main difference from copy is the aliasing precondition: theinput and output ranges do not aliasA special forall executes its body for all values of its iterationvariable in arbitrary order, possibly concurrently



void copy_parallel(I f, DistanceType(I) n, O r){

typedef DistanceType(I) N;forall(N i, N(0), n) {

sink(r + i) = source(f + i);}

}


copy_backward_n (from Chapter 6)

copy_backward is not realizable for indexed iteratorscopy_backward_n, which is often just as useful, is realizable



O copy_backward_n(I f, DistanceType(I) n, O r){


n = n - N(1);sink(r + n) = source(f + n);

}return r;

}


copy_from_segmented (from Chapter 6)

template <typename I, typename O>requires(Readable(I) && SegmentedIterator(I) &&

Writable(O) && Iterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))

O copy_from_segmented(I f, I l, O r){

while (f + end(f) != l + end(l)) {// f and l are in different segmentsr = copy(begin(f), end(f), r);f = f + end(f);

}// f and l are in the same segmentreturn copy(begin(f), begin(l), r);

}


Underlying type (from Chapter 7)

DefinitionFor any type T , the underlying type U is an affiliated type satisfying:

T and U are compatibleReferences with value types T and Umay be reinterpretively castinto each otherConstruction of type U and assignment to type U never throw anexceptionAn object of type Umay only be used to hold temporary valueswhile implementing a rearrangement of a range of T objectsA reference to an object of type Umay be reinterpretively cast to areference to T and passed as a const T& parameter

We denote the underlying type of T as UnderlyingType(T)


Motivation for underlying type (from Chapter 7)

The implementation of types T and U could exploit this restrictionto save timeAs we shall see when implementing containers, underlying typeallows multiple headers to point to the same data whileperforming rearrangements


UnderlyingType type function (from Chapter 7)

template <typename T> requires(Regular(T))struct underlying_type{

typedef T type; // default};

#define UnderlyingType(T) typename underlying_type<T>::type

template <typename I> requires(Readable(I) && Iterator(I))const UnderlyingType(ValueType(I))&underlying_source(I x){

return reinterpret_cast<const UnderlyingType(ValueType(I))&>(source(x));

}

template <typename I> requires(Writable(I) && Iterator(I))UnderlyingType(ValueType(I))&underlying_sink(I x){

return reinterpret_cast<UnderlyingType(ValueType(I))&>(sink(x));

}


cycle_2 (from Chapter 7)


void cycle_2(I x, I y){

UnderlyingType(ValueType(I)) t = underlying_source(x);underlying_sink(x) = underlying_source(y);underlying_sink(y) = t;

}


Underlying type and invariants (from Chapter 7)

To be correct, it is sufficient for a sequence of code usingUnderlyingType(T) to:

Be within a critical section: that is, avoid concurrent accessAvoid any exceptions being thrownRestore the class invariants of T by the end of the sequence

ReflectionDisciplined violation of invariants to enhance performance is perfectlylegitimate


cycle_left_3 (from Chapter 7)


void cycle_left_3(I x, I y, I z){

UnderlyingType(ValueType(I)) t = underlying_source(x);underlying_sink(x) = underlying_source(y);underlying_sink(y) = underlying_source(z);underlying_sink(z) = t;

}


swap (from Chapter 7)

We can interchange the contents of two variables usingunderlying type

#define UnderlyingRef(T) reinterpret_cast<UnderlyingType(T)&>


void swap(T& x, T& y){

UnderlyingType(T) tmp;tmp = UnderlyingRef(T)(x);UnderlyingRef(T)(x) = UnderlyingRef(T)(y);UnderlyingRef(T)(y) = tmp;

}


do_cycle_from (from Chapter 7)


Transformation(P) && Domain(P) == I)void do_cycle_from(I i, P p){

// Precondition: p is a from-permutation of the range containing iUnderlyingType(ValueType(I)) t = underlying_source(i);I f = i;I n = p(i);while (n != i) {

underlying_sink(f) = underlying_source(n);f = n;n = p(n);

}underlying_sink(f) = t;

}


swap_ranges_n (from Chapter 7)

template <typename I1, typename I2, typename N>requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&PartiallyCompatible(ValueType(I1), ValueType(I2)) &&Integer(N))

pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n){

while (n != N(0)) {cycle_2(f1, f2);++f1;++f2;n = n - N(1);


}


reverse_n_with_buffer (from Chapter 7)


Mutable(B) && BidirectionalIterator(B) &&UnderlyingType(ValueType(I)) == ValueType(B))

I reverse_n_with_buffer(I f, DistanceType(I) n, B b){

return reverse_copy_n(copy_n(f, n, b), n, f);}


reverse_n_adaptive (from Chapter 7)


Mutable(B) && BidirectionalIterator(B) &&UnderlyingType(ValueType(I)) == ValueType(B))

I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b){


return f + n;if (n <= n_b)

return reverse_n_with_buffer(f, n, b);I m = reverse_n_adaptive(f, h, b, n_b);m += r;I l = reverse_n_adaptive(m, h, b, n_b);swap_ranges_n(f, m, h);return l;

}


do_fused_cycles_from_with_buffer (from Chapter8)

template <typename I, typename P, typename B>requires(Mutable(I) && ForwardIterator(I) &&

Transformation(P) && Domain(P) == I &&Mutable(B) && ForwardIterator(B) &&UnderlyingType(ValueType(I)) == ValueType(B))

void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n){

// Precondition: p is a from-permutation on the range containing icopy_n(i, b, n);I f = i;I next = p(i);while (next != i) {

copy_n(next, f, n);f = next;next = p(next);

}copy_n(b, f, n);

}


rotate_indexed_helper_fused (from Chapter 8)


Transformation(P) && Domain(P) == I)I rotate_indexed_helper_fused(I f, I m, I l, P p){

// Precondition: p is a from-permutation on [f, l)typedef DistanceType(I) N;const N fusion_factor(16);UnderlyingType(ValueType(I)) buffer[fusion_factor];N d = gcd(m - f, l - m);N i(0);while (i + fusion_factor < d) {

do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);i = i + fusion_factor;

}do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);return f + (l - m);

};


swap_ranges (from Chapter 8)

template <typename I1, typename I2>requires(Mutable(I1) && Iterator(I1) &&

Mutable(I2) && Iterator(I2) &&PartiallyCompatible(ValueType(I1), ValueType(I2)))

pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2){

while (f1 != l1 && f2 != l2) {cycle_2(f1, f2);++f1;++f2;


}

Form ∈ [f, l) such that l−m = m− f, swap_ranges(f,m,m, l)rotates [f, l) aboutm


partition_copy_n (from Chapter 9)


Writable(O0) && Iterator(O0) &&Writable(O1) && Iterator(O1) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&PartiallyCompatible(ValueType(I), ValueType(O0)) &&PartiallyCompatible(ValueType(I), ValueType(O1)))

pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 r0, O1 r1, P p){


if (p(source(f))) {sink(r1) = source(f); ++r1;

} else {sink(r0) = source(f); ++r0;

}++f;n = n - N(1);

}return pair<O0, O1>(r0, r1);

}


single_cycle_partition (from Chapter 9)


UnaryPredicate(P) && Domain(P) == ValueType(I))I single_cycle_partition(I f, I l, P p){

f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;UnderlyingType(ValueType(I)) tmp = source(f);while (true) {

--l;sink(f) = source(l);f = find_if(successor(f), l, p);if (f == l) {

sink(l) = tmp;return f;

}sink(l) = source(f);l = assured_find_backward_if_not(l, p);

}}


underlying_forward_iterator (from Chapter 9)

template <typename I>requires(ForwardIterator(I))

struct underlying_forward_iterator{

typedef UnderlyingType(ValueType(I)) U;typedef DistanceType(I) N;typedef underlying_forward_iterator UFI;I i;underlying_forward_iterator() {}underlying_forward_iterator(const I& x) : i(x) {}operator I() { return i; }void operator++() { ++i; }UFI operator+(N n) { return i + n; }friend N operator-(UFI x, UFI y) { return x.i - y.i; }friend bool operator==(const UFI& x, const UFI& y) { return x.i == y.i; }friend const U& source(const UFI& x) { return underlying_source(x.i); }friend U& sink(UFI& x) { return underlying_sink(x.i); }

};#define UFI(I) underlying_forward_iterator


underlying_bidirectional_iterator (from Chapter 9)


struct underlying_bidirectional_iterator{

typedef UnderlyingType(ValueType(I)) U;typedef DistanceType(I) N;typedef underlying_bidirectional_iterator UBI;I i;underlying_bidirectional_iterator() {}underlying_bidirectional_iterator(const I& x) : i(x) {}operator I() { return i; }void operator++() { ++i; }void operator--() { --i; }UBI operator+(N n) { return i + n; }friend N operator-(UBI x, UBI y) { return x.i - y.i; }friend bool operator==(const UBI& x, const UBI& y) { return x.i == y.i; }friend const U& source(const UBI& x) { return underlying_source(x.i); }friend U& sink(UBI& x) { return underlying_sink(x.i); }

};#define UBI(I) underlying_bidirectional_iterator


Type functions for underlying_forward_iterator(from Chapter 9)


struct value_type<underlying_forward_iterator >{

typedef UnderlyingType(ValueType(I)) type;};


struct distance_type<underlying_forward_iterator >{

typedef DistanceType(I) type;};


struct iterator_category<underlying_forward_iterator >{

typedef forward_iterator_tag category;};


stable_partition_n_with_buffer (from Chapter 9)


Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))

pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p){

pair<UFI(I), B> r = partition_copy_n(UFI(I)(f), n, UFI(I)(f), b, p);return pair<I, I>(I(r.first), I(copy(b, r.m1, r.first)));

}


stable_partition_n_adaptive (from Chapter 9)


Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))

pair<I, I> stable_partition_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)

{typedef DistanceType(I) N;if (n == N(0))


return stable_partition_1(f, p);if (n <= n_b)

return stable_partition_n_with_buffer(f, n, b, p);N h = half_nonnegative(n);pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);pair<I, I> r1 = stable_partition_n_adaptive(r0.m1, n - h, b, n_b, p);return stable_partition_combine(r0, r1);

}


merge_copy_n (from Chapter 9)

template <typename I0, typename I1, typename O, typename R>requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&

Writable(O) && Iterator(O) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&PartiallyCompatible(ValueType(I0), ValueType(O)) &&PartiallyCompatible(ValueType(I1), ValueType(O)))

void merge_copy_n(I0 f0, DistanceType(I0) n0,I1 f1, DistanceType(I1) n1, O o, R r)

{typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;while (true) {

if (n0 == N0(0)) { copy_n(f1, n1, o); return; }if (n1 == N1(0)) { copy_n(f0, n0, o); return; }if (r(source(f1), source(f0))) {

sink(o) = source(f1); ++f1; n1 = n1 - N1(1);} else {

sink(o) = source(f0); ++f0; n0 = n0 - N0(1);}++o;

}}


underlying_compare (from Chapter 9)

template <typename T, typename R>requires(Regular(T) && Relation(R) && Domain(R) == T)

struct underlying_compare{

typedef UnderlyingType(T) U;R r;underlying_compare(R r) : r(r) {}bool operator()(const U& x, const U& y){

return r(reinterpret_cast<const T&>(x), reinterpret_cast<const T&>(y));}

};


merge_n_with_buffer (from Chapter 9)


Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, B b, R r)

{typedef underlying_compare<ValueType(I), R> UR;copy_n(UFI(I)(f0), n0, b);merge_copy_n(b, n0, UFI(I)(f1), n1, UFI(I)(f0), UR(r));

}


merge_n_adaptive (from Chapter 9)



void merge_n_adaptive(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,B b, DistanceType(I) n_b, R r)

{typedef DistanceType(I) N;if (n0 + n1 < N(8)) // ***** MEASURE AND TUNE

return insertion_merge_n(f0, n0, f1, n1, r);if (n0 <= n_b)

return merge_n_with_buffer(f0, n0, f1, n1, b, r);I i, j;if (n0 > n1) {

i = f0 + half_nonnegative(n0);j = lower_bound_n(f1, n1, source(i), r);

} else {j = f1 + half_nonnegative(n1);i = upper_bound_n(f0, n0, source(j), r);

}I m = rotate(i, f1, j);merge_n_adaptive(f0, i - f0, i, m - i, b, n_b, r);merge_n_adaptive(m, j - m, j, n1 - (j - f1), b, n_b, r);

} Stepanov, McJones Elements of Programming March 14, 2008 771 / 880

stable_sort_n_adaptive (from Chapter 9)



I stable_sort_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r)

{typedef DistanceType(I) N;if (n < N(16))

return binary_insertion_sort_n(f, n, r);N h = half_nonnegative(n);I m = stable_sort_n_adaptive(f, h, b, n_b, r);I l = stable_sort_n_adaptive(m, n - h, b, n_b, r);merge_n_adaptive(f, n / 2, m, n - h, b, n_b, r);return l;

}


Type functions for list (from Chapter 11)

template <typename T> requires(Regular(T))struct underlying_type< list<T> >{

typedef list_iterator<T> type; // or IteratorType(list<T>)};

template <typename C>requires(Container(C))

struct iterator_type{

typedef C type;};#define IteratorType(C) typename iterator_type<C>::type

template <typename T> requires(Regular(T))struct iterator_type< list<T> >{

typedef list_iterator<T> type;};


Type functions for array (from Chapter 11)

template <typename T> requires(Regular(T))struct underlying_type< array<T> >{

typedef struct { array_header<T>* p; } type;};

template <typename T> requires(Regular(T))struct iterator_type< array<T> >{

typedef T* type;};


reserve (from Chapter 11)


void reserve(array<T>& a, DistanceType(IteratorType(array<T>)) n){

if (n < size(a) || n == capacity(a)) return;typedef UnderlyingType<T> U;typedef UFI(IteratorType(array<T>)) UI;array tmp(n)copy(UI(begin(a)), UI(end(a)), array_push_iterator); // never throwsswap(tmp, a);

}


sort for array (from Chapter 11)


void sort(array<T>& x, R r){

typedef DistanceType(IteratorType(array<T>)) N;typedef UnderlyingType(T) U;N n = size(x) / N(10);array buffer(n, n, U());stable_sort_n_adaptive(begin(x), size(x), begin(buffer), n, r);

}

#endif // *****


Conclusions

. . .


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 Rotations




13 Iterative algorithms for divide-and-conquerReductionSortingStable partition redux


Contents III

ConclusionsReadingProject


15 C++ machinery

16 Acknowledgments

17 Index


Concept PartialSemigroupOperation

DefinitionPartialSemigroupOperation(Op)⇒

BinaryOperation(Op)

For all op ∈ Op and for all a,b, c,∈ Domain(Op), if

is_defined(op,a,b) ∧ is_defined(op,b, c)

then

is_defined(op,op(a,b), c) ∧ is_defined(op,a,op(b, c)) ∧

op(op(a,b), c) = op(a,op(b, c))


reduce_nonempty

template <typename I, typename Op>requires(Readable(I) && Iterator(I) &&

SemigroupOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce_nonempty(I f, I l, Op op){

// Precondition: f , lDomain(Op) r = source(f);while (true) {

++f;if (f == l) return r;r = op(r, source(f));

}}


reduce


MonoidOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce(I f, I l, Op op, const Domain(Op)& z){

if (f == l) return z;return reduce_nonempty(f, l, op);

}

In some situations the caller may need to supply a value for zother than identity_element(op)

***** Explain that there can be many monoids within Domain(Op)?????


find_not

template <typename I>requires(Readable(I) && Iterator(I))

I find_not(I f, I l, const ValueType(I)& x){

while (f != l && source(f) == x) ++f;return f;

}


reduce_nonzeroes


MonoidOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce_nonzeroes(I f, I l, Op op, const Domain(Op)& z){

f = find_not(f, l, z);if (f == l) return z;Domain(Op) r = source(f);while (true) {

f = find_not(successor(f), l, z);if (f == l) return r;r = op(r, source(f));

}}


add_to_counter

template <typename I, typename Op>requires(Mutable(I) && ForwardIterator(I) &&

BinaryOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) add_to_counter(I f, I l, Op op, Domain(Op) x, const Domain(Op)& z){

if (x == z) return z;while (f != l) {

if (source(f) != z) {x = op(source(f), x);sink(f) = z;

} else {sink(f) = x;return z;

}++f;

}return x;

}

Formally, this procedure requires an Iterator typeThe intent is to call it many times with the same mutable counterThus we specify the requirement as ForwardIteratorStepanov, McJones Elements of Programming March 14, 2008 786 / 880

tranpose_operation

template <typename Op>requires(BinaryOperation(Op))

struct transpose_operation{

Op op;transpose_operation(Op op) : op(op) { }typedef Domain(Op) T;T operator()(const T& x, const T& y){

return op(y, x);}

};

A more general version would allow functions with differentcodomain and domain


reduce_balanced

template <int k, typename I, typename Op>requires(Readable(I) && ForwardIterator(I) &&

MonoidOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce_balanced(I f, I l, Op op, const Domain(Op)& z){

// Precondition: 2k > l− ftypedef array_k<k, Domain(Op)> C;C counter;IteratorType(C) c_f = begin(counter), c_l = c_f;while (f != l) {

Domain(Op) carry = add_to_counter(c_f, c_l, op, source(f), z);if (carry != z) {

sink(c_l) = carry;++c_l;

};++f;

}return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), z);

}


merger_linkable


StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))struct merger_linkable{

typedef IteratorType(S) I;I l;R r;S set_link;merger_linkable(I l, R r, S set_link) : l(l), r(r), set_link(set_link) {}I operator()(I x, I y){

return merge_linkable_nonempty(x, l, y, l, r, set_link).first;}

};


sort_linkable

template <int k, typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))IteratorType(S) sort_linkable(IteratorType(S) f, IteratorType(S) l, R r, S set_link){

// Precondition: 2k > l− ftypedef merger_linkable<S, R> Op;Op op(l, r, set_link);typedef IteratorType(S) I;typedef array_k<k, I> C;C counter;IteratorType(C) c_f = begin(counter), c_l = c_f;while (f != l) {

I old_f = f;++f;set_link(old_f, l);I carry = add_to_counter(c_f, c_l, op, old_f, l);if (carry != l) {

sink(c_l) = carry;++c_l;

}}return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), l);


sort_forward_linkable

template <int k, typename I, typename R>requires(Readable(I) && LinkableForwardIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_forward_linkable(I f, I l, R r){

return sort_linkable<32>(f, l, r, forward_linker());}


sort_bidirectional_linkable

template <int k, typename I, typename R>requires(Readable(I) && LinkableBidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_bidirectional_linkable(I f, I l, R r){

f = sort_linkable<32>(f, l, r, forward_linker());restore_backward_links(f, l, backward_linker());return f;

}


combine_ranges

template <typename I>requires(ForwardIterator(I)) // but not Readable(I)

struct combine_ranges{

typedef pair<I, I> Pair;Pair operator()(const Pair& x, const Pair& y) const{

return Pair(rotate(x.first, x.second, y.first),y.second);

}};


partition_trivial

template <typename I, typename P>requires(ForwardIterator(I) &&UnaryPredicate(P) && Domain(P) == ValueType(I))

struct partition_trivial{

P p;partition_trivial(const P & p) : p(p) {}pair<I, I> operator()(I i) const {

// return stable_partition_1<I, P>(i, p);if (p(source(i)))

return pair<I, I>(i, i);else

return pair<I, I>(i, successor(i));}

};


value_iterator

template <typename I, typename F>requires(Incrementable(I) && Transformation(F) && Domain(F) == I)

struct value_iterator{

I i;F f;value_iterator() {}value_iterator(const I& i, const F& f) : i(i), f(f) {}void operator++() {

++i;}friend Codomain(F) source(const value_iterator& x) {

return x.f(x.i);}friend bool operator==(const value_iterator& x, const value_iterator& y) {

// Precondition: x.f = y.freturn x.i == y.i;

}friend bool operator!=(const value_iterator& x, const value_iterator& y) {

return !(x == y);}

};


stable_partition_iterative

template <typename I, typename P>requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))

I stable_partition_inplace_iterative(I f, I l, P p){

typedef partition_trivial<I, P> Fun;typedef value_iterator<I, Fun> RangeIterator;typedef combine_ranges Op;Fun fun(p);RangeIterator f1(f, fun);RangeIterator l1(l, fun);combine_ranges op;pair<I, I> z(l, l);return reduce_balanced<32, RangeIterator, Op>(f1, l1, op, z).first;

}

***** There is a bug in this code


Conclusions

***** reduce_balanced and Huffman monoid *****


Reading


Project

Project***** Write an iterative version of stable_partition_n andstable_partition_n_adaptive from Chapter 9 usingreduce_balanced

***** Give hint that many z’s per op will be required ?????

***** Compare the performance of the corresponding recursiveand iterative versions


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 Rotations







Contents III

15 C++ machinery

16 Acknowledgments

17 Index


Connectives

DefinitionIf P and Q are propositions, then so are ¬P (read as “not P”), P ∨Q (“Por Q”), P ∧Q (“P and Q”), P ⇒ Q (“P implies Q”), and P ⇔ Q (“P isequivalent to Q”), with these truth tables:

P Q ¬P P ∨Q P ∧Q P ⇒ Q P ⇔ Q

F F T F F T TF T T T F T FT F F T F F FT T F T T T T

For equivalence, we often write P if and only if Q, abbreviatedP iff Q


Existential and universal quantifiers

DefinitionIf P is a proposition and x is a variable, then (∃x)P is a proposition(read as "there exists x such that P")

DefinitionIf P is a proposition and x is a variable, then (∀x)P is a proposition(read as “for all x, P”); (∀x)P ⇔ (¬(∃x)¬P)


Sets and functions

We use the common vocabulary from naive set theory:a ∈ X (“a is an element of X”)X ⊂ Y (“X is a subset of Y”){a0, . . . ,an} (“the finite set with elements a0, . . . , and an”){a ∈ X|P(a)} (“the subset of X for which the predicate P holds”)X ∪ Y (“the union of X and Y”)X ∩ Y (“the intersection of X and Y”)X− Y (“the complement of Y in X”)X× Y (“the direct product of X and Y”)f : X→ Y (“f is a function from X to Y”)

X is the domain of fY is the codomain of f


Contents I

1 Introduction

2 Foundations



5 Orderings




Contents II


9 Rotations







Contents III

15 C++ machineryFoundationsDistanceType type functionSpecial cases of Integer proceduresDefault implementations for additive and multiplicative typesOrderingsIteratorsCategory dispatch, with reverse and reverse_n examplesIncreasing sequencesLinkable rangesBifurcate coordinatesCollocated typesfill_iterator

16 Acknowledgments


Contents IV

17 Index


Syntax for requires and models

#define requires(...)

#define models(...)


ArgumentType, Domain, and Codomain typefunctions

A C++ technique called a trait class can be used to simulate a typefunction:

template <typename T, int i>requires(Procedure(T))

struct argument_type;

#define ArgumentType(T, i) typename argument_type< T, i >::type#define Domain(T) ArgumentType(T, 0)

template <typename T>requires(Procedure(T))

struct codomain_type;

#define Codomain(T) typename codomain_type< T >::type

// The macros work only inside a template definition// because of the use of the keyword typename


DistanceType type function

template <typename T>requires(Countable(T)) /∗ ????? ∗/

struct distance_type{

typedef size_t type;};


struct distance_type<T*>{

typedef ptrdiff_t type;};#define DistanceType(T) typename distance_type< T >::type

template <>struct distance_type<short>{

typedef unsigned short type;};


Default implementations of special cases of Integerprocedures

// The successor defined in Chapter 6 for Iterator sufficestemplate <typename I> requires(Integer(I))I predecessor(I a) { --a; return a; }template <typename I> requires(Integer(I))I half_nonnegative(const I& a) { return a >> I(1); }template <typename I> requires(Integer(I))void halve_nonnegative(I& a) { a >>= I(1); }template <typename I> requires(Integer(I))I binary_scale_down_nonnegative(const I& a, const I& k) { return a >> k; }template <typename I> requires(Integer(I))I binary_scale_up_nonnegative(const I& a, const I& k) { return a << k; }template <typename I> requires(Integer(I))bool is_positive(const I& a) { return I(0) < a; }template <typename I> requires(Integer(I))bool is_negative(const I& a) { return a < I(0); }template <typename I> requires(Integer(I))bool is_zero(const I& a) { return a == I(0); }template <typename I> requires(Integer(I))bool is_even(const I& a) { return (a & I(1)) == I(0); }template <typename I> requires(Integer(I))bool is_odd(const I& a) { return (a & I(1)) != I(0); }


Default identity and inverse for additive types

template <typename T>requires(AdditiveSemigroup(T))

struct plus {T operator()(const T& x, const T& y) const { return x + y; }

};

template <typename T>requires(AdditiveMonoid(T))

T identity_element(const plus<T>&) { return T(0); }

template <typename T>requires(AdditiveGroup(T))

struct negate {T operator()(const T& x) const { return -x; }

};

template <typename T>requires(AdditiveGroup(T))

negate<T> inverse_operation(const plus<T>&) {return negate<T>();

}


Define ArgumentType(plusT , 0)

template <typename T>requires(AdditiveSemigroup(T))

struct argument_type<plus<T>, 0>{

typedef T type;};


Default identity and inverse for multiplicative types

template <typename T>requires(MultiplicativeSemigroup(T))

struct multiplies {T operator()(const T& x, const T& y) const { return x * y; }

};

template <typename T>requires(MultiplicativeMonoid(T))

T identity_element(const multiplies<T>&) { return T(1); }

template <typename T>requires(MultiplicativeGroup(T))

struct reciprocal {T operator()(const T& x) const { return T(1) / x; }

};

template <typename T>requires(MultiplicativeGroup(T))

reciprocal<T> inverse_operation(const multiplies<T>&) {return reciprocal<T>();

}


Default ordering for StrictTotallyOrdered


struct less{

bool operator()(const T& x, const T& y) const { return x < y; }};


struct argument_type<less<T>, 0>{

typedef T type;};


Default order selection for StrictTotallyOrdered


T& min(T& a, T& b){

return min(a, b, less<T>());}


Dereferenceable functions for T∗

// We will define this latertemplate <typename T>

requires(Regular(T))struct value_type;


struct value_type<T*> { typedef T type; };

#define ValueType(T) typename value_type< T >::type


const T& source(T* x) { return *x; }


T& sink(T* x) { return *x; }


More aliasing tests

template <typename T0, typename T1>requires(Writable(T0) && Writable(T1) && ValueType(T0) == ValueType(T1))

bool is_aliased_sinks(const T0& x0, const T1& x1){

// Return true⇔// For allU with Readable(U) and for all y ∈U,// is_aliased(x0, y) == is_aliased(x1, y)// For many types, this works:typedef const ValueType(T0)* P;return P(&sink(x0)) == P(&sink(x1));

}

template <typename T0, typename T1>requires(Readable(T0) && Readable(T1) && ValueType(T0) == ValueType(T1))

bool is_aliased_sources(const T0& x0, const T1& x1){

// Return true⇔// For allU with Writable(U) and for all y ∈U,// is_aliased(y, x0) == is_aliased(y, x1)// For many types, this works:typedef const ValueType(T0)* P;return P(&source(x0)) == P(&source(x1));

}


Default Dereferenceable functions for T


struct value_type{

typedef T type;};


const T& source(const T& x){

return x;}


T& sink(T& x){

return x;}


Mechanics of category dispatch

A tag type (containing no data) is defined for each categoryA type function is defined to obtain a type tag from a typeOverloading is used to select from multiple signatures differingonly by a tag type


Iterator tag types

It FI

BI

II

RI

struct iterator_tag {};struct forward_iterator_tag {};struct bidirectional_iterator_tag {};struct indexed_iterator_tag {};struct random_access_iterator_tag {};


IteratorCategory type function

template <typename T>requires(Iterator(T))

struct iterator_category{


#define IteratorCategory(T) typename iterator_category< T >::category


struct iterator_category<T*>{

typedef random_access_iterator_tag category;};


Category dispatch for reverse_n and reverse


void reverse_n(I f, DistanceType(I) n){

reverse_n(f, n, IteratorCategory(I)());}


void reverse(I f, I l){

reverse(f, l, IteratorCategory(I)());}


Category dispatch cases for reverse_n

template <typename I> requires(Mutable(I) && ForwardIterator(I))void reverse_n(I f, DistanceType(I) n, forward_iterator_tag){

reverse_n_forward(f, n);}

template <typename I> requires(Mutable(I) && BidirectionalIterator(I))void reverse_n(I f, DistanceType(I) n, bidirectional_iterator_tag){

reverse_n_bidirectional(f, n);}

template <typename I> requires(Mutable(I) && IndexedIterator(I))void reverse_n(I f, DistanceType(I) n, indexed_iterator_tag){

reverse_n_indexed(f, n);}

template <typename I> requires(Mutable(I) && RandomAccessIterator(I))void reverse_n(I f, DistanceType(I) n, random_access_iterator_tag){

reverse_n_indexed(f, n);}


Category dispatch cases for reverse

template <typename I> requires(Mutable(I) && ForwardIterator(I))void reverse(I f, I l, forward_iterator_tag){

reverse_forward(f, l);}

template <typename I> requires(Mutable(I) && BidirectionalIterator(I))void reverse(I f, I l, bidirectional_iterator_tag){

reverse_bidirectional(f, l);}

template <typename I> requires(Mutable(I) && IndexedIterator(I))void reverse(I f, I l, indexed_iterator_tag){

reverse_indexed(f, l);}

template <typename I> requires(Mutable(I) && RandomAccessIterator(I))void reverse(I f, I l, random_access_iterator_tag){

reverse_indexed(f, l);}


find_out_of_order


StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, forward_iterator_tag){

if (f == l) return l;I n(successor(f));while (n != l) {

if (r(source(n), source(f)))return n;

f = n;++n;

}return l;

}


find_out_of_order

template <typename I, typename R>requires(Readable(I) && Iterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, iterator_tag){

if (f == l) return l;I n(successor(f));ValueType(I) v = source(f);while (n != l) {

if (r(source(n), v))return n;

v = source(n);++n;

}return l;

}


Category dispatch for find_out_of_order

template <typename I, typename R>requires(Readable(I) && RandomAccessIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, random_access_iterator_tag){

return find_out_of_order(f, l, r, forward_iterator_tag());}

template <typename I, typename R>requires(Readable(I) && BidirectionalIterator(I) &&

StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, bidirectional_iterator_tag){

return find_out_of_order(f, l, r, forward_iterator_tag());}


StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r){

return find_out_of_order(f, l, r, IteratorCategory(I)());}


is_increasing


StrictWeakOrdering(R) && Domain(R) == ValueType(I))bool is_increasing(I f, I l, R r){

return l == find_out_of_order(f, l, r);}


partition_copy_n


Writable(O0) && Iterator(O0) &&Writable(O1) && Iterator(O1) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(O0) &&ValueType(I) == ValueType(O1))

pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 o0, O1 o1, P p){


if (p(source(f))) {sink(o1) = source(f); ++o1;

} else {sink(o0) = source(f); ++o0;

}++f;n = n - N(1);

}return pair<O0, O1>(o0, o1);

}


unstable_indexed_partition


UnaryPredicate(P) && Domain(P) == ValueType(I))I unstable_indexed_partition(I f, I l, P p){

DistanceType(I) i = 0;DistanceType(I) j = l - f;while (true) {

while (true) {if (i == j) return f + i;if (p(source(f + i))) break;i = i + 1;

}while (true) {

j = j - 1;if (i == j) return f + j + 1;if (!p(source(f + j))) break;

}cycle_2(f + i, f + j);i = i + 1;

}}


lower_bound_n


struct greater_than_or_equal_to_a{

const T& a;R r;greater_than_or_equal_to_a(const T& a, R r) : a(a), r(r) { }bool operator()(const T& x) const { return !r(x, a); }

};


StrictWeakOrdering(R) && Domain(R) == ValueType(I))I lower_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r){

greater_than_or_equal_to_a<ValueType(I), R> predicate(a, r);return partition_point_n(f, n, predicate);

}


upper_bound_n


struct greater_than_a{

const T& a;R r;greater_than_a(const T& a, R r) : a(a), r(r) { }bool operator()(const T& x) const { return r(a, x); }

};


StrictWeakOrdering(R) && Domain(R) == ValueType(I))I upper_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r){

greater_than_a<ValueType(I), R> predicate(a, r);return partition_point_n(f, n, predicate);

}


Category dispatch formerge_n_with_buffer

template <typename I, typename B, typename R>requires(Mutable(I) && RandomAccessIterator(I) &&

Mutable(B) && RandomAccessIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))

void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r,random_access_iterator_tag)

{merge_n_with_buffer(f0, n0, f1, n1, b, r, bidirectional_iterator_tag());

}



void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r)

{merge_n_with_buffer(f0, n0, f1, n1, b, r, IteratorCategory(I)());

}


IteratorType type function

template <typename T>requires(ImplementsIteratorType(T))

struct iterator_type;

#define IteratorType(T) typename iterator_type< T >::type


IteratorType for linkers

template <typename I> requires(LinkableForwardIterator(I))struct forward_linker;

template <typename I> requires(LinkableForwardIterator(I))struct iterator_type< forward_linker > { typedef I type; };

template <typename I> requires(LinkableBidirectionalIterator(I))struct backward_linker;

template <typename I> requires(LinkableBidirectionalIterator(I))struct iterator_type< backward_linker > { typedef I type; };

template <typename I> requires(LinkableBidirectionalIterator(I))struct bidirectional_linker;

template <typename I> requires(LinkableBidirectionalIterator(I))struct iterator_type< bidirectional_linker > { typedef I type; };


Domain for transpose_operation


struct transpose_operation;


struct argument_type< transpose_operation<Op>, 0 >{

typedef Domain(Op) type;};


Domain formerger_linkable


StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))struct merger_linkable;


StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))struct argument_type<merger_linkable<S, R>, 0>{

typedef IteratorType(S) type;};


Domain for combine_ranges


struct combine_ranges;


struct argument_type<combine_ranges, 0>{

typedef pair<I, I> type;};


Codomain for partition_trivial

template <typename I, typename P>

requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))struct partition_trivial;

template <typename I, typename P>

requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))struct codomain_type< partition_trivial<I, P> >{

typedef pair<I, I> type;};


WeightType type function

template <typename T>requires(WeakForwardBifurcateCoordinate(T))

struct weight_type{

typedef size_t type; // ***** What should the default be ?????};

template <typename T>requires(WeakForwardBifurcateCoordinate(T))

struct weight_type<T*>{

typedef ptrdiff_t type;};#define WeightType(T) typename weight_type< T >::type // ***** ?????


left_successor and right_successor forWeakForwardBifurcateCoordinate

template <typename C>requires(WeakForwardBifurcateCoordinate(C))

C left_successor(C f){

move_left(f);return f;

}

template <typename C>requires(WeakForwardBifurcateCoordinate(C))

C right_successor(C f){

move_right(f);return f;

}


pair

template <typename T0, typename T1>requires(Regular(T0) && Regular(T1))

struct pair{

T0 m0;T1 m1;pair() {}pair(T0 m0, T1 m1) : m0(m0), m1(m1) { }pair(const pair& x) : m0(x.m0), m1(x.m1) { }void operator=(const pair& x) { m0 = x.m0; m1 = x.m1; }friend void bool operator==(const pair& x, const pair& y){ return x.m0 == y.m0 && x.m1 == y.m1; }friend void bool operator<(const pair& x, const pair& y){ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1); }

};


triple

template <typename T0, typename T1, typename T2>requires(Regular(T0) && Regular(T1) && Regular(T2))

struct triple{

T0 m0;T1 m1;T2 m2;triple() {}triple(T0 m0, T1 m1, T2 m2) :

m0(m0), m1(m1), m2(m2) {}triple(const triple& x) : m0(x.m0), m1(x.m1), m2(x.m2) { }void operator=(const triple& x){ m0 = x.m0; m1 = x.m1; m2 = x.m2}friend void bool operator==(const triple& x, const triple& y){ return x.m0 == y.m0 && x.m1 == y.m1 && x.m2 == y.m2; }friend void bool operator<(const triple& x, const triple<T0, T1, T2>& y){ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1 ||

(!y.m1 < x.m1 && x.m2 == y.m2)); }};


lexicographic_equalityk

template <int k, typename I0, typename I1>requires(Readable(I0) && ForwardIterator(I0) &&

Readable(I1) && ForwardIterator(I1) &&ValueType(I0) == ValueType(I1))

struct lexicographic_equality_k{

bool operator()(I0 f0, I1 f1){

if (source(f0) != source(f1)) return false;return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);

}};

template <typename I0, typename I1>struct lexicographic_equality_k<0, I0, I1>{

bool operator()(I0, I1){

return true;}

};


lexicographic_orderingk

template <int k, typename I0, typename I1>requires(Readable(I0) && ForwardIterator(I0) &&

Readable(I1) && ForwardIterator(I1) &&ValueType(I0) == ValueType(I1))

struct lexicographic_ordering_k{

bool operator()(I0 f0, I1 f1){

if (source(f0) < source(f1)) return true;if (source(f0) > source(f1)) return false;return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);

}};

template <typename I0, typename I1>struct lexicographic_ordering_k<0, I0, I1>{

bool operator()(I0, I1){

return false;}

};


arrayk

***** Need const and non-const versions of begin and end ?????***** What about subscript: a[i] ?????template <int k, typename T>

requires(Regular(T))struct array_k{

typedef T* I; // IteratorType(arrayk,T )T a[k];array_k() { }array_k(const array_k& x) { basic_copy_k_indexed<k, I, O>(&x.a, &a); }friend void operator=(const array_k& x){ basic_copy_k_indexed<k, I, O>(&x.a, &a); }friend size_t size(const array_k& x) { return size_t(k); }friend I begin(array_k& x) { return &x.a[0]; }friend I end(array_k& x) { return &x.a[k]; }friend bool operator==(const array_k& x, const array_k& y){ return lexicographic_equality_k<k, I, I>(begin(x), begin(y)); }friend bool operator<(const array_k& x, const array_k& y){ return lexicographic_ordering_k<k, I, I>(begin(x), begin(y)); }

};


Forward declaration for array_k data structure

template <int k, typename T>requires(Regular(T))

struct array_k;

array_k is defined in Chapter 11


IteratorType for array_k

template <int k, typename T>requires(Regular(T))

struct iterator_type< array_k<k, T> >{

typedef T* type;};


fill_iterator

template <typename T, typename N>requires(Regular(T) && Integer(N))

struct fill_iterator{

const T* v;N n;fill_iterator() {}fill_iterator(const T& v, N n) : v(&v), n(n) {}friend void operator++(fill_iterator& x) { ++x.n; }friend const T& source(fill_iterator& x) { return source(x.v); }friend void operator+=(fill_iterator& x, N n) { x.n += n; }friend N operator-(fill_iterator x, fill_iterator y) { return x.n - y.n; }friend bool operator==(fill_iterator x, fill_iterator y) {

return x.n == y.n;}

};


Type functions for fill_iterator


struct value_type< fill_iterator<T, N> >{

typedef T type;};


struct distance_type< fill_iterator<T, N> >{

typedef N type;};


struct iterator_category< fill_iterator<T, N> >{

typedef indexed_iterator_tag category;};


Acknowledgments

We are grateful for comments and suggestions fromAndrei Alexandrescu,Matt Austern, Dave Abrahams, Jon Brandt,Boris Fomitchev, Kevlin Henney, Jussi Ketonen, Karl Malbrain,Larry Masinter, David Musser, Dave Parent, Sean Parent,Dmitry Polukhin, Jon Reid, Mark Ruzon, Geoff Scott,David Simons, Tony Van Eerd, Walter Vannini, John Wilkinson,and Oleg Zabluda


Index I

++definition space on range, 361

6standard definition, 242

>standard definition, 242

>standard definition, 242

absolute value, 272absorption

formin,max, 251abstraction, 19action

duality with transformation, 343Action concept, 342activity on entities, 19acyclic

descendants of bifurcate coordinate, 633AdditiveGroup concept, 200AdditiveMonoid concept, 198AdditiveSemigroup concept, 196algorithm

memory-adaptive, 426with buffer, 426

algorithmically-induced structure, 276aliasing, 338, 385amortized complexity, 347AmortizedConstantTime concept, 347


Index II

and (∧), 803Archimedean monoid

discrete, 287ArchimedeanMonoid concept, 276ArchimedeanOrderedField concept, 519area

constant, 82fixed, 82of an object, 82variable, 82

area function, 82argument, 40ArgumentType type function, 55Arity type attribute, 54assignment, 49

for Regular, 65associativity, 130

exploited by power, 145ofmin,max, 251of permutation composition, 412

asymmetric relation, 218attribute of entity, 19average cost, 84Axiom of Archimedes, 276, 277

backward movement in range, 533BackwardLinker concept, 601Banning, John, 358bidirectional binary trees, advantages, 641


Index III

BidirectionalBifurcateCoordinate concept, 629BidirectionalIterator concept, 374BidirectionalLinker concept, 602BifurcateCoordinate concept, 628bignum, 139bin-based rearrangement, 422binary gcd, 327binary search, 553BinaryOperation concept, 129bisection technique, 518Bolzano, Bernard, 518bounded integer type, 322buffer

with buffer algorithm, 426

CancellableOperation concept, 195category dispatch

mechanism, 822Cauchy, Augustin Louis, 518closed bounded range, 357closed counted range, 357closed interval, 243clusters

of derived procedures, 238code transformations

enabled by regular types, 81, 181codomain of function, 805codomain of procedure, 42Codomain type function, 55


Index IV

CommutativeOperation concept, 194CommutativeSemiringconcept, 298commutativity

ofmin,max, 251Compatible concept, 732complement (−), 805complement of converse of relation, 239complement of relation, 239complexity

algorithm selection, 175amortized, 347counting operations, 176in specifications, 175of generalized algorithm, 175slow_power versus power, 177, 178to determine feasibility, 175

compositionof permutations, 412

concept, 56ArchimedeanMonoid, 276Action, 342AdditiveGroup, 200AdditiveMonoid, 198AdditiveSemigroup, 196AmortizedConstantTime, 347ArchimedeanOrderedField, 519BackwardLinker, 601BidirectionalBifurcateCoordinate, 629BidirectionalIterator, 374


Index V

BidirectionalLinker, 602BifurcateCoordinate, 628BinaryOperation, 129CancellableOperation, 195CommutativeOperation, 194CommutativeSemiring, 298Compatible, 732Dereferenceable, 340DiscreteArchimedeanMonoid, 287DiscreteArchimedeanRing, 319DiscreteArchimedeanSemiring, 317EquivalenceRelation, 219EuclideanMonoid, 293EuclideanSemimodule, 304EuclideanSemiring, 306examples from C++, 59ForwardIterator, 371ForwardLinker, 600Function, 60GroupOperation, 137HomogeneousFunction, 61IndexedIterator, 380Integer, 139Iterator, 348LinkableBidirectionalBifurcateCoordinate, 644LinkableBidirectionalIterator, 606LinkableBifurcateCoordinate, 643LinkableForwardIterator, 605Merger, 567


Index VI

MonoidOperation, 136MultiplicativeGroup, 201MultiplicativeMonoid, 199MultiplicativeSemigroup, 197Mutable, 339NonnegativeDiscreteArchimedeanSemiring, 318Operation, 88OrderedAdditiveGroup, 271OrderedAdditiveMonoid, 271OrderedAdditiveSemigroup, 271OrderedCancellableMonoid, 273Ordering, 215PartiallyCompatible, 728PartialSemigroupOperation, 781Predicate, 214Procedure, 60ProperProcedure, 60QRSemimodule, 302RandomAccessIterator, 387Readable, 336refinement, 57Regular, 63RegularAction, 346RegularFunction, 79Relation, 214relational concept, 726SameGenus, 727SegmentedIterator, 399SemigroupOperation, 130


Index VII

Semimodule, 300StrictTotallyOrdered, 237StrictTotalOrdering, 223StrictWeakOrdering, 226Transformation, 89type concept, 58UnaryPredicate, 504univalent, 320WeakBifurcateCoordinate, 626weakening, 57WeaklyHalvableMonoid, 283Writable, 337

constant area, 82constructor, 52

type constructor, 53container, 668converse of relation, 239coordinate structure, 594copy constructor

for Regular, 68copying rearrangement, 425cost

average, 84fixed, 84uniform, 84worst-case, 84

cost function, 84cost of a procedure call, 84cycle in a permutation, 415


Index VIII

cyclic permutation, 417

default constructorfor Regular, 67

default ordering, 236definition space of procedure, 41Dereferenceable concept, 340derived relations, 239descendants

of bifurcate coordinate, 633description, 18destructor, 50

for Regular, 66DifferenceType type function, 387direct product (×), 805discrete Archimedean monoid, 287DiscreteArchimedeanMonoid concept, 287DiscreteArchimedeanRing concept, 319DiscreteArchimedeanSemiring concept, 317discreteness, 317distance type, 110DistanceType type function, 348distributivity

of semiring, 298divisibility lattice of integers, 466divisibility on an Archimedean monoid, 285division

Egyptian, 279division on multiplicative group, 201


Index IX

domain of function, 805Domain type function, 61Dudzinski, Krzysztof, 576Dydek, Andrzej, 576

Egyptian division, 279element (∈), 805empty range, 360entity

activity on, 19attribute, 19part, 19particulars, 20relationship, 19

equalityfor Regular, 64in the real world, 33of objects, 38on containers, 670, 671properties of, 31unique on a type, 234

equality (=), 30equational reasoning, 32

in mathematics, 32EquivalenceRelation concept, 219equivalent (⇔), 803essential object, 39Euclidean domain, 297Euclidean function, 306


Index X

EuclideanMonoid concept, 293EuclideanSemimodule concept, 304EuclideanSemiring concept, 306exclusive-or of links, 646exists (∃), 804

Fibonaccimatrices, 185numbers, 182

Fiduccia, Charles M., 206finite set, 418finite tree

descendents of bifurcate coordinate, 634first in a range, 358fixed area, 82fixed cost, 84fixed point, 411for all (∀), 804ForwardIterator concept, 371ForwardLinker concept, 600from-permutation, 431function

one-to-one, 408onto, 407

function (→), 805Function concept, 60function procedure, 47


Index XI

Gaussian integersdefined, 166example of Euclidean domain, 297Stein’s algorithm, 328

gcdand lcm, 465binary, 327greatest common divisor, 288subtractive, 290

generalized link rearrangement, 610generalized procedure, 71generalized rearrangement, 609genus, 35global state, 40goto

considered not harmful, 655greatest common divisor, 288group

of permutations, 414GroupOperation concept, 137

half-open interval, 243Ho, Wilson, 482, 545Hoare, C.A.R., 543HomogeneousFunction concept, 61

idea, 18idempotency


Index XII

ofmin,max, 251identity

in the real world, 29of objects, 29

identity permutation, 411implies (⇒), 803improper procedure, 46in place, 426in situ, 426increasing order, 248increasing range, 499index permutation, 419indexed iterator

equivalent to random access iterator, 391IndexedIterator concept, 380inequality

standard definition, 241Integer concept, 139interface

for rotate, 469useful variations, 180

interoperability, 62interpretatation of object, 25intersection (∩), 805interval, 243invariant, 655invariants

breaking, 622inverse


Index XIII

of permutation, 413, 417is_defined predicate, 43isomorphic types, 320iterator

linkable, 598Iterator concept, 348

k-partition, 504key function, 499, 569key-increasing range, 499key-sorting, 569

Lakshman, T.K., 523last in a range, 358lattice, 466lcm

and gcd, 465least common multiple, 464

least common multiple, 464Leibniz’s Law, 30lexicographic ordering, 225, 229limit in a range, 358linear ordering

importance of, 213linear recurrences, 206link rearrangement, 603linkable iterator, 598linkable range, 598


Index XIV

linkable structuresforward versus bidirectional, 625

LinkableBidirectionalBifurcateCoordinate concept, 644LinkableBidirectionalIterator concept, 606LinkableBifurcateCoordinate concept, 643LinkableForwardIterator concept, 605linking operation, 599links

exclusive-or of, 646reversing, 645

Lo, Raymond, 482, 545local state, 40locality of reference, 476, 618located sequence, 356Lomuto, Nico, 526loop fusion, 476lower bound, 506

implementation, 554

Mauchly, John W., 586memory, 335memory-adaptive algorithm, 426merge, 555

algebraic properties, 559complexity property, 560copying, 556melding, 556mutating, 556stability, 555


Index XV

mergeable range, 556Merger concept, 567models

models (|=), 56type models concept, 56

MonoidOperation concept, 136MultiplicativeGroup concept, 201MultiplicativeMonoid concept, 199MultiplicativeSemigroup concept, 197Mutable concept, 339mutable range, 362mutative rearrangement, 425mutator procedure, 47

natural total ordering, 235< reserved for, 237

Noether, Emmy, 307non-terminating procedure, 44nonessential object, 39NonnegativeDiscreteArchimedeanSemiring concept, 318not (¬), 803not last in a range, 358

object, 24, 25interpretation, 25state, 25

one-to-one function, 408onto function, 407


Index XVI

open interval, 243Operation concept, 88optimization

enabled by regular types, 81loop fusion, 476using segmented iterator, 398

or (∨), 803orbit, 92OrderedAdditiveGroup concept, 271OrderedAdditiveMonoid concept, 271OrderedAdditiveSemigroup concept, 271OrderedCancellableMonoid concept, 273ordering

reflexive, 217strict, 217weakly reflexive, 217

Ordering concept, 215ordering-based rearrangement, 422out-of-order execution, 385overlapping ranges, 377overloading, 190, 599, 642own state, 40

partialprocedure, 43representation, 27

partial orderingexample, 231topological sort defined on, 233


Index XVII

partially-formed, 51PartiallyCompatible concept, 728PartialSemigroupOperation concept, 781particulars of entity, 20partition, 504

k-partition, 504partition algorithm, 508partition algorithms, origin of, 543partition point, 505

finding, 515lower bound, 506upper bound, 506

partition rearrangementsemistable, 525

partitioned prefix of a range, 509parts of an entity, 19permutation, 409

composition, 412cycle, 415cyclic, 417from, 431identity, 411index, 419inverse, 413, 417product of its cycles, 417reverse, 436rotation, 463to, 431

permutation group, 414


Index XVIII

pointersexclusive-or of, 646

polynomialsexample of Euclidean domain, 297

position-based rearrangement, 422powers

non-positive exponent, 138of same element commute, 132positive exponent, 131

Predicate concept, 214procedure, 40

function, 47generalized, 71improper, 46mutator, 47non-terminating, 44partial, 43proper, 46semi-terminating, 44terminating, 44total, 43

Procedure concept, 60proper procedure, 46ProperProcedure concept, 60

QRSemimodule concept, 302quicksort, 528, 543


Index XIX

random access iteratorequivalent to indexed iterator, 391

RandomAccessIterator concept, 387range, 356

backward movement, 533closed bounded, 357closed counted, 357empty, 360first, 358increasing, 499k-partition, 504key-increasing, 499last, 358limit, 358linkable, 598lower bound, 506mergeable, 556mutable, 362not last, 358overlapping, 377partition, 504partition point, 505partitioned prefix, 509readable, 362semi-open bounded, 357semi-open counted, 357size, 359uniform random shuffle, 545upper bound, 506


Index XX

writable, 362reachability of bifurcate coordinates, 632reachable, 90Readable concept, 336readable range, 362rearrangement, 420

bin-based, 422copying, 425generalized, 609generalized link, 610key-sorting, 569link, 603mutative, 425ordering-based, 422partition, 508position-based, 422reverse, 437rotation, 469sorting, 569time complexity, 427, 434

refinement of concept, 57reflexive ordering, 217reflexivity of equality, 31regular

allows code transformations, 81, 181required for predicate with assured find, 544

Regular concept, 63RegularAction concept, 346RegularFunction concept, 79


Index XXI

relationcomplement, 239complement of converse, 239converse, 239symmetric, 218symmetric complement of, 222

Relation concept, 214relational concept, 726relations

derived, 239relationship between entities, 19representation, 27

partial, 27total, 27unique, 28

requires clause, 71, 72result space of procedure, 42returning useful information, 365, 367, 394, 439, 469, 482, 523, 533, 537, 553, 563, 735, 737reverse

permutation, 436rearrangement, 437

reversing links, 645rotation

permutation, 463rearrangement, 469

Russian peasant algorithm, 144

same genus, 35SameGenus concept, 727


Index XXII

Scalar type function, 300segmented iterator

intuition, 398SegmentedIterator concept, 399SegmentIterator type function, 399semi

usage convention, 232semi-open bounded range, 357semi-open counted range, 357semi-terminating procedure, 44SemigroupOperation concept, 130Semimodule concept, 300semistable partition rearrangement, 525sequence

located, 356sequence of iterators, 355sets, 805sink, 337size

of range, 359sort (type) of value, 21sorting

stability, 569sorting algorithm, 569space complexity

forward iterator reverse conjecture, 455in place, 426memory adaptive, 426with buffer, 426


Index XXIII

stability, 247ofmin,max, 250of rearrangement, 423of sorting algorithm, 569

stable adaptive merge and sort, importance of, 585stable in-place merge and sort, importance of, 585stable merge, 555state of object, 25Stein, Josef, 327storage efficiency, 83strict lower bound, 434strict ordering, 217strictly increasing order, 248StrictTotallyOrdered concept, 237StrictTotalOrdering concept, 223StrictWeakOrdering concept, 226subset (⊂), 805subtraction on additive group, 200subtractive gcd, 290symmetric complement of a relation, 222symmetric relation, 218symmetry of equality, 31

terminating procedure, 44thing, 18Tighe, Joseph, 469time complexity

of rearrangement, 427, 434to-permutation, 431


Index XXIV

totalprocedure, 43representation, 27

trait class, 811transformation

duality with action, 343fixed point of, 411self-composable, 90

Transformation concept, 89transitivity, 215transitivity of equality, 31traversal

of tree, recursive, 637tree

descendants of bifurcate coordinate, 633trichotomy law

for total ordering, 224for weak ordering, 228

trivial cycle, 415type, 36

constant area, 82fixed area, 82intuition, 37isomorphism, 320variable area, 82

type attribute, 54Arity, 54

type concept, 58type constructor, 53


Index XXV

type expression, 72type function, 55

ArgumentType, 55Codomain, 55DifferenceType, 387DistanceType, 348Domain, 61example, 109Scalar, 300SegmentIterator, 399simulated in C++ via trait class, 811UBI, 764UFI, 763UnderlyingType, 747ValueType, 336, 337WeightType, 626

UBI type function, 764UFI type function, 763UnaryPredicate concept, 504underlying type, 429, 747UnderlyingType type function, 747uniform cost, 84uniform random shuffle, 545union (∪), 805unique representation, 28univalent concept, 320upper bound, 506

implementation, 554


Index XXVI

value, 19sort (type), 21

value type, 336ValueType type function, 336, 337van der Waerden, Bartel Leendert, 307variable area, 82

weakusage convention, 232

WeakBifurcateCoordinate concept, 626weakening of concept, 57weakly reflexive ordering, 217WeaklyHalvableMonoid concept, 283WeightType type function, 626well-formed object state, 26with buffer algorithm, 426worst-case cost, 84Writable concept, 337writable range, 362

Zabluda, Oleg, 205

Documents

Lecture All