Lattice path enumeration and factorizationpeople.brandeis.edu/~gessel/homepage/students/lemusthesis.pdf · I would like to thank my advisor, Ira Martin Gessel, for the constant support,

Lattice path enumeration and factorization

A Dissertation

Presented to

The Faculty of the Graduate School of Arts and Sciences

Brandeis University

Department of Mathematics

Ira Martin Gessel, Advisor

In Partial Fulfillment

of the Requirements for the Degree

Doctor of Philosophy

by

Cristobal Lemus Vidales

July, 2017

This dissertation, directed and approved by Cristobal Lemus Vidales’s committee, has been

accepted and approved by the Faculty of Brandeis University in partial fulfillment of the

requirements for the degree of:

DOCTOR OF PHILOSOPHY

Eric Chasalow, Dean of Arts and Sciences

Dissertation Committee:

Ira Martin Gessel, Dept. of Mathematics, Chair

Olivier Bernardi, Dept. of Mathematics

James Gary Propp, Dept. of Mathematics, University of Massachusetts Lowell

c© Copyright by

Cristobal Lemus Vidales

2017

Dedication

A mis padres.

Ad astra per aspera.

iv

Acknowledgments

I would like to thank my advisor, Ira Martin Gessel, for the constant support, patience,

guidance, and insight. His generosity sharing with me many great ideas that helped me

develop this document are invaluable. I am very grateful to Olivier Bernardi and James

Propp for serving on my dissertation committee. I would also like to thank Susan Field

Parker for being an excellent teaching mentor and a friend.

The Brandeis Mathematics Department has always being a welcoming community to

me. I owe thanks to the faculty, to my fellow students and friends, and to the staff. In

particular, I wish to thank Catherine Broderick for diligently helping me to comply with all

the graduation deadlines and requirements in a timely fashion.

Before arriving to Brandeis, I received advice and support from many Mathematics and

Physics professors. I would like to thank in particular: Vera Serganova, Arthur Ogus, Hugo

Alarcon, Carlos Hinojosa, and Julio Cesar Gutierrez-Vega.

v

Abstract

Lattice path enumeration and factorization

A dissertation presented to the Faculty of theGraduate School of Arts and Sciences of Brandeis

University, Waltham, Massachusetts

by Cristobal Lemus Vidales

In this thesis we develop several examples of lattice path enumeration. The first chapter

introduces all the basic concepts we will use during the subsequent chapters. Chapter Two

deals with multi-colored Motzkin paths, gives an interpretation to several identities between

Catalan and Motzkin number generating functions and sets up the framework for a gener-

alized Touchard’s identity in the following chapter. In Chapter Three we count Dyck paths

by occurrences of three types of strings simultaneously, which together also count paths

by semilength and number of peaks. These strings are short peaks, triple rises, and rising

hooks. Special cases of this enumeration give several well-known families of numbers such as

Catalan, Motzkin, Narayana, Riordan, and Schroder numbers.

In Chapter Four we discuss paths that have generating functions which are in some way

related to the Delannoy number generating function. We obtain some interesting polynomials

and some relationships among the generating functions. In Chapter Five we apply the

factorization method developed by Ira Gessel in [Ges80] to the paths from Chapter Four. For

each type of path we obtain three types of restricted subpaths. We give explicit formulas for

the generating functions of the subpaths and compute the coefficients in the most interesting

cases. Many of our cases give back the generating functions for Schroder, small Schroder,

and Narayana numbers.

vi

Contents

List of Figures ix

List of Tables x

Chapter 1. Introduction 1

1.1. Preliminary definitions 1

Chapter 2. Generalized Motzkin numbers and Touchard’s identity 8

Chapter 3. Enumeration of paths via occurrences of distinguished strings 13

3.1. Dyck paths 13

3.2. Ballot paths and left incomplete Dyck paths 18

3.3. Prime Dyck paths 19

3.4. Several special cases counted by F 23

3.5. Generalized Touchard’s identity 29

Chapter 4. Miscellany of lattice path enumeration 31

4.1. Introduction 31

4.2. Jacobi’s change of variables formula 31

4.3. Delannoy numbers 32

4.4. Ordinary paths 35

4.5. Delannoy paths and Delannoy polynomials 35

4.6. Ordinary paths by left turns 38

4.7. Paths with arbitrarily long horizontal steps 40

vii

4.8. Paths with tailed long horizontal steps 45

4.9. Slanted paths 47

4.10. Paths with quasi-diagonal steps 49

Chapter 5. Lattice path factorization and diagonal restrictions 51

5.1. Introduction 51

5.2. Factorization of lattice paths 51

5.3. Computing f−, f0 and f+ in the quadratic case 55

5.4. Restricted Delannoy paths 56

5.5. Restricted paths by left turns 60

5.6. Restricted paths by right turns 62

5.7. Restricted long horizontal paths 71

5.8. Restricted tailed long horizontal paths 72

5.9. Restricted slanted paths 73

5.10. Restricted quasi-diagonal paths 74

Bibliography 75

viii

List of Figures

1.1 A Dyck path. 3

1.2 First return decomposition. 4

1.3 Prime decomposition. 4

3.1 Deutsch’s decomposition of a Dyck path. 15

3.2 Cases for G. 22

3.3 Example of the bijection sending rising hooks to up steps. 24

3.4 Example of the bijection µ. 26

4.1 A sample of Delannoy paths from (0, 0) to (3, 4). 36

4.2 A path and its associated complementary path. 39

4.3 The 9 LHPs from (0, 0) to (2, 2). 40

4.4 Bijection for slanted paths. 49

5.1 Path factorization. 52

5.2 Bijection for small Schroder paths. 67

ix

List of Tables

3.1 Narayana numbers. 25

4.1 Coefficients of f1(x, y): Delannoy numbers. 33

4.2 Coefficients of f2(x, y): asymmetrical Delannoy numbers. 33

4.3 Dm,n(w). 37

4.4 Em,n(w). 37

4.5 Number of LHP from (0, 0) to (m,n). 41

4.6 [xmyn]F3(x, y, w). 42

4.7 Number of TLHP from (0, 0) to (m,n). 46

4.8 Gm,n(w). 46

5.1 Coefficients of Schroder polynomials. 59

x

CHAPTER 1

Introduction

1.1. Preliminary definitions

1.1.1. Lattice paths. Let d be a positive integer and S ⊂ Zd. We say that S is a set

of steps in Zd. Geometrically s ∈ S is represented by an edge from 0 ∈ Zd to s rather than

by the point s itself.

A lattice path in Zd with k ≥ 1 steps is a sequence a0, a1, . . . , ak ∈ Zd such that for

each i, 1 ≤ i ≤ k, ai − ai−1 ∈ S. We will often refer to a lattice path simply as a path.

Geometrically a lattice path is represented by the edges between the consecutive vertices of

the path. Thus we can say that a path is a sequence of steps. A path with no steps is a

point. When the choice of point is clear (usually (0, 0)), we will call this path the empty

path.

Note: Throughout this document we will focus on lattice paths in the plane and hence

we will assume d = 2 from now on.

Given two paths p1 and p2, we define their concatenation p1p2 as the sequence of steps of p1

followed by the sequence of steps of p2. That is if p1 is equal to the sequence a0, a1, . . . , ak and

p2 is equal to the sequence b0, b1, . . . , bl then p1p2 is equal to the sequence a0, a1, . . . , ak, b1−

b0 + ak, b2 − b0 + ak, . . . , bl − b0 + ak. A concatenation of n paths is defined similarly.

A subpath of a path p is a sequence of consecutive steps belonging to p. Thus p is a

subpath of p if and only if there exist (possibly empty) paths q and r such that p = qpr.

1

CHAPTER 1. INTRODUCTION

1.1.2. Words associated to paths. Given a set of steps S we associate to each step

s ∈ S a unique letter s. The set of letters thus obtained, S, is called the alphabet associated

to S.

A word is a sequence of letters belonging to an alphabet. The unique word consisting of

no letters is called the empty word and denoted by ε.

A subword of a word w is a sequence of consecutive letters belonging to w.

The bijection S → S induces a bijection between the set of paths with steps in S and

the the set of words with alphabet S. More generally, if P is a set of paths with steps in S

subject to certain conditions then there is a bijection between P and the set of words in S

subject to an equivalent set of conditions.

In view of this bijection, subpaths of p correspond to subwords of p. We will use the

term string to denote either a subpath or a subword depending on context.

1.1.3. Free monoids. A monoid is a tuple (M, ?, e), where M is a set, ? is an associative

binary operation on M and e ∈M is an identity element. That is, for all a, b, and c in M ,

(a ? b) ? c = a ? (b ? c),

and for all a in M ,

e ? a = a ? e = a.

Let S be a set. The free monoid on S is the monoid whose elements are finite words on

the alphabet S, with its operation equal to word concatenation, and with the empty word as

the identity element. We will denote it by S∗. It is easy to check that S∗ is indeed a monoid

under this definition.

At times we will need to consider only nonempty words. The set S∗−{ε} will be denoted

by S+. Though we will not go into details it is a subsemigroup of S∗ and can be independently

defined as the free semigroup on A.

2


A monoid M is said to be free if it is isomorphic to a free monoid S∗.

1.1.4. Dyck paths. A Dyck path is a sequence of up (U = (1, 1)) and down (D =

(1,−1)) steps on a square integer lattice starting at (0,0) and ending at (2n, 0) which

never goes below the x-axis. A Dyck path with 2n steps is said to have semilength n.

A Dyck word is word whose letters correspond to the steps in a Dyck path. In Figure

1.1 we show an example of a Dyck path of semilength 10. Its associated Dyck word is

UUDUDDUUDUUUDDUDDDUD.

Figure 1.1. A Dyck path.

An ascent (respectively descent) is a maximal sequence of consecutive up (respectively

down) steps.

Define a peak as the string UD and a valley as the string DU .

The height of a point in a path is its y-coordinate. We define the level of a string in a

path as the height of the first point in the string. In the case of Dyck paths, the height of a

point is equal to the number of U ’s minus the number of D’s immediately before it.

For any path p, we define the elevation of p as the path p = UpD. Note that if a string

s has level l in p, then it has level l + 1 in p.

A Dyck path is prime if it is nonempty and its first return to the x-axis occurs at its

endpoint. Equivalently, a prime Dyck path is the elevation of a Dyck path.

Given a nonempty Dyck path P , we can decompose it as P = UP1DQ1, where P1 and

Q1 are possibly empty Dyck paths. To see this, locate the first down step to return to the

x-axis. The steps in between the starting U and this D constitute P1 while the steps after

3


this D constitute Q1. This is usually known as the first return decomposition. Figure 1.2

shows the first return decomposition of the Dyck path in Figure 1.1.

Figure 1.2. First return decomposition.

Note that the first return decomposition of P produces a prime Dyck path P1 followed by

an arbitrary Dyck path Q1. As long as Q1 is not empty we can decompose it as a prime Dyck

path P2 followed by an arbitrary Dyck path Q2. Successively decomposing the remaining

path until it is empty, we obtain the prime decomposition of P . That is P = P1 · · · Pm.

Figure 1.3 shows the prime decomposition of the Dyck path in Figure 1.1.

Figure 1.3. Prime decomposition.

1.1.5. Generating functions. Let K be a commutative ring and let (an)∞n=0 be a

sequence of elements in K. A formal power series in x with coefficients in K is a formal sum

a(x) =∞∑n=0

anxn,

where x is an indeterminate. In a way, a formal power series is simply a different notation for

a sequence but it has the advantage that we can perform operations on formal power series in

the same way one adds and multiplies polynomials. Define the addition and multiplication

of formal power series by

∞∑n=0

anxn +

∞∑n=0

bnxn =

∞∑n=0

(an + bn)xn,

4


and∞∑n=0

anxn ×

∞∑n=0

bnxn =

∞∑n=0

(n∑k=0

akbn−k

)xn.

This product is often called Cauchy product. It is easy to check that the set of formal power

series K[[x]] forms a ring under these operations. One can define formal power series in

several variables in a similar way.

A generalization of formal power series are formal Laurent series

b(x) =∞∑

n=n0

bnxn,

where n0 is allowed to be a negative integer. The analogous definitions for addition and

multiplication make the set of formal Laurent series K((x)) a ring. Note that the Cauchy

product is well defined because∑k∈Z

akbn−k has only finitely many nonzero terms. In practice

we will let K be Q or C throughout this document.

Given a family of finite sets {An} indexed by a “size” parameter that runs through the

natural numbers, we can encode the cardinalities of these sets in a sequence (#An)∞n=0 and

thus as a formal power series

A(x) =∞∑n=0

#Anxn.

We say that A(x) is the generating function for {An}.

More generally given a (possibly infinite) set A, a power series ring B, and a weight

function w : A → B such that only finitely many elements of A have the same weight, we

define the generating function for A with weight w by

A(x) =∑a∈A

w(a).

5


In the previous case when a has size n(a) set w(a) = xn(a). Thus the generating functions

coincide

A(x) =∑a∈A

w(a) =∞∑n=0

∑a∈An

xn =∞∑n=0

#Anxn.

Let A and B have generating functions A and B, respectively. For A × B define the

weight function by w((a, b)) = w(a)w(b). Then A×B has generating function A · B.

If C = A∗ then its generating function is C = (1−A)−1.

When assigning weights to lattice paths, we assign a weight to the steps. Then the weight

of a lattice path is the product of the weights of its steps, and the weight of a set of paths

is the sum of the weights of its paths.

Example 1.1.1. Denote the set of all Dyck paths by D and the set of Dyck paths of

semilength n by Dn weight a U step by x and a D step by 1. Then x measures semilength.

Let c(x) be the generating function for D. Then

c(x) =∑p∈D

xn(p) =∞∑n=0

#Dnxn.

From the first return decomposition we see that c(x) = 1 + xc(x)2 and from the prime

decomposition we obtain c(x) = (1− xc(x))−1. Solving either of these equations gives

c(x) =1−√

1− 4x

2x,

which is the generating function for the Catalan numbers Cn. The first few Catalan numbers

are 1, 1, 2, 5, 14, 42, 132, 429, 1430, . . . See [Slo, A000108] for a larger list. For an in depth

treatment of Catalan numbers and the many sets they count see [Sta15].

We can also assign weights to strings in a path rather than to individual steps. For exam-

ple one can weight peaks in a Dyck path by w. Simultaneously keeping track of semilength

6


and number of peaks leads to the Narayana generating function and the Narayana numbers

as given in Section 3.4.2.

1.1.6. Motzkin paths. A Motzkin path is a sequence of up (1, 1), down (1,−1), and

flat (1, 0) steps on a square integer lattice starting at (0, 0) and ending at (n, 0) which never

goes below the x-axis. The length of a Motzkin path is the number of steps. The number of

Motzkin paths of length n is the n-th Motzkin number Mn. Denote by m(x) the Motzkin

generating function, where x keeps track of length. That is,

m(x) =∞∑n=0

Mn xn.

7

CHAPTER 2

Generalized Motzkin numbers and Touchard’s identity

Let Mn,k be the number of Motzkin paths of length n with k flat steps. Given a Motzkin

path one can weight its flat steps by υ. Thus the total weight of a path with k flat steps is

υk. Summing the weights of all Motzkin paths of length n results in a polynomial

Mn(υ) =n∑k=0

Mn,k υk

Call the family of polynomials thus obtained the Motzkin polynomials.

Let q be a positive integer. Define a q-colored Motzkin path to be a Motzkin path in

which the flat steps can take any of q colors. The color of the up and down steps remains

irrelevant and can be set to black. Note that by setting υ equal to a positive integer, Mn(υ)

gives a positive integer, which can be interpreted as the number of υ-colored Motzkin paths

of length n. It is thus natural to call these numbers generalized Motzkin numbers.

Denote by mυ(x) the Motzkin polynomial generating function, where x keeps track of

length and υ keeps track of the number of flat steps. That is,

mυ(x) =∞∑n=0

Mn(υ)xn.

A Motzkin path can be decomposed into primes by cutting at each return to the x-axis.

Thus a Motzkin prime is either a single flat step or an elevated Motzkin path. From this we

deduce the following functional equation for mυ(x):

(2.1) mυ(x) =1

1− υx− x2mυ(x),

8

CHAPTER 2. GENERALIZED MOTZKIN NUMBERS AND TOUCHARD’S IDENTITY

or equivalently

(2.2) mυ(x) = 1 + υxmυ(x) + x2mυ(x)2.

Let c(x) be the Catalan generating function and let Cn denote the n-th Catalan number.

Lemma 2.0.1. The Catalan number Cn+1 counts 2-colored Motzkin paths of length n.

Proof. We give a combinatorial proof. Given a nonempty Dyck path of semilength

n + 1, remove the up step at the beginning and the down step at the end. Then take the

remaining steps of this path two at a time. Replace a pair UU with a U , a pair DD with a D,

a peak UD with a flat step colored red and replace a valley DU with a flat step colored blue.

This gives a 2-colored Motzkin path of length n. This process is clearly reversible. Thus

we have a bijection from the set of Dyck paths of semilength n + 1 to the set of 2-colored

Motzkin paths of length n. �

Note: Olivier Bernardi has informed me that the proof above was previously published by

Delest and Viennot [DV84, p. 179]. They use Dyck words and Motzkin words but otherwise

the bijection is identical.

One can restate Lemma 2.0.1 as Cn+1 = Mn(2) or in terms of generating functions as

c(x)− 1 = xm2(x).

Note that Lemma 2.0.1 gives a simple proof of Touchard’s identity [Tou24]:

(2.3) Cn+1 =

bn/2c∑k=0

2n−2k(n

2k

)Ck.

To see this note that a 2-colored Motzkin path of length n with k up steps has k down steps

and n− 2k flat steps. There are(

nn−2k

)ways to select the position of the flat steps and 2n−2k

ways to select their coloring. The remaining 2k positions can be filled in Ck ways because

they must form a Dyck path of semilength k when the flat steps are ignored. The number

9


of up steps can range from 0 to bn/2c. Summing over k gives the total number of 2-colored

Motzkin path of length n. Both Shapiro [Sha76] and Zeilberger et al. [RSZ] have given

proofs of Touchard’s identity but counting different objects. Shapiro’s proof makes use of

generating functions and it is not bijective. Zeilberger gives a bijective proof which is just a

little less simple than the one provided here.

Let us consider the family of generating functions c

(x

1 + lx

)where l is a nonnegative

integer. If we compute the coefficients of each of their expansions for l = 0, 1, 2, 3 and 4, we

observe that they can be expressed in terms of Catalan or Motzkin numbers. In particular

we have

(2.4) c

(x

1 + x

)= 1 + xm1(x).

(2.5) c

(x

1 + 2x

)= 1 + xc(x2).

(2.6) c

(x

1 + 3x

)= 1 + xm1(−x).

(2.7) c

(x

1 + 4x

)= 2− c(−x).

We will explain these formulas, but first we rewrite them in a uniform way.

First we show that

(2.8) mυ(x) = m−υ(−x).

Note that changing the weight of each step in a Motzkin path from x to −x is equivalent to

changing υ to −υ. This is because the number of U steps is equal to the number of D steps.

10


Their corresponding minus signs cancel and the minus sign of a flat step can be absorbed by

υ.

In particular m1(x) = m−1(−x). So equation (2.4) becomes

c

(x

1 + x

)= 1 + xm−1(−x).

Giving a weight of 0 to flat steps is equivalent to counting only Motzkin paths without flat

steps, namely Dyck paths. These are counted by c(x2) when both U ’s and D’s have a weight

of x. Thus equation (2.5) gives

c

(x

1 + 2x

)= 1 + xc(x2) = 1 + xm0(x) = 1 + xm0(−x).

From Lemma 2.0.1, we have

2− c(−x) = 1− (c(−x)− 1) = 1− (−xm2(−x)) = 1 + xm2(−x)

Thus equation (2.7) becomes

c

(x

1 + 4x

)= 1 + xm2(−x).

To explain the above formulas we give a combinatorial proof of the general case:

Theorem 2.0.2. Let υ be an indeterminate. We have

(2.9) c

(x

1 + (υ + 2)x

)= 1 + xmυ(−x).

Proof. Define Q(x) by

(2.10) Q(x) = 1− c(

−x1− (υ + 2)x

).

11


Proving (2.9) is equivalent to showing Q(x) = xmυ(x). We do this via a sign-reversing

involution.

To interpret Q(x), start with a 2-colored Motzkin path of length n with colors light red

and light blue for the flat steps. We assign a weight of xn+1 to this path so that its weight is

the same as the weight of the corresponding Dyck path as in Lemma 2.0.1. Insert anywhere

any number of flat steps in any of 3 colors: dark red, dark blue and green. Weight the green

flat steps by υ. When we replace x with−x

1− (υ + 2)x, we will be multiplying the weight of

a path of length n by1

(1− (υ + 2)x)n+1, which accounts for the n + 1 places where these

extra steps can be inserted. The −x is accounted for by multiplying the weight of a 2-colored

Motzkin path of length n by (−1)n+1. Thus Q(x) counts these paths (except for the empty

path) with a weight of (−1)k, where k is the number of steps which are U , D, or a flat step

colored with a light color. Again, since U ’s and D’s come in pairs we can consider k to be

the number of light colored steps.

To define a sign reversing involution on these paths, locate the first red or blue step and

convert it from a light shade to a dark shade or vice versa. This will cancel all of the paths

with a red or blue step. The only paths left are Motzkin paths in which the flat steps are

all green. Since green flat steps have a weight of υ, this gives xmυ(x). �

12

CHAPTER 3

Enumeration of paths via occurrences of distinguished strings

In this chapter we will count paths by occurrences of several strings in Dyck paths. Recall

that a string is a sequence of consecutive steps belonging to a path. There is an extensive

literature on lattice path enumeration where authors count the occurrence of a particular

string in Dyck paths, sometimes also keeping track of whether such a string occurs at even or

odd height. This has been done, for example, by Deutsch in [Deu99] where “peaks”, “valleys”,

and “doublerises” are studied and as a result give Narayana numbers. Strings of length 3

are also considered by Deutsch. In [STT07] the authors count paths by occurrences of any

given string of length 4 and by semilength. In his thesis [Wan11] Wang counts occurrences

of “peaks” and “UUD” as an application of the Goulden-Jackson cluster method.

Here we count three types of substrings simultaneously which together also count paths

by semilength and number of peaks. Special cases of this enumeration give several well-known

families of numbers such as Catalan, Motzkin, Narayana, and Schroder numbers.

3.1. Dyck paths

Recall that a Dyck path is a sequence of up (U = (1, 1)) and down (D = (1,−1)) steps

on a square integer lattice starting at (0,0) and ending at (2n, 0) which never goes below the

x-axis. A Dyck path with 2n steps is said to have semilength n. Other important definitions

such as ascent, descent, peak, level, elevation and first return decomposition can be found in

section 1.1.4.

Define a short peak as an occurrence of UD not preceded by U , i.e., a peak at the be-

ginning of the path or a peak following a D. Define a triple rise as the subsequence UUU

13

CHAPTER 3. ENUMERATION OF PATHS VIA OCCURRENCES OF DISTINGUISHEDSTRINGS

and a rising hook as the subsequence UUD. Throughout this section we will refer to these

strings collectively as distinguished strings.

Let j(P ) be the number of short peaks, let m(P ) be the number of triple rises, and let

l(P ) be the number of rising hooks in P . Denote the set of all Dyck paths by D. Then set

(3.1) F (x, y, z) =∑P∈D

xj(P )ym(P )zl(P ).

Thus F (x, y, z) is the generating function counting Dyck paths by the number of short peaks

(weighted by x), triple rises (weighted by y), and rising hooks (weighted by z).

Lemma 3.1.1. The semilength of a path is given by n(P ) = j(P ) + m(P ) + 2 l(P ). The

number of peaks (occurrences of UD) is recovered as the sum j(P ) + l(P ).

Proof. Since the semilength n(P ) of P is equal to the number of up steps in P it suffices

to add the lengths of each ascent. An ascent of length one corresponds to a short peak. An

ascent of length two corresponds to a rising hook. Thus each rising hook contributes two

to the semilength. An ascent of length three or greater has two final U ’s as part of a rising

hook and an occurrence of UUU for each extra U . Thus each UUU contributes one to the

semilength. This gives the first statement. By definition a peak is either a short peak or the

end of a rising hook. Thus the second statement follows. �

Theorem 3.1.2. The series F (x, y, z) defined by (3.1) satisfies the functional equation

(3.2) F = 1 + xF +zF 2

1− yF.

Proof. Given a nonempty Dyck path P , we use the first return decomposition to get

P = UP1DQ1, where P1 and Q1 are possibly empty Dyck paths. If P1 is not empty we

can apply the decomposition again to obtain P1 = UP2DQ2 and thus P = UUP2DQ2DQ1.

14


Successively decomposing Pi until Pk = ∅ we obtain P = UkDQkDQk−1 · · ·DQ1. This is

illustrated in Figure 3.1. We will refer to this decomposition as Deutsch’s decomposition.

Figure 3.1. Deutsch’s decomposition of a Dyck path.

Let D denote the set of all Dyck paths and let Ak be the set of all Dyck paths whose

first ascent has length k (A0 consisting of the empty path). Deutsch’s decomposition shows

that the Ak form a partition of D. We consider three cases for P depending on k such that

P belongs to Ak.

1) The empty path (case k = 0).

2) A short peak followed by an arbitrary Dyck path (case k = 1).

3) A run of k−2 U ’s followed by a rising hook followed by k arbitrary Dyck paths connected

by D’s (case k ≥ 2).

The left side of equation (3.2) counts all paths in D while the terms on the right side count

paths in A0, A1 and⋃k≥2

Ak, respectively. Each term corresponds to one of the above three

cases. The terms for the empty path and a short peak followed by an arbitrary Dyck path are

clear. For each k ≥ 2, zyk−2F k counts paths in Ak because if P = UkDQkDQk−1 · · ·DQ1,

then the short peaks of P are those of Qk, . . . , Q1. The triple rises of P are the k − 2

occurring in the initial segment plus those of Qk, . . . , Q1. The rising hooks of P are those of

Qk, . . . , Q1 plus one more . Note that every nonempty Qi ends with a D and distinct Qi’s

are separated by D’s thus making it impossible to have extra occurrences of distinguished

15


strings having steps belonging to two adjacent Qi’s. Therefore∑k≥2

zyk−2F k =zF 2

1− yFcounts

paths in⋃k≥2

Ak. �

Note: In [STT07] the authors use Deutsch’s decomposition to count paths by occurrences

of UUUU and by semilength. More generally, they give a generating function counting by

occurrences of strings U r for r ≥ 2.

The coefficient of xjymzl in F and more generally F k can be computed by means of

Lagrange inversion. See [Ges16] and [Sta04] for a survey on the method of Lagrange inversion.

First we introduce a new variable t and consider the equation

F = t

(1 + xF +

zF 2

1− yF

),

which determines a unique formal power series F in t, x, y and z. Note that F is obtained by

setting t = 1 in F . Then F = tg(F ), where g(s) = 1 + xs+ zs2/(1− ys). Thus by Lagrange

inversion, for n > 0 we have

[tn]F k =k

n[un−k]

(1 + xu+

zu2

1− yu

)n=k

n[un−k]

∑i+j+l=n

n!

i! j! k!(xu)j

(zu2

1− yu

)l=k

n[un−k]

∑j, l

n!

(n− j − l)! j! l!xjuj+2lzl

∑m

(l +m− 1

m

)(yu)m

=k

n[un−k]

∑j, l, m

n!

(n− j − l)! j! l!

(l +m− 1

m

)xjymzluj+2l+m.

16


Thus

F k =∑j, l, m

k(j + 2l +m+ k)!

(j + 2l +m+ k)(l +m+ k)! j! l!

(l +m− 1

m

)xjymzltj+2l+m+k.

Setting t = 1 we get

(3.3) [xjymzl]F k =k(j + 2l +m+ k − 1)!

(l +m+ k)! j! l!

(l +m− 1

m

).

This expression makes sense for all nonnegative integers j,m, l and k > 0. When k is

negative, it still makes sense provided that l +m+ k ≥ 0.

When l ≥ 1 we can write

(3.4) [xjymzl]F k =k(j + 2l +m+ k − 1)! (l +m− 1)!

(l +m+ k)! j! l!m! (l − 1)!.

When l = 0, [xjymz0]F k = 0 unless m = 0, which gives

(3.5) [xjy0z0]F k =

(j + k − 1

j

).

By setting k = 1 in formula (3.4) we obtain the following result.

Theorem 3.1.3. The number of Dyck paths with exactly j short peaks, m triple rises,

and l rising hooks (where l ≥ 1) is

(j + 2l +m)!

(l +m+ 1)(l +m)j! l!m! (l − 1)!.

A rising-hook-free Dyck path necessarily is also free of triple rises, thus it is a sequence

of short peaks. There is a unique such path for each j. This is consistent with equation (3.4)

when k = 1.

17


3.2. Ballot paths and left incomplete Dyck paths

A ballot path is a sequence of nu up and nd down steps on a square integer lattice starting

at (0, 0) and ending at (nu + nd, nu − nd) which never goes below the x-axis. The height of

a ballot path is h = nu − nd. Thus Dyck paths are ballot paths of height zero.

It is well known that a ballot path of height h can be decomposed as a sequence of h+ 1

Dyck paths separated by h up steps. Namely, P = Q0UQ1U · · ·Qh, where each Qi is a Dyck

path.

One cannot use the above decomposition directly to count short peaks, rising hooks and

triple rises in ballot paths because the U ’s in the decomposition generate extra occurrences

of these strings. However, we will count reversals of ballot paths by these parameters.

Define a left incomplete Dyck path L as a sequence of U ’s and D’s starting at (0, k) such

that UkL is a Dyck path.

One can transform a ballot path into a left incomplete Dyck path by starting at (0, h),

reading the steps from right to left and switching every U with D and vice versa. In other

words one flips the path around the y-axis and then shifts it horizontally to the first quadrant.

Since such an incomplete Dyck path is a sequence of h + 1 Dyck paths separated by down

steps, it provides an interpretation for F h+1. Applying formula (3.4) gives the following

result.

Theorem 3.2.1. The number of left incomplete Dyck paths starting at (0, h) with exactly

j short peaks, m triple rises and l rising hooks (where l ≥ 1) is

(h+ 1)(j + 2l +m+ h)! (l +m− 1)!

(l +m+ h+ 1)! j! l!m! (l − 1)!.

When such a path is rising-hook-free (and consequently UUU -free) it is a combination

of j short peaks and h separating down steps. Thus there are(j+hj

)such paths. This is

consistent with formula (3.5).

18


3.3. Prime Dyck paths

By algebraic manipulation of equation (3.2) we obtain the following closed forms for F .

F (x, y, z) =1− x+ y −

√(1− x− y)2 − 4z

2(y + z − xy)(3.6)

=2

1− x+ y +√

(1− x− y)2 − 4z(3.7)

=1

1− x− z

1− x− yc

(z

(1− x− y)2

) ,(3.8)

where c(x) = 1−√1−4x2x

is the generating function for the Catalan numbers.

Let G(x, y, z) be defined by

G(x, y, z) =z

1− x− yc

(z

(1− x− y)2

).(3.9)

Then by equation (3.8)

F (x, y, z) =1

1− (x+G(x, y, z)).

This shows that x+G(x, y, z) counts prime Dyck paths weighted by the number of short

peaks, triple rises, and rising hooks. Since x corresponds to a short peak (the only prime of

semilength one), G counts primes of semilength greater than or equal to two. In particular

any such prime must contain a rising hook. That is [z0]G = 0. Solving for G in terms of F

gives G = 1− x− F−1. Therefore [z0]F−1 = 1− x.

When l ≥ 1 we can use equation (3.4) to compute [xjymzl]F−1 and thus obtain [xjymzl]G:

[xjymzl]G = −[xjymzl]F−1 =(j + 2l +m− 2)!

j! l!m! (l − 1)!,

where j,m ≥ 0 and l ≥ 1. In other words we obtain the following result.

19


Theorem 3.3.1. The number of prime Dyck paths with exactly j short peaks, m triple

rises and l rising hooks (where l ≥ 1) is

(j + 2l +m− 2)!

j! l!m! (l − 1)!.

The first few terms of G are

G(x, y, z) = z + (xz + yz + z2) + (x2z + 2xyz + 3xz2 + y2z + 3yz2 + 2z3) + . . .

Using the fact that c(x) = 1 + xc2(x) one can manipulate equation (3.9) algebraically to

obtain

(3.10) G = z + (x+ y)G+G2.

Now we give a combinatorial interpretation of this equation. Since a path counted by G

is a prime of semilength at least two, it can be decomposed as UP1UP2DD, where P1 and

P2 are possibly empty Dyck paths. There are four cases depending on whether each of P1

and P2 is empty or nonempty. This is illustrated in Figure 3.2.

Case 1) P1 = P2 = ∅. Then the prime is simply UUDD with a weight of z.

Case 2) P1 = ∅, P2 6= ∅. Every such prime is of the form UUP2DD. Note that UP2D is

a prime of semilength at least two so it is counted by G. However, since P2 starts with a U

there is an extra occurrence of UUU not counted by G. This gives a total weight of yG for

all paths of this form.

Case 3) P1 6= ∅, P2 = ∅. These primes are of the form UP1UDD. Note that moving the

last D to the left to the position right after P1 gives a new path UP1DUD with the exact

same number of occurrences of substrings counted by x, y and z but in this way it is clear

that UP1D is counted by G and UD contributes an x giving a total weight of xG for this

case.

20


Case 4) P1 6= ∅, P2 6= ∅. As in case 3 we modify the path UP1UP2DD to obtain

UP1DUP2D without changing the weight. Each of UP1D and UP2D is counted by G. Thus

paths in case 4 are counted by G2.

Observe that G = z + (x+ y)G+G2 is symmetric with respect to x and y. This suggest

there should be an involution on prime Dyck paths (of semilength greater than or equal to

two) sending short peaks to triple rises and vice versa.

We define this involution recursively on prime paths by means of the decomposition with

four cases as above. However for simplicity we define the map φ on paths consisting of a

prime path counted by G with the last D removed. Thus we define

φ(UUD) = UUD,

φ(UUP2D) = φ(UP2)UD,

φ(UP1UD) = Uφ(UP1)D,

φ(UP1UP2D) = φ(UP1)φ(UP2)D,

where P1 and P2 are nonempty Dyck paths. One can check that φ2 is the identity. At each

step of the recursion φ leaves the case of a path in case 1 and 4 unchanged and switches

cases 2 and 3. Thus φ preserves rising hooks and sends a short peak to a triple rise and vice

versa.

21


Case 1) weight z

P1 = ∅P2 = ∅

Case 2) weight yG

P1 = ∅ +

Case 3) weight xG

−→

Case 4) weight G2

−→

Figure 3.2. Cases for G.

22


3.4. Several special cases counted by F

3.4.1. Catalan numbers. Recall that c(x) = 1−√1−4x2x

is the generating function for

the Catalan numbers. It satisfies the functional equations

c = 1 + xc2

=1

1− xc.

By setting F (x) = F (x, x, x2) and substituting in equation (3.2) we obtain

F = 1 + xF +x2F

2

1− xF=

1

1− xF.

Thus F = c(x). This is expected because

F = F (x, x, x2) =∑P∈D

xj(P )xm(P )x2l(P ) =∑P∈D

xj(P )+m(P )+2l(P ),

and j(P ) +m(P ) + 2l(P ) is equal to the semilength of the Dyck path P .

Now let F (x) = F (0, 0, x) and substitute in equation (3.2) to obtain F (x) = 1 + xF (x)2.

Therefore the coefficients of F (x) are the Catalan numbers. By definition F is counting

Dyck paths composed only of rising hooks and down steps. Since the difference in height

between the initial point and the final point of a rising hook is one, the number of down

steps is equal to the number of rising hooks and at no point can there be more down steps

than rising hooks in a subpath starting at the origin. Each rising hook has two up steps,

hence a path with j rising hooks has semilength 2j. By replacing each rising hook with an

up step we obtain a bijection from the set of Dyck paths composed exclusively of j rising

hooks (and j down steps) to the set of Dyck paths of semilength j. Figure 3.3 illustrates

how a path with 4 rising hooks is sent to a Dyck path of semilength 4.

23


Figure 3.3. Example of the bijection sending rising hooks to up steps.

3.4.2. Narayana numbers. The Narayana numbers Nar(n, k) count the number of n-

Dyck paths with k peaks. Let ηw(x) be the Narayana generating function, where x keeps

track of semilength and w keeps track of the number of peaks. Consider the substitution

x→ xw, y → x, and z → x2w in F (x, y, z). We have

F (xw, x, x2w) =∑P∈D

(xw)j(P )xm(P )(x2w)l(P ) =∑P∈D

xj(P )+m(P )+2l(P )wj(P )+l(P ).

Since j(P ) + m(P ) + 2l(P ) is the semilength of P and j(P ) + l(P ) is the total number of

peaks in P , it follows that F (xw, x, x2w) = ηw(x). Thus (3.6) gives an explicit formula for

ηw(x),

ηw(x) =1− xw + x−

√(1− xw − x)2 − 4x2w

2x.

The first few Narayana numbers are presented in Table 3.1. For a larger list consult [Slo,

A001263].

3.4.3. Motzkin numbers and Motzkin polynomials. Recall that a Motzkin path is

a sequence of up, down and (short) flat (1, 0) steps on a square integer lattice starting at

(0, 0) and ending at (n, 0) which never goes below the x-axis. The length of a Motzkin path

is the number of steps. Let M(x) be the Motzkin number generating function, where x keeps

24


n\k 1 2 3 4 5 6 7 8

1 1 0 0 0 0 0 0 0

2 1 1 0 0 0 0 0 0

3 1 3 1 0 0 0 0 0

4 1 6 6 1 0 0 0 0

5 1 10 20 10 1 0 0 0

6 1 15 50 50 15 1 0 0

7 1 21 105 175 105 21 1 0

8 1 28 196 490 490 196 28 1

Table 3.1. Narayana numbers.

track of length. It satisfies the functional equations

M(x) = 1 + xM(x) + x2M(x)2(3.11)

=1

1− x− x2M(x).(3.12)

In closed form we have

M(x) =1− x−

√1− 2x− 3x2

2x2.

By substituting in equation (3.2) and comparing with equation (3.11), one can verify that

F (x, 0, x2) = M(x).

We can interpret F (x, 0, x2) as counting Dyck paths without any triple rises and where x

keeps track of semilength. We can explicitly give a bijection from these Dyck paths to

Motzkin paths. Send a short peak to a flat step, send a rising hook to an up step, and leave

unchanged any remaining down step. For convenience call this bijection µ. Note that under

25


µ a Dyck path of semilength n goes to a Motzkin path of length n. Also µ sends primes into

primes. Note this bijection can be found in [Cal04].

µ

Figure 3.4. Example of the bijection µ.

Furthermore, one can weight flat steps by υ to obtain Motzkin polynomials. Denote by

Mυ(x) the Motzkin polynomial generating function, where x keeps track of length and υ

keeps track of the number of flat steps. Equation (3.12) follows from the decomposition of

a Motzkin path into primes. A Motzkin prime is either a single flat step or an elevated

Motzkin path. Thus the corresponding functional equation for Mυ(x) is

(3.13) Mυ(x) =1

1− υx− x2Mυ(x),

or equivalently

(3.14) Mυ(x) = 1 + υxMυ(x) + x2Mυ(x)2.

We have that

F (υx, 0, x2) = Mυ(x).

The interpretation of this equation is the same as that of F (x, 0, x2) = M(x), with the

additional information that short peaks and flat steps have a weight of υ.

26


3.4.4. Riordan numbers. A Riordan path is a Motzkin path with no flat steps on the

x-axis (also called level zero). Let R(x) be their generating function, where x keeps track of

length.

R(x) =1 + x−

√1− 2x− 3x2

2x(1 + x)

We can obtain a substitution similar to the Motzkin case which gives

F (0, x, x2) = R(x).

We interpret F (0, x, x2) as counting Dyck paths without any short peaks and where x keeps

track of semilength.

We now give a bijection from these restricted Dyck paths to Riordan paths. Starting

with P , one of our restricted Dyck paths with no short peaks, first we decompose it into

primes. Thus P = P1P2 · · ·Pk, where each Pi is a prime. Since P has no short peaks, each

prime is of semilength at least 2. Recall that in Section 3.3 we gave an involution on such

prime Dyck paths. In particular φ transforms a prime path with no short peaks, m triple

rises and l rising hooks into a prime path with no triple rises, m short peaks and l rising

hooks. If we apply the involution φ to Pi we get a path which might have short peaks but

not at level zero. Then using the bijection µ of Section 3.4.3 we get a prime Motzkin path

µφ(Pi) with no flat steps at level zero. In other words µφ(Pi) is a Riordan prime. Finally

µφ(P ) = µφ(P1)µφ(P2) · · ·µφ(Pk) is a Riordan path. It is easy to check that φµ−1 is the

inverse bijection.

3.4.5. Schroder polynomials. A Schroder path is a sequence of up, down and (long)

flat (2, 0) steps on a square integer lattice starting at (0, 0) and ending at (2n, 0) which never

goes below the x-axis. A Schroder path path ending at (2n, 0) is said to have semilength n.

The Schroder polynomials arise from giving each path a weight of υ to the number of flat

27


steps. Let rυ(x) be its generating function (where x keeps track of the semilength). Just

as with Dyck paths and Motzkin paths, a Schroder path can be decomposed into primes.

A prime is either a flat step or an elevated Schroder path. Thus we obtain the following

functional equation:

rυ(x) =1

1− υx− xrυ(x).

This can be solved algebraically to obtain a closed form for rυ(x).

rυ(x) =1− υx−

√(1− υx)2 − 4x

2x.

By making the appropriate substitution in equation (3.6) we can see that F (υx, 0, x) = rυ(x).

We can interpret F (υx, 0, x) as counting Dyck paths without any triple rises and where short

peaks have a weight of υ. We can explicitly give a bijection from these paths to Schroder

paths. Send a short peak to a flat step, send a rising hook to an up step, and leave unchanged

any remaining down step.

3.4.6. Small Schroder polynomials. A small Schroder path is a Schroder path with

no flat steps on the x-axis. The generating function for these paths sυ(x) is given by:

sυ(x) =1 + υx−

√(1− υx)2 − 4x

2(υ + 1)x.

We can obtain a substitution similar to the Riordan case which gives

F (0, υx, x) = sυ(x).

Here F (0, υx, x) is counting Dyck paths without any short peaks and where triple rises have

a weight of υ. The bijection between these paths and small Schroder paths is analogous to

the one described in the Riordan case.

28


3.5. Generalized Touchard’s identity

Recall from equation (3.9) that

G(x, y, z) =z

1− x− yc

(z

(1− x− y)2

).

We now give a different interpretation for G by adapting the proof of Touchard’s identity in

Chapter 2 to steps with weights x, y and z.

Recall that the level of a string in a path is the height of the first point in the string.

Theorem 3.5.1. The generating functionG(x, y, z) counts prime Dyck paths of semilength

at least 2 by the number of peaks at even level (weighted by x), valleys DU at even level

(weighted by y), and double rises UU at even level (weighted by z).

Proof. First start with a Dyck path and assign a weight of z to each U . Double each up

and down step. Before or after any pair of U ’s insert an arbitrary number of UD’s weighted

by x or DU ’s weighted by y. Then raise the path by adding UU at the begining and DD at

the end. The initial UU gets a weight of z. The result is a prime Dyck path of semilength

at least 2. There is a unique way to obtain each prime Dyck path of semilength at least

2 by this method. By construction we get the generating function on the right hence G is

counting these paths. Observe that UD’s and DU end at the same height they have started

and hence any sequence of such strings won’t alter the level of the next string. The initial

UU starts at height 0 and ends at height 2. Hence the following pair starts at height 2 and

will end at an even height (the same height if it is UD or DU or the next even if it is UU).

If the path goes down via D’s then it will go down by an even number of D’s before the next

pair or until it reaches the end of the path, thus not altering the fact that UU ’s with weight

z, UD’s with weight x and DU ’s with weight y, all occur at even level. �

29


Theorem 3.5.2. The generating function F (x, y, z) counts Dyck paths by the number

of peaks at even height (weighted by x), valleys DU at even height (weighted by y), and

double rises UU starting at even height (weighted by z).

Proof. Note that these parameters are compatible with the decomposition of a Dyck

path into primes. That is the total number of peaks or valleys or double rises at even height

of a Dyck path is the sum of the corresponding number of peaks or valleys or double rises

at even height in each of its prime paths. Since F (x, y, z) =1

1− (x+G(x, y, z)), the result

follows from Theorem 3.5.1. �

30

CHAPTER 4

Miscellany of lattice path enumeration

4.1. Introduction

In this chapter, we discuss a miscellany of lattice path enumerations. These paths have

generating functions that are in some way related to the Delannoy number generating func-

tion. We start with versions of these power series without weights and then modify them

to include some weights. As a result we obtain some interesting polynomials and some re-

lationships among the generating functions. Whenever possible we provide combinatorial

interpretations to these relationships.

This chapter should be viewed as a precursor to Chapter 5, where we use a factorization

method to study paths analogous to the ones in this chapter but with additional restrictions.

4.2. Jacobi’s change of variables formula

In this section we recall an important tool used to extract coefficients of Laurent series.

For a Laurent series f(t), let Res f(t) denote its residue, i.e., the coefficient of t−1. Similarly,

let CT(f) stand for constant term of f .

Lemma 4.2.1. Jacobi’s change of variables formula [Ges87, p. 186], [GJ83, p. 15]. If

g(y) is a power series with g(0) = 0 and g′(0) 6= 0, then Res f(t) = Res f(g(t)) · g′(t) and

CT f(t) = Res f(t)/t.

31

CHAPTER 4. MISCELLANY OF LATTICE PATH ENUMERATION

Proof. The residue is a linear operator hence we only need to prove the lemma in the

case f(t) = tk, where k is an integer. If k 6= −1, then Res tk = 0. Note that

d

dt

g(t)k+1

k + 1= g(t)k · g′(t)

and since the residue of a derivative of a Laurent series is 0, we must have

Res g(t)k ·g′(t) = 0. In the case when k = −1, we have Res t−1 = 1. Let g(t) = g1t+g2t2+· · · ,

where g1 6= 0. Then

g′(t)

g(t)=g1 + 2g2t+ · · ·g1t+ g2t2 + · · ·

=1

t· g1 + 2g2t+ · · ·g1 + g2t+ · · ·

.

Thus

Res f(g(t)) · g′(t) = Resg′(t)

g(t)= Res

1

t= 1.

�

Example 4.2.2. Let us consider the transformation g(t) =t

1 + at. We have

Res f(t)/t = Res f

(t

1 + at

)· 1 + at

t· ddt

t

1 + at= Res f

(t

1 + at

)· 1

t(1 + at),

thus

(4.1) CT f(t) = CT f

(t

1 + at

)· 1

1 + at.

4.3. Delannoy numbers

Let us consider the following two rational functions,

(4.2) f1(x, y) =1

1− x− y − xy,

32


(4.3) f2(x, y) =1

1− 2x− y + xy.

In Tables 4.1 and 4.2 we present the first few coefficients of f1 and f2, respectively, where

the (m,n) entry is the coefficient of xmyn.

m\n 0 1 2 3 4 5 6

0 1 1 1 1 1 1 1

1 1 3 5 7 9 11 13

2 1 5 13 25 41 61 85

3 1 7 25 63 129 231 377

4 1 9 41 129 321 681 1289

5 1 11 61 231 681 1683 3653

6 1 13 85 377 1289 3653 8989

Table 4.1. Coefficients of f1(x, y): Delannoy numbers.

m\n 0 1 2 3 4 5 6

0 1 1 1 1 1 1 1

1 2 3 4 5 6 7 8

2 4 8 13 19 26 34 43

3 8 20 38 63 96 138 190

4 16 48 104 192 321 501 743

5 32 112 272 552 1002 1683 2668

6 64 256 688 1520 2972 5336 8989

Table 4.2. Coefficients of f2(x, y): asymmetrical Delannoy numbers.

Observe that the diagonal entries are equal for both functions. We would like to prove

this fact. First we give an algebraic proof and then we generalize our functions to give a

33


combinatorial interpretation. Note that these numbers were studied by Hetyei in [Het06].

The coefficients in Table 4.1 are the well-known Delannoy numbers, while Hetyei calls the

numbers of Table 4.2 asymmetrical Delannoy numbers [Slo, A049600]. Hetyei observed the

equality of the diagonal entries, which are known as central Delannoy numbers.

Theorem 4.3.1. The diagonal entries of Tables 4.1 and 4.2 are equal. Equivalently,

[xmym]1

1− x− y − xy= [xmym]

1

1− 2x− y + xy.

Proof. Since we are only interested in the diagonal elements of the above tables, we

group together terms for which m − n is constant. Then we select the m − n = 0 group.

Following Gessel’s method in [Ges80, p. 322], given a formal power series in x and y we

introduce a new variable t and replace x by t and y by y/t. That is, if

F(x, y) =∑m,n≥0

amnxmyn,

then

G(t, y) = F(t, y/t) =∑m,n≥0

amntm−nyn =

∞∑l=−∞

tl∑

n≥max(0,−l)

an+l,nyn.

We can clearly recover F as F(x, y) = G(x, xy).

To f1 and f2 we apply the t substitution to get f1 =1

1− t− y/t− yand f2 =

1

1− 2t− y/t+ y.

Now we use example 4.2.2 with f = f1 and a = −1. This gives

f1

(t

1− t

)· 1

1− t= f2.

Hence by equation (4.1) the constant term of f1 is equal to the constant term of f2 and the

result is proven. �

34


4.4. Ordinary paths

We define an ordinary path to be a sequence of unit horizontal, (1, 0), and vertical, (0, 1),

steps from (0, 0) to (m,n). If we denote a horizontal step by X and a vertical step by Y ,

we can associate a unique word in {X, Y }∗ to each ordinary path. Here {X, Y }∗ denotes

the free monoid on the set {X, Y }. By giving a weight of x to X and y to Y , we obtain the

generating function for ordinary paths, (1 − x − y)−1. Now we compute the coefficient of

xmyn in (1− x− y)−1:

1

1− x− y=∑k≥0

(x+ y)k =∑k≥0

xk(1 + y/x)k

=∑k≥0

xk∑n≥0

(k

n

)ynx−n =

∑m,n≥0

(m+ n

n

)xmyn.

Thus

[xmyn]1

1− x− y=

(m+ n

n

).

Therefore the number of ordinary paths from (0, 0) to (m,n) is(m+nn

)This can be seen

directly because there are a total of m+ n steps in any such path. Choosing the position of

the n vertical steps automatically defines a unique path, thus there are(m+nn

)such paths.

4.5. Delannoy paths and Delannoy polynomials

It is widely known that f1(x, y) = (1− x− y− xy)−1 is the generating function for paths

in the first quadrant from (0, 0) to (m,n) with unit horizontal, vertical, and diagonal steps,

(1, 0), (0, 1), and (1, 1). See for example [Sul03]. Such paths are called Delannoy paths

and the Delannoy number Dm,n is defined to be the number of such paths, equivalently the

coefficient of xmyn in the Taylor expansion of f1(x, y). See Table 4.1. As an example, we

present 4 out of the 129 Delannoy paths from (0, 0) to (3, 4) in Figure 4.1.

35


Figure 4.1. A sample of Delannoy paths from (0, 0) to (3, 4).

To further analyze f1(x, y) let us weight the diagonal steps by w− 1, so we define a new

generating function

(4.4) F1(x, y, w) =1

1− x− y − (w − 1)xy.

To obtain the analogue of f2, we replace x by t and y by y/t in F1 and apply the transfor-

mation of Example 4.2.2 with a = −1. Then we reverse the transformation to return to x

and y. The resulting function is

(4.5) F2(x, y, w) =1

1− wx− y + (w − 1)xy.

By construction, the diagonal coefficients of F1 and F2 are the same. Note that we recover

f1 and f2 when w = 2. Also, when w = 1 we recover the generating function for ordinary

paths from both F1 and F2.

As before, we proceed to compute the coefficients of xmyn, which now are polynomials

in w. Let us denote them by Dm,n(w) and Em,n(w) for F1 and F2, respectively. See Tables

4.3 and 4.4.

Theorem 4.5.1. The coefficients of the polynomials Dm,n(w) and Em,n(w) are the same

but in reversed order.

Proof. By making the appropriate substitution in F1, we arrive at the equation

(4.6) F1(wx, y, w−1) =

1

1− wx− y + (w − 1)xy= F2(x, y, w).

36


m\n 0 1 2 3 4

0 1 1 1 1 1

1 1 w + 1 2w + 1 3w + 1 4w + 1

2 1 2w + 1 w2 + 4w + 1 3w2 + 6w + 1 6w2 + 8w + 1

3 1 3w + 1 3w2+6w+1 w3+9w2+9w+1 4w3+18w2+12w+1

4 1 4w + 1 6w2+8w+1 4w3+18w2+12w+1 w4+16w3+36w2+16w+1

Table 4.3. Dm,n(w).

m\n 0 1 2 3 4

0 1 1 1 1 1

1 w 1 + w 2 + w 3 + w 4 + w

2 w2 2w + w2 1 + 4w + w2 3 + 6w + w2 6 + 8w + w2

3 w3 3w2+w3 3w+6w2+w3 1+9w+9w2+w3 4+18w+12w2+w3

4 w4 4w3+w4 6w2+8w3+w4 4w+18w2+12w3+w4 1+16w+36w2+16w3+w4

Table 4.4. Em,n(w).

This shows that when we replace every w with its reciprocal in the coefficient of xmyn of

F1 and then multiply by wm we obtain the corresponding coefficient of F2. In symbols,

(4.7) wmDm,n(w−1) = Em,n(w).

�

Notice that the polynomials in the diagonal entries of Table 4.3 are invariant under the

reversal operation, which explains why they coincide with the diagonal entries of Table 4.4.

But why are these polynomials symmetric? In the next section we compute an explicit

formula for Dm,n(w) and then give a combinatorial interpretation to this symmetry.

37


4.6. Ordinary paths by left turns

Denote a horizontal step by X and a vertical step by Y . We define a left turn to be a

horizontal step followed by a vertical step (as a word it is written XY ). Similarly, a right

turn is a vertical step followed by a horizontal step (Y X). Let F1 and F2 be defined by

equations (4.4) and (4.5) respectively.

Theorem 4.6.1. The generating function F1 counts ordinary paths with a weight of w

to the number of left turns. Similarly, F2 counts paths with a weight of w raised to the

number of X steps minus the number of left turns.

Proof. Let υ = w − 1, so that F1 counts Delannoy paths with diagonal steps weighted

by υ. We can transform a Delannoy path into an ordinary path by replacing its diagonal

steps with left turns. Thus given an ordinary path with k left turns, there are 2k Delannoy

paths corresponding to it, depending on whether each of the left turns came from a diagonal

step or was originally a left turn. The sum of the weights of such paths is thus (1+υ)k = wk,

which then is the weight of the ordinary path. The second assertion follows from interpreting

the content of equation (4.6). �

Lemma 4.6.2. The number of paths from (0, 0) to (m,n) with k left turns is(m

k

)(n

k

).

Proof. We note that a path is uniquely determined by the position of its left turns,

{(a1, b1), . . . , (ak, bk)}, where 1 ≤ a1 < · · · < ak ≤ m and 0 ≤ b1 < · · · < bk ≤ n − 1. There

are(mk

)choices for the ai and

(nk

)choices for the bi, giving the result. This argument can be

found in [Mac84, p. 169]. �

38


In view of the previous lemma, we arrive at the explicit formula

(4.8) Dm,n(w) =m∑k=0

(m

k

)(n

k

)wk.

So we obtain

wmDm,m(w−1) =m∑k=0

(m

k

)2

wm−k =m∑k=0

(m

m− k

)2

wk =m∑k=0

(m

k

)2

wk = Dm,m(w),

which shows the desired symmetry.

In terms of paths, this means that the number of paths from the origin to (m,m) with

k left turns is the same as the number of such paths with m− k left turns. We also can get

an explicit bijection directly from interpreting the above formulas. To a path, we assign its

complementary path. That is, if the original path has left turns at points (a1, b1), . . . , (ak, bk),

where the ai’s and bi’s are ascending, A = {a1 < · · · < ak} and B = {b1 < · · · < bk}; then its

complementary path has coordinates (a′1, b′1), . . . , (a

′m−k, b

′m−k), where a′1 < · · · < a′m−k, A

′ =

{a′1, . . . , a′m−k} = [m] − A and b′1 < · · · < b′m−k, B′ = {b′1, . . . , b′m−k} = {0, . . . ,m − 1} − B.

Figure 4.2 illustrates the bijection.

←→

A = {1, 2, 4, 6, 7, 9} A′ = {3, 5, 8, 10}

B = {0, 2, 3, 5, 6, 8} B′ = {1, 4, 7, 9}

Figure 4.2. A path and its associated complementary path.

39


Note: All the arguments of this section can be made for right turns instead of left turns

and the results are identical after the proper modifications.

4.7. Paths with arbitrarily long horizontal steps

Now we consider paths from (0, 0) to (m,n) with unit vertical steps (0, 1) and arbitrarily

long horizontal steps (k, 0), k ≥ 1. We will call these paths long horizontal paths or LHP for

brevity. See Figure 4.3 for some examples of LHPs.

Figure 4.3. The 9 LHPs from (0, 0) to (2, 2).

It is easy to see that the generating function for such paths is

(4.9) f3(x, y) =1

1− x

1− x− y

=1− x

1− 2x− y + xy.

Table 4.5 shows the first few coefficients of the Taylor expansion of f3(x, y).

Observe that the coefficients of the first sub-diagonal are identical to those of Table 4.1,

namely [xn+1yn]f3(x, y) = [xn+1yn]f1(x, y).

Theorem 4.7.1. The number of LHP from (0, 0) to (n+ 1, n) is equal to the number of

Delannoy paths to (n+ 1, n).

It is easy to prove this fact using a slight modification of the method of Jacobi’s change

of variables formula introduced before. However, we would like to give a more combinatorial

40


m\n 0 1 2 3 4 5 6

0 1 1 1 1 1 1 1

1 1 2 3 4 5 6 7

2 2 5 9 14 20 27 35

3 4 12 25 44 70 104 147

4 8 28 66 129 225 363 553

5 16 64 168 360 681 1182 1925

6 32 144 416 968 1970 3653 6321

Table 4.5. Number of LHP from (0, 0) to (m,n).

approach so first we introduce the parameter w. Define

(4.10) F3(x, y, w) =1− (w − 1)x

1− wx− y + (w − 1)xy,

which we will show also satisfies

(4.11) [xn+1yn]F3(x, y, w) = [xn+1yn]F1(x, y, w).

Furthermore, one can readily verify that F3(x, y, 2) = f3(x, y). Since F1(x, y, 2) = f1(x, y),

Theorem 4.7.1 will follow from (4.11).

The first few coefficients of xmyn in F3 are shown in Table 4.6.

Before continuing with LHP’s, we would like to interpret F3 as counting ordinary paths

with some weight depending on w. In this case, the weight is w raised to the number of

occurrences of consecutive horizontal steps. Let us think of a path (with ordinary steps) as

a word in X and Y , where (1, 0) is replaced with X and (0, 1) with Y . By cutting a word

before each occurrence of a Y we see that it consists of a possibly empty sequence of X’s

then possibly followed by a Y which is followed by a possibly empty sequence of X’s and

41


so on. In other words, we have a decomposition X∗(Y X∗)∗, where ∗ denotes an arbitrary

number of repetitions.

For ordinary paths this decomposition gives the generating function identity,

1

1− x· 1

1− y(

1

1− x

) =1

1− x− y.

If we want to weight every occurrence of XX by w we replace 1/(1− x) with

1 +x

1− wx= 1 + x+ wx2 + w2x3 + · · ·

A quick computation verifies that this replacement gives a factorization for F3,

F3(x, y, w) =

(1 +

x

1− wx

)· 1

1− y(

1 +x

1− wx

) .Therefore F3 counts paths weighted by w to the number of XX’s.

A horizontal segment of a path is a sequence of horizontal steps not preceded nor followed

by horizontal steps. The analog for words is what we call a run of X’s. In a horizontal

segment, the total number of X’s is the number of occurrences of XX plus one. Thus the

total number of X’s in a path is is the number of XX plus the number of horizontal segments.

m\n 0 1 2 3 4

0 1 1 1 1 1

1 1 2 3 4 5

2 w 2w + 1 3w + 3 4w + 6 5w + 10

3 w2 2w2+2w 3w2+6w+1 4w2+12w+4 5w2+20w+10

4 w3 2w3+3w2 3w3+9w2+3w 4w3+18w2+12w+1 5w3+30w2+30w+5

Table 4.6. [xmyn]F3(x, y, w).

42


Lemma 4.7.2. The number of ordinary paths to (m,n) with k horizontal segments is(m−1k−1

)(n+1k

).

Proof. We have(n+1k

)ways to choose the ordinates of the k segments. Then the lengths

of the segments are determined by choosing the abscissas of the ending points of the first

k − 1 segments since the starting point of the first segment has abscissa x = 0 and the

last segment has ending point with abscissa x = m. This gives(m−1k−1

)ways to choose the

endpoints and a total of(m−1k−1

)(n+1k

)ways to chose a path. �

Thus if we have k occurrences of XX and m X’s in total, the number of horizontal

segments is m− k, so there are(

m−1m−k−1

)(n+1m−k

)such paths. Therefore,

[xmyn]F3 =m−1∑k=0

(n+ 1

m− k

)(m− 1

m− k − 1

)wk =

m−1∑k=0

(n+ 1

n+ 1−m+ k

)(m− 1

k

)wk.

Setting m = n+ 1 gives,

[xn+1yn]F3 =n∑k=0

(n+ 1

k

)(n

k

)wk = Dn+1,n(w).

This shows the validity of equation (4.11).

Proof of Theorem 4.7.1. Set w = 2 in equation (4.11). Then

[xn+1yn]f3(x, y) = [xn+1yn]f1(x, y) = Dn+1,n

and the theorem follows. �

Now we establish the connection between LHP’s and F3. A calculation shows that

F3(x, y, w) =

(1− x

1− (w − 1)x− y)−1

.

43


If we set w = 1 + υ, then

F3(x, y, 1 + υ) =

(1− x

1− υx− y)−1

,

which is the generating function for LHP’s such that each horizontal step (k, 0) has a weight

of υk−1.

This result can be proven combinatorially. Given an ordinary path, we can choose to

glue or not glue consecutive horizontal unit steps to obtain an LHP. At each occurrence of

XX, if we glue we assign a weight of υ to the pair, otherwise we assign a weight of 1. In

this way, a block of k X’s all glued together arose from gluing k − 1 times and has a weight

of υk−1 and corresponds to the step (k, 0) in an LHP. An ordinary path with l occurrences

of XX corresponds to 2l LHP’s with weights ranging from 1 to υl and whose total weight

is (1 + υ)l = wl. Since F3(x, y, w) counts these paths by wl it follows that F3(x, y, 1 + υ)

counts LHP’s with weight υk−1 for each (k, 0) step. If an LHP has q horizontal steps then

its weight is υ to the power (k1 − 1) + · · ·+ (kq − 1) = m− q.

Note that the definition of LHP can be modified by switching the roles of horizontal and

vertical steps to obtain the following definition.

A path with arbitrarily long vertical steps is a path from (0, 0) to (m,n) with unit

horizontal steps (1, 0) and arbitrarily long vertical steps (0, k), k ≥ 1. Call these paths long

vertical paths or LVP for brevity. We let

F 3(x, y, w) = F3(y, x, w) =1− (w − 1)y

1− x− wy + (w − 1)xy=

(1− x− y

1− (w − 1)y

)−1be the weighted generating function for such paths. It is easy to see that all the arguments

for F3 can be made for F 3 and the results are identical after the proper modifications.

44


4.8. Paths with tailed long horizontal steps

Now we consider paths from (0, 0) to (m,n) with steps of the form (k, 1), k ≥ 0, with red

color and (k + 1, 1), k ≥ 0, with blue color. Alternatively these steps can be seen as a block

of k horizontal steps followed by a vertical step in the first case or a diagonal step in the

second case. When k > 0, the final vertical or diagonal step can be regarded as the ‘tail’ of

a long horizontal step. Thus it makes sense to call these paths tailed long horizontal paths

or TLHP for brevity.

Clearly any TLHP can be transformed into a Delannoy path by cutting the path at every

vertex. Furthermore, any nonempty Delannoy path whose last step is a non-horizontal step

can be decomposed into tailed long horizontal steps by joining all steps and then cutting at

the end of each diagonal or vertical step. The generating function for steps with a vertical

tail is y/(1− x) and for steps with a diagonal tail is xy/(1− x). If we give a weight of w− 1

to the diagonal tails only, then the generating function for TLHP’s is

F4(x, y, w) =

(1− y

1− x− (w − 1)xy

1− x

)−1=

1− x1− x− y − (w − 1)xy

= (1− x) · F1(x, y, w).(4.12)

Recall from Theorem 4.6.1 that F1(x, y, w) counts ordinary paths with a weight of w to

the number of left turns. Thus xF1 counts ordinary paths where the last step is a horizontal

step, with the same weight, since a left turn cannot end with X. Therefore, F4 = (1− x)F1

counts ordinary paths where the last step is a non-horizontal step with a weight of w to the

number of left turns.

45


LetGm,n(w) be the coefficient of xmyn in F4. In view of equation (4.12) we haveG1,n(w) =

D1,n(w) = 1 and for m ≥ 1,

(4.13) Gm,n(w) = Dm,n(w)−Dm−1,n(w).

Thus we can easily compute these polynomials by subtracting consecutive rows from

Table 4.3, the result is given in Table 4.8. When w = 2 we are essentially ignoring the

weight of the diagonal tails so we obtain the number of TLHP from (0, 0) to (m,n). These

numbers are given in Table 4.7. These are also the sequence [Slo, A266213].

m\n 0 1 2 3 4 5 6

0 1 1 1 1 1 1 1

1 0 2 4 6 8 10 12

2 0 2 8 18 32 50 72

3 0 2 12 38 88 170 292

4 0 2 16 66 192 450 912

5 0 2 20 102 360 1002 2364

6 0 2 24 146 608 1970 5336

Table 4.7. Number of TLHP from (0, 0) to (m,n).

m\n 0 1 2 3 4

0 1 1 1 1 1

1 0 w 2w 3w 4w

2 0 w w2 + 2w 3w2 + 3w 6w2 + 4w

3 0 w 2w2 + 2w w3 + 6w2 + 3w 4w3 + 12w2 + 4w

4 0 w 3w2 + 2w 3w3 + 9w2 + 3w w4 + 12w3 + 18w2 + 4w

Table 4.8. Gm,n(w).

46


Observe that the diagonal entries of Table 4.8 are related to the subdiagonal entries of

Tables 4.3 and 4.4. Explicitly, we have

Theorem 4.8.1. For m ≥ 1,

Gm,m(w) = wmDm,m−1(w−1) = Em,m−1(w).

Proof. The second equality follows from (4.7). To show the first equality we use equa-

tions (4.13) and (4.8) to compute

Gm,n(w) =m∑k=0

(m

k

)(n

k

)wk −

m−1∑k=0

(m− 1

k

)(n

k

)wk(4.14)

=m−1∑k=0

(m− 1

k

)(n

k + 1

)wk+1.

In particular when n = m,

Gm,m(w) =m−1∑k=0

(m− 1

k

)(m

k + 1

)wk+1

=m−1∑j=0

(m− 1

j

)(m

j

)wm−j = wmDm,m−1(w

−1).

�

4.9. Slanted paths

Now let us consider paths from (0, 0) to (m,n) with arbitrarily long slanted steps, that is,

steps of the form (p, q), with p, q ∈ P. We will call these paths slanted paths. The generating

function for the steps is then

x

1− x· y

1− y.

47


Thus the generating function for the slanted paths is

f5(x, y) =

(1− xy

(1− x)(1− y)

)−1.

After a quick algebraic manipulation we see that

f5(x, y) = 1 +xy

1− x− y.

Since (1− x− y)−1 is the generating function for ordinary paths, we arrive at the following

conclusion.

Theorem 4.9.1. The number of non-empty slanted paths from (0, 0) to (m,n) is equal

to the number of ordinary paths from (0, 0) to (m− 1, n− 1).

In fact we can give a direct bijection between these sets of paths. Given any nonempty

slanted path, to each of its steps draw a right angled triangle below it with catheti parallel

to the x-axis and y-axis. Cut each of these sides into unit horizontal and vertical steps

and replace the path with the sequence of unit steps thus obtained. Now observe that any

such path must start with an X step and end with a Y step. So we delete these extremal

steps and shift the whole path one unit to the left to obtain an arbitrary ordinary path to

(m− 1, n− 1). Clearly we can reverse the process starting with an ordinary path to obtain

a unique slanted path. Figure 4.4 illustrates the bijection.

Now we give a weight of w to each slanted step. The generating function becomes

F5(x, y, w) =

(1− wxy

(1− x)(1− y)

)−1,

and counts slanted paths by the number of steps. A computation shows that

(4.15) F5(x, y, w) = 1 +wxy

1− x− y − (w − 1)xy= 1 + wxy · F1(x, y, w).

48


←→ ←→

Figure 4.4. Bijection for slanted paths.

We know that F1 counts either left turns or right turns of an ordinary path. Under the

bijection above we see that the right turns of an ordinary path occur exactly at the points

where consecutive slanted steps meet. Thus the number of right turns in an ordinary path is

one less than the number of steps in its associated slanted path. This gives an interpretation

to equation (4.15). We do not provide a table for the coefficients of F5 since its entries are

practically a shift of the entries in Table 4.3 multiplied by w.

4.10. Paths with quasi-diagonal steps

For the next set of paths let the steps be of the form (n+1, n) and (n, n+1). Since these

steps look like a long diagonal step but are offset by one unit to the right or one unit up we

will say that these steps are quasi-diagonal. We will then call paths made with these steps

quasi-diagonal paths or QDP for brevity. We assign a weight of (w− 1)n to each of (n+ 1, n)

and (n, n+ 1).

Their steps generating function is

x

1− (w − 1)xy+

y

1− (w − 1)xy.

49


Thus the generating function for QDPs is

F6(x, y, w) =

(1− x+ y

1− (w − 1)xy

)−1(4.16)

=1− (w − 1)xy

1− x− y − (w − 1)xy= [1− (w − 1)xy]F1(x, y, w).

50

CHAPTER 5

Lattice path factorization and diagonal restrictions

5.1. Introduction

We are interested in studying paths analogous to the ones described in Chapter 4 but

restricted by the diagonal line y = x. One can start with a path with steps from a given set S

without restrictions and use the factorization method described below to obtain three paths.

One of the resulting paths will have the property of starting and ending on the diagonal and

never going above it. The other two paths will have similar interesting properties.

The decomposition into three types of subpaths gives rise to a factorization of the gener-

ating function. In general, calculating the factors of the generating function is complicated.

However, we focus on cases where finding the factors only requires solving a quadratic equa-

tion. We give explicit formulas for the generating functions of the subpaths and compute

the coefficients in the most interesting cases. Many of our cases give back the generating

functions for Catalan, Schroder, and Narayana numbers. We also describe combinatorial

interpretations when available.

5.2. Factorization of lattice paths

Given two paths p1 and p2, recall that their concatenation p1p2 is the sequence of steps

of p1 followed by the sequence of steps of p2. If a path can be written as a successive

concatenation of paths p = p1 · · · pn, we say that the sequence p1, . . . , pn is a factorization

(or decomposition) of p. Note that we regard each pi as a path starting at (0, 0).

51

CHAPTER 5. LATTICE PATH FACTORIZATION AND DIAGONAL RESTRICTIONS

Now we review a factorization for paths described in [Ges80]. The diagonal height of a

point (m,n) is the number n −m. Given any path p starting at (0, 0), let h be the largest

integer such that the line y = x + h intersects the path. Cut p at the first and last place it

meets this line. Call these points A and B respectively. Let p− be the subpath of p from

(0, 0) to A, p0 the translation to the origin of the subpath from A to B and p+ the translation

to the origin of the subpath starting at B and ending at the same point as p. Thus we have

a factorization of p as p−, p0, p+. See Figure 5.1. Note that h is equal to the maximum

diagonal height of the points in p. Thus we can say that h is the diagonal height of the path.

A

B

p

p− p0 p+

Figure 5.1. Path factorization.

By definition p− is a path that meets the line y = x+ b for the first time at its endpoint

A. Therefore it is either the empty path or a path whose endpoint is above the diagonal

and has diagonal height greater than that of any other point. We call a path of this form a

minus-path. We see that p0 is a path that starts and ends on the main diagonal and never

goes above it. A path of this form is called a zero-path. Finally, p+ is a path that always

52


stays below the main diagonal after (0, 0). A path with this property is called a plus-path.

Throughout this chapter we will use the adjective restricted to denote a path of one of these

three forms.

It is clear that if a path has a factorization p1p2p3 where p1, p2, and p3 are a minus-,

zero-, and plus-path respectively, then we must have p1 = p−, p2 = p0 and p3 = p+. Our

use of plus and minus might seem a little counterintuitive; a minus-path has an endpoint

above the diagonal while a plus-path has an endpoint below the diagonal. However the

convenience of this terminology will be evident in the next section. Alternatively, one can

see some geometric intuition by flipping p around the diagonal and transforming the path

so that the diagonal becomes the x-axis.

The following lemma gives a factorization for a power series in y with coefficients in

C((t)) by collecting powers of t.

Lemma 5.2.1. [Ges80, Thm. 4.1] Let f be a power series in y with coefficients in C((t))

such that [y0]f ∈ 1+tC[[t]]. Then f has a unique factorization f = f−f0f+, where f−, f0, and

f+ are in C((t))[[y]] and the terms are of the form f− = 1+∑

i,j>0 ai,jyit−j, f0 = 1+

∑i>0 aiy

i,

and f+ = 1 +∑

i≥0,j>0 ai,jyitj.

Proof. Let h = log f . Collect the terms of h with negative powers of t, constant with

respect to t, and with positive powers of t. Call these sums h−, h0, and h+ respectively, and

set f− = eh− , f0 = eh0 , and f+ = eh+ . The construction shows that f−, f0, and f+ have the

required form and uniqueness is verified by taking the logarithm of any two decompositions

and comparing term by term. �

Now we can apply this factorization to generating functions for paths. For a path p we

define m(p) and n(p) as the x-coordinate and y-coordinate, respectively, of the last point in

53


the path. Thus, if Q is a set of paths we can define its generating function in x and y as

∑q∈Q

xm(q)yn(q).

For our purposes it will be more convenient to replace x by t and y by y/t. Thus we define

Γ(Q) =∑q∈Q

tm(q)(yt

)n(q)=∑q∈Q

tm(q)−n(q)yn(q).

Let S be a set of steps for building paths starting at (0, 0). Let P = S∗ be the set of

paths with steps in S, and let P−, P0 and P+, be the sets of minus-, zero-, and plus-paths

with steps in S.

The generating function for P is

Γ(P ) =

(1−

∑s∈S

tm(s)−n(s)yn(s)

)−1

Lemma 5.2.2. We have Γ(P ) = Γ(P−)Γ(P0)Γ(P+).

Proof. Since for each p in P there is a unique decomposition p = p−p0p+ with p− ∈ P−,

p0 ∈ P0, and p+ ∈ P+, the result follows. �

Theorem 5.2.3. [Ges80, Thm. 4.4] We have Γ(P−) = Γ(P )−, Γ(P0) = Γ(P )0, Γ(P+) =

Γ(P )+.

Proof. Lemma 5.2.1 gives the factorization Γ(P ) = Γ(P )−Γ(P )0Γ(P )+, while Lemma

5.2.2 gives Γ(P ) = Γ(P−)Γ(P0)Γ(P+). The empty path is in all of P , P−, P0 and P+ and

corresponds to the term 1 in each of the generating functions. Other than the term 1, we

realize that Γ(P−) has only terms with negative powers of t since the paths end above the

diagonal, Γ(P0) has no powers of t since all paths end in the diagonal and Γ(P+) has only

54


terms with positive powers of t since paths end below the diagonal. Thus the result follows

by uniqueness of the factorization. �

Theorem 5.2.3 gives a powerful tool for computing generating functions of restricted

paths in particular cases when the algebra of the factorization is particularly simple. One

such case is what we call the quadratic case, which we describe in the following section.

5.3. Computing f−, f0 and f+ in the quadratic case

Lemma 5.3.1. Let

f =1

1− t− y/t− z,

where z is a power series in y with constant term 0. Then

f− =

(1−

1− z −√

(1− z)2 − 4y

2t

)−1,

f0 =1− z −

√(1− z)2 − 4y

2y,

f+ =

(1− t

1− z −√

(1− z)2 − 4y

2y

)−1.

Proof. Since z has no terms in t, we can rewrite the denominator of f as a quadratic

polynomial in t.

f =t

t− t2 − y − zt=

t

−(t2 − (1− z)t+ y).

Let r1 and r2 be the roots of the polynomial t2 − (1− z)t+ y. Thus we have

f =t

−(t− r1)(t− r2)=

t

(t− r1)(r2 − t)

=1

(1− r1/t)· 1

(r2 − t)=

1

(1− r1/t)· 1

r2· 1

(1− t/r2)

=1

(1− r1/t)· r1y· 1

(1− tr1/y).

55


In the last line of the previous equation array we eliminated r2 using the fact that r1r2 = y.

For r1/y to be a power series in y, we must have

r1 =1− z −

√(1− z)2 − 4y

2,

as opposed to the other root. This is because 1 − z −√

(1− z)2 − 4y has constant term 0

and 1 − z +√

(1− z)2 − 4y has constant term 2. It is now easy to see that (1 − r1/t)−1,

r1/y, and (1− tr1/y)−1 have the desired form as in Lemma 5.2.1. �

5.4. Restricted Delannoy paths

Let us go back to our study of Delannoy paths, which have the step set S = {(1, 0), (0, 1), (1, 1)}

and generating function f1(x, y) = (1 − x − y − xy)−1. We apply the factorization to

f1(t, y/t) = (1− t− y/t− y)−1 and set z = y in Lemma 5.3.1 to obtain

Γ(P−) =

(1− 1− y −

√1− 6y + y2

2t

)−1,

Γ(P0) =1− y −

√1− 6y + y2

2y,

Γ(P+) =

(1− t 1− y −

√1− 6y + y2

2y

)−1.

Recall that a Schroder path of order n in “Cartesian coordinates” is a path from (0, 0)

to (n, n) with horizontal, vertical and diagonal unit steps that never goes above the main

diagonal. Thus by definition the type zero Delannoy paths are the Schroder paths. The

number of Schroder paths of order n is called the n-th Schroder number. The first few

Schroder numbers are 1, 2, 6, 22, 90, 394, 1806, 8558, . . . See [Slo, A006318].

56


As expected, the generating function Γ(P0) is the Schroder number generating function,

given in Section 3.4.5 as1− x−

√(1− x)2 − 4x

2x.

In fact in Section 3.4.5 we discussed Schroder polynomials, rυ which arise from giving each

path a weight of υ to the number of diagonal steps. Let P υ = {(1, 0), (0, 1), (1, 1)}∗, where

(1, 1) has a weight of υ. From Lemma 5.3.1 with z = υy we obtain

Γ(P υ) =1

1− t− y/t− υy,

Γ(P υ−) =

(1−

1− υy −√

(1− υy)2 − 4y

2t

)−1,

Γ(P υ0 ) =

1− υy −√

(1− υy)2 − 4y

2y,

Γ(P υ+) =

(1− t

1− υy −√

(1− υy)2 − 4y

2y

)−1.

Let rυ = Γ(P υ0 ), and observe that

Γ(P υ−) =

(1− y

trυ

)−1=∞∑l=0

yl

tlrlυ

and

Γ(P υ+) = (1− trυ)−1 =

∞∑l=0

tlrlυ.

So we need only compute the coefficients of rlυ to count all three types of subpaths.

Taking the logarithm of both sides of the equation Γ(P υ) = Γ(P υ−)Γ(P υ

0 )Γ(P υ+) we obtain

log1

1− t− y/t− υy= log

(1− y

trυ

)−1+ log rυ + log (1− trυ)−1 .

57


The first two logarithms on the right do not contain positive powers of t. Thus for l ≥ 1,

[tl] log1

1− t− y/t− υy= [tl] log

1

1− trυ,

[tl]∞∑m=1

1

m(t+ y/t+ υy)m = [tl]

∞∑m=1

tm

mrmυ =

1

lrlυ.

Expanding the multinomial we get

∞∑m=1

1

m(t+ y/t+ υy)m =

∑i+j+k=m

1

m

(m

i, j, k

)ti−jyj+kυk.

Set l = i− j and n = k+ j, so j = n− k, i = l+ n− k, and m = l+ 2n− k. Extracting the

tl coefficient gives

rlυ = l∑n,k

1

l + 2n− k

(l + 2n− k

l + n− k, n− k, k

)ynυk.

Thus

(5.1) [ynυk]rlυ =l(l + 2n− k − 1)!

(l + n− k)!(n− k)!k!=

l

l + n− k

(l + 2n− k − 1

n

)(n

k

).

The numbers in (5.1) are what we call generalized Schroder numbers.

In particular, the coefficient of yn in Γ(P υ0 ) is the n-th Schroder polynomial Schn(υ). Let

Schn(υ) =n∑k=0

Sch(n, k)υk.

From (5.1) with l = 1 we obtain the explicit formula

(5.2) Sch(n, k) =1

n− k + 1

(2n− kn

)(n

k

).

Theorem 5.4.1. The number of Schroder paths of order n with exactly k diagonal steps

is1

n− k + 1

(2n− kn

)(n

k

).

58


Proof. This follows immediately from the definition of Schn(υ) and the previous for-

mula. �

Note: An alternative proof of Theorem 5.4.1 follows from inserting k diagonal steps into

a type zero ordinary path from (0, 0) to (n− k, n− k) (in other words a Dyck path). There

are Cn−k such paths and there are(2n−kk

)ways of inserting the diagonal steps. This gives a

total of

Cn−k

(2n− kk

)=

1

n− k + 1

(2n− 2k

n− k

)(2n− kk

)=

1

n− k + 1· (2n− 2k)!

(n− k)!(n− k)!· (2n− k)!

(2n− 2k)!k!· n!

n!

=1

n− k + 1· (2n− k)!

(n− k)!n!· n!

(n− k)k!

=1

n− k + 1

(2n− kn

)(n

k

).

The coefficients of the first few Schroder polynomials are shown in Table 5.1. For a larger

list of Schroder polynomials consult [Slo, A060693].

n\k 0 1 2 3 4 5 6 7

0 1 0 0 0 0 0 0 0

1 1 1 0 0 0 0 0 0

2 2 3 1 0 0 0 0 0

3 5 10 6 1 0 0 0 0

4 14 35 30 10 1 0 0 0

5 42 126 140 70 15 1 0 0

6 132 462 630 420 140 21 1 0

7 429 1716 2772 2310 1050 252 28 1

Table 5.1. Coefficients of Schroder polynomials.

59


Note that summing over the n-th row is equivalent to computing Schn(1) which is the

n-th Schroder number.

5.5. Restricted paths by left turns

Recall that given a path with horizontal, vertical and diagonal steps (a Delannoy path),

we can construct an ordinary path by replacing all diagonal steps with left turns. Conversely

to go from ordinary paths to Delannoy paths just replace a subset of the left turns by diagonal

steps. Furthermore, we have shown in Section 4.5 that

F1(x, y, w) =1

1− x− y − (w − 1)xy

counts ordinary paths with a weight of w to the number of left turns. (See Theorem 4.6.1.)

In fact the parameter number of left turns is compatible with the factorization into negative,

zero, and positive parts. That is, if one cuts the path into its corresponding subpaths, the

cuts are never on the middle of a left turn, and thus the corresponding number of left turns

in each of the subpaths remains unchanged. That is, (F1)−, (F1)0, and (F1)+ count negative,

zero, and positive paths by number of left turns. We can compute (F1)−, (F1)0, and (F1)+ by

factorization or simply by substituting υ = w− 1 in Γ(P υ−), Γ(P υ

0 ), and Γ(P υ+), respectively.

In particular Schroder paths correspond to paths that stay below the diagonal by number of

left turns. Recall that the Narayana numbers

Nar(n, k) =1

n

(n

k

)(n

k − 1

),

count (among other objects) the number of n-Dyck paths with k peaks. By an appropriate

rotation of the diagonal and flipping the paths we see that this is equivalent to counting

ordinary zero-paths by number of left turns. Let ηw stand for the Narayana generating

60


function. Namely,

ηw = 1 +∞∑n=1

n∑k=1

Nar(n, k)wkyn.

The substitution υ = w − 1 gives

Γ(P ) =1

1− t− y/t− (w − 1)y,

ηw = Γ(P0) =1− (w − 1)y −

√(1− y − wy)2 − 4wy2

2y,

Γ(P−) =(

1− y

tηw

)−1,

Γ(P+) = (1− tηw)−1 .

Thus taking logarithms we arrive at

ηlw = l [tl] log1

1− t− y/t− (w − 1)y

= l [tl] log

1

(1− t)(1− y/t)

1

1− wy

(1− t)(1− y/t)

= l [tl]

[− log(1− t)− log(1− y/t)− log

(1− wy

(1− t)(1− y/t)

)]= 1 + l [tl]

∞∑k=1

1

k

wkyk

(1− t)k(1− y/t)k.

Thus

[wkyn]ηlw = [yntl]l

k

yk

(1− t)k(1− y/t)k

= [yn−ktl]l

k

1

(1− t)k(1− y/t)k

=l

k[tl]

1

(1− t)k

(n− 1

n− k

)1

tn−k

61


=l

k

(n− 1

n− k

)[tl+n−k]

1

(1− t)k

=l

k

(n− 1

n− k

)(l + n− 1

l + n− k

)=

l

n

(n

k

)(l + n− 1

k − 1

).

When l = 1, we recover the Narayana numbers Nar(n, k). When l = 2, we obtain the

sequence [Slo, A108838].

5.6. Restricted paths by right turns

We define a right turn to be a vertical step followed by a horizontal step. Let f−, f0 and

f+ count paths of negative, zero and positive type, respectively, weighted by w raised to the

number of right turns. The same argument used to show that

F1(x, y, w) =1

1− x− y − (w − 1)xy

counts ordinary paths by number of left turns can be used to show it counts paths by right

turns. However, applying the factorization directly to this generating function does not

work, since an extra right turn occurs where consecutive nonempty subpaths meet. That is,

if p is a path and p−, p0, and p+ are its respective subpaths, then every right turn of p−, p0

or p+ is a right turn of p, but p also has a right turn whenever a nonempty subpath p− or

p0 is immediately followed by a nonempty p0 or p+. So the number of extra right turns of a

nonempty p is one less than the number of nonempty components of p.

We define g−, g0 and g+ by f− = 1 + g−, f0 = 1 + g0 and f+ = 1 + g+ that is, the g

generating functions do not count the empty path. In view of the previous remark we have

that

F1 = 1 + (g− + g0 + g+) + w(g−g0 + g0g+ + g−g+) + w2g−g0g+

= 1 +(1 + wg−)(1 + wg0)(1 + wg+)− 1

w.

62


Rearranging we arrive at

(1 + wg−)(1 + wg0)(1 + wg+) = 1− w + wF1(5.3)

= 1− w +w

1− x− y − (w − 1)xy,

which is the desired factorization. An alternative way to see this is to declare that every

nonempty path has a virtual right turn at the endpoint. This new weight by total number

of right turns is then compatible with the factorization since the virtual turns are preserved

when breaking the path. Multiplying f−, f0, f+ and F1 by w and then adjusting for the

empty path gives equation (5.3). Observe that

1− w +w

1− x− y − (w − 1)xy=

[1 + y(w − 1)][1 + x(w − 1)]

1− x− y − (w − 1)xy.

The (x, y)→ (t, y/t) substitution gives

Γ(P ) =[1 + (y/t)(w − 1)][1 + t(w − 1)]

1− t− y/t− (w − 1)y,

So clearly the factors of Γ(P ) are those of the Narayana case but modified by the factors in

the numerator each corresponding to the minus and plus parts. Thus letting ηw be as in the

previous section.

Γ(P−) =(

1 +y

t(w − 1)

)(1− y

tηw

)−1,

Γ(P0) = ηw,

Γ(P+) = [1 + t(w − 1)] (1− tηw)−1 .

63


After a minor abuse of notation we write f0 in terms of the variables following the substitu-

tion, namely w and y. We have

(5.4) f0 = 1 + g0 = 1 + (ηw − 1)/w = 1 +∞∑n=1

n∑k=1

Nar(n, k + 1)wkyn.

Theorem 5.6.1. The number of zero-paths from (0, 0) to (n, n) with k right turns is the

shifted Narayana number

Nar(n, k + 1) =1

n

(n

k

)(n

k + 1

).

Proof. This follows immediately from equation (5.4). �

Note: An alternative proof of this theorem follows from the interpretation of zero-paths

as Dyck paths. Left turns correspond to peaks and right turns correspond to valleys. The

result follows from the fact that the number of peaks in a Dyck path is equal to the number

of valleys plus one.

Now we compute

[tl]Γ(P+) = ηlw + ηl−1w (w − 1),

so

[wkyntl]Γ(P+) = [wkyn]ηlw + [wk−1yn]ηl−1w − [wkyn]ηl−1w

=l

n

(n

k

)(l + n− 1

k − 1

)+l − 1

n

(n

k − 1

)(l + n− 2

k − 2

)− l − 1

n

(n

k

)(l + n− 2

k − 1

).

64


Let l = m− n (the number of horizontal steps minus vertical steps). Then

l

n

(n

k

)(l + n− 1

k − 1

)=m− nn

(n

k

)(m− 1

k − 1

)=

(n− 1

k − 1

)(m

k

)−(n

k

)(m− 1

k − 1

)So we can rewrite the previous equation as

[wkxmyn]Γ(P+) =

(n− 1

k − 1

)(m

k

)−(n

k

)(m− 1

k − 1

)+

(n− 1

k − 2

)(m− 1

k − 1

)−(

n

k − 1

)(m− 2

k − 2

)−(n− 1

k − 1

)(m− 1

k

)+

(n

k

)(m− 2

k − 1

).

Pair the first and fifth terms, and the second and last, and use Pascal’s recurrence to obtain(n− 1

k − 1

)(m− 1

k − 1

)−(n

k

)(m− 2

k − 2

)+

(n− 1

k − 2

)(m− 1

k − 1

)−(

n

k − 1

)(m− 2

k − 2

).

Now pair the first and third terms, and the second and fourth terms to get(n

k − 1

)(m− 1

k − 1

)−(n+ 1

k

)(m− 2

k − 2

).

Finally recall that since f+ = 1 + g+ = 1 + (Γ(P+)− 1)/w, we need to shift the index k by

one. Thus we arrive at the following theorem.

Theorem 5.6.2. The number of paths from (0, 0) to (m,n) with m > n that always stay

below the diagonal with k right turns is

R(m,n, k) =

(n

k

)(m− 1

k

)−(n+ 1

k + 1

)(m− 2

k − 1

).

For other proofs of this theorem see [Kra97] and [GS03].

We do not compute the coefficients for f− since in this case there is a symmetry between

f+ and f− resulting from interchanging x and y.

We know that f0 counts paths that start at (0, 0), end on the diagonal and do not go

above the diagonal, weighted by w to the number of right turns. The substitution w = 1 +υ

65


corresponds to replacing a subset of the right turns with diagonal steps. Observe that this

process won’t give any diagonal steps on the diagonal line since the endpoints of a right turn

must be below the diagonal.

A Schroder path with no diagonal steps on the diagonal x = y is called a small Schroder

path. Thus f0(1 + υ) counts small Schroder paths weighted by υ to the number of diagonal

steps. Let us call this generating function sυ. Using the notation established above we have

sυ = f0(1 + υ) = 1 +rυ − 1

1 + υ=rυ + υ

1 + υ=

1 + υy −√

(1− υy)2 − 4y

2(1 + υ)y.

In particular when υ = 1 we are simply counting small Schroder paths so s1 = f0(2) =

(1 + r1)/2 gives the small Schroder numbers. This gives rise to the following theorem.

Theorem 5.6.3. The number of small Schroder paths of order n > 0 is equal to half

the n-th Schroder number. When n = 0, both the small Schroder number and the Schroder

number are equal to one, corresponding to the empty path.

Note: We can give a combinatorial proof of Theorem 5.6.3 as in [Ges09] by constructing

a bijection between the set D of Schroder paths of order n > 0 with at least one diagonal

step on the diagonal and the set N of Schroder paths of order n with no diagonal steps on

the diagonal. Given a path in D we can factor it as PDQ where D is the last diagonal step

on the diagonal, so Q is in N. Now we replace PDQ with XPY Q where X is a horizontal

step and Y is a vertical step. Then XPY Q is in N. Given a path in N we locate the first

vertical step to return to the diagonal line and let this be the Y in XPY Q, then simply

reverse the steps to get a path in D. The result follows. Figure 5.2 illustrates the bijection.

Now we compute the coefficients for f+(1 + υ). Recall that f+(w) counts paths that

start at (0, 0) and stay below the diagonal, weighted by w to the number of right turns. So

f+(1 +υ) counts these paths after a subset of the right turns is replaced with diagonal steps.

Since the corners of a right turn can’t touch the diagonal, at best they can intersect the line

66


P

Q ←→

P

Q

Figure 5.2. Bijection for small Schroder paths.

y = x−1. Even in such a case the endpoints of a right turn must be below the line y = x−1.

Thus this process won’t give any diagonal steps on the line y = x− 1 or above it.

f+(w) = 1 +

(1 + t(w − 1)

1− tηw− 1

)1

w.

So

f+(1 + υ) = 1 +

(1 + tυ

1− trυ− 1

)1

1 + υ

= 1 +

(∑l≥1

tlrlυ +∑l≥1

tlυrl−1υ

)1

1 + υ

= 1 +∑l≥1

tlrυ + υ

1 + υrl−1υ

= 1 +∑l≥1

tlsυrl−1υ .

Thus for l ≥ 1, [tl]f+(1 + υ) = sυrl−1υ and [ynυktl]f+(1 + υ) = [ynυk]sυr

l−1υ . Rather than

computing [ynυk]sυrl−1υ directly, we compute the sightly more general case [ynυk]saυr

bυ and

specialize afterwards.

From the decomposition of a small Schroder path into primes we know that

sυ =1

1− yrυ.

67


To compute [ynυk]saυrbυ = [ynυk](1− yrυ)−arbυ we use the method of Lagrange inversion.

Specifically, we use Formula (2.1.2) of Theorem 2.1.1 in [Ges16] which we copy here as a

lemma.

Lemma 5.6.4 (Lagrange inversion). Let R(t) be a power series not involving x. Then

there is a unique power series f = f(x) such that f(x) = xR(f(x)), and for any Laurent

series φ(t) not involving x and any integer m we have

(5.5) [xm]φ(f) = [tm] (1− tR′(t)/R(t))φ(t)R(t)m.

Recall that rυ satisfies rυ = (1− υy − yrυ)−1. Thus we let

R(t) =1

1− υy − yt.

Then f(x, υ, y) = xrυ(υ/x, xy) satisfies f(x) = xR(f(x)) so we can use the lemma to compute

[xm]φ(f), where

φ(t) =tb

(1− yt)a.

Then we set x = 1 since f(1) = rυ. We have

R′(t) =y

(1− υy − yt)2

and

R′(t)

R(t)=

y

1− υy − yt= yR.

Equation (5.5) gives

[xm]φ(f) = [tm] (1− tyR(t))tb

(1− yt)aR(t)m(5.6)

= [tm]tb

(1− yt)aR(t)m − [tm]

tb+1y

(1− yt)aR(t)m+1

68


= [tm−b]R(t)m

(1− yt)a− y[tm−b−1]

R(t)m+1

(1− yt)a.

We compute

[tc]R(t)d

(1− yt)a= [tc]

1

(1− yt)a· 1

(1− υy − yt)d

= [tc]1

(1− yt)a+d· 1

[1− υy/(1− yt)]d

= [tc] (1− yt)−(a+d)(

1− υy

1− yt

)−d= [tc] (1− yt)−(a+d)

∑k≥0

(d+ k − 1

k

)υkyk

(1− yt)k

= [tc]∑k≥0

(d+ k − 1

k

)υkyk

(1− yt)a+d+k

= [tc]∑k,j≥0

(d+ k − 1

k

)(a+ d+ k + j − 1

j

)υkyk+jtj

=∑k≥0

(d+ k − 1

k

)(a+ d+ k + c− 1

c

)υkyk+c.

Thus equation (5.6) becomes

[xm]φ(f) =∑k≥0

(m+ k − 1

k

)(a+m+ k +m− b− 1

m− b

)υkyk+m−b(5.7)

− y∑k≥0

(m+ 1 + k − 1

k

)(a+m+ 1 + k +m− b− 1− 1

m− b− 1

)υkyk+m−b−1

=∑k≥0

[(m+ k − 1

k

)(a+ 2m+ k − b− 1

m− b

)

−(m+ k

k

)(a+ 2m+ k − b− 1

m− b− 1

)]υkyk+m−b.

69


Setting x = 1 in equation (5.7) we recover rυ from f and saυrbυ from φ(f). Thus

saυrbυ =

∑k≥0

[(m+ k − 1

k

)(a+ 2m+ k − b− 1

m− b

)

−(m+ k

k

)(a+ 2m+ k − b− 1

m− b− 1

)]υkyk+m−b.

Set n = k +m− b, so m = n− k + b. Extracting coefficients in the previous equation gives

the following lemma.

Lemma 5.6.5. The coefficient of ynυk in saυrbυ is(

n+ b− 1

k

)(a+ b+ 2n− k − 1

n− k

)−(n+ b

k

)(a+ b+ 2n− k − 1

n− k − 1

).

If we specialize Lemma 5.6.5 to the case a = 1, b = l − 1 we obtain

[ynυk]sυrl−1υ =

(n+ l − 2

k

)(2n+ l − k − 1

n− k

)−(n+ l − 1

k

)(2n+ l − k − 1

n− k − 1

).

In particular when l = 1 we obtain the coefficients of the small Schroder polynomials

[ynυk]sυ =

(n− 1

k

)(2n− kn− k

)−(n

k

)(2n− kn− k − 1

)=

1

n+ 1

(n− 1

k

)(2n− kn

).

Assume m > n and let l = m − n (the number of horizontal steps minus vertical steps).

Reversing the (x, y)→ (t, y/t) substitution gives

[xmynυk]f+(x, y, 1 + υ) = [ynυk]sυrm−n−1υ

=

(m− 2

k

)(m+ n− k − 1

n− k

)−(m− 1

k

)(m+ n− k − 1

n− k − 1

).

Thus we arrive at the following theorem.

70


Theorem 5.6.6. The number of paths from (0, 0) to (m,n) with m > n > 0, with

horizontal and vertical steps that always stay below the diagonal, and with k diagonal steps

all located below the line y = x− 1 is(m− 2

k

)(m+ n− k − 1

n− k

)−(m− 1

k

)(m+ n− k − 1

n− k − 1

).

Note that when k = 0 in Theorem 5.6.6 one recovers the ballot numbers

B(m,n) =m− nm+ n

(m+ n

m

).

The computations for f−(1 + υ) are omited due to symmetry.

5.7. Restricted long horizontal paths

Recall that an LHP is a path from (0, 0) to (m,n) with unit vertical steps (0, 1) and

arbitrarily long horizontal steps (k, 0), k ≥ 1. We give a weight of 1 to the vertical steps

and a weight of υk−1 to a horizontal step (k, 0). Thus the steps generating function is

x/(1− υx)− y, and the generating function for LHPs is(1− x

1− υx− y)−1

.

The substitution (x, y)→ (t, y/t) gives

Γ(P ) =

(1− t

1− υt− y/t

)−1.

We apply the factorization method to obtain

Γ(P−) =(

1− y

tsυ

)−1,

Γ(P0) = sυ,

71


Γ(P+) =1− υt

1− t(υ + rυ).

We see that zero-paths are counted by small Schroder polynomials and minus-paths are

counted by powers of the small Schroder polynomial generating function. The coefficient of

ynυk in saυ can be obtained from Lemma 5.6.5. We have

[ynυk]saυ =

(n− 1

k

)(a+ 2n− k − 1

n− k

)−(n

k

)(a+ 2n− k − 1

n− k − 1

).

In particular setting υ = 1 we arrive at the following theorem.

Theorem 5.7.1. The number of restricted long horizontal paths of zero type from (0, 0)

to (n, n) is equal to the n-th small Schroder number.

5.8. Restricted tailed long horizontal paths

Recall that a tailed long horizontal path, TLHP, has steps of the form (n, 1), n ≥ 0 with

red color and (n + 1, 1), n ≥ 0 with blue color with a weight of u. Alternatively these steps

can be seen as a block of n horizontal steps followed by a vertical step in the first case or a

diagonal with weight u in the second case. The steps generating function is

S =y

1− x+

uxy

1− x.

The generating functions for restricted paths are

Γ(P−) = (1− yrυ/t)−1 ,

Γ(P0) = rυ,

Γ(P+) =1− t

1− trυ.

Any nonempty minus- or zero-path ends with a diagonal or vertical step, and the cutting and

pasting process of Section 4.8 explains why the first two generating functions coincide with

72


the generating functions for Delannoy paths. Applying the transformation to plus-paths

gives Delannoy plus-paths that do not end with a horizontal step and thus are counted by

(1− t)/(1− trυ).

In particular for zero-paths we conclude with the following theorem.

Theorem 5.8.1. The number of restricted tailed long horizontal paths of zero type from

(0, 0) to (n, n) is equal to the n-th Schroder number.

5.9. Restricted slanted paths

Recall that slanted paths are paths from (0, 0) to (m,n) with steps of the form (p, q),

with p, q ∈ P, where we give a weight of w to each step. The generating function is

F5(x, y, w) =

(1− wxy

(1− x)(1− y)

)−1,

which can be rewritten as

F5(x, y, w) =(1− x)(1− y)

1− x− y − (w − 1)xy= (1− x)(1− y)F1(x, y, w).

Thus

F5(t, y/t, w) = (1− t)(1− y/t)F1(t, y/t, w),

and we have

Γ(P−) =1− y/t

1− y/tηw,

Γ(P0) = ηw,

Γ(P+) =1− t

1− tηw.

We arrive at the following conclusion.

Theorem 5.9.1. The number of restricted slanted paths of zero type from (0, 0) to (n, n)

with k steps is equal to the Narayana number Nar(n, k).

73


5.10. Restricted quasi-diagonal paths

Recall that a quasi-diagonal path or QDP is a path whose steps are of the form (n+1, n)

and (n, n + 1) each weighted by (w − 1)n. Let υ = w − 1 so that a quasi-diagonal step of

“length” n has weight un. Substituting w = υ + 1 in equation (4.16) gives

(5.8) F6(x, y, υ + 1) = (1− υxy)F1(x, y, υ + 1).

Thus the generating functions for the restricted subpaths are those of the restricted Delannoy

paths but modified by the factor (1 − υxy), which belongs to the zero-type generating

function. Therefore the generating functions are

Γ(P−) =(

1− y

trυ

)−1,

Γ(P0) = (1− υy)rυ,

and

Γ(P+) = (1− trυ)−1 .

For the zero-type QDPs we have

[ynυk](1− υy)rυ = [ynυk]rυ − [ynυk]υyrυ = Sch(n, k)− Sch(n− 1, k − 1).

In particular setting υ = 1 we conclude with the following theorem.

Theorem 5.10.1. The number of restricted quasi-diagonal paths of zero type from (0, 0)

to (n, n) with n > 0 is equal to the difference of the n-th Schroder number minus the n−1-th

Schroder number.

74

Bibliography

[Cal04] D. Callan, Two bijections for Dyck path parameters, ArXiv Mathematics e-prints(2004).

[Deu99] Emeric Deutsch, Dyck path enumeration, Discrete Math. 204 (1999), no. 1-3, 167–202.

[DV84] Marie-Pierre Delest and Gerard Viennot, Algebraic languages and polyominoes enu-meration, Theoret. Comput. Sci. 34 (1984), no. 1-2, 169–206. MR 774044

[Ges80] Ira M. Gessel, A factorization for formal Laurent series and lattice path enumeration,Journal of Combinatorial Theory, Series A 28 (1980), no. 3, 321–337.

[Ges87] , A combinatorial proof of the multivariable Lagrange inversion formula, Jour-nal of Combinatorial Theory, Series A 45 (1987), no. 2, 178–195.

[Ges09] , Schroder numbers, large and small, http://people.brandeis.edu/

~gessel/homepage/slides/schroder.pdf, 2009.[Ges16] , Lagrange inversion, Journal of Combinatorial Theory, Series A 144 (2016),

212–249.[GJ83] Ian P. Goulden and David M. Jackson, Combinatorial Enumeration, Wiley, New

York, 1983.[GS03] I. P. Goulden and Luis G. Serrano, Maintaining the spirit of the reflection principle

when the boundary has arbitrary integer slope, J. Combin. Theory Ser. A 104 (2003),no. 2, 317–326.

[Het06] Gabor Hetyei, Central Delannoy numbers and balanced Cohen-Macaulay complexes,Annals of Combinatorics 10 (2006), no. 4, 443–462.

[Kra97] C. Krattenthaler, The enumeration of lattice paths with respect to their numberof turns, Advances in combinatorial methods and applications to probability andstatistics, Stat. Ind. Technol., Birkhauser Boston, Boston, MA, 1997, pp. 29–58.

[Mac84] Percy Alexander MacMahon, Combinatory Analysis, Courier Dover Publications,1984.

[RSZ] Amitai Regev, Nathaniel Shar, and Doron Zeilberger, A very short (bijective!)proof of Touchard’s Catalan identity, http://www.math.rutgers.edu/~zeilberg/mamarim/mamarimPDF/touchard.pdf.

[Sha76] Louis W. Shapiro, A short proof of an identity of Touchard’s concerning Catalannumbers, Journal of Combinatorial Theory, Series A 20 (1976), no. 3, 375–376.

[Slo] N. J. A. Sloane, The On-Line Encyclopedia of Integer Sequences, http://oeis.org.[Sta04] Richard P. Stanley, Enumerative Combinatorics. Volume 2, Cambridge University

Press, Cambridge, U.K.; New York, 2004.

75

http://people.brandeis.edu/~gessel/homepage/slides/schroder.pdf

http://people.brandeis.edu/~gessel/homepage/slides/schroder.pdf

http://www.math.rutgers.edu/~zeilberg/mamarim/mamarimPDF/touchard.pdf

http://www.math.rutgers.edu/~zeilberg/mamarim/mamarimPDF/touchard.pdf

http://oeis.org

[Sta15] , Catalan numbers, Cambridge University Press, 2015.[STT07] A. Sapounakis, I. Tasoulas, and P. Tsikouras, Counting strings in Dyck paths,

Discrete Mathematics 307 (2007), no. 23, 2909–2924.[Sul03] Robert A. Sulanke, Objects counted by the central Delannoy numbers, J. Integer Seq.

6 (2003), no. 1, Article 03.1.5, 19 pp.[Tou24] J. Touchard, Sur certaines equations fonctionnelles, Proc. Int. Math. Congress

Toronto 1 (1924), 465–472.[Wan11] Chao-Jen Wang, Applications of the Goulden-Jackson cluster method to counting

Dyck paths by occurrences of subwords, ProQuest LLC, Ann Arbor, MI, 2011, Thesis(Ph.D.)–Brandeis University. MR 2942232

76

Documents

Lattice path enumeration and factorizationpeople.brandeis.edu/~gessel/homepage/students/lemusthesis.pdf · I would like to thank my advisor, Ira Martin Gessel, for the constant support,