Journal of Combinatorial Theory, Series A 130 (2015) 129–149

Contents lists available at ScienceDirect

Journal of Combinatorial Theory, Series A

www.elsevier.com/locate/jcta

Signed excedance enumeration in classical and affine Weyl groups

Pietro Mongelli 1

a r t i c l e i n f o a b s t r a c t

Article history:Received 25 March 2013Available online 19 November 2014

Keywords:Signed excedanceAffine Weyl groupsColored excedanceAbsolute excedance

Based on the notions of colored and absolute excedances introduced by Bagno and Garber and their affine versions introduced by Mongelli, we compute the signed generating function of such statistics. Moreover, whenever possible, we derive a combinatorial interpretation of the coefficients of such generating functions. This paper is inspired by a paper of S. Sivasubramanian in which the author enumerates signed statistics on the group of classical permutations.

© 2014 Published by Elsevier Inc.

0. Introduction

The symmetric group Sn has many interesting permutation statistics. The most im-portant ones are probably inversions, descents, excedances and the major index. It is well-known that the descent statistic and the excedance statistic are equidistributed, and the same is true for the inversion statistic and the major index. Some of these statistics, namely inversions and descents, have natural analogues in all Coxeter groups. Generalizing the other two and the corresponding results has been much more difficult and, already for the group of signed permutations SB

n (also known as the hyperoc-tahedral group), has occupied a number of mathematicians for a number of years

E-mail address: mongelli@mat.uniroma1.it.1 This paper is part of the author’s Ph.D. thesis written under the direction of Prof. F. Brenti at the

Univ. “la Sapienza” of Rome, Italy.

http://dx.doi.org/10.1016/j.jcta.2014.11.0010097-3165/© 2014 Published by Elsevier Inc.

130 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

(see e.g. [3,6,8,7,13,14]). Recently, the excedance statistic was generalized also to the affine Weyl groups An, Cn, Bn and Dn (see [9,17]).

Several signed enumeration results over permutations are known (see e.g. [15,11,23,19,20,1,14]). Recently, Sivasubramanian [21] proved results on signed-excedance enumer-ation in the set of classical permutations and its subset of all derangements. The main idea used by the author is to compute the signed generating functions of the statistics via determinants.

The purpose of this paper is to extend the study of signed-excedance statistic done by Sivasubramanian to all finite and affine Weyl groups and to all derangements of such groups.

The organization of this paper is as follows. In the next section we collect several definitions, notation and results that will be used in the sequel. In Section 2 we study properties of the signed-excedance statistics in the finite Weyl groups Bn and Dn. In Section 3 we compute the signed generating function for the affine groups An, Bn, Cn

and Dn. In some cases we give combinatorial interpretations to the coefficients of such generating functions.

1. Notation, definitions and preliminaries

In this section we collect some definitions, notation and results that will be used in the rest of this paper. We let P := {1, 2, 3, . . .}, N := P ∪ {0}, Z be the ring of integers, Z∗ := Z \ {0} and R be the field of real numbers. Given n, m ∈ Z, n ≤ m, we let

[n, m] := {n, n +1, . . . , m}. The cardinality of a set A will be denoted by |A|. We denote by χ the indicator function.

Given a non-negative integer i, its q-analogue is defined as [i]q = 1 +q+q2+ · · ·+qi−1, where q is an indeterminate and [0]q = 0.

Given a sequence σ = (a1, . . . , an) ∈ Zn we say that an index i ∈ [1, n] is an excedance

of σ if ai > i. We denote by exc(σ) the number of excedances of σ. We say that the pair (i, j) is an inversion of σ if i < j and ai > aj . We denote by inv(σ) the number of inversions of σ. Given a set T we let S(T ) be the set of all bijections π : T → T , and Sn := S([1, n]). If σ ∈ Sn then we write σ = [σ1, . . . , σn] to mean that σ(i) = σi, for i = 1, . . . , n. We call such sequence the window notation of σ to distinguish it from the disjoint cycle notation (see e.g. [22, p. 17]). It is well-known that Sn is generated by s1, . . . , sn−1 where si = [1, . . . , i − 1, i + 1, i, . . . , n] for all i ∈ [1, n − 1]; moreover, if we write a permutation σ as a product of generators, then the minimal number of factors used in any such product is equal to inv(σ) (in Coxeter theoretic language it is known as the length of σ).

In [15] and [21] the following result is shown.

Theorem 1.1. For n ≥ 1, the signed generating function of the excedance statistic on Sn

is given by ∑

σ∈S (−1)inv(σ)qexc(σ) = (1 − q)n−1.

n

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 131

We follow [4, Chapter 8] for combinatorial descriptions of the classical Weyl and affine Weyl groups. In particular, we denote by DSn the set of all derangements in Sn, i.e. permutations without fixed point. In [16] and [21] the following result is shown.

Theorem 1.2. For n ≥ 2 the signed generating function of the excedance statistic on the set DSn is given by

∑σ∈DSn

(−1)inv(σ)qexc(σ) = (−1)n−1q[n − 1]q.

We denote by SBn the group of all bijections σ in S([−n, n] \ {0}) such that

σ(−i) = −σ(i) for all i ∈ [−n, n] \ {0}, with composition as the group operation. This group is usually known as the group of “signed permutations” on [1, n], or as the hy-peroctahedral group of rank n. We have that SB

n is a Weyl group of type Bn (see e.g. [4, Proposition 8.1.3]). If σ ∈ SB

n then we write σ = [a1, . . . , an] to mean that σ(i) = aifor i = 1, . . . , n. We refer to it as the window notation of σ. The group SB

n is gener-ated by sB1 , . . . , s

Bn−1 (the window notation of sBi is the same of that of si in Sn, for

i = 1, . . . , n − 1) and by sB0 = [−1, 2, . . . , n]. We let

excA(σ) := exc(a1, . . . , an)

neg(σ) :=∣∣{i ∈ [1, n]

∣∣ ai < 0}∣∣. (1.1)

We denote by SDn the group of all bijections σ ∈ SB

n such that neg(σ) ≡ 0 (mod 2). It is a Weyl group of type Dn (see e.g. [4, Proposition 8.2.3]), generated by sD1 , . . . , sDn−1(where sDi = sBi for i = 1, . . . , n − 1) and by sD0 = [−2, −1, 3, . . . , n].

We denote by Sn the set of all bijections π ∈ S(Z) such that π(i + n) = π(i) + n

for all i ∈ Z and ∑n

i=1(π(i) − i) = 0, with composition as the group operation. The group Sn is an affine Weyl group of type An−1 (see e.g. [4, Proposition 8.3.3]). If π ∈ Sn

then we write π = [a1, . . . , an] to mean that π(i) = ai for i = 1, . . . , n. We refer to this as the window notation of π. Moreover, sometimes we will write π = (rπ, σπ), with rπ = (r1, . . . , rn) ∈ Z

n, σπ ∈ Sn to mean that π(i) = σπ(i) +nri. Note that any such pair (r, σ) uniquely determines one element in Sn. The group Sn is generated by s1, . . . , sn, where sn = [0, 2, . . . , n − 1, n + 1] and si = [1, . . . , i − 1, i + 1, i, . . . , n] for i ∈ [1, n − 1].

We denote by SCn the set of all bijections π ∈ S(Z) such that π(−i) = −π(i) and

π(i +2n +1) = π(i) +2n +1 for all i ∈ Z, with composition as the group operation. The group SC

n is an affine Weyl group of type Cn (see e.g. [4, Proposition 8.4.3]). If π ∈ SCn

then we write π = [a1, . . . , an] to mean that π(i) = ai for i = 1, . . . , n. We refer to this as the window notation of π. Moreover, sometimes we will write π = (rπ, σπ), with rπ =(r1, . . . , rn) ∈ Z

n, σπ ∈ SBn to mean that π(i) = σπ(i) +ri(2n +1). Note that any such pair

(r, σ) uniquely determines one element in SCn . The group SC

n is generated by sC0 , . . . , sCn ,

where sC0 = [−1, 2, . . . , n], sCn = [1, . . . , n − 1, n + 1] and sCi = [1, . . . , i − 1, i + 1, i, . . . , n]for i ∈ [1, n − 1].

We denote by SBn the subgroup of SC

n consisting of all bijections π such that |{i ≤ n |π(i) > n}| ≡ 0 (mod 2). The group SB

n is well defined and it is an affine Weyl group of

132 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

type Bn, generated by sC0 , . . . , sCn−1 and by sBn = [1, . . . , n − 2, n + 1, n + 2]. Note that

π = (rπ, σπ) ∈ SCn is in SB

n if and only if

n∑i=1

ri ≡ 0 (mod 2), (1.2)

as can be proved by inspection. In the following we say that the vector r ∈ Zn is even if it satisfies (1.2), otherwise we say that r is odd. Finally, we denote by SD

n the subgroup of SBn consisting of all bijections π such that |{i > 0 | π(i) < 0}| ≡ 0 (mod 2). The group SD

n

is well defined and it is an affine Weyl group of type Dn, generated by sC1 , . . . , sCn−1, s

Bn

and by sD0 = [−2, −1, 3, . . . , n]. Note that π = (rπ, σπ) ∈ SBn is in SD

n if and only if

σπ ∈ SDn . (1.3)

The Eulerian polynomials (see e.g. [22, Chapter 1]) are defined by A0(q) = 1 and for n ≥ 1

An(q) :=n−1∑h=0

A(n, h)qh+1 :=∑σ∈Sn

qexc(σ)+1.

In what follows we will find it convenient to use the following notation:

An(q) :={

1, if n = 0,1qAn(q) otherwise. (1.4)

A bijection π : T → T , with T ⊆ Z is a derangement if π(i) = i for all i ∈ T . Brenti [5] defined the derangement polynomials (of type A) by

dn(q) :=∑

σ∈DSn

qexc(σ), n ≥ 1, (1.5)

and d0(q) = 1. The following formula is given in [7, Theorem 1.1] and it is derived from [5].

Theorem 1.3. For n ≥ 0,

dn(q) =n∑

k=0

(−1)n−k

(n

k

)Ak(q).

For any permutation π ∈ SWn or π ∈ SW

n (W = C, B, D), we define the length of π as the minimum number of factors such that π can be written as product of the generators of the group. Since for each such group the inversion statistic (defined for example in [4, Chapter 8]) and length are equal we denote such statistic as inv(π).

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 133

For all groups SWn and SW

n (W = C, B, D) we denote by DSWn and DS

W

n respectively, the sets of all their derangements. Finally, given π ∈ SW

n or π ∈ SWn we denote by |π|

the unique permutation such that |π|(i) = |π(i)| for all i ∈ [1, n].Let s be a statistic on SW

n or SWn and let F (n) =

∑σ q

s(σ) be its generating function. Suppose that s is invariant under deletion of an index and renumbering the remaining ones. Then the Principle of Inclusion–Exclusion (see e.g. [22, Sections 3.7, 3.8]) allows us to compute the generating function on the set of derangements DSW

n or DSW

n as follows

∑σ∈DSW

n or DSW

n

qs(σ) =n∑

k=0

(−1)n−k

(n

k

)F (k) (1.6)

2. Signed-excedance enumeration in finite Weyl groups

In this section we introduce and study three different but similar excedance statistic on the set of signed permutations SB

n and its subset SDn . These statistics will be dealt

with one by one in the next following subsections.

2.1. Colored excedance

In [3] Bagno and Garber introduced a definition of excedance on the set of signed permutations, called colored excedance.

Definition 2.1. For any σ ∈ SBn we set

excClr(σ) := 2 excA(σ) + neg(σ).

For example, if σ = [4, 3, −1, −5, 2], then neg(σ) = 2, excA(σ) = 2 and excClr(σ) = 6. The colored excedance statistic is also known as flag-excedance statistic (see e.g. [12–14]). It is known (see [14, Theorem 5.3] or [17, Proposition 3.3]) that the generating function of the colored excedance statistic is given by

∑σ∈SB

n

q2 excA(σ)tneg(σ) = An

(q2 + t

t + 1

)(1 + t)n,

in particular ∑σ∈SB

n

qexcClr(σ) = An(q)(1 + q)n.

For the set of all derangements it is known (see [17, Proposition 3.4]) that

∑B

q2 excA(σ)tneg(σ) =n∑(

n

h

)(t + 1)htn−hdh

(q2 + t

t + 1

),

σ∈DSnh=0

134 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

in particular

∑σ∈DSB

n

qexcClr(σ) =n∑

h=0

(n

h

)(q + 1)hqn−hdh(q).

Here we derive analogous formulas for the signed generating functions. We start with a useful result.

Lemma 2.2. Let σ ∈ SWn , with W = B, D. Then inv(σ) ≡ inv(|σ|) + neg(σ) (mod 2).

Proof. We argue the proof by induction on inv(σ). If inv(σ) = 0 then σ = [1, . . . , n] and the claim is trivial. Now let inv(σ) > 0 and i ∈ [0, n − 1] (if W = D, i ∈ [1, n]) such that inv(σ) > inv(sWi σ). For i = 0 we have by induction

inv(σ) = 1 + inv(sWi σ

)≡ 1 + inv

(∣∣sWi σ∣∣) + neg

(sWi σ

)= 1 +

(inv

(|σ|

)± 1

)+ neg

(sWi σ

)≡ inv

(|σ|

)+ neg(σ)

where the last step follows since si has an even number of negative signs in its window notation. If i = 0 then inv(|sB0 σ|) = inv(|σ|) and neg(sB0 σ) = neg(σ) ± 1 and the proof is similar. �

Now we give the generating function of the signed colored excedances on the group SBn .

Theorem 2.3. For n ≥ 1 we have∑σ∈SB

n

(−1)inv(σ)q2 excA(σ)tneg(σ) =(1 − q2)n−1(1 − t). (2.1)

In particular, ∑σ∈SB

n

(−1)inv(σ)qexcClr(σ) = (1 − q)n(1 + q)n−1.

Proof. We claim that the sum on the left-hand side of (2.1) is the determinant of the following n × n matrix

Ln =

⎛⎜⎜⎜⎜⎜⎜⎜⎝

1 − t q2 − t q2 − t · · · q2 − t

1 − t 1 − t q2 − t · · · q2 − t

1 − t 1 − t 1 − t · · · q2 − t

......

.... . .

...

⎞⎟⎟⎟⎟⎟⎟⎟⎠

1 − t 1 − t 1 − t · · · 1 − t

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 135

Given a matrix M = (mi,j)i,j≤n, detM =∑

σ∈Sn(−1)inv(σ)mi,σ(i). The signed permu-

tations in SBn differ from the permutations in Sn only by the signs of their evaluations at

1, . . . , n. In particular, a negative sign at place i in σ changes the parity of the number of its inversions (by Lemma 2.2) and deletes the excedance if σ(i) > i. This justifies the summand −t in each entry of Ln. Moreover if σ(i) > i we have a contribution of q2. It is routine to compute the determinant of Ln. �Theorem 2.4. For the set of all derangements in SB

n we have∑σ∈DSB

n

(−1)inv(σ)q2 excA(σ)tneg(σ) = (−1)n−1(q2[n− 1]q2 − t[n]q2). (2.2)

In particular, ∑σ∈DSB

n

(−1)inv(σ)qexcClr(σ) = (−1)nq[2n− 1]−q.

Proof. The argument is similar to the proof of Theorem 2.3. Namely, it suffices to com-pute the determinant of the following matrix⎛⎜⎜⎜⎜⎜⎜⎜⎝

−t q2 − t q2 − t · · · q2 − t

1 − t −t q2 − t · · · q2 − t

1 − t 1 − t −t · · · q2 − t

......

.... . .

...1 − t 1 − t 1 − t · · · −t

⎞⎟⎟⎟⎟⎟⎟⎟⎠�

Note that (2.1) and (2.2) are linear in t. This fact can also be seen directly as follows: let σ be a permutation with at least two negative elements in its window notation. Let a1, a2 be the minimal (in absolute value) negative elements. Then the permutation σ′

whose window notation is obtained from that of σ by exchanging a1, a2 has excClr(σ′) =excClr(σ) and inv(σ′) = − inv(σ), thus their contributions vanish.

Since the group SDn is the subgroup of SB

n consisting of all permutations σ with neg(σ) even, it is immediate to compute the signed generating functions for SD

n from the previous results.

Corollary 2.5. For all n ≥ 4 we have∑σ∈SD

n

(−1)inv(σ)qexcClr(σ)tneg(σ) =(1 − q2)n−1

,

and ∑σ∈DSD

n

(−1)inv(σ)qexcClr(σ)tneg(σ) = (−1)n−1q2[n− 1]q2 .

136 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

2.2. Absolute excedance

In [3] another excedance statistic was introduced. Given σ ∈ SBn recall that |σ| is the

permutation in Sn such that |σ|(i) = |σ(i)| for all i ∈ [1, n].

Definition 2.6. For each σ ∈ SBn , the number of absolute excedances of σ is given by

excAbs(σ) = exc(|σ|

)+ neg(σ).

For example, if σ = [4, 3, −1, −5, 2] then exc(|σ|) = 3, neg(σ) = 2 and excAbs(σ) = 5.In [17] it is shown that∑

σ∈SBn

qexc(|σ|)tneg(σ) = (1 + t)nAn(q) (2.3)

and that

∑σ∈DSB

n

qexc(|σ|)tneg(σ) =n∑

h=0

(n

h

)dh(q)(t + 1)htn−h. (2.4)

In particular ∑σ∈SB

n

qexcAbs(σ) = (1 + q)nAn(q)

and

∑σ∈DSB

n

qexcAbs(σ) =n∑

h=0

(n

h

)dh(q)(q + 1)hqn−h.

Note that the two statistics excAbs and excClr are equidistributed on both the sets SBn

and DSBn .

We now obtain the signed generating function of the absolute excedance statistic.

Theorem 2.7. For all n ≥ 2 we have∑σ∈SB

n

(−1)inv(σ)qexcA(|σ|)tneg(σ) = (1 − t)n(1 − q)n−1.

In particular, ∑σ∈SB

n

(−1)inv(σ)qexcAbs(σ) = (1 − q)2n−1.

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 137

Proof. The generating function is given by the determinant of

⎛⎜⎜⎜⎜⎜⎜⎜⎝

1 − t q − qt q − qt · · · q − qt

1 − t 1 − t q − qt · · · q − qt

1 − t 1 − t 1 − t · · · q − qt

......

.... . .

...1 − t 1 − t 1 − t · · · 1 − t

⎞⎟⎟⎟⎟⎟⎟⎟⎠. �

Theorem 2.8. For all n ≥ 2 we have

∑σ∈DSB

n

(−1)inv(σ)qexcAbs(|σ|)tneg(σ) = (−1)n (q + t− qt)n − q

1 − q

In particular,

∑σ∈DSB

n

(−1)inv(σ)qexcAbs(σ) = (−1)n (2q − q2)n − q

1 − q

Proof. The generating function is given by the determinant of

⎛⎜⎜⎜⎜⎜⎜⎜⎝

−t q − qt q − qt · · · q − qt

1 − t −t q − qt · · · q − qt

1 − t 1 − t −t · · · q − qt

......

.... . .

...1 − t 1 − t 1 − t · · · −t

⎞⎟⎟⎟⎟⎟⎟⎟⎠. �

By taking the even powers of t in Theorems 2.7 and 2.8 we obtain the following corollaries.

Corollary 2.9. We have that

∑σ∈SD

n

(−1)inv(σ)qexcAbs(|σ|)tneg(σ) = (1 − t)n + (1 + t)n

2 (1 − q)n−1,

and

∑σ∈SD

n

(−1)inv(σ)qexcAbs(σ) = (1 − q)n + (1 + q)n

2 (1 − q)n−1.

138 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

Corollary 2.10. We have that∑σ∈DSD

n

(−1)inv(σ)qexcAbs(σ) = (−1)n (q + t− qt)n + (q − t + qt)n − 2q2(1 − q)

and ∑σ∈DSD

n

(−1)inv(σ)qexcAbs(σ) = (−1)n (2 − q)nqn + q2n − 2q2(1 − q) .

2.3. Type B excedance

Finally we consider a third variant of excedance for the signed permutations. Brenti [6] introduced the type B excedance as follows.

Definition 2.11. Given σ ∈ SBn , an element i ∈ [1, n] is a type B excedance of σ if

σ(i) = −i or σ(|σ(i)|) > σ(i).

For example, if σ = [4, 3, −1, −2, −5, 6] then there are 3 excedances of type B.This statistic has the same distribution as the descent number in the Coxeter group

theoretic language (see [6] for more details). For this reason we denote this excedance number of a signed permutation σ ∈ SB

n by excCox(σ). For a good exposition about the excedance of type B and derangements with respect to this statistic see the paper of Chen, Tang and Zhao [7]. See also the work of Chow [8] which studies essentially the same excedance (an element i ∈ [1, n] is a weak excedance of σ if σ(i) = i or σ(|σ(i)|) > σ(i)) with similar results.

The signed-generating function for the excedance number of type B is given by the following result.

Theorem 2.12. For n ≥ 1 we have∑σ∈SB

n

(−1)inv(σ)qexcCox(σ)tneg(σ) = (1 − q)n−1 − qtn(1 − q)n−1.

In particular, ∑σ∈SB

n

(−1)inv(σ)qexcCox(σ) = (1 − q)n. (2.5)

Proof. In this case we cannot apply the technique involving determinants. Let σ ∈ SBn be

a signed permutation. We write σ in cycle notation, σ = (c1,1, . . . , c1,l1) · · · (ck,1, . . . , ck,lk)with σ(|ci,j |) = ci,j+1 for i ≤ k and j < li and σ(|ci,li |) = ci,1. For example, if σ = [6, −2, 3, −1, −4, 5], then we write σ = (−1, 6, 5, −4)(−2)(3). Then the number of excedances of type B of σ is equal to the total number of ascents in all cycles (i.e. all the

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 139

pairs ci,j < ci,j+1 or ci,li < ci,1 in all cycles of length ≥ 2) plus the number of cycles of length one with a negative element. The length of σ is equal modulo 2 to the number of negative elements (by Lemma 2.2) plus the sum of the lengths of all cycles decreased by 1 (as for the classical permutations).

Now fix k ∈ [1, n] and let a1 < · · · < ak be k distinct elements in [1, n]. Let {b1, . . . , bn−k} be the complementary set of {a1, . . . , ak} in [1, n], b1 < · · · < bn−k. Con-sider all permutations σ ∈ SB

n with |σ(i)| = i for all i ∈ [1, n] and with the elements in their window notation given by −a1, . . . , −ak, b1, . . . , bn−k in any order. By considering the cycle structure, it is easy to verify that the signed enumerator of the excedances of type B on this subset of SB

n is equal to the signed enumerator of the excedances on the set of classical derangements (for this purpose use the bijection which maps −ak+1−i to i and bi to k + i and note that it preserves the number of excedances since there is no cycle of length 1, by assumption). By Theorem 1.2 we have that

∑σ∈SB

nσ(i) �=±i (∀i)

(−1)inv(σ)qexcCox(σ)tneg(σ) =n∑

k=0

(n

k

)(−1)n−1q[n− 1]q(−t)k

= −q[n− 1]q(t− 1)n

For σ ∈ SBn , let h(σ) be the number of elements i in [1, n] such that σ(i) = ±i. Let

[−1]q = −q−1, then we have that

∑σ∈SB

n

(−1)inv(σ)qexcCox(σ)tneg(σ) =n∑

h=0

∑σ∈SB

nh(σ)=h

(−1)inv(σ)qexcCox(σ)tneg(σ)

=n∑

h=0

(n

h

)(−q[h− 1]q

)(t− 1)h(1 − qt)n−h

= − q

q − 1

n∑h=0

(n

h

)(qh−1 − 1

)(t− 1)h(1 − qt)n−h

= − 1q − 1

(q(t− 1) + 1 − qt

)n + q

q − 1(t− 1 + 1 − qt)n

= (1 − q)n−1 − qtn(1 − q)n−1.

(Note that the factors (1 −qt) in the second line of the previous equation are determined by the cycles of length 1 and each negative sign in them increases the length and the number of excedances.) �Theorem 2.13. For n ≥ 1 we have

∑B

(−1)inv(σ)qexcCox(σ)tneg(σ) = q(t− tq − 1)n − (−q)n

q − 1 .

σ∈DSn

140 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

In particular, ∑σ∈DSB

n

(−1)inv(σ)qexcCox(σ) = (−q)n. (2.6)

Proof. With the same arguments of the proof of Theorem 2.12 we have

∑σ∈DSB

n

(−1)inv(σ)qexcCox(σ)tneg(σ) =n∑

h=0

(n

h

)(−q[h− 1]q

)(t− 1)h(−qt)n−h

= − q

q − 1

n∑h=0

(n

h

)(qh−1 − 1

)(t− 1)h(−qt)n−h

= − 1q − 1

(q(t− 1) − qt

)n + q

q − 1(t− 1 − qt)n

= q(t− tq − 1)n − (−q)n

q − 1 . �Note that (2.5) and (2.6) can also be computed combinatorially as follows: let σ ∈ SB

n

and suppose there exists i such that |σ(i)| = i. Consider the minimal such i. Then let σ′

be the permutation obtained by changing the sign of σ(i) in the window notation of σ. It is easy to check that excCox(σ′) = excCox(σ) and inv(σ′) ≡ 1 + inv(σ) (mod 2), resulting in cancellation. Therefore, the signed enumerator of the excedances of type B can be restricted to the set of all permutations π ∈ SB

n such that |π(i)| = i for all i ∈ [1, n]which is (1 − q)n (or (−q)n in the case of derangements).

By taking even powers of t in the previous result we obtain the signed generating function for SD

n and its derangement set.

Corollary 2.14. We have

∑σ∈SD

n

(−1)inv(σ)qexcCox(σ)tneg(σ) ={

(1 − q)n−1 − qtn(1 − q)n−1 if n is even(1 − q)n−1 if n is odd.

In particular, ∑σ∈SD

n

(−1)inv(σ)qexcCox(σ) = (1 − q)2�n2

where a� denotes the integer part of a.

Corollary 2.15. We have

∑D

(−1)inv(σ)qexcCox(σ)tneg(σ) = q(t− tq − 1)n + q(tq − t− 1)n − 2(−q)n

2(q − 1)

σ∈DSn

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 141

In particular,

∑σ∈DSB

n

(−1)inv(σ)qexcCox(σ) = 12q(q − 2)n + (q − 2)(−q)n

q − 1 .

3. Signed excedance enumeration in affine Weyl groups

Some excedance statistics are also defined for the affine Weyl groups. In this section we summarize them and give some results on signed enumeration.

3.1. Excedance in affine Weyl group Sn

In their recent work [9] Clark and Ehrenborg introduced the following definition of excedance statistic for the group of affine permutations Sn.

Definition 3.1. Let π ∈ Sn. The excedance statistic of π is

exc(π) :=n∑

i=1

∣∣∣∣⌈π(i) − i

n

⌉∣∣∣∣.In the same paper, the authors showed that if π = (rπ, σπ) ∈ Sn then exc(π) =

‖rπ − pσπ‖ where pσπ

is the vector in Nn whose i-th entry equals −1 if i is an excedance of σπ ∈ Sn and 0 otherwise. The symbol ‖ · ‖ denotes the 1-norm in Rn.

We begin by listing some useful properties which we need to compute the signed generating function of the excedance statistic on the group Sn.

Lemma 3.2. Let π = (rπ, σπ) ∈ Sn. Then

inv(π) ≡ inv(σπ) (mod 2)

Proof. We prove the claim by induction on the length of π. If inv(π) = 0 then π = σπ

and the result is trivial. Now we consider siπ, i ∈ [n], such that inv(siπ) > inv(π). By induction we have

inv(siπ) = 1 + inv(π) ≡ 1 + inv(σπ) ≡ inv(σsiπ),

since σsiπ is the product of a transposition and σπ. �Theorem 3.3. We have that∑

π∈Sn

(−1)inv(π)qexc(π)

= 1(1 − q2)n−1

n−1∑(n− 1k

) n−1−k∑ (n− 1 − k

i

)(n− 1 + k

n− 1 − i

)(−q)2i+k

k=0 i=0

142 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

Proof. We first prove that

∑π∈Sn

(−1)inv(π)qexc(π) =n−1∑k=0

(−1)k(n− 1k

) ∑r∈Ln

q‖r−pk‖

where pk = (−1, . . . ,−1︸ ︷︷ ︸k times

, 0, . . . , 0) ∈ Rn and Ln = {(x1, . . . , xn) ∈ Z

n |∑n

i=1 xi = 0}.

Let ρ ∈ Sn be a permutation with k excedances and consider the set of all π ∈ Sn with σπ = ρ. By permuting the coordinates of the vector r, we have that∑

r∈Ln

q‖r−pρ‖ =∑r∈Ln

q‖r−pk‖.

By Lemma 3.2 we have that∑π∈Sn

(−1)inv(π)qexc(π) =∑σ∈Sn

∑r∈Ln

(−1)inv(σ)qexc(r,σ)

=∑σ∈Sn

(−1)inv(σ)∑r∈Ln

q‖r−pσ‖

=n−1∑k=0

(−1)k(n− 1k

) ∑r∈Ln

q‖r−pk‖

where the last equation follows from Theorem 1.1.In [9] it is shown that

∑r∈Ln

q‖r−pk‖ = 1(1 − q2)n−1

n−1−k∑i=0

(n− 1 − k

i

)(n− 1 + k

n− 1 − i

)q2i+k.

The result follows. �One may prove that for all n ≥ 1 the following identity holds

∑π∈Sn

(−1)inv(π)qexc(π) =∑n−1

k=0(n−1k

)2(−q)k

(1 + q)n−1 . (3.1)

To show the previous identity use Theorem 3.3 and prove that the two polynomials

n−1∑k=0

(n− 1k

) n−1−k∑i=0

(n− 1 − k

i

)(n− 1 + k

n− 1 − i

)(−q)2i+k

and

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 143

n−1∑k=0

(n− 1k

)2 n−1∑i=0

(n− 1i

)(−q)k+i

are equal. It suffices to apply the Gosper–Zeilberger algorithm2 (see e.g. [18, Section II]) to the coefficients of qα, with α ≤ 2n − 2, and get

n−1∑i=α

(n− 1α− 2i

)(n− 1 − α + 2i

i

)(n− 1 + α− 2i

n− 1 − i

)=

n−1∑i=α

(n− 1α− i

)2(n− 1i

).

The right-hand side of (3.1) is the generating function of coordination sequences for An−1 (see e.g. [2]) in the indeterminate −q. Recall that given a graph G with a root O the coordination sequence is given by (S(0), S(1), S(2), . . .) where S(i) denotes the number of vertices in G having distance i from O. The graph considered here is the vector star of the root system An−1, i.e. the graph obtained by the rule that a lattice point has as neighbors all other lattice points that can be reached by a root vector (for more details see [10]). Therefore we have

Proposition 3.4. For all n ≥ 2 the sequence of signless coefficients of ∑

π∈Sn(−1)inv(π) ×

qexc(π) is the coordination sequence for An−1.

The formula for the set of all derangements in Sn is not very elegant. It can be proved from Theorem 3.3 by the Principle of Inclusion–Exclusion.

Corollary 3.5. For all n ≥ 1 we have that

∑π∈DSn

(−1)inv(π)qexc(π) =n∑

h=0

(−1)n−i

(n

h

)1

(1 + q)h−1

h−1∑k=0

(h− 1k

)2

(−q)k

3.2. Colored excedance in affine Weyl groups

We now consider the other affine Weyl groups. With the same techniques used in the proof of Lemma 3.2 one can show the following result, whose proof we omit.

Lemma 3.6. Let π ∈ SWn , with W = B, C, D. Then

inv(π) ≡ inv(σπ) (mod 2)

In [17] the following excedance statistic for SCn is defined, called colored excedance.

2 The author thanks Prof. C. Krattenthaler for his hints towards proving the identity.

144 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

Definition 3.7. Let π ∈ SCn . Then the colored excedance statistic associated to π is the

number

excClr(π) = 2n∑

i=1

∣∣∣∣⌈π(i) − i

2n + 1

⌉∣∣∣∣ + neg(σπ).

In [17, Lemma 6.3], the following result is proved.

Lemma 3.8. For all π ∈ SCn , let pσπ

∈ {−1, 0}n be the vector with pσπ(i) = −1 if

σπ(i) > i and 0 otherwise. Then

excClr(π) = 2‖rπ − pσπ‖ + neg(σπ).

With this lemma it is possible to prove the following result.

Theorem 3.9. For all n ≥ 2 we have that∑σ∈SB

n

(−1)inv(σ)qexcClr(σ) =∑σ∈SC

n

(−1)inv(σ)qexcClr(σ) =∑

σ∈SDn

(−1)inv(σ)qexcClr(σ) = 0

Proof. Let W = B, C, or D. By Lemmas 2.2 and 3.6, we have that

SGFWn (t, q) :=

∑π∈SW

n

(−1)l(π)q2‖rπ−pσπ‖tneg(σπ)

=( ∑

σ∈SWn

(−1)inv(σ)tneg(σ))( ∑

r∈Zn

q2‖r−pσ‖)

(where we recall that SCn = SB

n ). The first factor in the last expression is always 0 when-ever n ≥ 2: in fact left multiplication by sW1 gives an involution on SW

n which preserves the statistic neg and changes the sign of the statistic (−1)l. The result follows. �

It is possible to compute SGFW1 (t, q): although SW

1 is not defined, it could be thought of as the set of all elements π ∈ SW

2 with π(1) = 1 and SGFW1 (t, q) is used in computing

the analogous formulas for the derangement set. Namely, one has

SGFC1 (t, q) = (1 − t)1 + q2

1 − q2

SGFB1 (t, q) = (1 − t)1 + q4

1 − q4

SGFD1 (t, q) = 1 + q4

1 − q4 .

By the Principle of Inclusion–Exclusion, we obtain immediately the following result.

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 145

Corollary 3.10. We have that

∑π∈DSC

n

(−1)inv(π)qexcClr(π) = (−1)n + (−1)n−1n1 + q2

1 − q

and

∑π∈DSB

n

(−1)inv(π)qexcClr(π) = (−1)n + (−1)n−1n1 + q4

(1 + q)(1 + q2)

∑π∈DSD

n

(−1)inv(π)qexcClr(π) = (−1)n + (−1)n−1n1 + q4

1 − q4

Since the colored excedance can be zero also for permutations which are not the identity, in [17] the following definition was given.

Definition 3.11. Let π ∈ SCn . Then the variant colored affine excedance of π is

excClr 2(π) =n∑

i=1

∣∣∣∣⌈π(i) − i

2n + 1

⌉∣∣∣∣ + excA(σπ) + neg(σπ).

It is known (see [17, Section 6]) that for π ∈ SCn

excClr 2(π) = ‖rπ − pσπ‖ + excA(σπ) + neg(σπ).

We then have the following result.

Theorem 3.12. We have that∑π∈SC

n

(−1)inv(π)qexcClr 2(π) = (1 + q)n

and

∑π∈SB

n

(−1)inv(π)qexcClr 2(π) = (1 + q)n+1 + (1 − q)n+1

2(1 + q)

∑π∈SD

n

(−1)inv(π)qexcClr 2(π) = (1 + q)n+1 + (1 − q)n+1

2(1 − q2)

Proof. We first compute the following generating function

SGFWn (q, s, t) =

∑˜C(−1)inv(π)qexcA(σπ)tneg(σπ)s‖rπ−pσπ‖,

π∈Sn

146 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

for W = C, B, D. By Theorem 2.3 we have that

SGFCn (q, s, t) =

∑σ∈SB

n

(−1)inv(σ)qexcA(σ)tneg(σ)∑r∈Zn

s‖r−pσ‖

= (1 − q)n−1(1 − t)(

1 + s

1 − s

)n

.

By setting t = s = q the first part of the result follows.We now consider the group SB

n . Since the vector r is even, we have that r− pσ is even if and only if excA(σ) is an even number. Therefore

SGFBn (q, s, t) =

∑σ∈SB

nexcA(σ) even

(−1)inv(σ)qexcA(σ)tneg(σ)∑

r evens‖r‖

+∑

σ∈SBn

excA(σ) odd

(−1)inv(σ)qexcA(σ)tneg(σ)∑r odd

s‖r‖

= (1 − t) (1 − q)n−1 + (1 + q)n−1

212

((1 + s

1 − s

)n

+(

1 − s

1 + s

)n)+ (1 − t) (1 − q)n−1 − (1 + q)n−1

212

((1 + s

1 − s

)n

−(

1 − s

1 + s

)n)= 1 − t

2

((1 − q)n−1

(1 + s

1 − s

)n

+ (1 + q)n−1(

1 − s

1 + s

)n),

and the second identity in the claim follows by setting s = t = q. For the last identity, take only the even powers of t in SGFB

n (q, s, t). �By the Principle of Inclusion–Exclusion we obtain the signed generating functions on

the sets of derangements.

Corollary 3.13. We have that∑π∈DSC

n

(−1)inv(π)qexcClr 2(π) = qn

and

∑π∈DSB

n

(−1)inv(π)qexcClr 2(π) = (−1)n

2(1 + q)((1 + q)(−q)n + (1 − q)qn + 2q

)∑

π∈DSBn

(−1)inv(π)qexcClr 2(π) = (−1)n

2(1 − q2)((1 + q)(−q)n + (1 − q)qn − 2q2)

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 147

3.3. Absolute excedance in affine Weyl groups

We now consider the last statistic introduced in [17]. We recall here the definition.

Definition 3.14. For any π = (rπ, σπ) ∈ SCn the absolute affine excedance of π is

excAbs(π) =n∑

i=1

∣∣∣∣⌈ |π(i)| − i

2n + 1

⌉∣∣∣∣ + neg(σπ).

In [17] it is shown that

∣∣∣∣⌈ |π(i)| − i

2n + 1

⌉∣∣∣∣ =

⎧⎪⎨⎪⎩ri + χ(σπ(i) > i) if ri > 0χ(|σπ(i)| > i) if ri = 0|ri| + χ(σπ(i) < −i) if ri < 0

(3.2)

where π = (rπ, σπ) is as usual. In particular, if ri runs over Z, the previous expression assumes values in {1, 1, 2, 2, 3, 3, . . .} if |σπ(i)| > i and in {0, 1, 1, 2, 2, 3, 3, . . .} otherwise. Moreover, if i is such that |σπ(i)| > i and ri runs over Z and it is always even or always odd then the previous expression assumes all values in P.

Theorem 3.15. We have that∑π∈SC

n

(−1)inv(π)qexcAbs(π)= (1 + q)(1 − q)n−1

and ∑π∈SB

n

(−1)inv(π)qexcAbs(π)= (1 + q)(1 − q)n−1

2 + (1 − q)2n

2(1 + q)n

∑π∈SD

n

(−1)inv(π)qexcAbs(π)= (1 − q)n−1

2 + (1 + q)n+1

4(1 − q) + (1 − q)2n

4(1 + q)n .

Proof. The set of all permutations π = (rπ, σπ) ∈ SCn can be obtained by taking all pairs

(r, σ), with r ∈ Zn and σ ∈ SB

n . All signed permutations σ ∈ SBn can be obtained by

choosing all permutations in Sn with exactly h excedances (h ranges from 0 to n − 1) and then changing all possible signs in their window notation. By (3.2) we therefore have that

∑π∈SC

n

(−1)inv(π)qexcAbs(π) =n−1∑h=0

∑σ∈Sn

exc(σ)=h

n∑k=0

(n

k

)(−q)k(−1)inv(σ)

∑r∈(Z∗)h×Zn−h

q‖r‖,

where we have used Lemmas 2.2 and 3.6. Hence

148 P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149

∑π∈SC

n

(−1)inv(π)qexcAbs(π) =n−1∑h=0

∑σ∈Sn

exc(σ)=h

(1 − q)n(−1)inv(σ) 2hqh(1 + q)n−h

(1 − q)n

= (1 + q)nn−1∑h=0

∑σ∈Sn

exc(σ)=h

(−1)inv(σ)(

2q1 + q

)h

= (1 + q)n(

1 − 2q1 + q

)n−1

,

where the last identity follows from Theorem 1.1. The result follows.We now consider the group SB

n . We recall that (r, σ) ∈ SCn is also in SB

n if and only if r is even. Suppose that there exists i ∈ [1, n − 1] such that |σ(i)| > i. Then r can be chosen arbitrarily on all the entries j = i and then set choose ri to be even or odd as necessary. By (3.2), we have that

P ={ri + χ

(σπ(i) > i

) ∣∣ ri > 0}∪{χ(∣∣σπ(i)

∣∣ > i) ∣∣ ri = 0

}∪{|ri| + χ

(σπ(i) < −i

) ∣∣ ri < 0},

where ri ranges over all even (resp. odd) integers.Therefore

∑π∈SB

n

(−1)inv(π)qexcAbs(π) =n−1∑h=1

∑σ∈Sn

exc(σ)=h

n∑k=0

(n

k

)(−q)k(−1)inv(σ)

∑r∈P×(Z∗)h−1×Zn−h

q‖r‖

+n∑

k=0

(n

k

)(−q)k

∑r∈Zn even

q‖r‖,

where the last summand is due to all permutations σ ∈ SBn such that exc(|σ|) = 0. Now

the computation is the same as before.For the group SD

n we have to consider only permutations in SBn with an even number

of negative signs. Since negative signs contribute in the signed generating function only for the factor (1 − q)n it is easy to verify that

∑π∈SD

n

(−1)inv(π)qexcAbs(π) = (1 − q)n + (1 + q)n

2(1 − q)n∑

π∈SBn

(−1)inv(π)qexcAbs(π). �

Corollary 3.16. We have that

∑˜ C

(−1)inv(π)qexcAbs(π)= (−1)n q

n(1 + q) − 2q1 − q

π∈DSn

P. Mongelli / Journal of Combinatorial Theory, Series A 130 (2015) 129–149 149

∑π∈DSB

n

(−1)inv(π)qexcAbs(π)= (−1)n

(qn(1 + q) − 2q

2(1 − q)+ qn(3 − q)n

2(1 + q)n

)∑

π∈DSDn

(−1)inv(π)qexcAbs(π)= (−1)n

(qn − 1

2(1 − q) + (1 + q)((−q)n − 1)4(1 − q) + qn(3 − q)n

4(1 + q)n + 34

).

References

[1] R.M. Adin, I.M. Gessel, Y. Roichman, Signed mahonians, J. Combin. Theory Ser. A 109 (2005) 25–43.

[2] M. Baake, U. Grimm, Coordination sequences for root lattices and related graphs, Z. Kristallogr. 212 (1997) 253–256.

[3] E. Bagno, D. Garber, On the excedance number of colored permutation groups, Sém. Lothar. Combin. 53 (2006), Art. B53f.

[4] A. Björner, F. Brenti, Combinatorics of Coxeter Groups, Grad. Texts in Math., vol. 231, Springer-Verlag, New York, 2005.

[5] F. Brenti, Unimodal polynomials arising from symmetric functions, Proc. Amer. Math. Soc. 108 (1990) 1133–1141.

[6] F. Brenti, q-Eulerian polynomials arising from Coxeter groups, European J. Combin. 15 (1994) 417–441.

[7] W.Y.C. Chen, R.L. Tang, A.F.Y. Zhao, Derangement polynomials and excedances of type B, Elec-tron. J. Combin. 16 (2) (2009) R15.

[8] C.O. Chow, On derangement polynomials of type B. II, J. Combin. Theory Ser. A 116 (4) (2009) 816–830.

[9] E. Clark, R. Ehrenborg, Excedance of affine permutations, Adv. in Appl. Math. 46 (2011) 175–191.[10] J.H. Conway, N.J.A. Sloane, Low-dimensional lattices, VII: Coordination sequences, Proc. R. Soc.

Lond. Ser. A Math. Phys. Eng. Sci. 453 (1997) 2369–2389.[11] J. Désarménienien, D. Foata, The signed Eulerian numbers, Discrete Math. 99 (1992) 49–58.[12] M. Fire, Statistics on wreath products, Masters thesis, arXiv:math/0409421v2, 2004.[13] D. Foata, G.-N. Han, Signed words and permutations, V: A sextuple distribution, Ramanujan J.

19 (1) (2009) 29–52.[14] D. Foata, G.-N. Han, The decrease value theorem with an application to permutation statistic, Adv.

in Appl. Math. 46 (2011) 296–311.[15] R. Mantaci, Statistiques Eulriennes sur les Groupes de Permutation, PhD thesis, Université Paris,

1991.[16] R. Mantaci, F. Rakotondrajao, Exceedingly deranging!, Adv. in Appl. Math. 30 (2003) 177–188.[17] P. Mongelli, Excedances in classical and affine Weyl groups, J. Combin. Theory Ser. A 120 (2013)

1216–1234.[18] M. Petkovšek, H.D. Wilf, D. Zeilberger, A = B, A.K. Peters, Wellesley, MA, 1996.[19] V. Reiner, Signed permutation statistics, European J. Combin. 14 (1993) 553–567.[20] V. Reiner, Descents and one-dimensional characters for classical Weyl groups, Discrete Math. 140

(1995) 129–140.[21] S. Sivasubramanian, Signed excedance enumeration via determinants, Adv. in Appl. Math. 47 (2011)

783–794.[22] R.P. Stanley, Enumerative Combinatorics, vol. 1, Cambridge University Press, 1997.[23] M. Wachs, An involution for signed Eulerian numbers, Discrete Math. 99 (1992) 59–62.