Combinatorics of the Basic Stratum

of Some Simple Shimura Varieties

Arno Kret

Received: June 28, 2013

Communicated by Patrick Delorme

Abstract. We express the cohomology of the basic Newton stratum of some unitary Shimura varieties associated to division algebras in terms of automorphic representations of the group in the Shimura datum.

2010 Mathematics Subject Classification: Primary 11G18 ; Secondary 11F70, 11F72. Keywords and Phrases: Shimura varieties, Arthur-Selberg Trace Formula, Langlands program.

Introduction

Let S be a Shimura variety of PEL type, defined over its reflex field E [8, 9]. The variety S parametrizes Abelian varieties with additional PEL structures (Polarization, Endomorphisms, Level structures) [17, §5]. In this article we compute for certain simple S the ℓ-adic cohomology of the basic stratum, also called supersingular locus, at a prime of good reduction. Let us make this more precise. Let 𝔭 be a prime of E where S has good reduction; this means in particular that the PEL type moduli problem extends over the ring of integers O_{E_{𝔭}} of E_{𝔭}, and that this extended problem is representable by a smooth and quasi-projective scheme S over O_{E_{𝔭}} [17, §5]. Let 𝔽_{q} be the residue field of E at 𝔭. Write S_{𝔭} := S ⊗ 𝔽_{q}, and write for each x ∈ S_{𝔭}(\overline{𝔽}_{q}), A_{x}/\overline{𝔽}_{q} for the corresponding Abelian variety with additional PEL structures. The Newton strata of S_{𝔭} are defined as follows: Two points x, y ∈ S_{𝔭}(\overline{𝔽}_{q}) lie in the same Newton stratum if and only if the p-divisible group A_{x}[p^{∞}] with its additional PE-structures is isogenous to A_{y}[p^{∞}]. Thus the Newton strata of S_{𝔭} are the loci in S_{𝔭} where the isogeny class of A_{x}[p^{∞}] is constant. The Newton strata of S_{𝔭} are varieties defined over 𝔽_{q}, and stable under the Hecke correspondences. Thus their cohomology carries an action of the Hecke algebra and the Frobenius operator. For various arithmetic applications, one is interested in computing this cohomology. There is one Newton stratum which is particularly important. This stratum is called the basic stratum, or supersingular locus. The cohomology of the other strata is expected to be induced from Levi subgroups of lower rank. Thus, in this article we focus on the basic stratum, although one could extend the methods of this article to also derive results for the non-basic strata [21]. Before we continue, we give the definition of the basic Newton stratum. For each point x ∈ S_{𝔭}(\overline{𝔽}_{q}), the isogeny class of the corresponding p-divisible group with PE structure A_{x}[p^{∞}] is determined by its rational Dieudonné module 𝔻(A_{x}[p^{∞}])_{ℚ}. This Dieudonné module is an “isocrystal with additional structure” and, as such, it is determined by its slope morphism ν [14]. If the centralizer of the slope morphism ν is G, then the isocrystal is called basic. Among (the isomorphism classes of) the isocrystals 𝔻(A_{x}[p^{∞}])_{ℚ} for x ∈ S(\overline{𝔽}_{q}) there is one unique basic isocrystal, and the locus in S_{𝔭} corresponding to this isocrystal is the basic stratum B ⊂ S_{𝔭}. For example consider the modular curve Y_{1}(N) (we take N ≥ 4), let p be prime number not dividing N and consider the supersingular locus B of Y_{1}(N)_{𝔽_{p}}. Then B(\overline{𝔽}_{p}) is the set of pairs (E, η) ∈ Y_{1}(N)(\overline{𝔽}_{p}) of elliptic curves E equipped with a point η of order N where E is supersingular. A famous result of Deligne-Rapoport [10] is a geometric description of the basic stratum B, as follows. Consider the modular curve Y associated to the congruence subgroup Γ_{1}(N) ∩ Γ_{0}(p) ⊂ GL_{2}(ℤ) (for notation, see [11]). Deligne and Rapoport construct a model for the curve Y over ℤ_{p} (they introduce Y as a moduli space), and in particular they may reduce Y modulo p. When reduced modulo p, the curve Y is isomorphic to the union of two copies of Y_{1}(N)_{𝔽_{p}}, fibered over the supersingular locus. Since the curve Y has semistable reduction, one may compare its cohomology over the generic fibre with the cohomology of the special fibre. This yields a description of the Hecke/Galois module H_{et}^{0}(B_{\overline{𝔽}_{p}}, \overline{ℚ}_{ℓ}) in terms of weight 2 modular forms. This description of H_{et}^{0}(B_{\overline{𝔽}_{p}}, \overline{ℚ}_{ℓ}) can also be deduced from the Langlands/Kottwitz method and application of the trace formula, as we explain in an introductory chapter to our thesis [19, Chap. 1]. For higher dimensional Shimura varieties it is difficult to describe the basic stratum geometrically. Several complications occur because usually the basic stratum B is a non-finite, non-smooth variety^{1}. There are geometric descriptions on the structure of the basic stratum, but only under very restricted conditions. For example Vollaard and Wedhorn [32] obtained a geomteric description of the basic stratum, if G_{ℝ} is the group GU(n − 1, 1), and the prime p of (good) reduction is non-split. These restricted classes of Shimura varieties are still very interesting as they are certain arithmetic quotients of the n-dimensional complex unit ball. However, one would like to know what the basic stratum looks like in general. Since general geometric results seem currently out of reach, it is therefore interesting to see that it is possible to describe the cohomology of the basic stratum for general classes of Shimura varieties. In this article we answer the question for the simple, unitary Shimura varieties of Kottwitz at split primes of good reduction. This article is a sequel to the article [20]. In [20] we have already studied the above problem, but under strong simplifying conditions on the signatures. The varieties are associated to certain division algebras over ℚ with involution of the second kind; we call such varieties Kottwitz varieties. In [20] we assumed that the signatures of the unitary group U ⊂ G are coprime with n. This hypothesis greatly simplifies the geometry and cohomology of B; completely new phenomena occur when the hypothesis is dropped. In this article we solve all the resulting problems when one removes the hypothesis from the theorem in case the prime p of reduction is split in the center of the division algebra defining the Kottwitz variety. A consequence of our final result is an automorphic expression for the ℓ-adic cohomology of the basic stratum of Kottwitz’s varieties at split primes of good reduction. The expressions are in terms of:

1On a positive side, the basic stratum B should always be a projective variety. However, the author must admit that he is unaware of a proof nor a reference of this fact for general non-projective Shimura varieties.

(1) Automorphic forms on the group G of the Shimura datum, (2) The factor at 𝔭 of their associated Galois representations, and (3) Polynomials in q^{α} of combinatorial nature, associated to certain non-crossing lattice paths in the plane ℚ^{2}. In particular we describe the zeta function of the basic stratum in terms of the objects (1), (2) and (3) above. The first part of the proof is the same as the one we carried out in [20], and we will be only very brief on that part of the argument. This first part roughly consists of 3 steps:

• Truncate the formula of Kottwitz for Shimura varieties of PEL-type [17] to count the number of points in the basic stratum.

• Pseudo-stabilize the expression using the arguments from [16] (cf. [15]).

• Compare the resulting stable expression with the geometric side of the trace formula.

Carrying out this program, we obtained in [20] the formula

(0.1) ∑_{i=0}^{∞}(−1)^{i}Tr (f^{∞p} × Φ_{𝔭}^{α}, H_{et}^{i}(B_{\overline{𝔽}_{q}}, \overline{ℚ}_{ℓ})) = |Ker^{1}(ℚ, G)| · Tr (f, A(G)).

In this formula we have

• A(G) = L^{2}(G(ℚ)\G(𝔸), ω) is the space of automorphic forms on G with respect to inverse ω of the central character of ξ;

• Φ_{𝔭} is the geometric Frobenius of \overline{𝔽}_{q} over 𝔽_{q};

• (one fixes the choice of an isomorphism \overline{ℚ}_{ℓ} ≅ ℂ to compare the left hand side with the right hand side);

• The function f is a tensor product f_{∞} ⊗ f^{∞p} ⊗ f_{p}, of a function f_{∞} on G(ℝ), a function f^{∞p} on G(𝔸_{f}^{p}), and a function f_{p} on G(ℚ_{p}), where – f^{∞p} is an arbitrary K^{p}-bi-invariant, locally constant compactly

supported complex valued function on G(𝔸_{f}^{p}).

– The function f_{∞} on G(ℝ) is up to scalar the Euler-Poincaré function corresponding to the representation ξ (for the value of the scalar, see [16, Lem. 3.2]).

– The component f_{p} is a the pointwise product χ_{c}^{G}f_{α}, where

∗ the function χ_{c}^{G} is the characteristic function on G(ℚ_{p}) of the set of compact elements in G(ℚ_{p});

∗ the function f_{α} is the function of Kottwitz. It arises by base change from G(E_{𝔭,α}) to G(ℚ_{p}) of a spherical function ϕ_{α} on G(E_{𝔭,α}); the function ϕ_{α} is the characteristic function of the subset G(O_{E_{𝔭,α}})μ(p^{−1})G(O_{E_{𝔭,α}}) ⊂ G(E_{𝔭,α}), with respect to the choice (any choice) of reductive model of G over ℤ_{p}.

• Ker^{1}(ℚ, G) is the set of classes σ ∈ H^{1}(ℚ, G) such that for all rational

places v of ℚ the restricted class σ_{v} ∈ H^{1}(ℚ_{v}, G) is trivial. Once Equation (0.1) is established, the second step is to compute Tr (χ_{c}^{G}f, π) for each automorphic representation π occurring in A(G), and it is this second step that we carried out only partially in [20]. The problem is local at p: One needs a satisfactory expression for the traces Tr (χ_{c}^{G}f_{α}, π_{p}) against the local representations π_{p} of G(ℚ_{p}). We will not solve this local problem completely in this article: We only solve the problem in sufficient generality for global applications. More precisely, we only compute the compact trace Tr (χ_{c}^{G}f_{α}, π_{p}) for the class of rigid representations of G(ℚ_{p}). All representations π_{p} that occur as a factor at p of an automorphic representation π of G are rigid. The rigid representations form a very restricted subclass of all smooth representations of G(ℚ_{p}). Using base change, Jacquet-Langlands and the Moeglin-Waldspurger description of the discrete spectrum of GL_{n}, we describe the rigid representations precisely. Then, we compute the compact traces of f_{α} against all rigid representations. Let us explain this last computation in more detail. Since our group G(ℚ_{p}) is isomorphic to a product of general linear groups, the computation quickly reduces to a computation with general linear groups. Furthermore, it is quite easy to see that Tr (χ_{c}^{G}f_{α}, π_{p}) vanishes unless π_{p} has an invariant vector for the action of the Iwahori subgroup (the representation π_{p} is semistable). Thus cuspidal representations π_{p} do not appear in the description. A semistable representation π of GL_{n}(ℚ_{p}) is called standard if it is isomorphic to a product (see §2) of essentially square integrable representations. The computation of the compact trace Tr (χ_{c}^{G}f_{α}, π) on a square-integrable representation is easy, and using van Dijk’s formula adapted for compact traces [20, Prop. 3], we easily deduce formulas for compact traces on the standard representations. Inside the Grothendieck group of smooth G(ℚ_{p})-representations of finite length, any semistable irreducible representation π may be^{2} (uniquely) written as a sum π = ∑_{I} c_{I} · I where I ranges over the standard representations, and the coefficients c_{I} ∈ ℂ are 0 for all but finitely many I. Consequently the compact trace Tr (χ_{c}^{G}f, π) equals ∑_{I} c_{I}Tr (χ_{c}^{G}f, I). There are two steps to compute Tr (χ_{c}^{G}f, π): (Prob1) Know the coefficients c_{I} and (Prob2) Make the sum ∑_{I} c_{I}Tr (χ_{c}^{G}f, I). The first problem (Prob1) is related to the Kazhdan-Lusztig conjecture^{3}. The “Kazhdan-Lusztig theorem" of Beilinson-Bernstein [3] (and [13]) interprets the multiplicity of any given irreducible representation π in the representation I, as the dimension of certain intersection cohomology spaces, and also as the value at q = 1 of certain Kazhdan-Lusztig polynomials. Combining all these theorems gives expressions for the sum ∑_{I} c_{I}Tr (χ_{c}^{G}f, I); however they are far from satisfactory. The coefficients c_{I}, even though they are known, depend on dimensions of complicated cohomology spaces. Furthermore, the coefficients c_{I} can be negative, and the sum ∑_{I} c_{I}Tr (χ_{c}^{G}f, I) has in general a lot of redundancy. Thus Problem 2 seems (to the author) very non-trivial. To compute the sum ∑_{I} c_{I}Tr (χ_{c}^{G}f, I) we restrict our attention to the rigid representations only. Any rigid representation is a product of unramified twists of Speh representations, and therefore we reduce to the class of Speh representations. A deep theorem of Tadic completely resolves the first problem (Prob1) for the Speh representations. The coefficients c_{I} turn out to be −1, 0 or 1 for these representations (precise statement in Theorem 2.3). Still the sum ∑_{I} c_{I}Tr (χ_{c}^{G}f, I) remains very redundant; in fact most of the terms cancel out against each other. To evaluate the sum, we give an interpretation of the traces Tr (χ_{c}^{G}f, I) as certain lattice paths in ℚ^{2}. Using a variation on a classical lemma in combinatorics [30, Thm. 7.2.1], we see that only the non-intersecting paths remain in the sum, all other terms cancel out.

2Zelevinsky proved in [33] that the standard representations form a basis of the Grothendieck group of the category of smooth GL_{n}(ℚ_{p})-representations of finite length.

Acknowledgements. I thank my thesis advisor Laurent Clozel for all his help. He gave me many ideas and checked several times the arguments and proofs in this article. I thank the anonymous referee for his/her long list of corrections as suggestions that have helped a lot to improve the paper. I also thank Laurent Fargues (my second advisor), Marco Deieso, Vito Mandorino, Alberto Minguez, Giovanni Rosso, Olivier Schiffmann and Peter Scholze for helpful discussions. This material is based upon work supported by the National Science Foundation under Grant No. DMS 1128155 (resp. Grant No. 0932078000) while the author was a member of the Institute for Advanced Study (resp. in residence at the Mathematical Sciences Research Institute in Berkeley, California, during the Fall 2014 semester).

1. Notation

In Sections 1–13 we will work (almost) exclusively with the group GL_{n} over a non-Archimedean local field. Let us set up the principal notation.

• p is a prime number.

• F is a non-Archimedean local field with residue characteristic equal to p.

• \overline{F} is a fixed algebraic closure of F.

3This conjecture is a theorem, see [5, Thm. 8.6.23].

• O_{F} is the ring of integers of F.

• ϖ_{F} ∈ O_{F} is a prime element of F.

• q = #(O_{F} /ϖ_{F}) is the cardinality of the residue field of F.

• n is a positive integer.

• G_{n} = GL_{n}(F) (often we write simply G = G_{n} when n is understood).

• K = GL_{n}(O_{F}).

• H(G) for the Hecke algebra of complex valued, locally compact constantly supported functions on G. The product f ∗ h of two functions f, h ∈ H(G) is defined by the convolution integral (f ∗ h)(g) = ∫_{G} f(x)h(x^{−1}g)dx with measure dx normalized so that the group K ⊂ G has volume 1. The algebra H(G) is associative, but it does not have a unit element.

• H_{0}(G) is the K-spherical Hecke algebra of G, i.e. the algebra of f ∈ H(G) such that f is invariant under left and right K-translations.

• We write 1_{K} for the characteristic function of the subset K ⊂ G. Then 1_{K} is the unit element of the algebra H_{0}(G).

• The group P_{0} ⊂ G is the standard upper triangular Borel subgroup of G with P = T N_{0}, where T is the diagonal torus of G, and N_{0} is the group of upper triangular unipotent matrices in G.

• a parabolic subgroup P of G standard if it is upper triangular, and we

write P = MN for its standard Levi decomposition. A partition of n is a finite, non-ordered list of non-negative numbers whose sum is equal to n. A composition of n is a finite, ordered list of positive numbers whose sum is equal to n. To each a composition (n_{a}) of n we attach the standard parabolic subgroup

P_{(n_{a})} =^{def} {( ..._{0  g_{k}}^{g_{1}  ∗} ) ∈ G_{n}| g_{a} ∈ G_{n_{a}}} ⊂ G_{n}.

We write M_{(n_{a})} = G_{n_{1}} × G_{n_{2}} × ⋯ × G_{n_{k}} for the standard Levi-subgroup of P_{(n_{a})}. Finally, to each parabolic subgroup P ⊂ G we attach the sign ε_{P} := (−1)^{dim A_{P} /A_{G}}, where A_{G} (resp. A_{P}) is the center of G (resp. P). For P = P_{(n_{a})} we have ε_{P} = (−1)^{n−k}.

2. Tadic’s determinantal formula

We recall an important character formula of Tadic for the Speh representations. This formula is a crucial ingredient for our computations. Let m, m′ ∈ ℤ_{≥1}. If π (resp. π′) is a smooth admissible representation of G_{m} (resp. G_{m′}), then we write π × π′ for the G_{m+m′}-representation parabolically induced (unitary induction) from the representation π ⊗π′ of the standard Levi subgroup consisting of two blocks, one of size m, and the other one of size m′. The tensor product π ⊗ π′ in the above formula is taken along the blocks of this Levi subgroup. We write R for the direct sum ⨁_{n∈ℤ_{≥0}} Groth(G_{n}) with the convention that G_{0} is the trivial group. The group G_{0} has one unique irreducible representation σ_{0} (the space ℂ, with trivial action). The operation “direct sum of representations” together with the product “×” turns the vector space R into a commutative ℂ-algebra with σ_{0} as unit element. We call it the ring of Zelevinsky. We write ν for the absolute value morphism from GL_{1}(F) = F^{×} to ℂ^{×}. By a segment S = ⟨x, y⟩ we mean a set of numbers {x, x + 1, …, y} where x, y ∈ ℚ, y − x ∈ ℤ, and where we need to explain the conventions in case y ≤ x. In case y is strictly smaller than x − 1, then ⟨x, y⟩ = ∅; in case x is equal to y, the segment ⟨x, y⟩ = {x} has one element. We have one unusual convention: For y = x − 1 we define the segment ⟨x, y⟩ to be the set {⋆} of one element containing a distinguishing symbol “⋆”. The length ℓ(S) of a segment S = ⟨x, y⟩ is defined to be y − x + 1. Thus the segment {⋆} has length 0, the segment {x} has length 1, the segment {x, x + 1} has length 2, etc. For any segment ⟨x, y⟩ with y ≥ x we write Δ⟨x, y⟩ for the unique irreducible quotient of the induced representation ν^{x} × ν^{x+1} × ⋯ × ν^{y}. We define Δ{⋆} to be σ_{0} (the one-dimensional representation of the trivial group GL_{0}(F)), and we define Δ⟨x, y⟩ to be 0 in case y < x − 1. For any segment S of non-negative length the object ΔS is a representation of the group GL_{n}(F), where n is the length of S. For the standard properties of segments we refer to Zelevinsky’s work [33] (cf. [28]), but note that our conventions are slightly different, because we allow rational numbers in the segments and we have the segment {⋆}. For any finite ordered list of segments S_{1}, S_{2}, …, S_{t} we have the product representation π := (ΔS_{1}) × (ΔS_{2}) × ⋯ × (ΔS_{t}). Observe that, due to our conventions, in case S_{a} = {⋆} for some a, then ΔS_{a} is the unit in R, and π = ∏_{b=1,b≠a}^{t} ΔS_{b} ∈ R. In case S_{a} = ∅ for some index a, then we have π = 0 in R.

Definition 2.1. We write St_{G} for the Steinberg representation of the group G. This representation is by definition the (unique) irreducible quotient of the space of locally constant compactly supported complex valued functions on P_{0}\G, on which G acts by right translations.

Definition 2.2. Let t, h be positive integers such that n = th. We define Speh(h, t) to be the (unique) irreducible quotient of the representation

St_{G_{h}}ν^{\frac{t−1}{2}} × ⋯ × St_{G_{h}}ν^{\frac{1−t}{2}} . This representation has t segments, S_{a} = ⟨x_{a}, y_{a}⟩, a = 1, …, t, where x_{a} = \frac{t−h}{2} − (a − 1), y_{a} = \frac{t+h}{2} − a. Observe that, for each index a, we have ℓS_{a} = h. Furthermore, for each index a < t, we have x_{a+1} = x_{a} − 1 and y_{a+1} = y_{a} − 1.

Let S_{1} = ⟨x_{1}, y_{1}⟩, S_{2} = ⟨x_{2}, y_{2}⟩, …, S_{t} = ⟨x_{t}, y_{t}⟩ be an ordered list of segments defining a representation of the group G = GL_{n}(F). Let 𝔖_{t} be the symmetric group on {1, 2, …, t}. For any w ∈ 𝔖_{t} we define the number n_{a}^{w} to be y_{a} − x_{w(a)} + 1. One checks easily that ∑_{a=1}^{k} n_{a}^{w} = n. The numbers n_{a}^{w} need not be positive. We define 𝔖′_{t} ⊂ 𝔖_{t} to be subset consisting of those permutations w ∈ 𝔖_{t} such that the numbers n_{a}^{w} are positive or 0. If the permutation w lies in the subset 𝔖′_{t} ⊂ 𝔖_{t}, then (n_{a}^{w}) is a composition of n. Assuming that w ∈ 𝔖′_{t} we will write P_{w} = M_{w}N_{w} for the parabolic subgroup of G corresponding to the composition (n_{a}^{w}). Let w ∈ 𝔖′_{t}. We define the segments S_{1}^{w} := ⟨x_{w(1)}, y_{1}⟩, S_{2}^{w} := ⟨x_{w(2)}, y_{2}⟩, …, S_{t}^{w} := ⟨x_{w(t)}, y_{t}⟩. We have ℓ(S_{a}^{w}) = n_{a}^{w}. We let Δ_{w} be the representation of M_{w} defined by (ΔS_{1}^{w}) ⊗ ⋯ ⊗ (ΔS_{t}^{w}), where the tensor product is taken along the blocks of M_{w}. The representation I_{w} is defined to be the product ΔS_{1}^{w} × ΔS_{2}^{w} × ⋯ × ΔS_{t}^{w}, i.e. it is the (unitary) parabolic induction Ind_{P_{w}}^{G}Δ_{w} of Δ_{w} to G. In case w ∈ 𝔖_{t}\𝔖′_{t} we define both Δ_{w} and I_{w} to be 0.

Remark. It is possible that S_{a}^{w} = {⋆} for some permutation w. In that case the representation ΔS_{a}^{w} is the unit element σ_{0} of R, and thus can be left out of the product that defined I_{w}.

In this notation we have the following theorem:

Theorem 2.3 (Tadic). Let π be a Speh representation of G and let S_{1} = ⟨x_{1}, y_{1}⟩, S_{2} = ⟨x_{2}, y_{2}⟩, … S_{t} = ⟨x_{t}, y_{t}⟩ be its segments. The representation π satisfies Tadic’s determinantal formula

(2.1)  π = ∑_{w∈𝔖_{t}} sign(w)I_{w} ∈ R.

Proof. This theorem was first proved by Tadic in [31] for Speh representations with a difficult argument. Chenevier and Renard give a simpler proof of Theorem 2.3 in [4]. Also Badulescu gives a simpler proof in the note [2] using the Moeglin-Waldspurger algorithm [25]. Recently Lapid and Minguez [23, Thm. 1] extended the formula to the larger class of ladder representations. □

Remark. By the definition of the subset 𝔖′_{t} ⊂ 𝔖_{t} we have for all w ∈ 𝔖_{t} that I_{w} ≠ 0 if and only if w ∈ 𝔖′_{t}, and thus we may as well index over the elements w ∈ 𝔖′_{t} in the sum in the above theorem. In the cases where the inclusion 𝔖′_{t} ⊂ 𝔖_{t} is strict, the subset 𝔖′_{t} is practically never a subgroup of 𝔖_{t}, it will neither be closed under composition nor contain inverses of elements.

The reason we call the formula of Theorem 2.3 the determinantal formula of Tadic, is the observation of Chenevier and Renard [4], who recognized that Equation (2.1) can be written as a determinant of a t × t-matrix over the ring R:

π = det (\begin{matrix}Δ⟨x_{1}, y_{1}⟩ & Δ⟨x_{1}, y_{2}⟩ & ⋯ & Δ⟨x_{1}, y_{t}⟩ \\ Δ⟨x_{2}, y_{1}⟩ & Δ⟨x_{2}, y_{2}⟩ & ⋯ & Δ⟨x_{2}, y_{t}⟩ \\ ⋮ & ⋮ & & ⋮ \\ Δ⟨x_{t}, y_{1}⟩ & Δ⟨x_{t}, y_{2}⟩ & ⋯ & Δ⟨x_{t}, y_{t}⟩ \end{matrix}) ∈ R.

3. The Satake transform

Let P = MN ⊂ G be a standard parabolic subgroup of G, and let f ∈ H(G) be a locally constant function on G. Then we define the constant term f^{(P)} ∈ H(M) by the formula

f^{(P)}(m) = δ_{P}^{1/2}(m) · ∫_{N} f(mn)dn,

for all m ∈ M, and where • δ_{P} is the modulus character of P, i.e. δ_{P} (m) = | det(m, Lie(N))|;

• the integral is taken with respect to the left Haar measure of N giving

the group K ∩ N measure 1. In case f is K-spherical, then f^{(P)} is K_{M} = M(O_{F})-spherical, and the constant term f ↦ f^{(P)} defines a morphism of algebras H_{0}(G) → H_{0}(M). In case T = M we have the obvious isomorphism H_{0}(T) →^{∼} ℂ[X_{1}^{±1}, …, X_{n}^{±1}], under which the characteristic function of the subset q^{−e_{1}}O_{F}^{×}×q^{−e_{2}}O_{F}^{×}×⋯ q^{−e_{n}}O_{F}^{×} ⊂ (F^{×})^{n} = T, corresponds to the monomial X^{e_{1}}X^{e_{2}} ⋯ X^{e_{n}} in ℂ[X_{1}^{±1}, …, X_{n}^{±1}] for all (e_{i}) ∈ ℤ^{n}. The composition of the constant term H_{0}(G) → H_{0}(T) with the isomorphism H_{0}(T) ≅ ℂ[X_{1}^{±1}, …, X_{n}^{±1}] is called the Satake morphism, and is denoted S_{G} or simply S if G is understood. Satake proved [29] that the map is injective, and has image equal to the algebra A = ℂ[X_{1}^{±1}, …, X_{n}^{±1}]^{𝔖_{n}} where 𝔖_{n} is the Weyl group of T in G; it is the symmetric group on the set {1, …, n} acting on X_{1}, …, X_{n} through its natural permutation action. If P = MN ⊂ G is a standard parabolic subgroup corresponding to the composition (n_{a}), then we have a commutative diagram

[figure: f; H_{0}(G); S_{G}; ℂ[X_{1}^{±1}, …, X_{n}^{±1}]^{𝔖_{n}}; f^{(P)}; H_{0}(M); S_{M}; ℂ[X_{1}^{±1}, …, X_{n}^{±1}]^{𝔖_{n_{1}} ×𝔖_{n_{2}} ×···×𝔖_{n_{k}}}]

The conceptual way to think about the algebra ℂ[X_{1}^{±1}, …, X_{n}^{±1}] is as the symmetric algebra ℂ[X_{∗}(T)] on the lattice of cocharacters of T. Similarly A is the algebra of Weyl group invariant elements in ℂ[X_{∗}(T)]. Write Ĝ, T̂, M̂ for the complex dual groups of G, T, M. Then ℂ[X_{∗}(T)] is the algebra of regular algebraic functions on T̂, A is the algebra of algebraic functions on T̂/𝔖_{n}, and the Hecke algebra of M is the space of regular algebraic functions on T̂/𝔖_{n_{1}} × 𝔖_{n_{2}} × ⋯ × 𝔖_{n_{k}}. If π is an unramified representation of G, then the space of K-invariants π^{K} is one-dimensional, and induces a surjection H_{0}(G) → End(π^{K}) = ℂ. We get an element φ_{G,π} ∈ SpecMax(H_{0}(G)) = T̂/𝔖_{n} which may be identified with a semisimple element φ_{G,π} ∈ Ĝ, well-defined up to conjugacy. This element is called the Hecke matrix of π. For example, if π is the trivial representation of G, then an element f ∈ H_{0}(G) acts on π by multiplication with the degree deg(f) = ∫_{G} f(g)dg. By the Iwasawa decomposition G = KP_{0} we have

∫_{G} f(g)dg = ∫_{P_{0}} ∫_{K} f(kp)dpdk = ∫_{P_{0}} f(p)dp = ∫_{T_{0}} δ_{P_{0}}^{−1/2}(t) · f^{(P_{0})}(t)dt

= ⟨S(f), ρ_{G}⟩,

where ρ_{G} = (\frac{n−1}{2}, \frac{n−1}{2}, …, \frac{1−n}{2}) ∈ X^{∗}(T) ⊗ ℤ[1/2] is the half sum of the positive roots of G. Equivalently, ⟨S(f), ρ_{G}⟩ is the evaluation of S(f) ∈ ℂ[T̂/𝔖_{n}] at the point

(3.1)  φ_{G,Triv} = (q^{\frac{n−1}{2}} , …, q^{\frac{1−n}{2}} ) ∈ T̂,

(the Hecke matrix of the trivial representation).

4. Weyl chambers and truncation of Hecke functions

Let P ⊂ G be a parabolic subgroup corresponding to the composition (n_{a}) of n. Let k be the length of n_{a}. Write 𝔞_{P} for the space X_{∗}(A_{P})⊗ℝ, where A_{P} ⊂ P is the center of P. We write 𝔞_{0} = 𝔞_{P_{0}}. Let 𝔞_{P} ⊂ 𝔞_{0} be the set of (x_{i}) ∈ ℝ^{n} so that x_{1} = … = x_{n_{1}}, …, x_{n_{1}+...+n_{k−1}+1} = … = x_{n}. The acute Weyl chamber 𝔞_{P}^{+} in 𝔞_{P} is the set of (x_{a}) ∈ ∏_{a=1}^{k} ℝ^{n_{a}} = ℝ^{n} such that for all indices a = 1, …, k − 1 we have x_{a} ∈ ℝ^{n_{a}}, and \frac{1}{n_{a}} ∑_{i=1}^{n_{a}} x_{a,i} > \frac{1}{n_{a+1}} ∑_{i=1}^{n_{a+1}} x_{a+1,i}. The obtuse Weyl chamber ^{+}𝔞_{P} in 𝔞_{P} is the set of (x_{i}) ∈ ℝ^{n} so that for all a we have

x_{1} + … + x_{n_{a}} > \frac{n_{1} + n_{2} + … + n_{a}}{n} (x_{1} + x_{2} + … + x_{n}).

Let X = X_{1}^{e_{1}} ⋯ X_{n}^{e_{n}} ∈ ℂ[X_{1}^{±1}, …, X_{n}^{±1}] = ℂ[X_{∗}(T)] = A be a monomial. We may view X_{∗}(T) as a lattice inside the space 𝔞_{0}. Let π_{P} : 𝔞_{0} → 𝔞_{P} be the canonical projection. Let f ↦ χ_{N}f be the endomorphism of A sending X ∈ X_{∗}(T) to X if π_{P} (X) ∈ 𝔞_{P}^{+} and to 0 if π_{P} (X) ∉ 𝔞_{P}^{+}. Similarly, let f ↦ χ̂_{N}f be the endomorphism of A sending X ∈ X_{∗}(T) to X if π_{P} (X) ∈ ^{+}𝔞_{P}, and to 0 if π_{P} (X) ∉ ^{+}𝔞_{P}). Via the Satake isomorphism H_{0}(T) ≅ A we transport the endomorphisms f ↦ χ_{N}f, f ↦ χ̂_{N}f to endomorphisms of H_{0}(T).

5. Functions of Kottwitz

We introduce the functions of Kottwitz f_{nαs} that will be used later in the paper:

Definition 5.1. Let n and α be positive integers, and let s be a non-negative integer with s ≤ n. We call the number s the signature, and we call the number α the degree. The function f_{nαs} ∈ H_{0}(G) is the spherical function whose Satake transform is

q^{αs(n−s)/2} ∑_{ν∈𝔖_{n}·μ_{s}}[ν]^{α} = q^{αs(n−s)/2} ∑_{I⊂{1,...,n},#I=s} ∏_{i∈I} X_{i}^{α} ∈ A. (5.1)

We put f_{nαs} = 0 when n, α, s ∈ ℤ_{≥0} are such that n < s. We will call f_{nαs} a simple Kottwitz function.

6. Compact traces

We introduce the notion of compact trace for the group G (cf. [6], [20, §2.2], [20, p. 509]). An element g ∈ G is compact if the roots α_{i} ∈ \overline{F} of its characteristic polynomial charpol(g) = det(1 − Xg) ∈ F [X] all have the same absolute value. We write χ_{c}^{G} for the characteristic function of the set of compact elements in G. If g lies inside a standard Levi subgroup M of G, then there are two inequivalent notions of compactness: (1) g ∈ M can be M-compact, (2) g ∈ G can be G-compact. Write M = ∏_{i} G_{n_{a}}, then the element g ∈ M is M-compact if g_{a} ∈ G_{n_{a}} is compact for all a. We write χ_{c}^{M} for the characteristic function on M of the set of M-compact elements. There is a notion of compactness and compact traces for general unramified reductive groups G′: A semisimple element γ ∈ G′ is compact if for some (any) maximal torus T′ in G′ containing γ the absolute value |α(γ)| equals 1 for all roots α of T′ in Lie(G′). A non-semisimple element of G′ is compact if its semisimple part is compact. Let π be a smooth G-representation of finite length and f a locally constant, compactly supported function on G. Harish-Chandra proved that there is a locally integrable function θ_{π} on G such that for all f ∈ H(G) the trace Tr (f, π) is given by the integral ∫_{G_{rs}} f(g)θ_{π}(g)dg where G_{rs} ⊂ G is the subset of regular semisimple elements. In this paper we are interested in the compact trace, Tr _{c}(f, π), which is defined by taking the integral over the set G_{c} compact, regular semisimple elements, ∫_{G_{c}} f(g)θ_{π}(g)dg (cf. [6]). Equivalently, write χ_{c}^{G}f for the pointwise product of f with χ_{c}^{G}; then Tr _{c}(f, π) = Tr (χ_{c}^{G}f, π).

7. Lattice paths and the Steinberg representation

Fix an integer s with 0 ≤ s ≤ n, and α ∈ ℤ_{≥1}. We express the compact trace of the functions f_{nαs} on the Steinberg representation in terms of certain lattice paths in ℚ^{2}. Let A^{+} be the polynomial ring ℂ[q^{a}|a ∈ ℚ] of rational, formal powers of the variable q. Equivalently, A^{+} is the complex group ring ℂ[ℚ^{+}] of the additive group ℚ^{+} underlying ℚ.

Definition 7.1. A path L in ℚ^{2} is a sequence of points v⃗_{0}, v⃗_{1}, v⃗_{2}, …, v⃗_{r} such that v⃗_{i+1} − v⃗_{i} = (1, 0) (east), or v⃗_{i+1} − v⃗_{i} = (1, 1) (north-east). The starting point of L is v⃗_{0} and the end point is v⃗_{r}; the number r is the length. An eastward step (1, 0) has weight 1 and a north-eastward step (a, b) → (a + 1, b + 1) has weight q^{−α·a} ∈ A^{+}. The weight of the path L is defined to be the product in A^{+} of the weights of its steps.

Remark. We allow paths of length zero; such a path consists of one point v⃗_{0} and no steps. The weight of a path of length 0 is equal to 1. The paths of length 0 correspond to compact traces on the special segments {⋆} introduced earlier.

Definition 7.2. Let L be a path in ℚ^{2}. Connect the starting point v⃗_{0} of L with its end point v⃗_{r} via a straight line ℓ. Then L is called a Dyck path if all of its points v⃗_{a} lie on or below the line ℓ in the plane ℚ^{2}. The Dyck path is called strict if none of its points v⃗_{a} other than the initial and end point, lies on the line ℓ. Let x⃗, y⃗ be two points in ℚ^{2}. Then we write Dyck_{s}(x⃗, y⃗) ∈ A^{+} for the sum of the weights of all the strict Dyck paths that go from the point x⃗ to the point y⃗. We call the polynomial Dyck_{s}(x⃗, y⃗) the strict Dyck polynomial.

There are also non-strict Dyck polynomials Dyck(\overline{x}, \overline{y}) but we are not concerned with those in this section; they will become important later, when we compute the compact traces on the trivial representation. Using obtuse and acute Weyl chambers we will define truncations of the Satake transforms of elements in H_{0}(G). Those truncations will be best understood graphically. We first extend the notion of a path slightly to the concept of a graph.

Definition 7.3. A graph in ℚ^{2} is a sequence of points v⃗_{0}, v⃗_{1}, …, v⃗_{r} with v⃗_{i+1}− v⃗_{i} = (1, e_{i}), where e_{i} is an integer.

Let ℓ_{s,n} ⊂ ℚ^{2} be the line of slope \frac{s}{n} going through the origin. We will often abuse notation, and write simply ℓ = ℓ_{s,n}. If x ∈ ℚ, then we write ℓ(x) for the point (x, \frac{s}{n}x) on the line ℓ.

Definition 7.4. To a monomial X  = X_{1}^{e_{1}}X_{2}^{e_{2}} ⋯ X_{n}^{e_{n}}  ∈

ℂ[X_{1}^{±1}, X_{2}^{±1}, …, X_{n}^{±1}], with e_{i} ∈ ℤ and ∑_{i=1}^{n} e_{i} = s we associate the graph G_{X} with points v⃗_{0} := ℓ(\frac{1−n}{2}), and for i = 1, …, n, the point v⃗_{i} is defined by

v⃗_{0} + (i, e_{n} + e_{n−1} + … + e_{n−(i+1)}) ∈ ℚ^{2}.

Definition 7.5. We define the weight of a step (a, b) → (a + 1, b + e) to be q^{−α·e·a} ∈ A^{+}, and the weight of a graph is the product of the weights of its steps.

Lemma 7.6. Let X = X_{1}^{e_{1}}X_{2}^{e_{2}} ⋯ X_{n}^{e_{n}} ∈ ℂ[X_{1}^{±1}, X_{2}^{±1}, …, X_{n}^{±1}] be a monomial of degree ∑_{i=1}^{n} e_{i} = s. Let G = G_{X} be the graph of X. Then

• the end point of the graph is ℓ(\frac{n−1}{2} + 1) ∈ ℚ^{2};

• the evaluation of X at the point φ_{G,Triv} ∈ T̂ is equal the weight of the

graph G_{X}. Furthermore, the following 3 statements are equivalent:

(i) χ̂_{N_{0}}X = X; (ii) e_{1} + e_{2} + … + e_{i} > \frac{s}{n}i, for all indices i < n; (iii) the graph G_{X} lies strictly below the line ℓ.

Proof. The fact stated in the first bullet point is trivial. The second point follows from Equation (3.1). The equivalence of (i)-(iii) is also formal (details can be found in the proof of [20, Prop. 5]). □

Lemma 7.7. Consider the representation π = 1_{T} (δ_{P_{0}}^{1/2}) of the group T. Let f be a function in the spherical Hecke algebra of T. Then the trace of f against π is equal to the evaluation of S(f) ∈ A at the point (q^{(1−n)/2}, q^{(3−n)/2}, …, q^{(n−1)/2}) ∈ T̂.

Proof. The character δ_{P_{0}}^{1/2} on T maps an element (t_{1}, t_{2}, …, t_{n}) to the complex number |t_{1}|^{(n−1)/2}|t_{2}|^{(n−3)/2} ⋯ |t_{n}|^{(n−1)/2}. To any (rational) cocharacter ν ∈ X_{∗}(T) we may associate the composition (δ_{P_{0}}^{1/2} ∘ ν): F^{×} → T → ℂ^{×}. We evaluate this composition at the prime element ϖ_{F} ∈ F^{×}. Thus we have an element of the set Hom(X_{∗}(T), ℂ^{×}) = T̂(ℂ) where the last isomorphism is given by X_{∗}(T̂) ⊗_{ℤ} ℂ^{×} ∋ ν ⊗ z ↦ ν(z) ∈ T̂(ℂ). We have T = (F^{×})^{n} and thus we have the standard basis e_{i} on X_{∗}(T). This corresponds to the standard basis e_{i} on X_{∗}(T̂). If we take ν = e_{i} in (δ_{P_{0}}^{1/2} ∘ ν)(ϖ_{F}) then we get (δ_{P_{0}}^{1/2} ∘ e_{i})(ϖ_{F}) = |ϖ_{F} |^{(n−1)/2−i+1} = q^{(1−n)/2+i−1}. This completes the verification. □

Lemma 7.8. The compact trace Tr (χ_{c}^{G}f_{nαs}, St_{G}) on the Steinberg representation is equal to the polynomial (−1)^{n−1}q^{s(n−s)α/2} · Dyck_{s}(ℓ((1 − n)/2), ℓ((n − 1)/2 + 1)) ∈ A^{+}.

Proof. This lemma is a translation of [20, Prop. 7] using Lemma 7.6. □

Compact traces are compatible with twists:

Lemma 7.9. Let χ be an unramified character of F^{×}, π a smooth irreducible G representation. Then Tr (χ_{c}^{G}f_{nαs}, π ⊗ χ) = χ(ϖ_{F}^{αs}) · Tr (χ_{c}^{G}f_{nαs}, π).

Proof. [20, Lem. 2]. □

Lemma 7.10. Assume that π is an essentially square integrable representation of the form ΔS, where S = ⟨x, y⟩ is a segment of length n. The compact trace Tr (χ_{c}^{G}f_{nαs}, Δ⟨x, y⟩) equals (−1)^{n−1} · q^{s(n−s)α/2} · Dyck_{s}(ℓ(x), ℓ(y + 1)).

Proof. The representation (ΔS) ⊗ ν^{−x+(1−n)/2} is the Steinberg representation, and so Lemma 7.8 applies to it. The result then follows from Lemma 7.9. □

8. Lattice t-paths and standard representations

We describe the compact traces on the standard representations of G using ‘t-paths’.

Definition 8.1. Let t be a positive integer. Let x⃗ = (x⃗_{a}) and y⃗ = (y⃗_{a}) be two ordered lists of points in ℚ^{2}, both of length t. A t-path from x⃗ to y⃗ is the datum consisting of, for each index a ∈ {1, 2, …, t}, a path L_{a} from the point x⃗_{a} to the point y⃗_{a}.

A t-path (L_{a}) is called a Dyck t-path if all the paths L_{a} are Dyck paths. The Dyck path (L_{a}) is called strict if, for each index a, no point v⃗_{i} of L_{a} other than v⃗_{0} and v⃗_{r} lies on the line ℓ. The weight weight(L_{a}) of a t-path (L_{a}) is the product of the weights of the paths L_{a}, where a ranges over the set {1, 2, …, t}. We extend the definition of the strict Dyck polynomial Dyck_{s}(x⃗, y⃗) ∈ A^{+} also to tpaths: The polynomial Dyck_{s}(x⃗, y⃗) ∈ A^{+} is by definition the sum of the weights of the strict Dyck t-paths from the points (x⃗_{a}) to the points (y⃗_{a}). The Dyck polynomial Dyck_{s}(x⃗, y⃗) decomposes into the product ∏_{a=1}^{t} Dyck_{s}(x⃗_{a}, y⃗_{a}) ∈ A^{+}.

Lemma 8.2. Let S_{1} = ⟨x_{1}, y_{1}⟩, S_{2} = ⟨x_{2}, y_{2}⟩, …, S_{t} = ⟨x_{t}, y_{t}⟩ be a list of segments and let I be the representation (ΔS_{1}) × (ΔS_{2}) × ⋯ × (ΔS_{t}). The compact trace Tr (χ_{c}^{G}f_{nαs}, I) is equal to (−1)^{n−t}q^{s(n−s)α/2}Dyck_{s}(x⃗, y⃗), where for the indices a = 1, …, t we have x⃗_{a} := ℓ(x_{a}) and y⃗_{a} := ℓ(y_{a} + 1).

Proof. Let P be the parabolic subgroup of G corresponding to the composition n = ∑_{a=1}^{t} ℓ(S_{a}) of n. Let χ_{M}^{G} be the characteristic function on M of the subset of elements m = (m_{a}) ∈ ∏_{a=1}^{t} G_{ℓ(S_{a})} = M such that ∏_{a=1}^{b} | det(m_{a})| =

| det(m)|^{\frac{1}{n} ∑_{a=1}^{b} n_{a}} for all b ≤ t. Then χ_{c}^{G} = χ_{M}^{G}χ_{c}^{M} holds on M. By the integration formula of van Dijk for compact traces [20, Prop. 3] we have

Tr (χ_{c}^{G}f, I) = Tr (χ_{c}^{G}f_{nαs}^{(P)}, (ΔS_{1}) × (ΔS_{2}) × ⋯ × (ΔS_{t}))

= Tr (χ_{c}^{M}χ_{M}^{G}f_{nαs}^{(P)}, (ΔS_{1}) × (ΔS_{2}) × ⋯ × (ΔS_{t})) . (8.1)

By [20, Prop. 4] the constant term χ_{c}^{G}f_{nαs}^{(P)} is equal to q^{αC(n⃗,s⃗)}f_{nαs_{1}} ⊗ f_{nαs_{2}} ⊗ ⋯ ⊗ f_{nαs_{t}} where n⃗ := (n_{a})_{a=1}^{t}, s⃗ := (s_{a})_{a=1}^{t}, s_{a} := n_{a}s/n, and C(n⃗, s⃗) := \frac{1}{2}s(n − s) − \frac{1}{2} ∑_{a=1}^{t} s_{a}(n_{a} − s_{a}). The constant term χ_{c}^{G}f_{nαs}^{(P)} vanishes in case one of the numbers s_{a} is non-integral. We have (χ_{c}^{G}f_{nαs}^{(P)})^{(P_{0}∩M)} = χ_{M}^{G}χ_{c}^{M}f_{nαs}^{(P_{0})}. Consequently, one may rewrite the trace in Equation (8.1) to the product q^{αC(n⃗,s⃗)} ∏_{a=1}^{t} Tr (χ_{c}^{G_{na}} f_{n_{a}αs_{a}}, ΔS_{a}). By Lemma 7.10 we obtain

q^{αC(n⃗,s⃗)} ∏_{a=1}^{t}(−1)^{n_{a}−1}q^{s_{a}(n_{a}−s_{a})α/2}Dyck_{s}(ℓ(x_{a}), ℓ(y_{a} + 1)).

Note that the vertical distance between the point y⃗_{a} and the point x⃗_{a} has to be integral before paths can exist. Therefore the expression in this last Equation simplifies to the one stated in the lemma and the proof is complete. □

9. Non-crossing paths

We express the compact traces on Speh representations in terms of non-crossing lattice paths. We call a t-path (L_{a}) crossing if there exists a couple of indices a, b with a ≠ b such that the path L_{a} has a point v⃗ ∈ ℚ^{2} in common with the path L_{b}. There is an important condition:

• The point v⃗ of crossing must appear in the list of points v⃗_{a,i} that define L_{a} and it must also occur in the list of points v⃗_{b,i} that define L_{b}

(Because we work with rational coordinates, the point of intersection could a priori be a point lying halfway a step of a path (for example). We are ruling out such possibilities.) We write Dyck_{s}^{+}(x⃗, y⃗) for the sum of the weights of the non-crossing strict Dyck t-paths. Let π be the Speh representation of G associated to the Zelevinsky segments ⟨x_{1}, y_{1}⟩, ⟨x_{2}, y_{2}⟩, …, ⟨x_{t}, y_{t}⟩ with x_{1} > x_{2} > … > x_{t} and y_{1} > y_{2} > … > y_{t}. We define the points x⃗_{a} := ℓ(x_{a}) ∈ ℚ^{2} and y⃗_{a} := ℓ(y_{a} + 1) ∈ ℚ^{2}, for a = 1, 2, …, t. The group 𝔖_{t} acts on the free ℚ^{2}-module ℚ^{2t} = (ℚ^{2})^{t} by Figure 1. An example of a 3-path corresponding to the representation π of GL_{54}(F) defined by the segments ⟨3, 20⟩, ⟨2, 19⟩ and ⟨1, 18⟩. We take s = 27 and we take the permutation w = (13) ∈ 𝔖′_{3}. The 3 dots on the lower left hand corner are the points x⃗_{1}, x⃗_{2} and x⃗_{3} in ℚ^{2} respectively; the points y⃗_{1}, y⃗_{2} and y⃗_{3} are in the upper right corner. Observe that this 3-path is non-strict.

sending the a-th standard basis vector e_{a} ∈ (ℚ^{2})^{t} to the basis vector e_{w(a)} ∈ (ℚ^{2})^{t}. Thus if we have the vector x⃗ ∈ ℚ^{2t}, then we get the new vector x^{w} whose a-th coordinate x_{a}^{w} ∈ ℚ^{2t} is equal to w(a)-th coordinate of the vector x⃗. Note that the difference \frac{s}{n} · (y_{a} + 1) − \frac{s}{n} · x_{a} need not be integral, in this case there do not exist paths from the point x_{a}^{w} ∈ ℚ^{2} to y⃗_{a} ∈ ℚ^{2}. Let π be a Speh representation of type (h, t). The points x⃗_{a} ∈ ℚ^{2} and y⃗_{a} ∈ ℚ^{2} lie on the line ℓ ⊂ ℚ^{2}, and the point x⃗_{a} lies on the left of the point y⃗_{a} with horizontal distance y_{a}+1−x_{a} = ℓ(S_{a}) = h. The two lists of points may overlap: There could exist couples of indices (a, b) such that x⃗_{a} = y⃗_{b}. All points x⃗_{a} and y⃗_{b} are distinct if we have h ≥ t (cf. Figure 1). Assume h ≥ t. Then, because all the points x⃗_{a}, y⃗_{b} are distinct, there is no permutation w ∈ 𝔖_{t} such that one of the segments S_{a}^{w} = ⟨x_{w(a)}, y_{a}⟩ is empty or equal to {⋆} for some index a. In particular we have 𝔖′_{t} = 𝔖_{t}.

Definition 9.1. To any point v⃗ ∈ ℚ^{2} we associate the invariant ρ(v⃗) := p_{2}(v⃗) ∈ ℚ/ℤ where p_{2} : ℚ^{2} → ℚ is projection on the second coordinate.

Remark. The horizontal distance between the point x⃗_{b} and the point y⃗_{a} is integral for all indices. Therefore the invariant of the first coordinate is not of interest. However, the vertical distance is the number s_{a}^{w} = \frac{s}{n}n_{a}^{w} ∈ ℚ, which certainly need not be integral.

Using this invariant we define a particular permutation w_{0} ∈ 𝔖_{t}:

Definition 9.2. Assume h ≥ t and assume that for each invariant ρ ∈ ℚ/ℤ the number of indices a such that the point x⃗_{a} has invariant ρ is equal to the number of indices a such that the point y⃗_{a} has invariant ρ. The element w_{0} ∈ 𝔖_{t} is the unique permutation such that for all indices a, b we have

(a < b and ρ(x⃗_{a}) = ρ(x⃗_{b}))
⟹ (w_{0}^{−1}(a) > w_{0}^{−1}(b) and ρ(y⃗_{a}) = ρ(y⃗_{b}) = ρ(x⃗_{a})) .

Remark. Observe that the permutation w_{0} depends on the integer s because the heights of the points x⃗_{a}, y⃗_{a}, and therefore also their invariants depend on s.

Remark. If our assumption on the invariants ρ(x⃗_{a}) and ρ(y⃗_{a}) in Definition 9.2 is not satisfied, then the permutation w_{0} cannot exist because it has to induce bijections between sets of different cardinality.

One could also define the permutation w_{0} ∈ 𝔖_{t} inductively: First the index w_{0}^{−1}(t) ∈ {1, 2, 3, …, t} is the minimal index b such that the points x⃗_{t} and y⃗_{b} have the same invariant. Next, the index w_{0}^{−1}(t − 1) ∈ {1, 2, 3, …, t} is the minimal index b, different from w_{0}^{−1}(t), such that x⃗_{a} and y⃗_{b} have the same invariant. And so on: w_{0}^{−1}(t−i) ∈ {1, 2, 3, …, t} is the minimal index b different from the previously chosen indices w_{0}^{−1}(t), w_{0}^{−1}(t − 1), …, w_{0}^{−1}(t − i + 1), such that the points y⃗_{b} and x⃗_{t−i} have the same invariant.

Lemma 9.3. Let π be a Speh representation with parameters h, t with h ≥ t. Let d be the greatest common divisor of n and s and write m for the quotient n/d. Define the points x⃗_{a} := ℓ(x_{a}) and y⃗_{a} := ℓ(y_{a} + 1). The following two statements are equivalent:

(i) for each invariant ρ ∈ ℚ/ℤ the number of indices a such that the point x⃗_{a} has invariant ρ is equal to the number of indices a such that the point y⃗_{a} has invariant ρ

(ii) m divides t or m divides h

Proof. We first prove that “m|t ⇒ (i)”. We have

(9.1)  ρ(x⃗_{a+1}) = ρ(x⃗_{a}) − \frac{s}{n} ∈ ℚ/ℤ

and the same relation for the points y⃗_{a}. Therefore, if m divides t, then the possible classes of the points x⃗_{a} are equally distributed over the subset \frac{s}{n}ℤ/ℤ ⊂ ℚ/ℤ, and every invariant occurs precisely t/m times. The same statement also holds for the points y⃗_{a}, and in particular (i) is true. This proves the claim. We prove that “m|h ⇒ (i)”. Assume m|h. Then the invariants of x⃗_{a} and y⃗_{a} are the same for all indices a. Thus (i) is true. We prove that “(m |̸ t and m |̸ h) ⇒ ((i) is false)”. Assume m |̸ t and m |̸ h. We first reduce to the case where t < m. Assume t ≥ m. Consider the lists of elements ρ(x⃗_{1}), ρ(x⃗_{2}), …, ρ(x⃗_{m}), and ρ(y⃗_{1}), ρ(y⃗_{2}), …, ρ(y⃗_{m}) of ℚ/ℤ. By Equation (9.1) every possible class in \frac{s}{n}ℤ/ℤ occurs precisely once in both lists. Thus, the truth value of (i) is not affected if we remove the elements x⃗_{1}, x⃗_{2}, …, x⃗_{m} and y⃗_{1}, y⃗_{2}, …, y⃗_{m} from the respective lists. Renumber the indices and repeat this argument until t < m. Because we assumed that t did not divide m there remains a positive number of elements in the list x⃗_{a} and y⃗_{a}. We renumber so that the indices range from 1 to t. Then we have reduced to the case where 1 ≤ t < m. Now look at the two lists ρ(x⃗_{1}), ρ(x⃗_{2}), …, ρ(x⃗_{t}) and ρ(y⃗_{1}), ρ(y⃗_{2}), …, ρ(y⃗_{t}). In both lists every class in ℚ/ℤ occurs at most once. We assumed that m does not divide h, and therefore ρ(x⃗_{1}) ≠ ρ(y⃗_{1}). If there does not exist an index b such that ρ(x⃗_{1}) = ρ(y⃗_{b}), then (i) is false and we are done. Thus assume ρ(x⃗_{1}) = ρ(y⃗_{b}) for some 1 < b ≤ t. By Equation (9.1) we then have ρ(y⃗_{b−1}) = ρ(y⃗_{b}) + \frac{s}{n} = ρ(x⃗_{1}) + \frac{s}{n}. The invariant ξ := ρ(x⃗_{1}) + \frac{s}{n} ∈ ℚ/ℤ does not occur in the list ρ(x⃗_{1}), ρ(x⃗_{2}), …, ρ(x⃗_{t}). Thus, we have found an invariant, namely ξ, occurring once in the list of invariants of the elements y⃗_{a} and does not occur in the list of invariants of the elements x⃗_{a}. This contradicts (i) and completes the proof. □

Theorem 9.4. Let π be a Speh representation with parameters h, t with h ≥ t. Let d be the greatest common divisor of n and s and write m for the quotient n/d. Define the points x⃗_{a} := ℓ(x_{a}) and y⃗_{a} := ℓ(y_{a} + 1) (for x_{a}, y_{a}, see Def. 2.2). The compact trace Tr (χ_{c}^{G}f_{nαs}, π) on π is non-zero if and only if m divides t or m divides h, and if the compact trace is non-zero, then it is equal to (−1)^{n−t}sign(w_{0})q^{s(n−s)α/2}Dyck_{s}^{+}(x^{w_{0}}, y⃗), where the permutation w_{0} ∈ 𝔖_{t} depends on s and is defined in Definition 9.2.

Proof. For a technical reason we assume that 0 < s < n in this proof. In case s = 0 we have f_{nαs} = 1_{GL_{n}(O_{F})}. All the elements in GL_{n}(O_{F}) are compact and therefore χ_{c}^{G}f_{nα0} = f_{nα0}. The compact trace becomes the usual trace and the theorem is easy. A similar argument applies in case s = n. Thus we may indeed assume 0 < s < n. To ease notation, we write f = f_{nαs}. By Theorem 2.3 the compact trace Tr (χ_{c}^{G}f, π) is equal to the combinatorial sum ∑_{w∈𝔖_{t}} sign(w)Tr (χ_{c}^{G}f, I_{w}) for any Hecke operator f ∈ H(G). We apply it to the Kottwitz functions f = f_{nαs}. We have 𝔖′_{t} = 𝔖_{t} because h ≥ t. Let w ∈ 𝔖_{t}. Thus we have the formula

(9.2)  Tr (χ_{c}^{G}f, π) = ∑_{w∈𝔖_{t}} sign(w) · Tr (χ_{c}^{G}f^{(P_{w})}, I_{w})

By Lemma 8.2 we get for f = f_{nαs},

(9.3)  Tr (χ_{c}^{G}f, π) = q^{s(n−s)α/2} ∑_{w∈𝔖_{t}} sign(w) · ε_{P_{0}∩M_{w}} · Dyck_{s}(x^{w}, y⃗).

We apply a standard combinatorial argument^{4} to simplify the right hand side. Put the lexicographical order “<” on ℚ^{2}, i.e. for all u⃗, v⃗ ∈ ℚ^{2} we have u⃗ < v⃗ if and only if (u⃗_{1} < v⃗_{1} or (u⃗_{1} = v⃗_{1} and u⃗_{2} < v⃗_{2})). Let (L_{a}) be a strict Dyck t-path from the points x^{w} to the points y⃗, and assume that (L_{a}) has at least one point of crossing. In particular (L_{a}) is the datum consisting of, for each index a, a path from x⃗_{w(a)} to y⃗_{a}. Let v⃗ ∈ ℚ^{2} be the point chosen among the points of crossing minimal for the lexicographical order on ℚ^{2}. Let (a, b) a couple of different indices, minimal for the lexicographical order on the set of all such couples, such that v⃗ lies on the path L_{a} and also on the path L_{b}. We define a new path L′_{a}, defined by following the steps of L_{b} until the point v⃗ and then following the steps of the path L_{a}. We define L′_{b} by following L_{a} until the point v⃗ and then continuing the path L_{b}. For the indices c with c ≠ a, b we define L′_{c} := L_{c}. Observe that (L′_{a}) is a t-path from the points x^{(ab)w} to the points y⃗. Furthermore, it is a Dyck path (with respect to this new configuration of points), and we have weight(L_{a}) = weight(L′_{a}) because the weight is the product of the weights of the steps, and only the order of the steps has changed in the construction (L_{a}) ↦ (L′_{a}). The construction is self-inverse: If we apply the construction to the path (L′_{a}) then we re-obtain (L_{a}). Both paths (L_{a}) and (L′_{a}) occur in the sum of Equation (9.3). The sign ε_{P_{0}∩M_{w}} is equal to (−1)^{n−1}(−1)^{t−1}(−1)^{#{c∈{1,2,...,t} | x⃗_{w(c)}=y⃗_{c}}}. By the assumption that h ≥ t, the points in the list x⃗ are all different to the points in the list y⃗, and therefore the sign ε_{P_{0}∩M_{w}} equals (−1)^{n−t} (and does not depend on the permutation w). The sign of the permutation w is opposite to the sign of (ab)w. Consequently, the contributions of the paths (L_{a}) and (L′_{a}) to Equation (9.3) cancel, and only the non-crossing paths remain in the sum. We find

4The Lindström-Gessel-Viennot lemma. The argument appears in many (almost) equivalent forms in the literature. We learned and essentially copied it from Stanley’s book [30, Thm 2.7.1]. Note however that, strictly speaking, the Theorem 2.7.1 there does not apply as stated at this point in our argument. In the paragraph that follows we show that Stanley’s argument may be adapted so that it does apply to our situation.

(9.4)  Tr (χ_{c}^{G}f, π) = (−1)^{n−t}q^{s(n−s)α/2} ∑_{w∈𝔖_{t}} sign(w) · Dyck_{s}^{+}(x^{w}, y⃗).

We need a second notion of crossing paths, called topological intersection. Here we mean that, when the t-path L is drawn in the plane ℚ^{2} there is a point x⃗ ∈ ℚ^{2} lying on two paths L_{a}, L_{b} occurring in L. Because we allow rational coordinates, topological intersection is not the same as crossing: It is easy to give an example of a 2-path has one topological intersection point x⃗ ∈ ℚ^{2} but the point x⃗ does not occur in the lists of points v⃗_{1,0}, v⃗_{1,1}, …, v⃗_{1,r_{1}}, v⃗_{2,0}, v⃗_{2,1}, …, v⃗_{2,r_{2}} defining the 2-path. Such paths are considered non-crossing under our definition, even though they may have topological intersection points^{5}. We claim that there is at most one permutation w ∈ 𝔖_{t} such that the polynomial Dyck^{+}(x^{w}, y⃗) is non-zero, and that this permutation is the one we defined in Definition 9.2. Let 𝔖′′_{t} be the set of all permutations such that Dyck^{+}(x^{w}, y⃗) ≠ 0, and assume that 𝔖′′_{t} contains an element w ∈ 𝔖′′_{t}. We first make the following observation: (Obs) To any point v⃗ ∈ ℚ^{2} we associated the invariant ρ(v⃗) = p_{2}(v⃗) ∈ ℚ/ℤ.

The horizontal distance between the point x⃗_{w(a)} and the point y⃗_{a} is the number n_{a}^{w}. The vertical distance is the number s_{a}^{w} = \frac{s}{n}n_{a}^{w} ∈ ℚ. Because w ∈ 𝔖′′_{t} there exists a path from the point x⃗_{w(a)} to the point Figure 2. The leftmost point x⃗_{t} is connected to the third point y_{w^{−1}(t)}, and the second point x⃗_{a} is connected to the last point y⃗_{b}. Any 2-path staying below the line ℓ must selfintersect topologically.

5If one uses the wrong, topological notion of intersection, then the proof breaks at 9 lines below Equation (9.3): The constructed ‘path’ (L′_{a}) is not a path.

y⃗_{a}. Consequently s_{a}^{w} is integral. This implies that ρ(x⃗_{w(a)}) = ρ(y⃗_{a}) for all indices a and in particular the invariant of the point x⃗_{w(a)} is independent of w ∈ 𝔖′′_{t}.

By assumption Dyck^{+}(x^{w}, y⃗) ≠ 0 and thus there exists a non-crossing Dyck path (L_{a}) connecting x^{w} with y⃗. We show inductively that w is uniquely determined. We start with showing that the index w^{−1}(t) ∈ {1, 2, …, t} is determined. We claim that w^{−1}(t) ∈ {1, 2, …, t} is the minimal index such that the point y_{w^{−1}(t)} has the same invariant as x⃗_{t}. To see that this claim is true, suppose for a contradiction that it is false, i.e. assume the index w^{−1}(t) is not minimal. There is an index b strictly smaller than w^{−1}(t) such that y⃗_{b} has the same invariant as x⃗_{t}. By the observation (Obs) there exists an index a ≠ t such that x⃗_{a} has the same invariant as x⃗_{t} and such that x⃗_{a} is connected to y⃗_{b}. Draw a picture (see Figure 2) to see that the paths L_{a} and L_{t} must intersect topologically. But, by construction, the invariants of x⃗_{a} and x⃗_{t} are the same. Therefore, any topological intersection point of the paths L_{a} and L_{t} is a point of crossing. The paths L_{a} and L_{b} are crossing and thus (L_{a}) is crossing. This is a contradiction, and therefore the claim is true. Thus the value w^{−1}(t) is determined. We now look at the index t − 1. The point x⃗_{t−1} is connected to the point y_{w^{−1}(t−1)}. We claim that w^{−1}(t − 1) ∈ {1, 2, …, t} is the minimal index, different from w^{−1}(t), such that y_{w^{−1}(t−1)} has the same invariant as x⃗_{t−1}. The proof of this claim is the same as the one we explained for the index t. We may repeat the same argument for the remaining indices t − 2, t − 3, etc. Consequently w is uniquely determined by its properties, and equal to the permutation w_{0} defined in Definition 9.2. We proved that if the set 𝔖′′_{t} is non-empty, then it contains precisely one element, and this element is equal to w_{0}. Therefore, if the compact trace does not vanish, then m must divide t or m divides h by Lemma 9.3. Inversely, assume that m divides t or m divides h. The permutation w_{0} ∈ 𝔖_{t} exists by Lemma 9.3. We claim that Dyck_{s}^{+}(x^{w_{0}}, y⃗) ≠ 0, so that w_{0} ∈ 𝔖′′_{t}. To prove Figure 3. An example of a non-crossing 4-path (L_{a}) in case \frac{s}{n} = \frac{1}{2} ∈ ℚ/ℤ and t = 4. For each a, the path L_{a} first takes n_{a}^{w_{0}} − s_{a}^{w_{0}} horizontal steps and then s_{a}^{w_{0}} vertical steps. Note that paths with the same invariant do not intersect.

this, it suffices to construct one non-crossing t-path from the points x^{w_{0}} to the points y⃗. This is easy (see Figure 3): Let a be an index, and write n_{a}^{w_{0}} for the horizontal distance between x_{a}^{w_{0}} and y⃗_{a} and s_{a}^{w_{0}} for the vertical distance. The path L_{a} from x_{a}^{w_{0}} to y⃗_{a} is defined to be the path taking n_{a}^{w_{0}} − s_{a}^{w_{0}} eastward steps, and then s_{a}^{w_{0}} northeastward steps. Then (L_{a}) is a strict non-crossing t-path and therefore Dyck_{s}^{+}(x^{w_{0}}, y⃗) is non-zero. This completes the proof. □

10. A dual formula

We also want to compute the compact trace on the representation Speh(h, t) in case h ≤ t. The argument for Theorem 9.4 extends to the case where h ≤ t. This computation is more complicated, because the permutation w ∈ 𝔖_{t} that contributes to Equation (9.4) is no longer unique and the sign ε_{P_{0}∩M_{w}} in Equation (9.3) depends on w. We don’t reproduce the computation here, because there is a more elegant approach using the duality of Zelevinsky. The ring of Zelevinsky R has an involution ι, called the Zelevinsky involution which was first defined by Zelevinsky in [33]. Aubert [1] gave a refined definition of this involution via a character formula involving Jacquet modules (which, moreover, makes sense for all reductive groups). Aubert defines the involution by X^{ι} := ∑_{P =MN} ε_{P} Ind_{P}^{G}(X_{N}(δ_{P}^{−1/2})), for all X ∈ R, where

• X_{N} is the non-normalized Jacquet module, i.e. to a G-representation X it attaches the space of N-coinvariants X_{N} (which is an M-repre-sentation). The functor X ↦ X_{N} is additive, and thus extends to all objects X ∈ ℝ.

• Ind_{P}^{G} is unitary induction (sending unitary representations of M to unitary representation of G)

• ε_{P} = (−1)^{dim(A_{P} /A_{G})}, where A_{P} is the split center of P and A_{G} is the split center of G

(With ‘involution’ we mean that ι is an automorphism of the complex algebra R and it is of order two: ι^{2} = Id_{R}. )

The Zelevinsky dual of a Speh representation with parameters (h, t) is a Speh representation with the role of the parameters inversed, thus of type (t, h). Furthermore, taking the Zelevinsky dual of the formula of Tadic yields a new character formula, now in terms of duals of standard representations. Of course, the Zelevinsky dual of a standard representation is not standard, rather it is an unramified twist of products in R of one dimensional representations. Therefore, we compute first the compact trace on the one dimensional representations, then use van Dijk’s theorem [20, Prop. 3] to obtain formulas for products in R of one dimensional representations, and finally use the dual of Tadic’s formula to compute the compact traces on Speh representations with h ≤ t (opposite inequality to Theorem 9.4). We will then have computed the formula for all Speh representations.

11. The trivial representation

We compute the compact traces of spherical Hecke operators acting on the trivial representation of G. In Section 12 we will use this result, and the dual of Tadic’s formula, to compute the compact traces of the Kottwitz functions against the representations that were exlcuded in Theorem 9.4. We work with general unramified reductive groups G. In this paper we need the result only for G = GL_{n}(F), but the computation is valid in general. We fix notation, conventions and definitions on roots and convexes for unramified groups (cf. [24, §1] and [22, Chap. 1]):

• Fix P_{0} ⊂ G a Borel subgroup with Levi decomposition P_{0} = T N_{0}. Standardize the parabolic subgroups of G and their Levi-decomposition with respect to P_{0} = T N_{0}.

• Let P be a standard parabolic subgroup of G, then we write – A_{P} for the split center of P;

– ε_{P} = (−1)^{dim(A_{P} /A_{G})};

– 𝔞_{P} := X_{∗}(A_{P}) ⊗ ℝ.

• If P ⊂ P′ then we have A_{P′} ⊂ A_{P} and thus an induced map 𝔞_{P′} → 𝔞_{P}.

• 𝔞_{P}^{P′} is the quotient of 𝔞_{P} by 𝔞_{P′}.

• 𝔞_{0} = 𝔞_{P_{0}} and 𝔞_{0}^{G} = 𝔞_{P_{0}}^{G}.

• Δ is the set of simple roots of A_{P_{0}} occurring in the Lie algebra of N_{0}.

• For each root α in Δ we have a coroot α^{∨} in 𝔞_{0}^{G}.

• We write Δ_{P} ⊂ Δ for the subset of α ∈ Δ acting non-trivially on A_{P}.

• For α ∈ Δ_{P} ⊂ Δ we send the coroot α^{∨} ∈ 𝔞_{0}^{G} to the space 𝔞_{P}^{G} via the canonical surjection 𝔞_{0}^{G} → 𝔞_{P}^{G}. The set of these restricted coroots α^{∨}|_{𝔞_{P}^{G}} with α ranging over Δ_{P} form a basis of the vector space 𝔞_{P}^{G}. By definition the set of fundamental weights {ϖ_{α}^{G} ∈ 𝔞_{P}^{G∗} | α ∈ Δ_{P} } is the basis of 𝔞_{P}^{G∗} = Hom(𝔞_{P}^{G}, ℝ) dual to the basis {α_{𝔞_{P}^{G}}^{∨}} of coroots.

• We have the acute and obtuse Weyl chambers of G.

– The acute chamber 𝔞_{P}^{G+} is the set of x ∈ 𝔞_{P}^{G} such that ⟨α, x⟩ > 0 for all roots α ∈ Δ_{P}.

– The obtuse chamber ^{+}𝔞_{P}^{G} is the set of x ∈ 𝔞_{P}^{G} such that we have the inequality ⟨ϖ_{α}^{G}, x⟩ > 0 for all fundamental weights ϖ_{α}^{G}, associated to α ∈ Δ_{P}.

– We need another chamber ^{≤}𝔞_{P}^{G} defined to be the set of x ∈ 𝔞_{P}^{G} such that for all α ∈ Δ_{P} the pairing ⟨ϖ_{α}^{G}, x⟩ is non-positive. We call this chamber the closed opposite obtuse Weyl chamber.

• Let ^{≤}τ_{P}^{G} be the characteristic function on 𝔞_{P} of the closed opposite obtuse chamber.

• Let H_{M}: M → 𝔞_{P} be the Harish-Chandra mapping, normalized such that |χ(m)|_{p} = q^{−⟨χ,H_{M} (m)⟩} for all rational characters χ of M.

• We define the following characteristic functions on T:

– ξ_{c}^{G} = ^{≤}τ_{P_{0}}^{G} ∘ (𝔞_{P_{0}} → 𝔞_{P_{0}}^{G}) ∘ H_{M_{0}};

– χ_{N} = τ_{P}^{G} ∘ (𝔞_{P} → 𝔞_{P}^{G}) ∘ H_{M};

– χ̂_{N} = τ_{P}^{G} ∘ (𝔞_{P} → 𝔞_{P}^{G}) ∘ H_{M}.

• If f ∈ H_{0}(G) is a function whose Satake transform is the function h ∈ A, then we often abuse notation, and write ξ_{c}^{G}h for the Satake transform of the function ξ_{c}^{G}f^{(P_{0})}, and similarly for the functions χ_{N}f and χ̂_{N}f if f ∈ H_{0}(M) (cf. Section 4).

Proposition 11.1. Let f be a function in the Hecke algebra H_{0}(G). The compact trace Tr (χ_{c}^{G}f, 1_{G}) is equal to Tr (ξ_{c}^{G}f^{(P_{0})}, 1_{T} (δ_{P_{0}}^{−1/2})).

Proof. For comfort we prove the proposition under the additional assumption that G is its own derived group. We have

Tr (χ_{c}^{G}f, 1) = ∑_{P =MN} ε_{P} Tr (χ̂_{N}f^{(P)}, 1(δ_{P}^{−1/2})).

Write Â_{P} for the complex dual torus of A_{P}. We write φ_{M,ρ} ∈ Â_{P} for the Hecke matrix of a representation ρ of M. The Hecke matrix φ_{M,δ_{P}^{−1/2}} is conjugate in

M̂ to the Hecke matrix φ_{T,δ_{P}^{−1/2}δ_{P_{0}∩M}^{−1/2}} = δ_{T,δ_{P_{0}}^{−1/2}} ∈ Â_{P_{0}} ⊂ M̂. The trace on the

trivial representation Tr (χ̂_{N}f^{(P)}, 1(δ_{P}^{−1/2})) is equal to S(χ̂_{N}f^{(P_{0})})(φ_{T,δ_{P_{0}}^{−1/2}}).

Using linearity of the Satake transform we obtain

Tr (χ_{c}^{G}f, 1) = S ( ∑_{P =MN} ε_{P} χ̂_{N}f^{(P_{0})}) (φ_{T,δ_{P_{0}}^{−1/2}}).

Thus we have to compute the function ∑_{P =MN} ε_{P} χ̂_{N} on the group T. By definition, the function χ̂_{N} equals τ_{P}^{G} ∘ H_{M}. Let W_{M} be the rational Weyl group of T in M. Let t ∈ T. Then H_{M}(t) = \frac{1}{#W_{M}} ∑_{w∈W_{M}} wH_{T} (t). Thus χ̂_{N}(t) = 1 if and only if, for all roots α ∈ Δ_{P} we have ∑_{w∈W_{M}} ⟨ϖ_{α}^{G}, wH_{T} (t)⟩ > 0. We have for all α ∈ Δ_{P} the inequality ⟨ϖ_{α}^{G}, H_{T} (t)⟩ > 0 if and only if we have ⟨ϖ_{α}^{G}, wH_{T} (t)⟩ > 0 for all w ∈ W_{M}. Therefore, we have on the group T that χ̂_{N} = τ_{P}^{G} ∘ H_{T}. Thus the alternating sum ∑_{P =MN} ε_{P} χ̂_{N} is equal to (∑_{P =MN} ε_{P} τ_{P}^{G}) ∘ H_{T}. By inclusion-exclusion we have ∑_{P =MN} ε_{P} τ_{P}^{G} = ^{≤}τ_{P}^{G}. This proves the proposition in case G = G_{der}. (For a group G with G ≠ G_{der}, one may repeat the above argument, replacing eaching occurance of A_{P} with A_{P} /A_{G}) □

12. The dual formula

Using the computations from Section 11 we give the dual version of Theorem 9.4.

Lemma 12.1. Let T_{1} = ⟨u_{1}, v_{1}⟩, T_{2} = ⟨u_{2}, v_{2}⟩, …, T_{h} = ⟨u_{h}, v_{h}⟩ be a list of segments and consider the representation J := (ΔT_{1})^{ι} × (ΔT_{2})^{ι} × ⋯ × (ΔT_{h})^{ι}. Then Tr (χ_{c}^{G}f_{nαs}, π) is equal to q^{s(n−s)α/2}Dyck(u⃗, v⃗), where u⃗_{a} = ℓ(u_{a}) and v⃗_{a} = ℓ(v_{a} + 1) for a = 1, 2, …, t.

Proof. The proof is the same as the proof for Lemma 7.8 (but now use Proposition 11.1). We repeat the argument for verification purposes. Assume first that h = 1 and that π is the trivial representation of G. In this case the trace Tr (χ_{c}^{G}f_{nαs}, π) is equal to Tr (ξ_{c}^{G}f_{nαs}^{(P_{0})}, 1_{T} (δ_{P_{0}}^{−1/2})). To a monomial

X = X_{1}^{e_{1}}X_{2}^{e_{2}} ⋯ X_{n}^{e_{n}} ∈ ℂ[X_{1}^{±1}, X_{2}^{±1}, … X_{n}^{±1}] with e_{i} ∈ ℤ and ∑_{i=1}^{n} e_{i} = s we associate the graph G_{X} with points v⃗_{0} := ℓ((1 − n)/2), and for i = 1, 2, …, n, v⃗_{i} equals v⃗_{0} + (i, e_{1} + e_{2} + … + e_{i}) ∈ ℚ^{2}. We have ξ_{c}^{G}X = X if and only if

(12.1)  e_{1} + e_{2} + ⋯ + e_{i} ≤ \frac{s}{n}i,

for all indices i < n, and ξ_{c}^{G}X = 0 otherwise. The evaluation of X at the point

(12.2)  (q^{(n−1)/2}, q^{(n−3)/2}, …, q^{(1−n)/2})

equals the weight of the graph G_{X}. The trace of f_{nαs} against the representation 1_{T} (δ_{P_{0}}^{−1/2}) is equal to the evaluation of f_{nαs} at the point in Equation (12.2) (use Lemma 7.7 but notice that the signs are different). The monomials X occurring S(f_{nαs}) yield paths from the point ℓ((1 − n)/2) ∈ ℚ^{2} to the point ℓ((n−1)/2+1). The condition in Equation (12.1) is true if and only if the graph G_{X} lies (non-strictly) below the line ℓ. Therefore the trace Tr (χ_{c}^{G}f_{nαs}, 1_{G}) is equal to q^{s(n−s)/2}Dyck(ℓ((1 − n)/2), ℓ((n − 1)/2 + 1)). By twisting with the character ν^{−x+(1−n)/2} as we did in Lemma 7.10 the trace Tr (χ_{c}^{G}f, (Δ⟨u, v⟩)^{ι}) equals q^{s(n−s)/2}Dyck(ℓ(x), ℓ(y + 1)) for all segments ⟨u, v⟩. Finally use van Dijk’s induction formula for compact traces to find the compact traces on duals of standard representations as stated in the lemma (this argument is the same as the argument in Lemma 8.2). □

Theorem 12.2. Let π be a Speh representation with parameters h, t with h ≤ t. Let d be the greatest common divisor of n and s and write m for the quotient n/d. Let T_{a} = ⟨u_{a}, v_{a}⟩ be the segments of π^{ι}. Define the points u⃗_{a} := ℓ(u_{a}) and v⃗_{a} := ℓ(v_{a} + 1). The compact trace Tr (χ_{c}^{G}f_{nαs}, π) is non-zero if and only if m divides h or m divides t. Assume that the compact trace is non-zero, then it is equal to (−1)^{t−1}(−1)^{n−h}sign(w_{0})q^{s(n−s)α/2}Dyck^{+}(u^{w_{0}}, v⃗), where the permutation w_{0} ∈ 𝔖_{h} is defined in Definition 9.2.

Proof. Let π^{ι} be the dual of the representation π. After dualizing the formula of Tadic for π^{ι} we obtain π = ∑_{w∈𝔖_{h}} sign(w)I_{w}^{ι} ∈ R. The involution ι on R commutes with products. Therefore, if T_{1}, …, T_{h} are the Zelevinsky segments of the dual representation π^{ι}, then I_{w}^{ι} is equal to (ΔT_{1})^{ι} × (ΔT_{2})^{ι} × ⋯ (ΔT_{h})^{ι}. By Lemma 12.1 the trace Tr (χ_{c}^{G}f_{nαs}, I_{w}^{ι}) equals q^{s(n−s)α/2}Dyck(u^{w}, v⃗). One may now repeat the proof for Theorem 9.4, but using the dual Tadic formula instead; one only has to interchange t with h and every occurrence of the word “strict Dyck t-path” with “Dyck h-path”. □

13. Return to Shimura varieties

At this point most of the original arguments of this paper are completed. In Sections 13 and 14 we shows how the computations from Sections 1–12 combine with arguments from our earlier article [20] to give results on the basic stratum in new cases. In our article [20] we proved a formula for the basic stratum of certain Shimura varieties associated to unitary groups, subject to a technical condition on the Newton polygon of the basic stratum (that it has no non-trivial integral points). In the previous sections we have completely resolved the combinatorial issues that arise if you remove this condition in case p is totally split in the center of the division algebra. We may now essentially repeat the argument from [20] to obtain the description of the cohomology if there is no condition on the Newton polygon of the basic stratum. A large part of the argument remains the same, that part will only be sketched and we refer to [20] for the details. We will work with a Kottwitz variety (cf. [16]). More precisely, we fix the following list of notation and assumptions:

The Shimura datum.

(1) Let D be a division algebra over ℚ; write n ∈ ℤ_{≥0} for the positive integer such that dim_{F} (D) = n^{2}.

(2) F is the center of D; we assume that F is a CM field of the form F = KF^{+} ⊂ \overline{ℚ}, where F^{+} is totally real, and K/ℚ is quadratic imaginary.

(3) ∗ is an anti-involution on D inducing complex conjugation on F.

(4) G is the ℚ-group with G(R) = {x ∈ D_{R}^{×}|g^{∗}g ∈ R^{×}} for every commutative ℚ-algebra R.

(5) h is an ℝ-algebra morphism h: ℂ → D_{ℝ} such that h(z)^{∗} = h(\overline{z}) for all z ∈ ℂ.

(6) We assume that the involution x ↦ h(i)^{−1}x^{∗}h(i) on D_{ℝ} is positive.

(7) We write X for the G(ℝ) conjugacy class of the restriction of h to ℂ^{×} ⊂ ℂ.

(8) μ ∈ X_{∗}(G) is the restriction of h ⊗ ℂ: ℂ^{×} × ℂ^{×} → G(ℂ) to the factor ℂ^{×} of ℂ^{×} × ℂ^{×} indexed by the identity isomorphism ℂ →^{∼} ℂ.

(9) E ⊂ \overline{ℚ} is the reflex field of the Shimura datum (G, X, h^{−1}).

The special fibre and the basic stratum.

(10) p is a prime number where the PEL datum fixed in (1)-(9) is unramified in the sense of [17, §5]; we assume additionally that p is split in the extension K/ℚ. In fact, for most of Sections 13 and 14 we make the following stronger assumption: (10b) From Hypothesis 14.4 onwards, we assume that p is split in the

extension F/ℚ.

(11) K ⊂ G(𝔸_{f}) is a compact open subgroup, small enough that the moduli problem is representable by a smooth and quasi-projective scheme over O_{E} ⊗ ℤ_{p}, and such that K decomposes as K^{p}K_{p} where K^{p} is a compact open subgroup of G(𝔸_{f}^{p}) and K_{p} is a hyperspecial compact open subgroup of G(ℚ_{p}).

(12) S = S_{K}/O_{E} ⊗ ℤ_{(p)} is the Shimura variety of level K corresponding to the PEL moduli problem.

(13) ν_{p}: \overline{ℚ} → \overline{ℚ}_{p} is a fixed embedding, ν_{∞}: \overline{ℚ} → ℂ is another fixed embedding, the fields F, F^{+}, E, K are all embedded into ℂ.

(14) 𝔭 is the E-prime induced by ν_{p}.

(15) 𝔽_{q} is the residue field of E at the prime 𝔭 and \overline{𝔽}_{q} is the residue field of \overline{ℚ} at ν_{p}; for every positive integer α, E_{𝔭,α} ⊂ \overline{ℚ}_{p} is the unramified extension of E_{𝔭} of degree α; 𝔽_{q^{α}} is the residue field of E_{𝔭,α}.

(16) ι: B ↪ S_{K,𝔽_{q}} is the basic stratum [26] (cf. [12, 17, 18, 27]).

Local system.

(17) ξ is an (any) irreducible algebraic representation over \overline{ℚ} of G\overline{ℚ}.

(18) ℓ is a prime number and \overline{ℚ}_{ℓ} is an algebraic closure of ℚ_{ℓ} together with an embedding \overline{ℚ} ⊂ \overline{ℚ}_{ℓ}.

(19) L is the ℓ-adic local system on S_{K}/O_{E} ⊗ ℤ_{(p)} associated to the representation ξ ⊗ \overline{ℚ}_{ℓ} of G_{\overline{ℚ}_{ℓ}} [17, p. 393].

Test functions and harmonic analysis.

(20) We consider functions of the form f = f_{∞} ⊗ f^{∞p} ⊗ f_{p}, with f_{∞} a function on G(ℝ), f^{∞p} a function on G(𝔸_{f}^{p}), f_{p} a function on G(ℚ_{p}), where:

• The function f_{∞} at infinity has its stable orbital integrals prescribed by the identities of Kottwitz in [15]. The function f_{∞} can be taken to be a scalar multiple of an Euler-poincaré function [16] (cf. [7]). In [16, p. 657, Lem. 3.2]) is explained that the function has the following property: Let π_{∞} be an (𝔤, K_{∞})-module occurring as the component at infinity of an automorphic representation π of G. The trace of f_{∞} against π_{∞} is equal to the Euler-Poincaré characteristic ∑_{i=0}^{∞} N_{∞}^{−1}(−1)^{i} dim H^{i}(𝔤, K_{∞}; π_{∞} ⊗ ξ), where N_{∞} is the following constant. Let Π_{∞}^{0} be the L-packet of isomorphism classes of irreducible admissible representations of G(ℝ) having the same central character and infinitesimal characters as the contragredient of ξ. Then N_{∞} is equal to the product of the number of representations in Π_{0}^{∞} and the number of connected components (for the real topology) of the group G(ℝ)/Z(ℝ), where Z is the center of G.

• The function f_{p} ∈ H(G(ℚ_{p})) at p is a pointwise product of the

form χ_{c}^{G}f_{α}, where

– χ_{c}^{G} is the characteristic function on G(ℚ_{p}) of the subset of compact elements (cf. [6]).

– The function f_{α} is the function of Kottwitz [15] associated to μ (cf. [20, Prop. 9]). Since the group G is unramified over ℚ_{p} we fix a smooth reductive model G_{ℤ_{p}} of G over ℚ_{p}. Define ϕ_{α} ∈ H_{0}(G(E_{𝔭,α})) to be the characteristic function of the subset G_{ℤ_{p}}(O_{E_{𝔭,α}})μ(p^{−1})G_{ℤ_{p}}(O_{E_{𝔭,α}}). By definition, the function f_{α} ∈ H_{0}(G(ℚ_{p})) is the function obtained from ϕ_{α} by the base change morphism from H_{0}(G(E_{𝔭,α})) to H_{0}(G(ℚ_{p})).

• f^{∞p} is an arbitrary function in the Hecke algebra H(G(𝔸_{f}^{p})) which is K^{p}-biinvariant.

The signatures of the unitary group.

(21) U ⊂ G is the subgroup of elements with trivial factor of similitudes.

(22) For each infinite F^{+}-place v, we have signatures p_{v}, q_{v} such that U(ℝ) ≅ ∏_{v} U(p_{v}, q_{v}). We define s_{v} := min(p_{v}, q_{v}).

(23) The embedding \overline{ℚ} ⊂ \overline{ℚ}_{p} induces an action of the group Gal(\overline{ℚ}_{p}/ℚ_{p}) on the set of infinite F^{+}-places. For each Gal(\overline{ℚ}_{p}/ℚ_{p})-orbit ℘ we define the number s_{℘} := ∑_{v∈℘} s_{v}, and we write σ_{℘} for the partition (s_{v})_{v∈℘} of the number s_{℘}.

The space of automorphic forms.

(24) A(G) := L^{2}(G(ℚ)\G(𝔸), ω), where ω is the inverse of the central character of ξ.

14. Combining the results

We compute the factors Tr (χ_{c}^{G}f_{α}, π_{p}) occurring in Theorem 14.3 below. We need to introduce two classes of representations:

Definition 14.1. Consider the general linear group G_{n} over a non-Archimedean local field. Then a representation π of G_{n} is called a (semistable) rigid representation if it is equal to a product of the form ∏_{a=1}^{k} Speh(x_{a}, y)(ε_{a}) ∈ R, where y is a divisor of n and (x_{a}) is a composition of n/y, and ε_{a} are unramified unitary characters.

Definition 14.2. A representation π of the group G(ℚ_{p}) = ℚ_{p}^{×} × ∏_{℘|p} GL_{n}(F_{℘}^{+}) is called a rigid representation if for each F^{+}-place ℘ above p the component π_{℘} is a (semistable) rigid representation of GL_{n}(F_{℘}^{+}) in the previous sense: π_{℘} = ∏_{a=1}^{k} Speh(x_{℘,a}, y_{℘})(ε_{℘,a}) ∈ R, where two additional conditions hold: (1) y_{℘} = y_{℘′} for all ℘, ℘′|p, and (2) the factor of similitudes ℚ_{p}^{×} of G(ℚ_{p}) acts through an unramified character on the space of π. We write y := y_{℘} and call the set of data (x_{℘,a}, ε_{℘,a}, y) the parameters of π.

We recall a theorem from our previous article [20]:

Theorem 14.3. Let α be a positive integer. Let f^{∞p} ∈ H(G(𝔸_{f}^{p})) be a K^{p}- biinvariant function. Assume the conditions (1)-(24) from §5.1. Then

∑_{i=0}^{∞}(−1)^{i}Tr (f^{∞p} × Φ_{𝔭}^{α}, H_{et}^{i}(B_{\overline{𝔽}_{q}}, ι^{∗}L)) = |Ker^{1}(ℚ, G)| · ∑_{{π⊂A(G) \atop πp rigid}} Tr (f, π), (14.1)

the sum ranges over those irreducible subspaces π ⊂ A(G) such that π_{p} is a rigid representation. In Equation (14.1), the group Ker^{1}(ℚ, G) the Hasse invariant of G:

Ker^{1}(ℚ, G) := Ker(H^{1}(ℚ, G) → ∏_{v} H^{1}(ℚ_{v}, G)) ,

where the product ranges over all places of ℚ.

Erratum to [20]. I should mention that, unfortunately, I made a few typos in [20] regarding the constant |Ker^{1}(ℚ, G)|. The mistakes are easily corrected and occur only in body of the text (the statement of the main result [20, Thm. 1] is correct). Let me explain how to correct [20]. One should know that to the PEL datum that we fixed in Section 13, one can associate a priori two Shimura varieties: (1) The Shimura variety S_{1} that represents the PEL moduli problem, and (2) The Shimura variety S_{2} which is the canonical model, as in [9]. Kottwitz proves in [17, Sect. 5] that S_{1} ≅ ∐_{Ker^{1}(ℚ,G)} S_{2}. In some statements I forgot to multiply the expressions with |Ker^{1}(ℚ, G)|, thus I was counting points in the basic locus of S_{2,𝔭}, instead of S_{1,𝔭}. This mistake occurs at the following places of [20]: p. 490 lines 2 and 14, Prop. 10, Eqn. (19) and the equation above it, Eqn. (20), Eqn. (21), proof of Thm. 3, and Cor. 2.

Remark. In Theorem 14.3 we fixed the choice of an isomorphism \overline{ℚ}_{ℓ} ≅ ℂ, to compare the left and right hand side of Equation (14.1).

Proof. Write T (f^{∞p}, α) for the left hand side of Equation (14.1). We had already found in [20, Prop. 10] that T (f^{∞p}, α) = |Ker^{1}(ℚ, G)|Tr (f, A(G)), for all sufficiently large integers α. Let π ⊂ A(G) be an automorphic representation of G contributing to the trace Tr (f, A(G)). In [20, p. 20] we explained that π may be base changed to an automorphic representation BC(π) of the algebraic group K^{×} × D^{×}, and that, in turn, BC(π) may be sent to an automorphic representation Π := JL(BC(π)) of the ℚ-group G^{+} = K^{×} × GL_{n}(F). The representation Π is a discrete automorphic representation of the group G^{+}(𝔸), and Π is semistable at p. Using the Moeglin-Waldspurger classification of the discrete spectrum, a computation now shows that π_{p} is a rigid representation [20, Thm. 2]. □

The point of this article is to make the traces Tr (f_{p}, π_{p}) = Tr (χ_{c}^{G}f_{α}, π_{p}) occurring in Equation (14.1) explicit. We carried out the important arguments in the previous sections, here we now only have to collect the results.

Hypothesis 14.4. From this point onwards, we work under the condition that p is split in the center F of the algebra D. (cf. Condition (10b) in the list of conditions from Section 13.)

Hypothesis 14.4 implies that G(ℚ_{p}) ≅ ℚ_{p}^{×} × ∏_{v∈Hom(F^{+},\overline{ℚ})} GL_{n}(ℚ_{p}). We have by [20, Prop. 9] that

(14.2)  f_{α} = 1_{q^{−α}ℤ_{p}^{×}} ⊗ ⨂_{v∈Hom(F^{+},ℝ)} f_{nαs_{v}}^{GL_{n}(ℚ_{p})} ∈ H_{0}(G(ℚ_{p})),

where the numbers s_{v} are the signatures of the unitary group (cf. Section 13). Let π_{p} be a rigid representation of G(ℚ_{p}). Write π_{p} = ⨂_{v} π_{p,v}, where π_{p,v} = ⨂_{a=1}^{k} Speh(x_{v,a}, y)(ε_{v,a}) and where the factor of similitudes of G(ℚ_{p}) acts through the unramified character ε_{s} on the space of π_{p}. We compute

Tr (χ_{c}^{G}f_{α}, π_{p}) =

= ε_{s}(q^{−α}) ∏_{v∈Hom(F^{+},ℝ)} Tr (χ_{c}^{GL_{n}(ℚ_{p})}f_{nαs_{v}}, Ind_{P (ℚ_{p})}^{GL_{n}(ℚ_{p})} ⨂_{a=1}^{r} Speh(x_{v}, y)(ε_{v,a}))

= ε_{s}(q^{−α}) ( ∏_{v∈Hom(F^{+},ℝ)} ∏_{a=1}^{r} ε_{v,a}(q^{−s_{v} \frac{y·x_{a}}{n} α})) ·

· ∏_{v∈Hom(F^{+},ℝ)} Tr (χ_{c}^{GL_{n}(ℚ_{p})}f_{nαs_{v}}, Ind_{P (ℚ_{p})}^{GL_{n}(ℚ_{p})} ⨂_{a=1}^{r} Speh(x_{v,a}, y)) .

Write ζ_{π}^{α} ∈ ℂ for the product ∏_{v} ∏_{a} ε_{a}(q^{−s_{v} \frac{y·x_{a}}{n} α}). The polynomial (14.3)

∏_{v∈Hom(F^{+},ℝ)} Tr (χ_{c}^{GL_{n}(ℚ_{p})}f_{nαs_{v}}, Ind_{P (ℚ_{p})}^{GL_{n}(ℚ_{p})} ⨂_{a=1}^{r} Speh(x_{v,a}, y)) ∈ ℂ[q^{α}],

is computed in Theorems 9.4 and 12.2 to be a polynomial defined by the weights of certain non-intersecting lattice paths. In particular the polynomial in Equation (14.3) vanishes unless, for all indices a, for all v ∈ Hom(F^{+}, ℝ), the number

(14.4)  m_{v,a} := \frac{y · x_{v,a}}{gcd (y · x_{v,a}, \frac{y·x_{v,a}}{n} s_{v})} = \frac{y · x_{v,a}}{gcd(n, s_{v})},

is an integer, and divides either x_{v,a} or y. Assume that the compact trace Tr (χ_{c}^{G}f_{α}, π_{p}) is non-zero so that the divisibility relations at Equation (14.4) are satisfied. The number ζ_{π}^{α} ∈ ℂ is defined by:

(14.5)  ε_{s}(q^{−α}) · ∏_{v∈Hom(F^{+},ℝ)} ∏_{a=1}^{r} ε_{v,a} (q^{−s_{v} \frac{y·x_{a}}{n} α}) = ζ_{π}^{α},

(cf. [20, Lem. 9, Eq. (24)]).

Remark. Let us comment on the assumption that p splits completely in F (Hypothesis 14.4). If p splits only in K, then the group G_{ℚ_{p}} is not split, and further combinatorial complications arise. The main issue is that the function of Kottwitz f_{α} in Equation (14.2) is no longer a tensor product of the simple Kottwitz functions f_{nαs}: The function f_{α} is a tensor product of ‘composite’ Kottwitz functions. (The composite Kottwitz functions f_{nασ} are obtained from partitions σ of s as follows. Let σ = (σ_{1}, σ_{2}, …, σ_{r}) be a partition of s. Then f_{nασ} ∈ H_{0}(G) is the convolution product f_{nασ_{1}} ∗ f_{nασ_{2}} ∗ ⋯ ∗ f_{nασ_{r}} ∈ H_{0}(G).)

References

[1] A.-M. Aubert. Dualité dans le groupe de Grothendieck de la catégorie des représentations lisses de longueur finie d’un groupe réductif p-adique. Trans. Amer. Math. Soc., 347(6):2179–2189, 1995.

[2] A. I. Badulescu. On p-adic Speh representations. 2011.

[3] A. Beĭlinson and J. Bernstein. Localisation de g-modules. C. R. Acad. Sci. Paris Sér. I Math., 292(1):15–18, 1981.

[4] G. Chenevier and D. Renard. Characters of Speh representations and Lewis Caroll identity. Represent. Theory, 12:447–452, 2008.

[5] N. Chriss and V. Ginzburg. Representation theory and complex geometry. Modern Birkhäuser Classics. Birkhäuser Boston Inc., Boston, MA, 2010. Reprint of the 1997 edition.

[6] L. Clozel. The fundamental lemma for stable base change. Duke Math. J., 61(1):255–302, 1990.

[7] L. Clozel and P. Delorme. Pseudo-coefficients et cohomologie des groupes de Lie réductifs réels. C. R. Acad. Sci. Paris Sér. I Math., 300(12):385–387, 1985.

[8] P. Deligne. Travaux de Shimura. In Séminaire Bourbaki, 23ème année (1970/71), Exp. No. 389, pages 123–165. Lecture Notes in Math., Vol.

244. Springer, Berlin, 1971.

[9] P. Deligne. Variétés de Shimura: interprétation modulaire, et techniques de construction de modèles canoniques. In Automorphic forms, representations and L-functions (Proc. Sympos. Pure Math., Oregon State Univ., Corvallis, Ore., 1977), Part 2, Proc. Sympos. Pure Math., XXXIII, pages 247–289. Amer. Math. Soc., Providence, R.I., 1979.

[10] P. Deligne and M. Rapoport. Les schémas de modules de courbes elliptiques. In Modular functions of one variable, II (Proc. Internat. Summer School, Univ. Antwerp, Antwerp, 1972), pages 143–316. Lecture Notes in Math., Vol. 349. Springer, Berlin, 1973.

[11] F. Diamond and J. Shurman. A first course in modular forms, volume 228 of Graduate Texts in Mathematics. Springer-Verlag, New York, 2005.

[12] L. Fargues. Cohomologie des espaces de modules de groupes p-divisibles et correspondances de Langlands locales. Astérisque, (291):1–199, 2004. Variétés de Shimura, espaces de Rapoport-Zink et correspondances de Langlands locales.

[13] M. Kashiwara and T. Tanisaki. The Kazhdan-Lusztig conjecture for Kac-Moody Lie algebras. Sūrikaisekikenkyūsho Kōkyūroku, (895):44–66, 1995. Noncommutative analysis on homogeneous spaces (Japanese) (Kyoto, 1994).

[14] R. E. Kottwitz. Isocrystals with additional structure. Compositio Math., 56(2):201–220, 1985.

[15] R. E. Kottwitz. Shimura varieties and λ-adic representations. In Automorphic forms, Shimura varieties, and L-functions, Vol. I (Ann Arbor, MI, 1988), volume 10 of Perspect. Math., pages 161–209. Academic Press, Boston, MA, 1990.

[16] R. E. Kottwitz. On the λ-adic representations associated to some simple Shimura varieties. Invent. Math., 108(3):653–665, 1992.

[17] R. E. Kottwitz. Points on some Shimura varieties over finite fields. J. Amer. Math. Soc., 5(2):373–444, 1992.

[18] R. E. Kottwitz. Isocrystals with additional structure. II. Compositio Math., 109(3):255–339, 1997.

[19] A. Kret. Stratification de Newton des varietes de Shimura et formule des traces d’Arthur-Selberg. thesis, Paris-sud, 2012.

[20] A. Kret. The basic stratum of some simple Shimura varieties. Math. Ann., 356(2):487–518, 2013.

[21] A. Kret and E. Lapid. Jacquet modules of ladder representations. C. R. Math. Acad. Sci. Paris, 350(21-22):937–940, 2012.

[22] J.-P. Labesse and J.-L. Waldspurger. La formule des traces tordue d’apres le Friday morning seminar. Preprint, 2012.

[23] E. Lapid and A. Mínguez. On a determinantal formula of Tadić. Amer. J. Math., To appear.

http://www.ma.huji.ac.il/~erezla/publications.html.

[24] G. Laumon and M. Rapoport. The Langlands lemma and the Betti numbers of stacks of G-bundles on a curve. Internat. J. Math., 7(1):29–45, 1996.

[25] C. Mœglin and J.-L. Waldspurger. Sur l’involution de Zelevinski. J. Reine Angew. Math., 372:136–177, 1986.

[26] M. Rapoport. A guide to the reduction modulo p of Shimura varieties. Astérisque, (298):271–318, 2005. Automorphic forms. I.

[27] M. Rapoport and M. Richartz. On the classification and specialization of F -isocrystals with additional structure. Compositio Math., 103(2):153–181, 1996.

[28] F. Rodier. Représentations de GL(n, k) où k est un corps p-adique. In Bourbaki Seminar, Vol. 1981/1982, volume 92 of Astérisque, pages 201– 218. Soc. Math. France, Paris, 1982.

[29] I. Satake. Theory of spherical functions on reductive algebraic groups over p-adic fields. Inst. Hautes Études Sci. Publ. Math., (18):5–69, 1963.

[30] R. P. Stanley. Enumerative combinatorics. Volume 1, volume 49 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, second edition, 2012.

[31] M. Tadić. On characters of irreducible unitary representations of general linear groups. Abh. Math. Sem. Univ. Hamburg, 65:341–363, 1995.

[32] I. Vollaard and T. Wedhorn. The supersingular locus of the Shimura variety of GU(1, n − 1) II. Invent. Math., 184(3):591–627, 2011.

[33] A. V. Zelevinsky. Induced representations of reductive p-adic groups. II. On irreducible representations of GL(n). Ann. Sci. École Norm. Sup. (4), 13(2):165–210, 1980.

Arno Kret

arnokret@gmail.com 268

Documenta Mathematica 20 (2015)