YIFAN YANG
ABSTRACT. In this paper, we prove that, for an integer r with (r, 6) = 1 and 0 < r < 24 and a nonnegative even integer s, the set
{η(24τ )rf (24τ ) : f (τ ) ∈ M s(1)} is isomorphic to Snewr+2s−1 6, − 8 r , − 12 r ⊗ 12 ·
as Hecke modules under the Shimura correspondence. Here Ms(1) denotes the space
of modular forms of weight s on Γ0(1) = SL(2, Z), Snew2k (6, 2, 3) is the space of
newforms of weight 2k on Γ0(6) that are eigenfunctions with eigenvalues 2and 3for
Atkin-Lehner involutions W2and W3, respectively, and the notation ⊗ 12· means the
twist by the quadratic character 12· . There is also an analogous result for the cases (r, 6) = 3. 1. INTRODUCTION Let θ(τ ) =X n∈Z qn2, q = e2πiτ,
be the Jacobi theta function. Then Shimura’s theory of modular forms of half-integral weights can be described as follows. Let k be a positive integer, N a positive integer and χ a Dirichlet character modulo 4N . We say a holomorphic function f : H = {τ : Im τ > 0} → C is a modular form of half integral weight k + 1/2 on Γ0(4N ) with character χ if
it is holomorphic at each cusp and satisfies f (γτ )
f (τ ) = χ(d)
θ(γτ )2k+1
θ(τ )2k+1
for all γ = a b
c d ∈ Γ0(4N ). Let Mk+1/2(4N, χ) denote the space of these functions.
In [19], Shimura showed how the Hecke theory can be extended to these spaces. More importantly, he showed that if f ∈ Mk+1/2(4N, χ) is a Hecke eigenform, then there is
a corresponding Hecke eigenform of integral weight 2k with character χ2that shares the
same eigenvalues. Moreover, he conjectured that the level of this modular form of integral weight can be taken to be 2N . This conjecture was later proved by Niwa [16]. (See also [21].) In literature, this correspondence between modular forms of half-integral weights and modular forms of integral weights is called the Shimura correspondence.
In [19], the correspondence was proved by using Weil’s characterization of Hecke eigen-forms in terms of L-functions. From the representation-theoretical point of view, this cor-respondence amounts to a corcor-respondence from certain automorphic representations of the metaplectic double cover of GL(2, AQ) to automorphic representations of GL(2, AQ),
where AQdenotes the adele ring of Q. See [6, 10, 24] for more details. Date: October 3, 2011.
In general, the Shimura correspondence is not one-to-one. In particular, the multiplicity-one theorem does not hold for modular forms of half-integral weights in general. In order to get a multiplicity-one result, in the fundamental works [13, 14], Kohnen introduced a subspace of Mk+1/2(4N, χ) and developed a newform theory for this subspace that is
parallel to the Atkin-Lehner-Li theory of newforms for modular forms of integral weights. To state Kohnen’s result, let χ be a Dirichlet character modulo 4N and set = χ(−1). Let Sk+1/2+ (4N, 4
· χ) be the subspace consisting of cusp forms of half-integral weight
k + 1/2 and character 4
· χ on Γ0(4N ) whose Fourier expansions are of the form
X
(−1)kn≡0,1 mod 4
ane2πinτ.
Then Kohnen proved that, under the assumptions that N is odd and squarefree and χ is a quadratic character, the image of Sk+1/2+ (4N, 4· χ) under the Shimura correspondence is S2k(N ). Moreover, there is a canonically defined subspace Sk+1/2new (4N, 4· χ) ⊂
Sk+1/2+ (4N, 4· χ) such that Sk+1/2new (4N, 4· χ) ' Sk+1/2new (N ) as Hecke modules. In particular, the strong multiplicity-one theorem holds for Sk+1/2new (4N, 4
· χ). Kohnen’s
work was later extended by several authors in various directions [7, 8, 23].
Modular forms of half-integral weights are closely related to many problems in number theory. For example, let p(n) denote the number of ways to write a positive integer n as unordered sums of positive integers. Then the generating function of the partition function p(n) is equal to ∞ X n=0 p(n)qn= ∞ Y m=1 1 1 − qm.
If we set q = e2πiτ, then the infinite product above is essentially the reciprocal of the
Dedekind eta function, which is well-known to be a modular form of weight 1/2 on Γ0(576) with character 12·. Using this fact, along with the Shimura correspondence and
properties of Galois representations attached to cusp forms, Ono [18] proved that for every prime m greater than 3, there is a positive proportion of primes ` such that the congruence
p m`
3n + 1
24
≡ 0 mod m
holds for all integers n relatively prime to `. This result was later extended by several authors [1, 2, 28]. For example, in [28], the author of the present paper showed that for every prime m greater than 3 and every prime different from 2, 3, and m, there is an explicitly computable integer k such that
p m`
kn + 1
24
≡ 0 mod m
for all integers n relatively prime to `. A key ingredient in [28] is the Hecke invariance of the space
(1) Sr,s = {η(24τ )rf (24τ ) : f (τ ) ∈ Ms(1)},
where r is an odd integer between 0 and 24, s is a nonnegative even integer, and Ms(1)
is the space of modular forms of weight s on Γ0(1) = SL(2, Z). That is, even though
the space Mk+1/2(576, 12· ) itself has a huge dimension, it contains several subspaces
of small dimensions that are invariant under the action of Hecke algebra. The invariance of these spaces was first proved by Garvan [9] and later rediscovered by the author of the present paper independently. (See the arxiv version of [28] for a proof of the invariance.)
Now recall that a well-known result of Waldspurger [25] states that if f is a Hecke eigenform of half-integral weight k + 1/2 and F is the corresponding Hecke eigenform of integral weight 2k, then for squarefree n, the square of the nth Fourier coefficient of f is essentially proportional to the special value at s = k of L(F ⊗ χ(−1)kn, s), where
χ(−1)knis the Kronecker character of the quadratic field Q(p(−1)kn). (See also [15].)
Using this result of Waldspurger, Guo and Ono [11] related the arithmetic of the partition function p(n) to the arithmetic of certain motives. Specifically, let 13 ≤ ` ≤ 31 be a prime. Let r be the unique integer between 0 and 24 such that r ≡ −` mod 24 and let s = (` − r − 2)/2. Then the space Sr,sdefined in (1) is one-dimensional and spanned by
g`(τ ) = η(24τ )rEs(24τ ), where Es(τ ) denotes the Eisenstein series. It is known that
(2) g`(τ ) ≡ ∞ X n=0 p `n + 1 24 qnmod `.
Then Guo and Ono showed that if we let G`(τ ) be the unique Hecke eigenform in S`−3(6)
with Fourier expansion G`(τ ) = q + 8 r 2(`−5)/2q2+ 12 r 3(`−5)/2q3+ · · · ,
then the image of g`(τ ) under the Shimura correspondence is G` ⊗ 12·, which is a
newform of level 144. (Note that g`(τ ) is contained in Kohnen’s +-space.) In view of
Waldspurger’s result and (2), this means that the values of the partition function modulo ` are related to the values at the center point of the L-function of G`twisted by quadratic
Dirichlet characters. Thus, assuming the truth of the Bloch-Kato conjecture, the arithmetic properties of the partition function are related to those of certain motives associated to G`.
Now observe that, by [3, Theorem 3], the function G`(τ ) is contained in the
Atkin-Lehner eigensubspace of S`−3new(6) with eigenvalues − 8r and − 12r for the Atkin-Lehner involutions W2and W3, respectively. In other words, for the few cases considered in [11],
the Shimura correspondence yields an isomorphism Sr,s' Sr+2s−1new 6, − 8 r , − 12 r ⊗ 12 ·
as Hecke modules, where Snew
2k (6, 2, 3) denotes the space of newforms of weight 2k on
Γ0(6) that are eigenfunctions with eigenvalues 2and 3 for W2and W3. (Note that the
Hecke algebras on the two sides are isomorphic. Thus, we may talk about isomorphisms as Hecke modules.) It is natural to ask whether this isomorphism holds in general. The purpose of this paper is to prove that this is indeed the case.
Theorem 1. Let r be an integer satisfying (r, 6) = 1 and 0 < r < 24 and s be a nonneg-ative even integer. Let
Sr,s= {η(24τ )rf (24τ ) : f (τ ) ∈ Ms(1)} ⊂ Sr/2+s 576, 12 · ,
whereMs(1) denotes the space of modular forms of weight s on Γ0(1) = SL(2, Z). Then
the Shimura correspondence yields an isomorphism Sr,s' Sr+2s−1new 6, − 8 r , − 12 r ⊗ 12 · as Hecke modules.
Theorem 2. Let r be an odd integer satisfying 0 < r < 8 and s be a nonnegative even integer. Let S3r,s= {η(8τ )3rf (8τ ) : f (τ ) ∈ Ms(1)} ⊂ S3r/2+s 64, −4 · . Then the Shimura correspondence yields an isomorphism
S3r,s' S3r+2s−1new 2, − 8 r ⊗ −4 · as Hecke modules.
Corollary 1. The multiplicity-one property holds for the spaces Sr,sdefined in Theorems
1 and 2.
Remark 2. Note that the space Snew
2k (6, 2, 3) ⊗ 12· is contained in S2knew(144, −, −),
regardless of whether 2, 3 are 1 or −1. Also, S2knew(2, 2) ⊗ −4· is a subspace of
Snew
2k (16, −) for both 2= 1 and 2= −1.
It turns out that the Hecke invariance of Sr,sand the explicit Shimura correspondence in
Theorems 1 and 2 are best explained in terms of modular forms of half-integral weight of η-type. Namely, in Shimura’s setting, a function is called a modular form of half-integral weight if its transformation is comparable with the Jacobi theta function. In a similar way, we say a function f (τ ) is a modular form of η-type if its transformation is comparable with the Dedekind eta function, that is, if f (τ ) satisfies
f (γτ ) f (τ ) = (cτ + d) sη(γτ ) r η(τ )r for all γ = a b
c d in a subgroup Γ of SL(2, Z), where s is assumed to be a nonnegative
even integer and r is an odd integer between 0 and 24. Then it is easy to show that modular forms of η-type on SL(2, Z) are essentially just the functions in Sr,sdefined in Theorems
1 and 2. (See Proposition 6 below.) This explains the Hecke invariance of the spaces Sr,s.
At the first sight, the introduction of the notion of modular forms of η-type is superficial since if f (τ ) is such a function, then f (24τ ) is just a modular form of half-integral weight with character 12· in the sense of Shimura, and we do not get any new modular forms in this way. However, if a modular form of half-integral weight on a congruence subgroup Γ0(4N ) in the sense of Shimura happens to be a modular form of η-type on a larger group,
then the extra symmetries from this larger group will give us additional information about the function. This is the reason why we can determine such a precise image of Sr,s
un-der the Shimura correspondence. (In the proof of Theorems 1 and 2, we will work with modular forms of η-type on SL(2, Z) instead of modular forms on the much smaller group Γ0(576) in the sense of Shimura.)
Our proof of Theorems 1 and 2 is classical. That is, since the Hecke modules involved are all semisimple, to prove the theorems, it suffices to show that the traces coincide for all Hecke operators. It will be interesting to have a representation-theoretical proof of the results. Note that here we only prove Theorem 1; the proof of Theorem 2 is similar, but much simpler, and will be omitted.
The rest of the paper is organized as follows. In Section 2, we first define modular forms of η-type in more details. We then define Hecke operators and introduce several basic properties of them. We also describe Shimura’s abstract trace formula [20]. In Section 3, we compute the traces of Hecke operators on the space of modular forms of η-type. This constitutes the main bulk of the paper. In Section 4, we determine the traces of Hecke
operators on S2knew(6, 2, 3). In Section 5, we show that the traces coincide and thereby
establish Theorem 1.
2. PRELIMINARIES
In this section, we first give a more detailed definition of modular forms of η-type. We then define Hecke operators on these modular forms and review Shimura’s trace formula for Hecke operators.
2.1. Modular forms of (ηr, s)-type.
Notation 3. Throughout the rest of the paper, we let r and s be fixed integers with (r, 6) = 1, 0 < r < 24 and s even. Set also k = (r − 1)/2 + s. Let Gk+1/2be the group of
pairs (A, φ(τ )), where A = a b c d ∈ GL
+
(2, Q), φ(τ ) is a holomorphic function H 7→ C satisfying
|φ(τ )| = (det A)−k/2−1/4|cτ + d|k+1/2,
and the group law is defined by
(A, φ(τ ))(B, ψ(τ )) = (AB, φ(Bτ )ψ(τ )). Consider the subgroup Γ∗of Gk+1/2defined by
Γ∗= Γ∗r,s = γ,η(γτ ) r η(τ )r (cτ + d) s : γ =a b c d ∈ SL(2, Z)
For an element γ in SL(2, Z), we let γ∗denote the element in Γ∗whose first component is γ. Naturally, if G is a subgroup of SL(2, Z), then we let G∗be the subgroup {γ∗: γ ∈ G}.
Here let us recall a well-known formula for η(γτ )/η(τ ). Lemma 4 ([26, pp.125–127]). Let γ = a b
c d ∈ SL(2, Z) with c ≥ 0. Then we have
η(γτ ) η(τ ) = (a, b, c, d)(cτ + d) 1/2, where (3) (a, b, c, d) = ( d c i
(1−c)/2e2πi(bd(1−c2)+c(a+d)−3)/24, ifc is odd, c
d e
2πi(ac(1−d2)+d(b−c+3)−3)/24, ifc is even.
We now define modular forms of η-type.
Definition 5. Let f : H → C be a holomorphic function on the upper half-plane. We define the action of γ∗= (γ, φγ(τ )) on f by
(f |γ∗)(τ ) = φγ(τ )−1f (γτ ).
Let G be a subgroup of SL(2, Z) of finite index. If the function f satisfies (f |γ∗)(τ ) = f (τ )
for all γ∗∈ G∗and is holomorphic at each cusp of G, then we say f is a modular form of
(ηr, s)-type on G. If, in addition, f vanishes at each cusp of G, we say f is a cusp form of
(ηr, s)-type. The space of cusp forms of (ηr, s)-type on G will be denoted by S
r,s(G). If
G = Γ0(N ), we simply write it as Sr,s(N ).
Proposition 6. For N = 1, we have
Sr,s(1) = {η(τ )rf (τ ) : f ∈ Ms(1)},
whereMs(1) is the space of modular forms of weight s on SL(2, Z).
Proof. Assume that g(τ ) ∈ Sr,s(1). We have
g(τ + 1) = e2πir/24g(τ ).
Thus, the Fourier expansion of g(τ ) takes the form qr/24(a0+ a1q + · · · ). Since η(τ ) is
nonvanishing throughout H, the function g(τ )/η(τ )ris holomorphic on H. Moreover, it is easy to see that for γ = a b
c d ∈ SL(2, Z), one has
g(γτ )
η(γτ )r = (cτ + d) s g(τ )
η(τ )r.
Therefore, g(τ )/η(τ )ris a modular form of weight s on SL(2, Z). This proves the
propo-sition.
2.2. Hecke operators on Sr,s(N ).
Notation 7. Let N be a positive integer. For a positive integer n, let Mn(N ) = Γ0(N ) 1 0 0 n Γ0(N ) =a b c d : a, b, c, d ∈ Z, ad − bc = n, N |c, (a, N ) = 1, (a, b, c, d) = 1 , and letMn(N )∗denote the subset
Mn(N )∗= Γ0(N )∗ 1 0 0 n , nk/2+1/4 Γ0(N )∗ of Gk+1/2.
Lemma 8. If n is a positive integer relatively prime to 6, then for each γ ∈ Mn2(N ),
there exists a unique elementγ∗inM
n2(N )∗such that the first component ofγ∗isγ.
Proof. It suffices to prove the case γ = 1 00 n2. We are required to show that if A, B ∈
Γ0(N ) are matrices such that A 1 00 n2 B−1= 1 0 0 n2, then (4) A∗1 0 0 n2 , nk+1/2 =1 0 0 n2 , nk+1/2 B∗. Assume that A = a b c d. By Lemma 4, we have A∗1 0 0 n2 , nk+1/2 =a bn 2 c dn2 , (a, b, c, d)r(cτ /n2+ d)s+r/2nk+1/2 , where (a, b, c, d) is defined by (3). Now the assumption that A 1 00 n2 B−1 = 1 00 n2
implies that if A = a b c d, then B = a bn2 c/n2 d
. In particular, we have n2|c. Thus,
1 0 0 n2 , nk+1/2 B∗ =a bn 2 c dn2 , nk+1/2(a, bn2, c/n2, d)r(cτ /n2+ d)s+r/2 .
Since n is assumed to be relatively prime to 6, we have n2≡ 1 mod 24 and hence (a, b, c, d) = (a, bn2, c/n2, d)
This establishes (4) and the lemma.
The group Γ0(N ) acts onMn2(N ) by matrix multiplication on the left. It is clear that
if f ∈ Sr,s(N ) and α and β are two elements ofMn2(N ) that are equivalent under the left
action of Γ0(N ), then
f α∗= f β∗. Moreover, since 1 00 n2
−1
Γ0(N ) 1 00 n2 and Γ0(N ) are commensurable, there are finitely
many right cosets in Γ0(N )\Mn2(N ). Thus, for each positive integer n with (n, 6) = 1,
we can define a linear operator on Sr,s(N ).
Lemma 9. The mapping
[Mn2(N )∗] : f 7−→ f [Mn2(N )∗] = X γ∈Γ0(N )\Mn2(N ) f γ∗ is a linear operator onSr,s(N ).
We now define Hecke operators on Sr,s(N ).
Definition 10. For a positive integer n with (n, 6) = 1, the Hecke operator Tn2on Sr,s(N )
is defined by
Tn2 : f 7−→ nk−3/2
X
ad=n,a|d
af [M(d/a)2(N )]
Proposition 11. Let p be a prime such that p - 6N . Then for f (τ ) =P∞
n=1af(n)qn/24∈ Sr,s(N ), we have Tp2 : f (τ ) 7→ ∞ X n=1 af(p2n) + 12 p (−1)kn p pk−1af(n) + p2k−1af(n/p2) qn/24. Proof. One way to prove the proposition would be to utilize the standard coset representa-tives of Γ0(N )\Mp2(N ) given by p2 0 0 1 , p a 0 p , 1 b 0 p2 , a = 1, . . . , p − 1, b = 0, . . . , p2− 1, and then apply formulas similar to those in (5) in the proof of Lemma 17 below to get the conclusion. Here, however, because it is well-known that f (24τ ) is a modular form of half-integral weight on Γ0(576) with character 12· in the sense of Shimura, we can
actually skip those tedious computations. Indeed, from the commutativity of the diagram
Sr,s Sr,s Sk+1/2(576, 12 · ) Sk+1/2(576, 12 · ) -Tp2 ? (24 0 0 1) ? (24 0 0 1) -Tp2
and the formula given in [19, Page 450] for the Hecke operator Tp2 on Sk+1/2(576, 12 ·)
Remark 12. The linear operators [Mn(N )] can also be defined for nonsquare integers n
and integers that are not relatively prime to 6, but it turns out that they are actually the zero operator. The reason is that for such integers n, there are more than one elements in Mn(N ) with the same first component and the actions of these elements cancel out each
other.
2.3. Shimura’s trace formula. Here we state a trace formula of Shimura [20], adapted to our setting. Our description of the trace formula mostly follows that of Kohnen [14]. Definition 13. Let N and n be positive integers. For γ ∈Mn(N ), we say γ is
(1) a scalar element if γ = (a 0
0 a) for some integer a (this only happens when n = 1),
(2) a parabolic element if the fixed point of γ is a single cusp in P1
(Q),
(3) a hyperbolic element if the fixed points of γ is two distinct real numbers, and (4) an elliptic element if the fixed points of γ is a pair of conjugate complex numbers. Two elements γ1and γ2inMn(N ) are equivalent if
(1) γ1and γ2are scalars and γ1= γ2,
(2) γ1and γ2are hyperbolic or elliptic and there exists an element σ ∈ Γ0(N ) such
that σγ1σ−1= γ2, or
(3) γ1and γ2are parabolic and there exist σ ∈ Γ0(N ) and α in the stabilizer subgroup
inside Γ0(N ) of the cusp fixed by γ2such that σγ1σ−1= αγ2.
Now for γ ∈Mn2(N ), we define a number J (γ) as follows.
(1) If γ is a scalar, then we set J (γ) = 1 24 k −1 2 [SL(2, Z) : Γ0(N )].
(2) Assume that γ is parabolic with fixed point a/c ∈ P1
(Q). Let σ ∈ SL(2, Z) be a matrix such that σ∞ = a/c. Then the stabilizer subgroup of a/c inside Γ0(N )
is generated by σ (1 w
0 1) σ−1and −1 0
0 −1, where w = N/(c2, N ) is the width of
the cusp a/c. Now we write 1 w 0 1 ∗ =1 w 0 1 , e−2πiµ with 0 ≤ µ < 1. If σ∗−1γ∗σ∗= ±n nuw 0 n , η , then let J (γ) = ( −1 2ηe −2πiuµ(1 − 2µ), if u ∈ Z, −1 2ηe
−2πiuµ(1 − i cot πu), if u 6∈ Z.
(3) If γ is hyperbolic and the fixed points are not cusps, then set J (γ) = 0.
(4) Assume that γ is hyperbolic fixing (two distinct) cusps. Then the eigenvalues of γ are two integers λ and λ0. We assume that |λ| > |λ0|. Let (ac) be an eigenvector
associated to λ0with a, c ∈ Z and (a, c) = 1. Find an element σ ∈ SL(2, Z) such that σ = a b
c d. Then σ
−1γσ = λ0x
0 λ for some integer x. If
σ∗γ∗σ−1∗=λ 0 x 0 λ , η ,
then set J (γ) =1 2 η λ 0 λ − 1 −1 . (5) Assume that γ = a b
c d is elliptic. Then the eigenvalues of γ are a pair of
conju-gate complex numbers ρ and ρ. We assume that sgn Im ρ = sgn c. If γ∗=a b c d , u(cτ + d)k+1/2 , then we set J (γ) =wuρk−1/2(ρ − ρ) −1 ,
where w denotes the number of elements in Γ0(N ) that commute with γ.
Then according to Shimura’s formula [20, Page 273], the trace of the operator [Mn2(N )]
on Sr,s(N ) is as follows. (Cf. [14, Page 49].)
Proposition 14. For positive integers N and n with (n, 6N ) = 1, the trace of the linear operator[Mn2(N )∗] on Sr,s(N ) is given by
tr [Mn2(N )∗] =
X
γ
J (γ),
where the sum runs over representatives of equivalence classes as per Definition 13 and J (γ) are defined as in the paragraph preceding the proposition.
3. TRACES OFHECKE OPERATORS ONSr,s(1)
In this section, we will compute the trace of Hecke operators on Sr,s(1). The
contribu-tions of scalar, parabolic, hyperbolic, and elliptic classes will be determined separately in individual subsections.
Throughout this section, we write Mn2(1) and Mn2(1)∗ simply asMn2 andMn∗2,
respectively. All equivalences mentioned here refer to the equivalence relation described in Definition 13.
3.1. Scalar cases. Since any element a b
c d inMn2 satisfies (a, b, c, d) = 1. Scalar
ele-ments exist only inM1∗and they are (± (1 0 0 1) , 1).
Proposition 15. The contribution of scalar elements inMn2 to the trace of[Mn∗2] is
(1
12 k − 1
2 , if n = 1,
0, else.
3.2. Parabolic cases. The contribution of the parabolic classes to the trace ofMn2 is
summarized in Proposition 21. The proof is divided into several steps. Lemma 16. The inequivalent parabolic elements inMn2are
n a 0 n
, a = 1, . . . , n, (a, n) = 1.
Proof. Since SL(2, Z) has only one inequivalent cusp ∞, a parabolic element is conjugate to
±n b
0 n
for some b with (b, n) = 1. Since the stabilizer subgroup of ∞ inside SL(2, Z) is generated by (1 1
0 1) and −1 0
0 −1, we see that the inequivalent parabolic elements are given as in the
statement.
Lemma 17. Let a be an integer relatively prime to n. The contribution of the class of n a 0 n to the trace of[Mn∗2] on Sr,s(1) is −(r − 12)/24, ifn = a = 1, −(−i)r(n−1)/2 −a n e−2πi(r`+1)a/n 1 − e−2πia/n , ifn > 1. where` = (n2− 1)/24.
Proof. Assume first that n = a = 1. We have (1 1 0 1)
∗
= (1 1 0 1) , e
2πir/24. Then the
numbers µ, η, and u in the definition of J (γ) in Proposition 14 are µ = 24 − r
24 , η = e
2πir/24, u = 1,
respectively. Thus, the contribution to the trace is −(r − 12)/24.
Now assume that n > 1. We have (n, a) = 1. Let α and β be integers such that αn + βa = 1 and β > 0. We have
n a 0 n =a −α n β −1 0 0 −n2 −nβ −1 1 0 . By Lemma 4, we have a −α n β ∗ =a −α n β ,a n ir(1−n)/2e2πir(n(a+β)−3)/24(nτ + β)k+1/2 and −nβ −1 1 0 ∗ =−nβ −1 1 0 , e2πir(−nβ−3)/24τk+1/2 . Then −1 0 0 −n2 ∗−nβ −1 1 0 ∗ = nβ 1 −n2 0 , e2πir(−nβ−3)/24(nτ )k+1/2 and n a 0 n ∗ = n a 0 n ,a n ir(1−n)/2e2πir(na−6)/24 − 1 nτ k+1/2 (nτ )k+1/2 ! =n a 0 n ,a n e2πir(n(a−3)+3)/24 . (5)
Now the stabilizer of ∞ is generated by (1 1 0 1) , e
2πir/24 and −1 0
0 −1 , 1. Thus,
according to Proposition 14 and (5), the contribution of the class of (n a0 n) to the trace is
−1 2
a n
e−2πir(n(a−3)+3)/24e−2πia(24−r)/(24n)(1 − i cot(πa/n)).
The proof of Proposition 21 involves sums of the formP
u≤N/jχ(u) for some Dirichlet
character χ. Here we recall a formula from [22] for such sums. Note that for a Dirichlet character χ modulo M , the generalized Bernoulli numbers Bm,χare defined by the power
series M X a=1 χ(a)teat eM t− 1 = ∞ X m=0 Bm,χ tm m!.
If χ is the trivial character modulo 1, then Bm,χis just the Bernoulli numbers Bm. We
have (6) B0,χ= 1 M M X a=1 χ(a) = ( φ(M )/M, if χ is principal, 0, else.
Also, B1,χ = 0 if χ is an even character. Moreover, if χ is an imprimitive Dirichlet
character induced from χ1, then we have the relation
(7) B1,χ= B1,χ1 X d|M µ(d)χ1(d) = B1,χ1 Y p|M (1 − χ1(p)).
Also, if d < 0 is a fundamental discriminant, then we have
(8) B1,(d
·) = −H(d),
where H(d) is the Hurwitz class number.
Lemma 18 ([22, Equation (6), Page 276]). Let M be a positive integer and χ be a Dirichlet character moduloM . Let N > 0 be a multiple of M and j be a positive integer relatively prime toN . Then we have
X 0≤u<N/j χ(u) = −B1,χ+ χ(j) φ(j) X ψ mod j ψ(−N )B1,χψ(N ),
where the sum runs over all Dirichlet charactersψ modulo j and B1,χψ(N ) = B0,χψN + B1,χψ
= (
B1,χψ, ifχψ is a nonprincipal character modulo jM,
N φ(jM )/jM, ifχψ is the principal character modulo jM. Proof. If M > 1, then the formula is just Equation (6) of [22] on Page 276 with m = 1. If M = 1, that is, if χ(u) = 1 for all u ∈ Z, then we should modify the definition of Lχ(t)
on Page 274 of [22] to Lχ(t) =P∞n=0χ(n)e
nt. Then following the argument from Page
274 up to Equation (6) of Page 276 of [22], we see that our formula holds. The details of
proof are omitted.
Lemma 19. Let n be a positive odd integer greater than 1. If n ≡ 1 mod 4, then X a mod n a n 1 1 − e2πia/n = ( φ(n)/2, ifn is a square, 0, else. Ifn ≡ 3 mod 4, then X a mod n a n 1 1 − e2πia/n = i √ nH(−n).
HereH(−n) is the Hurwitz class number, i.e., H(−3) = 1/3, H(−4) = 1/2, and H(−n) is the ideal class number of the quadratic order of discriminant−n if n 6= 3, 4.
Proof. Assume that n ≡ 1 mod 4. Then n· is an even function. Let S denote the sum. We have 2S = X a mod n a n 1 1 − e2πia/n+ 1 1 − e−2πia/n = X a mod n a n .
If n is a square, then n· is the principal Dirichlet character modulo n and we have S = φ(n)/2. If n is not a square, then n· is a nonprincipal Dirichlet character modulo n and the sum S vanishes.
Now assume that n ≡ 3 mod 4. Set z = e2πi/n. We first consider the case n is squarefree. For an integer t, set
S(t) = X a mod n a n zta 1 − za, T (t) = X a mod n a n zta. The two sums are related by
S(t) − S(t + 1) = X a mod n a n zta(1 − za) 1 − za = X a mod n a n zta= T (t).
Since n is assumed to be squarefree, we have T (t) = nt i√n. It follows that
n X t=1 (S(0) − S(t)) = n X t=1 t−1 X a=0 T (a) = t−1 X a=0 T (a)(n − a) = i√n n−1 X a=0 a n (n − a) = in√nH(−n). Since n X t=1 S(t) = n X a=1 a n 1 + za+ · · · + z(n−1)a 1 − za = 0,
we conclude that S = S(0) = i√nH(−n) for the case n is squarefree. Now if n is not squarefree, we write n as m2n
0 with squarefree n0. Then using the
partial fraction decomposition ` 1 − x` = `−1 X j=0 1 1 − xe2πij/`, we get S =X d|m µ(d) n/d X a=1 ad n0 1 1 − zad =X d|m µ(d) d n0 n0−1 X a=1 a n0 n/dn0−1 X j=0 1 1 − e2πid(a+jn0)/n =X d|m µ(d) d n0 n0 X a=1 a n0 n/dn 0 1 − e2πia/n0 = im2√n0H(−n0) X d|m µ(d) d d n0 ,
where in the last step we use the result for the squarefree case computed earlier. Now recall that, for d > 0 with d ≡ 0, 3 mod 4, the Hurwitz class numbers H(−m2d) and H(−d) are related by the formula
H(−m2d) = H(−d)mY p|m 1 − 1 p −d p = H(−d)mX a|m µ(a) a −d a . (9) Therefore, S = im√n0H(−n) = i √ nH(−n).
This completes the proof of the lemma.
Lemma 20. For a positive integer n > 1 with (n, 6) = 1, let ` = (n2− 1)/24. If
n ≡ 1 mod 4, then 1 √ n X a mod n a n e2πi(r`+1)a/n 1 − e2πia/n = −1 8 24 n X u=−3,−4,−8,−24 u r 1 −un 2 1 −un 3 H(un). (10) Ifn ≡ 3 mod 4, then 1 i√n X a mod n a n e2πi(r`+1)a/n 1 − e2πia/n = 1 8 24 n X u=1,8,12,24 u r 1 − −un 2 1 − −un 3 H(−un). (11)
Proof. Let S denote the sum in question and for a positive integer `, let z`denote the `th
primitive root of unity e2πi/`. Also, write n as n = m2n
0with squarefree n0. We have
S = − X a mod n a n 1 − zn(r`+1)a 1 − za n + X a mod n a n 1 1 − za n = − r` X t=0 X a mod n a n ztan + X a mod n a n 1 1 − za n = − r` X t=0 T (t) + S0, (12)
say. The sum S0has been evaluated in Lemma 19. We now consider T (t). We have T (t) =X d|m µ(d) X a mod n/d ad n0 zntad =X d|m µ(d) d n0 X a mod n0 a n0 ztadn m2/d−1 X b=0 zmbt2/d.
The inner sum is 0 if (m2/d) - t. Then T (t) = m2 X d|m,(m2/d)|t µ(d) d d n0 X a mod n0 a n0 zat/(mn0 2/d) = m2√n0 X d|m,(m2/d)|t µ(d) d d n0 t/(m2/d) n0 , (13)
where = ( 1 if n ≡ 1 mod 4, i, if n ≡ 3 mod 4. From (13), we obtain r` X t=0 T (t) = m2√n0 X d|m µ(d) d d n0 X 0≤u≤r`/(m2/d) u n0 .
Now ` = (n2− 1)/24 and r is assumed to be in the range 0 < r < 24. Therefore, the sum above can also be written as
(14) r` X t=0 T (t) = m2√n0 X d|m µ(d) d d n0 X 0≤u<(rn2d/m2)/24 u n0 .
Now we apply Lemma 18 to (14) with χ = χn0 =
· n0
, M = n0, N = rn2d/m2, and
j = 24. We consider the three cases (1) n0= 1,
(2) n ≡ 1 mod 4 and n06= 1,
(3) n ≡ 3 mod 4, separately.
When n0= 1, an application of Lemma 18 yields r` X t=0 T (t) = m2X d|m µ(d) d −B1+ 1 8 X ψ mod 24 ψ(−rm2d)B 1,ψ(rm2d) =1 2mφ(m) + 1 8m 2X d|m µ(d) d X ψ mod 24 ψ(−rm2d)B 1,ψ(rm2d). (15)
If we let χ0denote the principal Dirichlet character modulo 24, then the Dirichlet
charac-ters ψ modulo 24 are given by (16) ψu= χ0(·) u · , u = 1, 8, 12, 24, −3, −4, −8, −24. By (6), (7), and (8), we have (17) B1,ψu(rm 2d) = rm2dφ(24)/24 = rm2d/3, if u = 1, 0, if u = 8, 12, 24, − 1 − u 2 1 − u 3 H(u), if u = −3, −4, −8, −24.
The contribution from the character ψ1to (15) is
(18) rm 4 24 X d|m µ(d) = 0,
since m = √n is assumed to be greater than 1. The contributions from ψu, for u =
−3, −4, −8, −24, are m2 8 u r 1 −u 2 1 −u 3 H(u)X d|m µ(d) d u d = m 8 u r 1 −u 2 1 −u 3 H(un), (19)
where we have utilized formula (9). Combining (12), (15), (17), (18), and (19) and using the formula for S0given in Lemma 19, we get formula (10) for the case n = m2.
Now let us consider the case n ≡ 1 mod 4 but not a perfect square. That is, n = m2n0
with squarefree n06= 1 and n0≡ 1 mod 4. In this case, we have B1, · n0 = B 1, · n0 ψu = 0 for u = 1, 8, 12, 24, where ψuare defined by (16). Then an application of Lemma 18 to
(14) yields r` X t=0 T (t) = m 2√n 0 8 X d|m µ(d) d d n0 24 n0 X u=−3,−4,−8,−24 ψu(−rn2d/m2)B1, · n0 ψu = −m √ n 8 24 n X d|m µ(d) d n0 d X u=−3,−4,−8,−24 u rd B1,(n0u · ) ×1 −un0 2 1 −un0 3 , where we have used (7). Then by (8) and (9), we get
r` X t=0 T (t) = √ n 8 24 n X u=−3,−4,−8,−24 u r 1 −un 2 1 −un 3 H(un).
This gives the evaluation of the sum of T (t) in (12). The term S0 in (12) is shown to be 0 in Lemma 19. This establishes (10) for the case n is not a square.
Now assume that n ≡ 3 mod 4. Then B1,(·
n)ψu = 0 for u = −3, −4, −8, −24 and an
application of Lemma 18 to (14) gives us
r` X t=0 T (t) = im√nX d|m µ(d) d d n0 − B 1,n0· +1 8 24 n0 X u=1,8,12,24 ψu(−rn2d/m2)B1, · n0 ψu
Using (7), (8), and (9) again, we get
r` X t=0 T (t) = i√n H(−n) −1 8 24 n X u=1,8,12,24 u r 1 − −un 2 1 − −un 3 H(−un) ! . Combining this, (12), and Lemma 19, we arrive at the claimed formula. This completes
the proof of the lemma.
Proposition 21. The total contribution of the parabolic classes ofMn2 to the trace of the
linear operator[Mn∗2] on Sr,s(1) is √ n 8 12 n X e=1,2,3,6 −4e r 1 − −en 3 (H(−4en) − H(−en))
where, for a negative integer−d, we let H(−d) denote the Hurwitz class number of the imaginary quadratic order of discriminant −d. (If −d is not a discriminant, then set H(−d) = 0.)
Proof. We first consider the case n = 1. By Lemmas 16 and 17, we find that the total contribution is −(r − 12)/24. We then verify case by case that
−r − 12 24 = 1 8 X e=1,2,3,6 −4e r 1 − −e 3 (H(−4e) − H(−e)).
We next consider the cases n > 1. Again, using Lemmas 16 and 17, we find that the total contribution is −(−i)r(n−1)/2 n X a=1 −a n e−2πi(r`+1)a/n 1 − e−2πia/n
Now for n ≡ 1 mod 4, we have
(−i)r(n−1)/2= 8 n
. Thus, by (10), the contribution to the trace is
√ n 8 12 n X u=−3,−4,−8,−24 u r 1 −un 2 1 −un 3 H(un). Since H(−12n) = H(−3n) 2 − −3n2 , H(−n) = H(−2n) = H(−6n) = 0, the formula above is equal to that in the statement of the proposition.
Now assume that n ≡ 3 mod 4. We have
i(−i)r(n−1)/2= − −4 r 8 n .
Then from Lemma 17, (11), and H(−4n) = H(−n) 2 − −n2 , we get the claimed
formula. This proves the proposition.
3.3. Hyperbolic cases.
Lemma 22. A complete set of representatives of inequivalent hyperbolic elements inMn2
whose fixed points are cusps is
±a b + ma
0 d
: b = 1, . . . , h, (b, h) = 1, m = 1, . . . , (d − a)/h with h = (a, d)
, where in the set we leta and d run over all integers satisfying ad = n2and0 < a < d.
Proof. Let M be a hyperbolic element in Mn2 whose fixed points are cusps. Then the
eigenvalues of M are two integers a and d satisfying ad = n2. Without loss of generality,
we assume that |d| > |a| > 0. Let α be the cusp corresponding to the eigenvalue a and choose an element γ ∈ SL(2, Z) such that σ∞ = α. Then
σM σ−1=a b 0 d
for some integer b. The integer b must satisfy (a, b, d) = 1. In other words, a hyperbolic element inMn2 whose fixed points are cusps is conjugate to a matrix of the form a b0 d
with ad = n2, |d| > |a| > 0, and (b, (a, d)) = 1. It is clear that if two such matrices
a b
0 d and a 0 b0
0 d0 are conjugate, then we must have a = a
0 and d = d0. Furthermore, by
considering the eigenvector (1
0), it is easy to see that two matrices a b0 d and a b 0 0 d are
conjugate if and only they are conjugate by (1 m
0 1 ) for some m ∈ Z, that is, if and only if
(d − a)|(b0− b). With these informations, it is straightforward to verify that the set in the lemma is a complete set of representatives. This proves the lemma.
Proposition 23. The total contribution of hyperbolic elements inMn2to the trace is0.
Proof. By Shimura’s formula, the contribution from hyperbolic elements whose fixed points are not cusp is 0. Furthermore, by Lemma 22, a complete set of inequivalent hyper-bolic elements whose fixed points are cusps is
±a b + ma
0 d
: b = 1, . . . , h, (b, h) = 1, m = 1, . . . , (d − a)/h with h = (a, d)
, where in the set we let a and d run over all integers satisfying ad = n2and 0 < a < d. Now we have a b + ma 0 d =a b 0 d 1 m 0 1 . Thus, if the contribution from the conjugacy class a b
0 d is A, then the contribution from
the class a b+ma 0 d is e
2πirm/24A. Since 24|(d − a)/h, as m runs from 1 to (d − a)/h,
the contributions from the classes a b+ma0 d cancel out each other. We therefore conclude that the total contribution from hyperbolic elements is 0. 3.4. Elliptic cases.
3.4.1. Quadratic forms. Assume that γ = a b
c d ∈ Mn2 is elliptic. Then t = a + d,
f = (b, c, d − a), and sgn c are invariants under conjugation by elements of SL(2, Z). This can be seen from the one-to-one correspondence between the set
Γn,t,f = a b c d ∈Mn2 : a + d = t, f = (b, c, d − a), c > 0
and the set Q(t2−4n2)/f2of all primitive positive definite quadratic forms of discriminant
(t2− 4n2)/f2, where
QD= {Ax2+ Bxy + Cy2: B2− 4AC = D, A > 0}.
and the correspondence is given by
(20) a b c d ←→ 1 f(cx 2+ (d − a)xy − by2).
Elements γ of SL(2, Z) act on Γn,t,f by conjugation and on Q(t2−4n2)/f2 by change of
variable (xy) 7→ γ (xy). Moreover, the two group actions are compatible with respect to the
correspondence above.
Lemma 24. The total contribution of elliptic elements inMn∗2 to the trace of the linear
operator[M∗ n2] on Sr,s(1) is 2X t,f X γ∈Γn,t,f/SL(2,Z) J (γ),
where the outer sum runs over all integerst with t2< 4n2and all positive integersf such
that(t2− 4n2)/f2 ≡ 0, 1 mod 4, the inner sum runs over all class representatives of
Γn,t,f under conjugation bySL(2, Z), and J(γ) is defined as in Definition 13.
Proof. From the discussion preceding the lemma, we easily see that every elliptic element inMn2falls in precisely one of Γn,t,f and −Γn,t,f. Also, it is clear that if γ1, . . . , γmare
class representatives of Γn,t,f, then −γ1, . . . , −γmare class representatives of −Γn,t,f.
Assume that γ = a b c d ∈ Γn,t,fand γ∗=a b c d , u(cτ + d)k+1/2 . Then by Proposition 14, the contribution of the class of γ to the trace is
J (γ) = ρ
1/2−k
wu(ρ − ρ), where ρ = (t +√t2− 4n2)/2. Now we have
(−γ)∗=−a −b −c −d , u(cτ + d)k+1/2 =−a −b −c −d , ue2πi(2k+1)/4(−cτ − d)k+1/2 . Thus, J (−γ) = (−ρ) 1/2−k wue2πi(2k+1)/4(−ρ + ρ) = e2πi(2k−1)/4ρ1/2−k wue2πi(2k+1)/4(−ρ + ρ)= J (γ).
This proves the lemma.
Remark 25. Since for a b
c d ∈ Mn2, we have (a, b, c, d) = 1, the integers n, t, and f
need to satisfy the following conditions.
(1) The common divisor (n, t, f ) must be 1.
(2) The integer (t, f ) can only be 1 or 2. Moreover, if (t, f ) = 2, then because (t2− 4n2
)/4 is a discriminant, we must have 2|t, but 4 - t.
(3) The integer (f, t + 2n, t − 2n) can only be 1, 2, or 4 and if the latter two cases occur, then 2|t, but 4 - t.
We first choose a suitable representative γ in each conjugacy class in Γn,t,f and
deter-mine γ∗.
Lemma 26. Each conjugacy class of Γn,t,f contains an element a bc d such that c = f p
for some primep > 4n and (c, d) = 1.
Moreover, in the case2|f (which occurs only when 2kt), we may further require that d satisfies the following congruences
d ≡ t0mod 8, if2kf, t0mod 16, if4kf,
t0mod 8, if8kf and (t2− 4n2)/64 is even,
t0+ 4 mod 8, if8kf and (t2− 4n2)/64 is odd,
t0mod 8, if16|f, wheret0= t/2.
Proof. It is well-known that a primitive positive definite quadratic form represents infinity many primes, which implies that each class of quadratic forms in Q(t2−4n2)/f2 contains
elements of the form px2+ uxy + vy2 for infinitely many primes p. The matrix
corre-sponding to this form under (20) is
(t − f u)/2 −f v f p (t + f u)/2
.
Now if p > 4n, then the relation
f2(u2− 4pv) = t2− 4n2
implies that p - (t + f u) and hence (f p, (t + f u)/2) = (f, (t + f u)/2). By Remark 25, this can only be 1 or 2. However, because the determinant of the matrix is the odd integer n2, (f p, (t + f u)/2) cannot be even. That is, (f p, (t + f u)/2) = 1. This proves the first part of the statement.
Now assume that 2kf . From
(21) 1 −u 0 1 a b c d 1 u 0 1 =a − cu −cu 2+ u(a − d) + b c d + cu
it is clear that we can further assume that d ≡ t0mod 8. Assume that 4kf . Since 32|(t2− 4n2
) and (t2− 4n2
)/16 is a discriminant, we must have 64|(t2− 4n2). Then the equality t2− 4n2= (d − a)2+ 4bc shows that 64|(d − a)2,
i.e., 8|(d − a). Then
d =1
2(t + (d − a)) ≡ t
0mod 4.
By (21) again, we may assume that d ≡ t0mod 16.
Assume that 8kf . We have 64|(t2− 4n2) and 8|b, c. Then (d − a)2= t2− 4n2− 4bc ≡
(t2− 4n2) mod 256. If (t2− 4n2)/64 is even, then d ≡ a mod 16 and d = (t + d − a)/2 ≡
t0mod 8. If (t2− 4n2)/64 is odd, then d ≡ a + 8 mod 16 and d ≡ t0+ 4 mod 8.
Finally, if 16|f , by a computation similar to the case 8kf , we find d ≡ t0mod 8. This
proves the lemma.
For our purpose, we also need to recall some properties of genus characters of integral binary quadratic forms. Let D < 0 be a discriminant, i.e., D ≡ 0, 1 mod 4. For each odd prime divisor p of D, one can associate to QDa character by
χ : Ax2+ Bxy + Cy27−→ A p = p ∗ A , p∗= (−1)(p−1)/2p. For D ≡ 0 mod 4, there may also exist characters of the form
Ax2+ Bxy + Cy27−→u A
, u ∈ {−4, 8, −8},
depending on the residue of D/4 modulo 8. Any product of these characters is called a genus character. All genus characters can be written as
χD1: Ax
2+ Bxy + Cy27−→ D1
A
for some (positive or negative) divisor D1of D such that D1and D/D1are both
discrim-inants and vice versa. General properties of genus characters can be found in [4, Chapter 1]. Here we only quote some properties relevant to our calculation of traces.
Lemma 27. Let D and D1 be as above andχD1 : QD → {±1} be a genus character.
Then we have the following properties.
(1) The value of χD1 for a quadratic formAx
2+ Bxy + Cy2∈ Q
Ddepends only on
the genus in which the quadratic form lies. In particular, every quadratic form in the same class takes the same value ofχD1.
(2) χD1 is a trivial character (i.e., mapping every quadratic form to1) if and only if
(3) The sum of χD1over a complete set of class representatives ofQD/SL(2, Z) is X Ax2+Bxy+Cy2∈Q D/SL(2,Z) D1 A = ( h(D), ifD1orD/D1is a square, 0, else, whereh(D) is the number of classes in QD.
Finally, the following formula frequently occurs in our computation.
Lemma 28. Let D be the discriminant of an imaginary quadratic order. Let u be a positive integer. Then we have
X f |u D f H u 2 f2D = uH(D). Proof. We have X f |u D f H u 2 f2D = H(D)X f |u D f u f X m|(u/f ) µ(m) m D m = H(D)X n|u D n u n X m|n µ(m) The sumP
m|nµ(m) is nonzero only when n = 1. From this, we get the claimed formula.
3.4.2. Preliminary calculation and notations.
Lemma 29. If a b
c d ∈Mn2satisfies(c, d) = 1 and c > 0, then
a b c d ∗ =a b c d , n−k−1/2(a, b, c, d)r(cτ + d)k+1/2 , where (a, b, c, d) = ( d c e 2πi(bd(1−c2)+c(a+d−3))/24 , ifc is odd, c d e 2πi(ac(1−d2)+d(b−c+3)−3)/24 , ifc is even. Proof. Let α, β ∈ Z be integers such that αd + βc = 1. We have
a b c d =1 aβ + bα 0 1 n2 0 0 1 α −β c d . Now α −β c d ∗ =α −β c d , (α, −β, c, d)r(cτ + d)k+1/2 and 1 aβ + bα 0 1 ∗ =1 aβ + bα 0 1 , e2πir(aβ+bα)/24 . Then from 1 aβ + bα 0 1 α −β c d =a + (1 − n 2)α b + (n2− 1)β c d we deduce that e2πir(aβ+bα)/24(α, −β, c, d)r= (a + (1 − n2)α, b + (n2− 1)β, c, d)r = (a, b, c, d)r.
All the 24th roots of unity can be expressed in terms of Jacobi symbols, which we give in the next lemma for future reference.
Lemma 30. If (t, 24) = 1, then e2πit/24= √ 2 4 8 t + √ 6 4 24 t −i √ 2 4 −8 t +i √ 6 4 −24 t . If(t, 12) = 1, then e2πit/12= √ 3 2 12 t + i 2 −4 t . If(t, 8) = 1, then e2πit/8= √ 2 2 8 t +i √ 2 2 −8 t . If(t, 6) = 1, then e2πit/6= 1 2+ i√3 2 −3 t . If(t, 4) = 1, then e2πit/4= i −4 t . If(t, 3) = 1, then e2πit/3= −1 2+ i√3 2 −3 t . Let class representatives a b
c d of Γn,t,f be chosen as in Lemma 26. Then by
Proposi-tion 14 and Lemma 29, the contribuProposi-tion of the class of a b
c d to the trace is (22) J (γ) = n k+1/2 wn,t,f (a, b, c, d)−rρ 1/2−k ρ − ρ, where (23) wn,t,f = 6, if (t2− 4n2)/f2= −3, 4, if (t2− 4n2)/f2= −4, 2, else,
is the number of elements in SL(2, Z) commuting with γ, (a, b, c, d) = d
c
e−2πic/8e2πi(bd(1−c2)+ct)/24
and ρ = (t +√t2− 4n2)/2. In view of Lemma 30, sums of the form P d c
e c, e =
±1, ±2, ±3, ±6, will appear frequently in our computation. So we first compute such sums.
Notation 31. For e = ±1, ±2, ±3, ±6 and nonnegative integers `, we set Le,`(n, t) := ρ1/2−k ρ − ρ X f :2`kf,(n,t,f )=1 1 wn,t,f X a b c d ∈Γn,t,f/SL(2,Z) d c0 e c0 − 3 X f :2`kf,3|f,(n,t,f )=1 1 wn,t,f X a b c d ∈Γn,t,f/SL(2,Z) d c0 e c0 ! , (24)
where in the inner sums, the representatives a b
c d are chosen according to Lemma 26,
c0 = c/2`(that is, c0is the odd part of c), and ρ = (t +√t2− 4n2)/2.
Moreover, for e = 1, 2, 3, 6 and integers u, we set Me,`(n, u) := (τ /√e)1−2k τ − τ X g:2`kg,(n,u,g)=1 H e 2u2− 4en g2 − 3 X g:2`kg,3|g,(n,u,g)=1 H e 2u2− 4en g2 ! , (25) and Me,`0 (n, u) := (τ / √ e)1−2k τ − τ X g:2`kg,(n,u,g)=1 H e 2u2− 4en g2 − 3 X g:2`kg,3|g,(n,u,g)=1 H e 2u2− 4en g2 ! , (26)
where τ = (eu+√e2u2− 4en)/2 and g runs over all positive integers satisfying the given
conditions such that (e2u2− 4en)/g2is a discriminant.
Lemma 32. Let n be a positive integer relatively prime to 6 and t be an integer satisfying t2< 4n2. Let e ∈ {±1, ±3}, if(t, 24) = 1, {±1, ±2, ±3, ±6}, if(t, 24) = 2, {±1}, if(t, 24) = 3, {±2, ±6}, if(t, 24) = 4 or (t, 24) = 8, {±1, ±2}, if(t, 24) = 6, {±2}, if(t, 24) = 12 or (t, 24) = 24.
Ife(t + 2n) is a discriminant, i.e., if e(t + 2n) ≡ 0, 1 mod 4, then there exists a rational numbers such that t2− 4n2decomposes into a product of two discriminantss2e(t + 2n)
and(t − 2n)/(es2).
Proof. Here we only consider the case (t, 24) = 2. When e = ±1, we have 4|(t+2n), (t− 2n) and the statement holds obviously. When e = ±2, we can decompose t2− 4n2
as 2(t + 2n) · (t − 2n)/2 or (t + 2n)/2 · 2(t − 2n) according to whether 8|(t − 2n) or 8|(t+2n). When e = ±3, the decomposition is 3(t+2n)·(t−2n)/3 or (t+2n)/3·3(t−2n) according to whether 3|(t − 2n) or 3|(t + 2n). When e = ±6, the rational number s can be one of 1, 1/2, 1/3, or 1/6, depending one whether 8|(t + 2n) and whether 3|(t + 2n).
We now evaluate Le,`(n, t). The computation mostly follows that in [14].
Lemma 33. Let n be a positive integer relatively prime to 6. For e ∈ {1, 2, 3, 6} and integersu, let µe(n, u) = (1, if3 - u, 1 +en 3 , if3|u. Then we have the following formulas.
(1) Let t and e be integers satisfying the assumptions in Lemma 32. Ife(t + 2n) ≡ 0, 1 mod 4, then
Le,0(n, t) =
1 2
(
0, ife(t + 2n) is not a square, µe(n, u)Me,0(n, u), ift + 2n = eu2.
(27)
whereu is the positive square root of (t + 2n)/e. Also, if−e(t + 2n) ≡ 0, 1 mod 4, then L−e,0(n, t) = i 2 −4 r (0,
ife(2n − t) is not a square, µe(n, u)Me,00 (n, u) if2n − t = eu
2,
(28)
whereu is the positive square root of (2n − t)/e. (2) Assume that 2kt and e ∈ {±1, ±3}. Then
Le,1(n, t) =
1
2Le,0(n, t). (3) Assume that 2kt, e ∈ {1, 3}, and ` ≥ 2. Then
Le,`(n, t) = 1 2
0, ife(t + 2n) is not a square,
µe(n, u)Me,`−1(n, u)/2, ift + 2n = eu2and2ku,
µe(n, u)Me,1(n, u)/2`−1, ift + 2n = eu2and2`−1ku,
µe(n, u)Me,0(n, u)/2`, ift + 2n = eu2and2`|u.
and L−e,`(n, t) = i 2 −4 r
0, ife(2n − t) is not a square,
µe(n, u)Me,`−10 (n, u)/2, if2n − t = eu
2and2ku,
µe(n, u)Me,10 (n, u)/2`−1, if2n − t = eu2and2`−1ku,
µe(n, u)Me,00 (n, u)/2`, if2n − t = eu2and2`|u,
Proof. We first prove (27). Note that in the sum defining Le,0(n, t), the integers f are
always odd and according to the choice of representatives given in Lemma 26, we have c0 = c. Then from Remark 25, we know that (f, t + 2n, t − 2n) = 1. Thus, if f1 =
(f, t + 2n) and f2= (f, t − 2n), then f = f1f2, f12|(t + 2n) and f22|(t − 2n). Write d c as d c = d c/f d f1 d f2 . Since d c/f t + 2n c/f = ad + 2nd + d 2 c/f = n 2+ 2nd + d2 c/f = 1, we have d c/f = (t + 2n)/f 2 1 c/f . By the same token, we also have
d f1 = (t − 2n)/f 2 2 f1 , d f2 = (t + 2n)/f 2 1 f2 , and hence (29) d c e c = e(t + 2n)/f 2 1 c/f e(t − 2n)/f2 2 f1 e(t + 2n)/f2 1 f2 .
Now e(t+2n) is assumed to be a discriminant. By Lemma 32, we can decompose t2−4n2
into a product es2(t + 2n) · (t − 2n)/es2of two discriminants. So by Lemma 27, 1 wn,t,f X a b c d ∈Γn,t,f/SL(2,Z) e(t + 2n)/f2 1 c/f = (
H((t2− 4n2)/f2)/2, if e(t + 2n) = (eu)2is a square,
0, else.
(30)
In the case e(t + 2n) = (eu)2, we have
(31) ρ =t + √ t2− 4n2 2 = p e(t + 2n) +pe(t − 2n) 2√e !2 = τ √ e 2 and (32) ρ − ρ =pt2− 4n2= upe2u2− 4en = u(τ − τ ),
where τ = (eu +√∆)/2 with ∆ = e2u2− 4en. Then for the first sumP
f defining Le,0(n, t) in (24), we have X (f,2)=1,(n,t,f )=1 1 wn,t,f X γ d c e c =1 2 X (f,2)=1,(n,t,f )=1 e(t + 2n)/f2 1 f2 e(t − 2n)/f2 2 f1 H t 2− 4n2 f2 =1 2 X (f2,2)=1,(n,t,f2)=1 X f1|u,(f1,2)=1,(n,t,f1)=1 ∆/f2 2 f1 H u 2 f2 1 ∆ f2 2 . (33)
The inner sum running over f1is the same as
(34) X f1|u ∆/f2 2 f1 H u 2 f2 1 ∆ f2 2 since if 2|f1or (n, t, f1) > 1 then ∆/f2 2 f1
= 0. Then by Lemma 28, the first sum in (24) is equal to (35) X (f,2)=1,(n,t,f )=1 1 wn,t,f X γ d c e c =u 2 X (f2,2)=1,(n,t,f2)=1 H ∆ f2 2 .
For the second sum in (24), we have two cases (
3|f1, 3 - f2, if 3|u,
3|f2, 3 - f1, if 3 - u.
If 3 - u, then a computation similar to (33)–(35) yields
(36) X (f,2)=1,3|f,(n,t,f )=1 1 wn,t,f X γ d c e c =u 2 X (f2,2)=1,3|f2,(n,t,f2)=1 H ∆ f2 2 .
If 3|u, then X (f,2)=1,3|f,(n,t,f )=1 1 wn,t,f X γ d c e c =1 2 X (f,6)=3,(n,t,f )=1 e(t + 2n)/f2 1 f2 e(t − 2n)/f2 2 f1 H t 2− 4n f2 =1 2 X (f2,2)=1,(n,t,f2)=1 X f1|u,(f1,6)=3,(n,t,f )=1 ∆/f2 2 f1 H u 2 f2 1 ∆ f2 2 .
By Lemma 28 the inner sum is equal to X h|(u/3),(h,2)=1,(n,t,h)=1 ∆/f2 2 3h H (u/3) 2 f2 1 ∆ f2 2 = u 3 ∆ 3 H ∆ f2 2 and we have (37) X (f,2)=1,3|f,(n,t,f )=1 1 wn,t,f X γ d c e c =u 6 ∆ 3 X (f2,2)=1,(n,t,f2)=1 H ∆ f2 2 .
Combining (29)–(32), (35), (36), and (37), we get the formula (27).
The proof of (28) follows the same line of calculation. Equation (29) continues to hold when we replace e by −e. By Lemma 27, (30) also remains valid, provided that the condition that e(t+2n) is a square is change to that e(2n−t) is a square. The computations in (32), (35), (36), and (37) are almost the same. The only significant difference lies at (31). Instead of (31), we have t +√t2− 4n2 2 = i pe(2n − t) − p−e(t + 2n) 2√e !2 = i√τ e 2 and t +√t2− 4n2 2 !1/2−k = i1−2k τ √ e 1−2k = i −4 r τ √ e 1−2k , (38)
where τ = (eu +√∆)/2 with ∆ = e2u2− 4en. Here in the last step, we have used the assumption k = (r − 1)/2 + s. This establishes (28).
We now prove the second part of the lemma. Here we only consider the case e = 1, 3. The integers f in the sum defining Le,1(n, t) satisfies 2kf . Let f0 = f /2 and set f1 =
(f0, t + 2n) and f2 = (f0, t − 2n). Then as before, we have f0 = f1f2, f12|(t + 2n),
f2
2|(t − 2n), and (29) remains valid if we replace c by c0and f by f0. For the first sum in
Le,`(n, t), the computation in (30)–(32) remain unchanged. Now instead of (33), we have
X 2kf,(n,t,f )=1 1 wn,t,f X γ d c e c = 1 2 X 2kf,(n,t,f )=1 e(t + 2n)/f2 1 f2 e(t − 2n)/f2 2 f1 H t 2− 4n2 f2 = 1 2 X (f2,2)=1,(n,t,f2)=1 X f1|u,(f1,2)=1,(n,t,f1)=1 ∆/f2 2 f1 H (u/2) 2 f2 1 ∆ f2 2 .
Now since 4|(∆/f22) = (e2u2− 4en)/f22, we have H (u/2) 2 f2 1 ∆ f2 2 = 1 2H u2 f2 1 ∆ f2 2 .
Following the remaining computation, we easily see that the first sum in Le,1(n, t) is equal
to one half of that of Le,0(n, t). Similarly, the second sum in Le,1(n, t) is equal to one half
of that of Le,0(n, t). This proves the second part of the lemma.
The proof of the third part is also similar. Again, we only consider the case e > 0. The computation for the first sum in Le,`(n, t) differs from that for the first sum in Le,0(n, t)
mainly at (33). In the case 2ku, instead of (33), we have X 2`kf,(n,t,f )=1 1 wn,t,f X γ d c e c =1 2 X (f2,2)=1,(n,t,f2)=1 X f1|u,(f1,2)=1,(n,t,f1)=1 ∆/f2 2 f1 H (u/2) 2 f2 1 ∆ (2`−1f 2)2 ,
where ∆ = e2u2− 4en. Then, by Lemma 28, it is equal to
X 2`kf,(n,t,f )=1 1 wn,t,f X γ d c e c =u 4 X (f2,2)=1,(n,t,f2)=1 H ∆ (2`−1f 2)2 ,
The computation for the case 2`−1ku is almost the same. In this case, we have X 2`kf,(n,t,f )=1 1 wn,t,f X γ d c e c = u 2` X (f2,2)=1,(n,t,f2)=1 H ∆ (2f2)2 ,
Now if 2`|u, then instead of (33), we have X 2`kf,(n,t,f )=1 1 wn,t,f X γ d c e c = 1 2 X (f2,2)=1,(n,t,f2)=1 X f1|u,(f1,2)=1,(n,t,f1)=1 ∆/f2 2 f1 H (u/2 `)2 f2 1 ∆ f2 2 . Since 4|(∆/f22), we have H (u/2 `)2 f2 1 ∆ f2 2 = 1 2`H u2 f2 1 ∆ f2 2 and by Lemma 28, X 2`kf,(n,t,f )=1 1 wn,t,f X γ d c e c = u 2`+1 X (f2,2)=1,(n,t,f2)=1 H ∆ f2 2 .
Together with (31) and (32), this completes the computation for the first sum in Le,`(n, t).
The computation for the second sum is analogous and is skipped. We now utilize Lemma 33 to compute the contribution of Γn,t,f to the trace of [Mn2].
This will be done according to the greatest common divisor of t and 24. To summarize our computation, we fix the following notations.
Notation 34. Given a positive integer n relatively prime to 6, e ∈ {1, 2, 3, 6}, and an integer u with e2u2< 4en, we let
(39) Pk(e, n, u) =
τ2k−1− τ2k−1
τ − τ , τ =
eu +√e2u2− 4en
2 .
For nonnegative integers ` and m, we set
A`,m(n) = 12 n X 3-u X g H ∆ g2 − 3X 3|g H ∆ g2 Pk(1, n, u) + 12 n X 3|u 1 − ∆ 3 X g H ∆ g2 Pk(1, n, u),
where ∆ = u2− 4n, the outer sums run over all integers u satisfying u2< 4n, 2`ku,
and the given conditions, and the inner sums run over all positive integers g such that ∆/g2≡ 0, 1 mod 4, 2mkg, (n, u, g) = 1,
and the specified conditions are met. We also set A∗`,m(n) = X `≤j<∞ Aj,m(n). Similarly, we define B`(n) = 1 2k−1 12 n 8 r X 3-u X g H ∆ g2 − 3X 3|g H ∆ g2 Pk(2, n, u) + 1 2k−1 12 n 8 r X 3|u 1 − ∆ 3 X g H ∆ g2 Pk(2, n, u),
where ∆ = 4u2− 8n, the outer sums run over all integers u such that 4u2< 8n, 2`ku,
and the inner sums run over all positive integers g such that ∆/g2≡ 0, 1 mod 4, (n, u, g) = 1. Also, we set B`∗(n) = X `≤j<∞ B`(n). Likewise, define C`,m(n) = 1 3k−1 12 n 12 r X 9u2<12n,2`ku X g H ∆ g2 Pk(3, n, u),
where ∆ = 9u2− 12n, and the inner sum runs over all positive integers g with
∆/g2≡ 0, 1 mod 4, 2mkg, (n, u, g) = 1, and let
C`,m∗ (n) = X
`≤j<∞
Finally, we let D`(n) = 1 6k−1 12 n 24 r X 36u2<24n,2`ku X g H ∆ g2 Pk(6, n, u),
where ∆ = 36u2− 24n, and the inner sum runs over all positive integer g such that
∆/g2≡ 0, 1 mod 4, (n, u, g) = 1. Also, let
D∗`(n) = X
`≤j<∞
Dj(n).
Here we give alternative expressions for A`,m(n) and B`(n), which will be used in a
later section.
Lemma 35. For a discriminant ∆ of an imaginary quadratic order, we let ∆0denote the
discriminant of the field Q(√∆). Then we have
A`,m(n) = β(n) 12 n X u 1 − ∆0 3 X g,3-∆/(∆0g2) H ∆ g2 Pk(1, n, u), where β(n) = ( 1, ifn ≡ 1 mod 3, 1/2, ifn ≡ 2 mod 3,
the double sum runs over the same u and g as in the definition of A`,m satisfying the
additional condition that3 does not divide ∆/(∆0g2), where ∆ = u2− 4n. Analogously,
we also have B`(n) = γ(n) 2k−1 12 n 8 r X u 1 − ∆0 3 X g,3-∆/(∆0g2) H ∆ g2 Pk(2, n, u), where γ(n) = ( 1/2, ifn ≡ 1 mod 3, 1, ifn ≡ 2 mod 3.
Proof. Here we prove only the case A`,m(n). Consider the case n ≡ 1 mod 3 first.
Assume that 3 - u. We have ∆ ≡ 0 mod 3. If 3k∆, then 3|∆0and the sumP3|gis empty,
which yields X g H ∆ g2 −X 3|g H ∆ g2 = 1 − ∆0 3 X g H ∆ g2
If 9|∆, we have X g H ∆ g2 − 3X 3|g H ∆ g2 =X h H(h2∆0) − 3 X 3|h H h 2∆ 0 9 = X (h,3)=1 H(h2∆0) + X 3|h H(h2∆0) − 3H h2∆ 0 9 = X (h,3)=1 H(h2∆0) − X 3|h h2∆ 0/9 3 H h 2∆ 0 9 = X (h,3)=1 H(h2∆0) − X (h,3)=1 h2∆ 0 3 H(h2∆0) = 1 − ∆0 3 X (h,3)=1 H(h2∆0).
Either way, we find that the first double sum in the definition of A`,m(n) is equal to
X 3-u 1 − ∆0 3 X g,3-∆/(g2∆0) H ∆ g2 Pk(1, n, u).
Now if 3|u, then ∆ is not a multiple of 3 and hence ∆ 3 = ∆0 3 .
There is nothing to do in these cases. This proves the case n ≡ 1 mod 3. Now assume that n ≡ 2 mod 3. We have 3 - ∆. Thus, the sumP
3|gis empty and 1 − u 2− 4n 3 = ( 2, if 3 - u, 0, if 3|u.
That is, the second sum in the definition of A`,m(n) vanishes. The assertion follows.
3.4.3. Case (t, 24) = 1.
Lemma 36. For the cases (t, 24) = 1, we have X (t,24)=1 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = −n 3/2−k 8 ( (A0,0(n) + C0,0(n)) , ifn ≡ 1 mod 3, C0,0(n), ifn ≡ 2 mod 3.
Here the outermost sums run over all integerst satisfying t2 < 4n2and(t, 24) = 1, the
middle sums run over positive integersf such that (t2− 4n2)/f2is a discriminant, and
the functionsA0,0(n) and C0,0(n) are defined as in Notation 34.
Proof. Since (t, 24) = 1, the integers f are always odd. Let class representatives a b c d of
Γn,t,f/SL(2, Z) be chosen as per Lemma 26. By Lemma 29, we have
a b c d ∗ =a b c d , n−k−1/2(a, b, c, d)r(cτ + d)k+1/2 , where (a, b, c, d) = d c e2πi(bd(1−c2)+c(a+d−3))/24
If 3 - f , then (c, 6) = 1 and bd(1 − c2) ≡ 0 mod 24. If 3|f , then 3|b and 8|(1 − c2). Either way, we get (a, b, c, d) = dc e2πict/24e−2πic/8and
(40) J (γ) =n k+1/2 wn,t,f d c e2πirc/8e−2πirct/24ρ 1/2−k ρ − ρ , where ρ = (t +√t2− 4n2)/2.
Now for 3 - f , we have, by the formulas in Lemma 30, e2πirc/8e−2πirct/24 = 1 4 8 rc + i −8 rc 8 −rct +√3 24 −rct − i −8 −rct + i√3 −24 −rct = 1 4 8 t − −8 t + i 4 −4 rc 8 t + −8 t + √ 3 4 12 rc 24 t + −24 t +i √ 3 4 −3 rc 24 t − −24 t = 1 2 8 t δ3(t) + iδ1(t) −4 rc + √ 3 2 24 t 12 rc δ1(t) + iδ3(t) −4 rc , where (41) δj(t) = ( 1, if t ≡ j mod 4, 0, else.
For 3|f , we have 3|c and e2πirc/8e−2πirct/24= 1 2 8 rc + i −8 rc 8 rct/3 − i −8 rct/3 = 1 2 − 8 t + −8 t − i 2 −4 rc 8 t + −8 t = −δ3(t) 8 t − iδ1(t) 8 t −4 rc . Thus, X (t,24)=1 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = n k+1/2 2 X (t,24)=1 δ3(t) 8 t L1,0(n, t) + iδ1(t) −4 r 8 t L−1,0(n, t) +√3δ1(t) 12 r 24 t L3,0(n, t) + i √ 3δ3(t) −3 r 24 t L−3,0(n, t) ! , (42)
where Le,`(n, t) are defined by (24). (Note that the second sums in the definition of
L3,0(n, t) and L−3,0(n, t) are empty.)
For the term δ3(t)L1,0(n, t), δ3(t) 6= 0 implies that t + 2n is a discriminant and Lemma
33 applies. That is, δ3(t)L1,0(n, t) is nonzero only when t + 2n = u2for some (odd)
integer u. If this situation occurs, then t + 2n ≡ 1 mod 8 and
(43) 8 t = (−1)(t2−1)/8= (−1)n(n−1)/2 = −4 n .
Also observe that since (t, 24) = 1, if n ≡ 2 mod 3, any integer u such that u2= t + 2n must be divisible by 3. These informations, together with Lemma 33, yield
X (t,24)=1 δ3(t) 8 t L1,0(n, t) = λ1(n) 2 −4 n X (u,6)=1 M1,0(n, u) +1 2 −4 n X (u,6)=3 1 +n 3 M1,0(n, u), (44)
where the sums run over all positive integers u such that u2 < 4n satisfying the given conditions, Me,`(n, u) are defined by (25) and for j = 1, 2,
(45) λj(n) =
(
1, if n ≡ j mod 3, 0, else.
Likewise, for the term δ1(t)L−1,0(n, t) in (42), δ1(t) 6= 0 implies that −(t + 2n) is a
discriminant. Therefore, assuming δ1(t) = 1, Lemma 33 shows that L−1,0(n, t) is nonzero
only when 2n − t = u2is a square. In such cases, (43) continues to hold. Then Lemma 33
yields X (t,24)=1 δ1(t) 8 t L−1,0(n, t) = i λ1(n) 2 −4 rn X (u,6)=1 M1,00 (n, u) + i 2 −4 rn X (u,6)=3 1 +n 3 M1,00 (n, u). (46)
The computation for the rest of terms in (42) is similar. We find for t with δ1(t) = 1,
3(t + 2n) is a discriminant. For such t, L3,0(n, t) is nonzero only when 3(t + 2n) = (3u)2
is a square. Then t + 2n ≡ 3 mod 24 and 24 t = −3 t −4 t 8 t = −3 n (−1)(t−1)/2+(t2−1)/8 = −3 n (−1)n(n−1)/2 = 12 n . Then by Lemma 33, X (t,24)=1 δ1(t) 24 t L3,0(n, t) = 1 2 12 n X (u,2)=1 M3,0(n, u) (47)
where the sum runs over all positive integers u satisfying 9u2< 12n and the given
condi-tions. By a similar computation and the same lemma, we also have X (t,24)=1 δ3(t) 24 t L−3,0(n, t) = i 2 −4 r 12 n X (u,2)=1 M3,00 (n, u). (48)
Combining (42), (44), (46), (47), and (48), we get X (t,24)=1 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = λ1(n)n k+1/2 4 −4 n X (u,6)=1 M1,0(n, u) − M1,00 (n, u) +n k+1/2 4 −4 n X (u,6)=3 1 +n 3 M1,0(n, u) − M1,00 (n, u) +3 knk+1/2 4 12 n 12 r X (u,2)=1 M3,0(n, u) − M3,00 (n, u) . (49)
Notice that for τ = (eu +√e2u2− 4en)/2 we have
τ1−2k− τ1−2k
τ − τ = −(en)
1−2kτ2k−1− τ2k−1
τ − τ . Now if n ≡ 1 mod 3, then −4n = 12
n and X (t,24)=1 X f X γ J (γ) = −n 3/2−k 8 (A0,0(n) + C0,0(n)) .
(Note that in the definition of A`,m(n), the integers u can be positive or negative, but in
(49), the integers u are always positive. This explains the additional factor 1/2 above.) If n ≡ 2 mod 3, then the factor 1 + n3 in the middle sum in (49) is equal to 0 and we have
X (t,24)=1 X f X γ J (γ) = −n 3/2−k 8 C0,0(n).
This completes the proof.
3.4.4. Case (t, 24) = 3. Lemma 37. We have X (t,24)=3 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = −n 3/2−k 4 ( 0, ifn ≡ 1 mod 3, A0,0(n), ifn ≡ 2 mod 3,
whereA0,0(n) is defined as in Notation 34.
Proof. Since (t, 24) = 3, we have (f, 6) = 1. Then for class representatives γ = a b c d
given in Lemma 26, we have, by Lemma 29,
J (γ) = n k+1/2 wn,t,f d c e2πirc/8e−2πirc(t/3)/8ρ 1/2−k ρ − ρ . Now e2πirc/8e−2πirc(t/3)/8= 1 2 8 rc + i −8 rc 8 rc(t/3) − i −8 rc(t/3) = 1 2 − 8 t + −8 t + i 2 −4 rc − 8 t − −8 t = −δ3(t) 8 t − iδ1(t) −4 rc 8 t ,
where δ1(t) and δ3(t) are defined as (41). Then X (t,24)=3 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = nk+1/2 X (t,24)=3 −δ3(t) 8 t L1,0(n, t) − iδ1(t) −4 r 8 t L−1,0(n, t) . By Lemma 33, the term δ3(t)L1,0(n, t) is nonzero only when t + 2n is a square and
δ1(t)L1,0(n, t) is nonzero only when 2n − t is a square. Since 3|t, if n ≡ 1 mod 3, then
2n ± t ≡ 2 mod 3 and 2n ± t can never be a square. Therefore, if n ≡ 1 mod 3, then the sum is 0. If n ≡ 2 mod 3 and 2n ± t is a square, we have, as in (43),
8 t = −4 n = − 12 n . Then following the computation in (44) and (46), we get
X (t,24)=3 X f X γ J (γ) = n k+1/2 2 12 n X (u,6)=1 M1,0(n, u) − M1,00 (n, u) = −n 3/2−k 4 A0,0(n).
(Note that here since 3|t, the integer u such that 2n ± t = u2 is never a multiple of 3, while the second sum in the definition of A0,0(n) is just 0. Also, since n ≡ 2 mod 3,
∆ = u2− 4n is never a multiple of 3. So the sum over g with 3|g in the definition of
A0,0(n) is empty.) This completes the proof.
3.4.5. Case (t, 24) = 4. Lemma 38. We have X (t,24)=4 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = −n 3/2−k 16 D0(n), ifn ≡ 1 mod 4, 0, ifn ≡ 7 mod 12, B0(n), ifn ≡ 11 mod 12.
Proof. Let class representatives a b
c d be chosen according to Lemma 26. Since (t, 24) =
4, the integers f must be odd, and the entries c in the class representatives are always odd. Regardless of whether 3|f or not, we have bd(1 − c2) ≡ 0 mod 24 so that the term J (γ) in Proposition 14 is equal to J (γ) = n k+1/2 wn,t,f d c e2πirc/8e−2πirc(t/4)/6ρ 1/2−k ρ − ρ , where ρ = (t +√t2− 4n2
)/2. Now if 3 - f , then 3 - c and e2πirc/8e−2πirc(t/4)/6 = 1 2√2 8 rc + i −8 rc 1 − i√3 −3 rc(t/4) = √ 2 4 8 rc + i √ 2 4 −8 rc + √ 6 4 24 rc −3 t − i √ 6 4 −24 rc −3 t If 3|f , then 3|c and e2πirc/8e−2πirc(t/4)/6= − √ 2 2 8 rc + i −8 rc .
Therefore, X (t,24)=4 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = n k+1/2 4 X (t,24)=4 √ 2 8 r L2,0(n, t) + i √ 2 −8 r L−2,0(n, t) +√6 24 r −3 t L6,0(n, t) − i √ 6 −24 r −3 t L−6,0(n, t) ! . (50)
By Lemma 33, L2,0(n, t) is nonzero only when (t + 2n)/2 is a square. Since 4kt, this
can possibly happen only when n ≡ 3 mod 4. Also, since 3 - t, when n ≡ 1 mod 3 and (t + 2n)/2 is indeed a square, this square must be a multiple of 3. By the same reasoning, L−2,0(n, t) can possibly be nonzero only when n ≡ 3 mod 4. Moreover, if n ≡ 1 mod 3
and (2n − t)/2 is a square, then this square is a multiple of 3. Thus, by Lemma 33, we have X (t,24)=4 √ 2 8 r L2,0(n, t) + i √ 2 −8 r L−2,0(n, t) =2 kδ 3(n) 2 8 r λ2(n) X (u,6)=1 M2,0(n, u) − M2,00 (n, u) + X (u,6)=3 1 −n 3 M2,0(n, u) − M2,00 (n, u) ! , (51)
where the sums run over all positive integers u satisfying 4u2 < 8n and the given
condi-tions, and δj(n) and λj(n) are defined by (41) and (45), respectively.
Similarly, L6,0(n, t) (respectively, L−6,0(n, t)) can possibly be nonzero only when n ≡
1 mod 4 and (t + 2n)/6 (respectively, (2n − t)/6) is a square. Moreover, when (t + 2n)/6 is a square, we have −3t = −3
n = 12
n and when (2n − t)/6 is a square, we have −3 t = − −3 n = − 12 n. Then, by Lemma 33, X (t,24)=4 √ 6 24 r −3 t L6,0(n, t) − i √ 6 −24 r −3 t L−6,0(n, t) = 6 kδ 1(n) 2 24 r 12 n X (u,2)=1 M6,0(n, u) − M6,00 (n, u) , (52)
where the sum runs over all positive odd integer u satisfying 36u2 < 24n. Substituting
(51) and (52) into (50) and simplifying, we obtain the claimed formula. (The reader is reminded again that the integers u in the sums in (51) and (52) are all positive, while the integers u in the definition of B`(n) and D`(n) can be positive or negative.)
3.4.6. Case (t, 24) = 8. Lemma 39. We have X (t,24)=4 X f X γ∈Γn,t,f/SL(2,Z) J (γ) = −n 3/2−k 16 0, ifn ≡ 1 mod 12, B0(n), ifn ≡ 5 mod 12, D0(n), ifn ≡ 3 mod 4.
