Obviously the choice of an appropriate circulant matrix to approximate a Toeplitz matrix is not unique, so we are free to choose a construction with the most desirable properties. It will, in fact, prove useful to consider two slightly different circulant approximations to a given Toeplitz matrix. Say we have an absolutely summable sequence {tk; k = 0,±1, ±2, · · ·} with
f (λ) =
X∞ k=−∞
tkeikλ
tk = (2π)−1
Z 2π
0
f (λ)e−ikλ
. (4.25)
Define Cn(f ) to be the circulant matrix with top row (c(n)0 , c(n)1 ,· · · , c(n)n−1) where
c(n)k = n−1
nX−1 j=0
f (2πj/n)e2πijk/n . (4.26)
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 35
Since f (λ) is Riemann integrable, we have that for fixed k
nlim→∞c(n)k = lim
and hence the c(n)k are simply the sum approximations to the Riemann integral giving t−k. Equations (4.26), (3.7), and (3.9) show that the eigenvalues ψn,m of Cn are simply f (2πm/n); that is, from (3.7) and (3.9)
Thus, Cn(f ) has the useful property (4.21) of the circulant approximation (4.15) used in the finite case. As a result, the conclusions of lemma 4.4 hold for the more general case with Cn(f ) constructed as in (4.26). Equation (4.28) in turn defines Cn(f ) since, if we are told that Cnis a circulant matrix
The fact that lemma 4.4 holds for Cn(f ) yields several useful properties as summarized by the following lemma.
Lemma 4.5
1. Given a function f of (4.25) and the circulant matrix Cn(f ) defined by (4.26), then
c(n)k =
X∞ m=−∞
t−k+mn , k = 0, 1,· · · , n − 1. (4.30) (Note, the sum exists since the tk are absolutely summable.)
2. Given Tn(f ) where f (λ) is real and f (λ)≥ mf > 0, then Cn(f )−1 = Cn(1/f ).
3. Given two functions f (λ) and g(λ), then Cn(f )Cn(g) = Cn(f g).
Proof.
1. Since e−2πimk/n is periodic with period n, we have that f (2πj/n) = and hence from (4.26) and (3.9)
c(n)k = n−1
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 37 2. Since Cn(f ) has eigenvalues f (2πk/n) > 0, by theorem 3.1 Cn(f )−1 has eigenvalues 1/f (2πk/n), and hence from (4.29) and the fact that Cn(f )−1 is circulant we have Cn(f )−1 = Cn(1/f ).
3. Follows immediately from theorem 3.1 and the fact that, if f (λ) and g(λ) are Riemann integrable, so is f (λ)g(λ).
2 Equation (4.30) points out a shortcoming of Cn(f ) for applications as a circulant approximation to Tn(f ) — it depends on the entire sequence {tk; k = 0,±1, ±2, · · ·} and not just on the finite collection of elements {tk; k = 0,±1, · · · , ±n − 1} of Tn(f ). This can cause problems in practi-cal situations where we wish a circulant approximation to a Toeplitz matrix Tn when we only know Tn and not f . Pearl [16] discusses several coding and filtering applications where this restriction is necessary for practical reasons.
A natural such approximation is to form the truncated Fourier series fˆn(λ) =
Note that both Cn(f ) and ˆCn= Cn( ˆfn) reduces to the Cn(f ) of (4.15) for an rth order Toeplitz matrix if n > 2r + 1.
The matrix ˆCn does not have the property (4.28) of having eigenvalues f (2πk/n) in the general case (its eigenvalues are ˆfn(2πk/n), k = 0, 1,· · · , n−
1), but it does not have the desirable property to depending only on the entries of Tn. The following lemma shows that these circulant matrices are asymptotically equivalent to each other and to Tm.
Lemma 4.6 Let Tn={tk−j} where
X∞ k=−∞
|tk| < ∞
and define as usual
f (λ) =
X∞ k=−∞
tkeikλ.
Define the circulant matrices Cn(f ) and ˆCn= Cn( ˆfn) as in (4.26) and (4.31)-(4.32). Then,
Cn(f )∼ ˆCn ∼ Tn. (4.34) Proof. Since both Cn(f ) and ˆCn are circulant matrices with the same eigen-vectors (theorem 3.1), we have from part 2 of theorem 3.1 and (2.14) and the comment following it that
|Cn(f )− ˆCn|2 = n−1
nX−1 k=0
|f(2πk/n) − ˆfn(2πk/n)|2.
Recall from (4.4) and the related discussion that ˆfn(λ) uniformly converges to f (λ), and hence given ² > 0 there is an N such that for n ≥ N we have for all k, n that
|f(2πk(n) − ˆfn(2πk/n)|2 ≤ ² and hence for n≥ N
|Cn(f )− ˆCn|2 ≤ n−1nX−1
i=0
² = ².
Since ² is arbitrary,
nlim→∞|Cn(f )− ˆCn| = 0
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 39
From (4.33) we have that
t0k =
Since the {tk} are absolutely summable,
nlim→∞|tn−1|2 = 0
and given ² > 0 we can choose an N large enough so that
X∞
Since ² is arbitrary,
nlim→∞|Tn− ˆCn| = 0 and hence
Tn ∼ ˆCn, (4.37)
which with (4.35) and theorem 2.1 proves (4.34). 2 Pearl [16] develops a circulant matrix similar to ˆCn (depending only on the entries of Tn) such that (4.37) holds in the more general case where (4.1) instead of (4.2) holds.
We now have a circulant matrix Cn(f ) asymptotically equivalent to Tn and whose eigenvalues, inverses and products are known exactly. We can now use lemmas 4.2-4.4 and theorems 2.2-2.3 to immediately generalize theorem 4.1
Theorem 4.2 Let Tn(f ) be a sequence of Toeplitz matrices such that f (λ) is Riemann integrable, e.g., f (λ) is bounded or the sequence tk is absolutely summable. Then if τn,k are the eigenvalues of Tn and s is any positive integer
nlim→∞n−1
nX−1 k=0
τn,ks = (2π)−1
Z 2π
0
f (λ)sdλ. (4.38) Furthermore, if Tn(f ) is Hermitian (f (λ) is real) then for any function F (x) continuous on [mf, Mf]
nlim→∞n−1
nX−1 k=0
F (τn,k) = (2π)−1
Z 2π
0
F [f (λ)] dλ. (4.39)
Theorem 4.2 is the fundamental eigenvalue distribution theorem of Szeg¨o [1]. The approach used here is essentially a specialization of Grenander’s ([13], ch. 7).
Theorem 4.2 yields the following two corollaries.
Corollary 4.1 Let Tn(f ) be Hermitian and define the eigenvalue distribution function Dn(x) = n−1 (number of τn,k ≤ x). Assume that
Z
λ:f (λ)=x
dλ = 0.
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 41
Then the limiting distribution D(x) = limn→∞Dn(x) exists and is given by D(x) = (2π)−1
Z
f (λ)≤x
dλ.
The technical condition of a zero integral over the region of the set of λ for which f (λ) = x is needed to ensure that x is a point of continuity of the limiting distribution.
Proof. Define the characteristic function
1x(α) =
1 mf ≤ α ≤ x 0 otherwise
.
We have
D(x) = lim
n→∞n−1
nX−1 k=0
1x(τn,k) .
Unfortunately, 1x(α) is not a continuous function and hence theorem 4.2 can-not be immediately implied. To get around this problem we mimic Grenander and Szeg¨o p. 115 and define two continuous functions that provide upper and lower bounds to 1x and will converge to it in the limit. Define
1+x(α) =
1 α ≤ x
1− α−x² x < α≤ x + ² 0 x + ² < α
1−x(α) =
1 α≤ x − ²
1− α−x+²² x− ² < α ≤ x
0 x < α
The idea here is that the upper bound has an output of 1 everywhere 1xdoes, but then it drops in a continuous linear fashion to zero at x + ² instead of immediately at x. The lower bound has a 0 everywhere 1x does and it rises linearly from x to x− ² to the value of 1 instead of instantaneously as does 1x. Clearly
1−x(α) < 1x(α) < 1+x(α)
for all α. 2
Since both 1+x and 1−x are continuous, theorem 4 can be used to conclude
These inequalities imply that for any ² > 0, as n grows the sample average n−1Pnk=0−11x(τn,k) will be sandwiched between
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 43 Since ² can be made arbitrarily small, this means the sum will be sandwiched between
(2π)−1
Z
f (λ)≤x
dλ and
(2π)−1
Z
f (λ)≤x
dλ− (2π)−1Z
f (λ)=x
dλ.
Thus if Z
f (λ)=x
dλ = 0, then
D(x) = (2π)−1
Z 2π
0
1x[f (λ)]dλ
= (2π)−1v
Z
f (λ)≤x
dλ .
Corollary 4.2 For Tn(f ) Hermitian we have
nlim→∞max
k τn,k = Mf
nlim→∞min
k τn,k = mf. Proof. From Corollary 4.1 we have for any ² > 0
D(mf + ²) =
Z
f (λ)≤mf+²
dλ > 0.
The strict inequality follows from the continuity of f (λ). Since
nlim→∞n−1{number of τn,k in [mf, mf + ²]} > 0
there must be eigenvalues in the interval [mf, mf+ ²] for arbitrarily small ².
Since τn,k ≥ mf by lemma 4.1, the minimum result is proved. The maximum
result is proved similarly. 2
We next consider the inverse of an Hermitian Toeplitz matrix.
Theorem 4.3 Let Tn(f ) be a sequence of Hermitian Toeplitz matrices such that f (λ) is Riemann integrable and f (λ) ≥ 0 with equality holding at most at a countable number of points.
1. Tn(f ) is nonsingular 2. If f (λ)≥ mf > 0, then
Tn(f )−1 ∼ Cn(f )−1, (4.40) where Cn(f ) is defined in (4.29). Furthermore, if we define Tn(f )− Cn(f ) = Dn then Tn(f )−1 has the expansion
Tn(f )−1 = [Cn(f ) + Dn]−1
= Cn(f )−1[I + DnCn(f )−1]−1
= Cn(f )−1hI + DnCn(f )−1+ (DnCn(f )−1)2+· · ·i (4.41) and the expansion converges (in weak norm) for sufficiently large n.
3. If f (λ)≥ mf > 0, then Tn(f )−1 ∼ Tn(1/f ) =
½
(2π)−1
Z π
−πdλei(k−j)λ/f (λ)
¾
; (4.42) that is, if the spectrum is strictly positive then the inverse of a Toeplitz matrix is asymptotically Toeplitz. Furthermore if ρn,k are the eigenval-ues of Tn(f )−1 and F (x) is any continuous function on [1/Mf, 1/mf], then
nlim→∞n−1
nX−1 k=0
F (ρn,k) = (2π)−1
Z π
−πF [(1/f (λ)] dλ. (4.43) 4. If mf = 0, f (λ) has at least one zero, and the derivative of f (λ) exists and is bounded, then Tn(f )−1 is not bounded, 1/f (λ) is not integrable and hence Tn(1/f ) is not defined and the integrals of (4.41) may not exist. For any finite θ, however, the following similar fact is true: If F (x) is a continuous function of [1/Mf, θ], then
nlim→∞n−1
nX−1 k=0
F [min(ρn,k, θ)] = (2π)−1
Z 2π
0
F [min(1/f (λ), θ)] dλ.
(4.44)
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 45
Proof.
1. Since f (λ) > 0 except at possible a finite number of points, we have from (4.9)
x∗Tnx = 1 2π
Z π
−π
¯¯¯¯
¯
nX−1 k=0
xkeikλ
¯¯¯¯
¯
2
f (λ)dλ > 0 so that for all n
mink τn,k > 0 and hence
det Tn =
nY−1 k=0
τn,k 6= 0 so that Tn(f ) is nonsingular.
2. From lemma 4.6, Tn ∼ Cn and hence (4.40) follows from theorem 2.1 since f (λ)≥ mf > 0 ensures that
k Tn−1 k, k Cn−1 k≤ 1/mf <∞.
The series of (4.41) will converge in weak norm if
|DnCn−1| < 1 (4.45) since
|DnCn−1| ≤k Cn−1 k ·|Dn| ≤ (1/mf)|Dn|n−→→∞ 0
(4.45) must hold for large enough n. From (4.40), however, if n is large enough, then probably the first term of the series is sufficient.
3. We have
|Tn(f )−1− Tn(1/f )| ≤ |Tn(f )−1− Cn(f )−1| + |Cn(f )−1− Tn(1/f )|.
From (b) for any ² > 0 we can choose an n large enough so that
|Tn(f )−1− Cn(f )−1| ≤ ²
2. (4.46)
From theorem 3.1 and lemma 4.5, Cn(f )−1 = Cn(1/f ) and from lemma 4.6 Cn(1/f ) ∼ Tn(1/f ). Thus again we can choose n large enough to ensure that
|Cn(f )−1− Tn(1/f )| ≤ ²/2 (4.47)
so that for any ² > 0 from (4.46)-(4.47) can choose n such that
|Tn(f )−1− Tn(1/f )| ≤ ²
which is (4.42). Equation (4.43) follows from (4.42) and theorem 2.4.
Alternatively, if G(x) is any continuous function on [1/Mf, 1/mf] and (4.43) follows directly from lemma 4.6 and theorem 2.4 applied to G(1/x).
4. When f (λ) has zeros (mf = 0) then from Corollary 4.2 lim
n→∞min
k τn,k= 0 and hence
k Tn−1 k= max
k ρn,k = 1/ min
k τn,k (4.48)
is unbounded as n → ∞. To prove that 1/f(λ) is not integrable and hence that Tn(1/f ) does not exist we define the sets
Ek = {λ : 1/k ≥ f(λ)/Mf > 1/(k + 1)}
= {λ : k ≤ Mf/f (λ) < k + 1} (4.49) since f (λ) is continuous on [0, Mf] and has at least one zero all of these sets are nonzero intervals of size, say, |Ek|. From (4.49)
Z π
−πdλ/f (λ)≥X∞
k=1
|Ek|k/Mf (4.50)
since f (λ) is differentiable there is some finite value η such that
¯¯¯¯
¯
df dλ
¯¯¯¯
¯≤ η. (4.51)
From (4.50) and (4.51)
Z π
−πdλ/f (λ) ≥ X∞
k=1
(k/Mf)(1/k− 1/(k + 1)/η
= (Mfη)−1
X∞ k=1
1/(k + 1)
, (4.52)
which diverges so that 1/f (λ) is not integrable. To prove (4.44) let F (x) be continuous on [1/Mf, θ], then F [min(1/x, θ)] is continuous on [0, Mf] and hence theorem 2.4 yields (4.44). Note that (4.44) implies that the eigenvalues of Tn−1 are asymptotically equally distributed up to any finite θ as the eigenvalues of the sequence of matrices Tn[min(1/f, θ)].
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 47 2 A special case of part 4 is when Tn(f ) is finite order and f (λ) has at least one zero. Then the derivative exists and is bounded since
df /dλ =
¯¯¯¯
¯¯
Xm k=−m
iktkeikλ
¯¯¯¯
¯¯
≤ Xm
k=−m
|k||tk| < ∞ .
The series expansion of part 2 is due to Rino [6]. The proof of part 4 is motivated by one of Widom [26]. Further results along the lines of part 4 regarding unbounded Toeplitz matrices may be found in [11]. Related results considering asymptotically equal distributions of unbounded sequences can be found in Tyrtyshnikov [25] and Trench [22]. These works extend Weyl’s definition of asymptotically equal distributions to unbounded sequences and develop conditions for and implications of such equal distributions.
Extending (a) to the case of non-Hermitian matrices can be somewhat difficult, i.e., finding conditions on f (λ) to ensure that Tn(f ) is invertible.
Parts (a)-(d) can be straightforwardly extended if f (λ) is continuous. For a more general discussion of inverses the interested reader is referred to Widom [26] and the references listed in that paper. It should be pointed out that when discussing inverses Widom is concerned with the asymptotic behavior of finite matrices. As one might expect, the results are similar. The results of Baxter [1] can also be applied to consider the asymptotic behavior of finite inverses in quite general cases.
We next combine theorems 2.1 and lemma 4.6 to obtain the asymptotic behavior of products of Toeplitz matrices. The case of only two matrices is considered first since it is simpler.
Theorem 4.4 Let Tn(f ) and Tn(g) be defined as in (4.5) where f (λ) and g(λ) are two bounded Riemann integrable functions. Define Cn(f ) and Cn(g) as in (4.29) and let ρn,k be the eigenvalues of Tn(f )Tn(g)
1.
Tn(f )Tn(g)∼ Cn(f )Cn(g) = Cn(f g). (4.53)
Tn(f )Tn(g)∼ Tn(g)Tn(f ). (4.54)
de-fined as in (4.29)
Ym
5. If ρn,k are the eigenvalues of
Ym
If the Tn(fi) are Hermitian, then the ρn,k are asymptotically real, i.e., the imaginary part converges to a distribution at zero, so that
nlim→∞n−1
4.3. ABSOLUTELY SUMMABLE TOEPLITZ MATRICES 49
Proof.
1. Equation (4.53) follows from lemmas 4.5 and 4.6 and theorems 2.1 and 3. Equation (4.54) follows from (4.53). Note that while Toeplitz matrices do not in general commute, asymptotically they do. Equation (4.55) follows from (4.53), theorem 2.2, and lemma 4.4.
2. Proof follows from (4.53) and theorem 2.4. Note that the eigenvalues of the product of two Hermitian matrices are real ([15], p. 105).
3. Applying lemmas 4.5 and 4.6 and theorem 2.1
|Tn(f )Tn(g)− Tn(f g)| = |Tn(f )Tn(g)− Cn(f )Cn(g) +Cn(f )Cn(g)− Tn(f g)|
≤ |Tn(f )Tn(g)− Cn(f )Cn(g)| +|Cn(f g)− Tn(f g)|n−→→∞0
.
4. Follows from repeated application of (4.53) and part (c).
5. Equation (4.58) follows from (d) and theorem 2.1. For the Hermitian case, however, we cannot simply apply theorem 2.4 since the eigenvalues ρn,k of Y
i
Tn(fi) may not be real. We can show, however, that they are asymptotically real. Let ρn,k = αn,k+ iβn,kwhere αn,k and βn,kare real.
Then from theorem 2.2 we have for any positive integer s
nlim→∞n−1
where ψn,k are the eigenvalues of Cn
Ãm
From (4.57), theorem 2.1 and lemma 4.4
Subtracting (4.61) for s = 2 from (4.61) yields
nlim→∞n−1
nX−1 k=1
βn,k2 ≤ 0.
Thus the distribution of the imaginary parts tends to the origin and hence Parts (d) and (e) are here proved as in Grenander and Szeg¨o ([13], pp.
105-106.
We have developed theorems on the asymptotic behavior of eigenvalues, inverses, and products of Toeplitz matrices. The basic method has been to find an asymptotically equivalent circulant matrix whose special simple structure as developed in Chapter 3 could be directly related to the Toeplitz matrices using the results of Chapter 2. We began with the finite order case since the appropriate circulant matrix is there obvious and yields certain desirable properties that suggest the corresponding circulant matrix in the infinite case. We have limited our consideration of the infinite order case to absolutely summable coefficients or to bounded Riemann integrable func-tions f (λ) for simplicity. The more general case of square summable tk or bounded Lebesgue integrable f (λ) treated in Chapter 7 of [13] requires sig-nificantly more mathematical care but can be interpreted as an extension of the approach taken here.