A Hilbert space is separable if it contains a countable dense subset. All finite-dimensional Hilbert spaces are separable, and Problem 3.39 shows that the space ℓ2 is separable. However, not every Hilbert space is separable; an ex-ample is given in Problem 3.41.
We will show that every separable Hilbert space contains an orthonormal basis. We begin with finite-dimensional spaces, where we can use the same Gram–Schmidt procedure that is employed to construct orthonormal se-quences in Rd or Cd.
Theorem 3.22. If H is a finite-dimensional Hilbert space and d is the vector space dimension of H, then H contains an orthonormal basis of the form {e1, . . . , ed}.
Proof. Since H is a d-dimensional vector space, it has a Hamel basis B that consists of d vectors, say B = {x1, . . . , xd}.
Set y1= x1, and note that f16= 0 since x1, . . . , xdare linearly independent and therefore nonzero. Define
M1 = span{x1} = span{y1}.
Note that M1 is closed since it is finite-dimensional.
If d = 1 then we stop here. Otherwise M1 is a proper subspace of H, and x2∈ M/ 1(because {x1, . . . , xd} is linearly independent). Let p2be the orthog-onal projection of x2onto M1. Then the vector y2= x2− p2is orthogonal to x1, and y26= 0 since x2∈ M/ 1. Therefore we can define
M2 = span{x1, x2} = span{y1, y2},
where the second equality follows from the fact that y1, y2 are linear combi-nations of x1, x2, and vice versa.
If d = 2 then we stop here. Otherwise we continue in the same manner.
At stage k we have constructed orthogonal vectors y1, . . . , yk such that Mk = span{x1, . . . , xk} = span{y1, . . . , yk}.
The subspace Mk has dimension k. Consequently, when we reach k = d we will have Md= H, and therefore
H = Md = span{x1, . . . , xd} = span{y1, . . . , yd}.
The vectors y1, . . . , yd are orthogonal and nonzero, so by setting ek = yk
kykk, k = 1, . . . , d, we obtain an orthonormal basis {e1, . . . , ed} for H. ⊓⊔
3.9 Existence of an Orthonormal Basis 53
Next we consider infinite-dimensional, but still separable, Hilbert spaces.
Theorem 3.23. If H is a infinite-dimensional, separable Hilbert space, then H contains an orthonormal basis of the form {en}n∈N.
Proof. Since H is separable, it contains a countable dense subset. This subset must be infinite, so let us say that it is Z = {zn}n∈N. However, Z need not be linearly independent, so we will extract a linearly independent subsequence as follows.
Let k1 be the first index such that zk1 6= 0, and set x1= zk1. Then let k2
be the first index larger than k1such that zk2 ∈ span{x/ 1}, and set x2= zk2. Then let k3 be the first index larger than k2 such that zk3 ∈ span{x/ 1, x2}, and so forth. In this way we obtain vectors x1, x2, . . . such that x16= 0 and for each n > 1 we have
xn∈ span{x/ 1, . . . , xn−1} and span{x1, . . . , xn} = span{z1, . . . , zkn}.
Therefore {xn}n∈N is linearly independent, and furthermore span¡{xn}n∈N¢ is dense in H.
Now we apply the Gram–Schmidt procedure utilized in the proof of Theo-rem 3.22, but without stopping. This gives us orthonormal vectors e1, e2, . . . such that for every n we have
span{e1, . . . , en} = span{x1, . . . , xn}.
Consequently span¡{en}n∈N¢ is dense in H. Therefore {en}n∈Nis a complete orthonormal sequence in H, and hence it is an orthonormal basis for H. ⊓⊔
The following result gives a converse to Theorems 3.22 and 3.23: only a separable Hilbert space can contain an orthonormal basis. This is a conse-quence of the fact that we declared in Definition 3.20 that an orthonormal basis must be a countable sequence.
Theorem 3.24. If a Hilbert space H is not separable, then H does not con-tain an orthonormal basis.
Proof. We will prove the contrapositive statement. Suppose that H contains an orthonormal basis. Such a basis is either finite or countably infinite; since both cases are similar let us assume that {en}n∈N is an orthonormal basis for H.
Say that a complex number is rational if both its real and imaginary parts are rational, and let
S =
½ N X
n=1
rnen: N > 0, r1, . . . , rN rational
¾ .
This is a countable subset of H, and we will show that it is dense.
Choose any x ∈ H and fix ε > 0. Since kxk2=P |hx, eni|2, we can choose N large enough that
∞
X
n=N +1
|hx, eni|2 < ε2 2 .
For each n = 1, . . . , N, choose a scalar rn that has real and imaginary parts and satisfies
|hx, eni − rn|2 < ε2 2N. Then the vector
z =
N
X
n=1
rnen
belongs to S, and by the Plancherel Equality we have
kx − zk2 =
N
X
n=1
|hx, eni − rn|2 +
∞
X
n=N +1
|hx, eni|2 < N ε2 2N +ε2
2 = ε2. Thus kx − zk < ε, so S is dense in H. As S is also countable, it follows that H is separable. ⊓⊔
Nonseparable Hilbert spaces do exist (see Problem 3.41 for one example).
An argument based on the Axiom of Choice in the form of Zorn’s Lemma shows that every Hilbert space, including nonseparable Hilbert spaces, con-tains a complete orthonormal set (for a proof, see [Heil11, Thm. 1.56]). How-ever, if H is nonseparable, then such a complete orthonormal set must be uncountable. An uncountable complete orthonormal set does have certain basis-like properties (e.g., see [Heil11, Exer. 3.6]), and for this reason some authors refer to a complete orthonormal set of any cardinality as an ortho-normal basis. In keeping with the majority of the Banach space literature, we prefer to reserve the word “basis” for use in conjunction with countable sequences only.
Problems
3.25. Prove Lemma 3.2.
3.26. Prove the continuity of the inner product: If H is an inner product space and xn→ x, yn→ y in H, then hxn, yni → hx, yi.
3.27. Prove that if a seriesP∞
n=1xn converges in an inner product space H, then
¿∞ X
n=1
xn, y À
=
∞
X
n=1
hxn, yi, y ∈ H.
3.9 Existence of an Orthonormal Basis 55
Note that this is not merely a consequence of the linearity of the inner product in the first variable; the continuity of the inner product is also needed.
3.28. Prove that the function h·, ·i defined in equation (3.6) is an inner prod-uct on C[a, b].
3.29. Prove that ℓ2 is complete with respect to the norm k · k2 defined in equation (3.2).
3.30. For each n ∈ N, let
yn = ¡1,12, . . . ,n1, 0, 0, . . .¢.
Note that yn∈ ℓ1for every n.
(a) Assume that the norm on ℓ1is its usual norm k·k1. Prove that {yn}n∈N
is not a Cauchy sequence in ℓ1 with respect to this norm.
(b) Now assume that the norm on ℓ1 is k · k2. Prove that {yn}n∈N is a Cauchy sequence in ℓ1 with respect to k · k2. Even so, prove that there is no vector y ∈ ℓ1such that ky − ynk2→ 0. Conclude that ℓ1is not complete with respect to the norm k · k2.
3.31. Prove Lemma 3.12.
3.32. Prove the remaining implications in Lemma 3.15.
3.33. Given a sequence {xn}n∈Nin a Hilbert space H, prove that the follow-ing two statements are equivalent.
(a) For each m ∈ N we have xm ∈ span¡{x/ n}n6=m¢ (such a sequence is said to be minimal ).
(b) There exists a sequence {yn}n∈N in H such that hxm, yni = δmn for all m, n ∈ N (we say that sequences {xn}n∈N and {yn}n∈N satisfying this condition are biorthogonal ).
Show further that in case statements (a), (b) hold, the sequence {yn}n∈N
is unique if and only if {xn}n∈Nis complete.
3.34. Formulate and prove analogues of Theorems 3.18 and 3.19 for finite orthonormal sequences.
3.35. Extend the Pythagorean Theorem to finite orthogonal sets of vectors.
That is, prove that if x1, . . . , xN ∈ H are orthogonal vectors in an inner product space H, then
°
°
°
°
N
X
n=1
xn
°
°
°
°
2
=
N
X
n=1
kxnk2.
Does the result still hold if we only assume that h·, ·i is a semi-inner product?
3.36. A d × d matrix A with scalar entries is said to be positive definite if Ax · x > 0 for all nonzero vectors x ∈ Cd, where x · y denotes the usual dot product of vectors in Cd.
(a) Suppose that S is an invertible d × d matrix and Λ is a diagonal matrix whose diagonal entries are all positive. Prove that A = SΛSH is a positive definite matrix, where SH= STis the complex conjugate of the transpose of S (usually referred to as the Hermitian of S).
Remark: In fact, it can be shown that every positive definite matrix has this form, but that is not needed for this problem.
(b) Show that if A is a positive definite d × d matrix, then hx, yiA = Ax · y, x, y ∈ Cd, defines an inner product on Cd.
(c) Show that if h·, ·i is an inner product on Cd, then there exists some positive definite d × d matrix A such that h·, ·i = h·, ·iA.
3.37. Let h·, ·i be a semi-inner product on a vector space H. Show that equal-ity holds in the Cauchy–Bunyakovski–Schwarz Inequalequal-ity if and only if there exist scalars α, β, not both zero, such that kαx + βyk = 0. In particular, if h·, ·i is an inner product, then either x = cy or y = cx where c is a scalar.
3.38. Let M be a closed subspace of a Hilbert space H, and let P be the orthogonal projection of H onto M. Show that I − P is the orthogonal pro-jection of H onto M⊥.
3.39. Prove that the set S defined in Problem 2.26 is a countable, dense subset of ℓ2. Conclude that ℓ2 is separable.
3.40. We say that a sequence {xn}n∈N in a Banach space X is ω-dependent if there exist scalars cn, not all zero, such that P∞
n=1cnxn = 0, where the series converges in the norm of X. A sequence is ω-independent if it is not ω-dependent.
(a) Prove that every Schauder basis is both complete and ω-independent.
(b) Let α, β ∈ C be fixed nonzero scalars such that |α| > |β|. Let {δn}n∈N
be the sequence of standard basis vectors, and define
x0 = δ1 and xn = αδn+ βδn+1, n ∈ N.
Prove that the sequence {xn}n≥0is complete and finitely linearly independent in ℓ2, but it is not ω-independent and therefore is not a Schauder basis for ℓ2. 3.41. Let I be an uncountable index set I, and let ℓ2(I) consist of all se-quences x = (xi)i∈I with at most countably many nonzero components such that
3.9 Existence of an Orthonormal Basis 57
kxk22 = X
i∈I
|xi|2< ∞.
(a) Prove that ℓ2(I) is a Hilbert space with respect to the inner product hx, yi = X
i∈I
xiyk.
(b) For each i ∈ I, define δi = (δij)j∈I, where δij is the Kronecker delta.
Show that {δi}i∈I is a complete orthonormal sequence in ℓ2(I).
(c) Prove that ℓ2(I) is not separable.
Index of Symbols
Sets
Symbol Description
∅ Empty set
Br(x) Open ball of radius r centered at x
C Complex plane
N Natural numbers, {1, 2, 3, . . . }
P Set of all polynomials
Q Rational numbers
R Real line
Z Integers, {. . . , −1, 0, 1, . . . }
Operations on Sets
Symbol Description
AC= X\A Complement of a set A ⊆ X A◦ Interior of a set A
A Closure of a set A
∂A Boundary of a set A
A × B Cartesian product of A and B dist(x, E) Distance from a point to a set inf(S) Infimum of a set of real numbers S span(A) Finite linear span of a set A
span(A) Closure of the finite linear span of A sup(S) Supremum of a set of real numbers S
59
Sequences
Symbol Description
{xn}n∈N A sequence of points x1, x2, . . . (xk)k∈N A sequence of scalars x1, x2, . . . δn nth standard basis vector
Operations on Functions
Symbol Description
f−1(B) Inverse image of B under f
Vector Spaces
Symbol Description
c00 Set of all “finite sequences”
Cb(X) Set of bounded continuous functions on X C[a, b] Set of continuous functions on [a, b]
ℓ1 Set of all absolutely summable sequences ℓ2 Set of all square-summable sequences
Metrics, Norms, and Inner Products
Symbol Description
A⊥ Orthogonal complement of a set A d(·, ·) Generic metric
h·, ·i Generic inner product
k · k Generic norm
kf ku Uniform norm of a function f x ⊥ y Orthogonal vectors
kxk1 ℓ1-norm of a sequence x
References
[Ben97] J. J. Benedetto, Harmonic Analysis and Applications, CRC Press, Boca Raton, FL, 1997.
[Con90] J. B. Conway, A Course in Functional Analysis, Second Edition, Springer-Verlag, New York, 1990.
[DM72] H. Dym and H. P. McKean, Fourier Series and Integrals, Academic Press, New York–London, 1972.
[Enf73] P. Enflo, A counterexample to the approximation problem in Banach spaces, Acta Math., 130 (1973), pp. 309–317.
[Fol99] G. B. Folland, Real Analysis, Second Edition, Wiley, New York, 1999.
[Heil11] C. Heil, A Basis Theory Primer, Expanded Edition, Birkh¨auser, Boston, 2011.
[Heil15] C. Heil, Metrics, Norms, Inner Products, and Topology, Birkh¨auser, Boston, to appear.
[Kat04] Y. Katznelson, An Introduction to Harmonic Analysis, Third Edition, Cam-bridge University Press, CamCam-bridge, UK, 2004.
[Kre78] E. Kreyszig, Introductory Functional Analysis with Applications, Wiley, New York, 1978.
[Mun75] J. R. Munkres, Topology: A First Course, Prentice-Hall, Englewood Cliffs, NJ, 1975.
[Rud76] W. Rudin, Principles of Mathematical Analysis, Third Edition. McGraw-Hill, New York, 1976.
[ST76] I. M. Singer and J. A. Thorpe, Lecture Notes on Elementary Topology and Geom-etry, Springer-Verlag, New York-Heidelberg, 1976 (reprint of the 1967 edition).
[SS05] E. M. Stein and R. Shakarchi, Real Analysis, Princeton University Press, Prince-ton, NJ, 2005.
[WZ77] R. L. Wheeden and A. Zygmund, Measure and Integral, Marcel Dekker, New York-Basel, 1977.
61
Index
absolutely summable, 2 accumulation point, 7
Baire Category Theorem, 28 Banach space, 23
basis Hamel, 27 orthonormal, 51 Schauder, 28 Basis Problem, 29 Bessel’s Inequality, 47 biorthogonal sequence, 55 boundary, 7
point, 7 bounded set, 7
Cauchy sequence, 3
Cauchy–Bunyakovski–Schwarz Inequality, 36
CBS Inequality, 36 closed
finite span, 26 set, 7
Closest Point Theorem, 43 closure, 7
compact set, 9 complete
inner product space, 37 metric space, 6 normed space, 23 sequence, 27 continuity, 12
of the inner product, 54 of the norm, 22 uniform, 14 convergence
absolute, 24
uniform, 29 convergent
sequence, 3 series, 24 convex
set, 23
δsequence, 2 dense set, 7 distance, 2, 21
Dvoretzky–Rogers Theorem, 26
finite
linear independence, 28 linear span, 26 function
continuous, 12
uniformly continuous, 14 fundamental sequence, 27
Gram–Schmidt orthogonalization, 52
Hamel basis, 27
Hausdorff metric space, 9, 18 Hermitian, 56
Hilbert space, 37
independence finite linear, 28 ω-independence, 56 induced
metric, 22 norm, 36 seminorm, 36 inner
product, 35 product space, 35
63