John E. Gilbert

### Lecture Notes Fall 1997

Version: August 1997

Contents

Overview, introductory articles.

Part I: Fourier analysis on Euclidean space.

1. Hilbert space, bases.

2. Operator theory.

3. Fourier series.

4. Fourier transforms.

5. Poisson summation formula.

Part II: Wavelet analysis.

1. Continuous wavelet transform.

2. Multi-resolution analysis.

3. Pre-historic wavelets.

4. Splines as pre-wavelets.

5. Daubechies construction.

Notes and Comments

Part I. Fourier Analysis on Euclidean Space

### 1. Hilbert space, bases.

A Hilbert space is the natural abstraction of very familiar ideas from 3-space:(1:1)

(i) the linear structure of vectors,

(ii) the inner product (or dot product) of vectors, (iii) the completeness property of real numbers.

The natural abstraction of (1.1)(i) is a vector space over the scalar eld of real numbers

R or complex numbers ^{C} which for us will always be ^{C} unless it is speci
cally identi
ed
as being ^{R}. To axiomatize (1.1)(ii), recall that in 3-space the basis-free form of the dot
product of vectors

### u

### v

is de ned by(1:2)

### u

:### v

=^{k}

### u

^{k}

^{k}

### v

^{k}cos

where^{k}

### u

^{k}

^{k}

### v

^{k}denote the length of

### u

### v

, and denotes the angle between### u

and### v

. Thus the dot product captures both the idea of length and of angle, and so adds the geometric structure of 3-space to the usual linear structure of vectors. Clearly(1:3)(i)

### u

:### u

^{}0

### u

:### u

= 0 if and only if### u

=### 0

### u

:### v

=### v

:### u

the geometric structure is linked to the linear structure through the third property (1:3)(ii) (

### u

^{1}+

### u

^{2}):

### v

=### u

^{1}:

### v

+### u

^{2}:

### v

(### u

):### v

=(### u

:### v

) (^{2}

^{R}) :

These properties are the ones used to impose an inner product structure on a vector space.

(1.4) Definition. Let^{V} be a complex vector space. An inner product on^{V} is a mapping
(::) :^{V}^{}^{V} ^{!}^{C} such that

(i) (ff)^{}0 , (ff) = 0 if and only if f = 0, (fg) = (gf),
(ii) (f+gh) = (fh) + (gh) (fg) =(fg)

1

for all fgh in ^{V} and in ^{C}. A vector space equipped with such an inner product is
called an inner product space or, sometimes, a pre-Hilbert space.

Reversing the procedure used for 3-space we de ne the length of an element f of an inner product space by

(1:5) ^{k}f^{k}= (ff)^{1}^{=}^{2} :

Clearly

(1:6) ^{k}f^{k}^{}0 ^{k}f^{k}=^{j}^{j}^{k}f^{k} ^{k}f^{k}= 0 if and only if f = 0 :
This notion of length also has all the usual properties of length in 3-space.

(1.7) Theorem. Let ^{V} be an inner product space. Then
(i) (Cauchy-Schwarz inequality)

j(fg)^{j}^{}^{k}f^{k}^{k}g^{k}
(ii) (triangle inequality)

kf+g^{k}^{}^{k}f^{k}+^{k}g^{k}
(iii) (parallelogram law)

kf +g^{k}^{2}+^{k}f^{;}g^{k}^{2}= 2(^{k}f^{k}^{2}+^{k}g^{k}^{2})
(iv) (polarization identity)

(fg) = ^{1}^{4} ^{k}f +g^{k}^{2}^{;}^{k}f^{;}g^{k}^{2}+i^{k}f+ig^{k}^{2}^{;}i^{k}f^{;}ig^{k}^{2}^{}
for all fg in ^{V}.

Proof. (i) When g= 0 the inequality is obvious, so we can assumeg = 0. Thus, by (1.6), it is enough to show that

f g

kg^{k}

kf^{k}
2

or, equivalently, that

(^{}) ^{j}(fg)^{j}^{}^{k}f^{k} (^{k}g^{k}= 1) :
Now, when ^{k}g^{k}= 1,

0^{}^{k}f^{;}(fg)g^{k}^{2}=^{;}f^{;}(fg)gf ^{;}(fg)g^{}

=^{k}f^{k}^{2}+^{j}(fg)^{j}^{2}^{;}^{;}f(fg)g^{}^{;}^{;}(fg)gf^{}:
But

;f(fg)g^{}+^{;}(fg)gf^{}= 2^{j}(fg)^{j}^{2} :
Consequently

kf^{k}^{2}^{;}^{j}(fg)^{j}^{2} ^{}0 (^{k}g^{k}= 1)
establishing (^{}).

(ii) For any fg in ^{V},

(^{}) ^{k}f +g^{k}^{2} = (f+gf +g) =^{k}f^{k}^{2}+^{k}g^{k}^{2}+ (fg) + (gf):
Consequently, by the Cauchy-Schwarz inequality,

kf +g^{k}^{2} ^{}^{k}f^{k}^{2}+^{k}g^{k}^{2}+ 2^{j}(fg)^{j}^{}(^{k}f^{k}+^{k}g^{k})^{2}
establishing the triangle inequality.

(iii) Analogous to (^{}),

(^{}^{}^{}) ^{k}f^{;}g^{k}^{2} =^{k}f^{k}^{2}+^{k}g^{k}^{2}^{;}(fg)^{;}(gf):

After adding (^{}) and (^{}^{}^{}) we obtain the parallelogram law for an inner product space.

(iv) left as exercise.

Properties (1.6) and (1.7)(ii) show that an inner product space is always a normed space in the following sense.

3

(1.8) Definition. Let^{V} be a complex vector space. Then ^{V} is said to be a normed space
when there is a function ^{k}:^{k}:^{V} ^{!}^{R} such that

(i) ^{k}f^{k}^{}0 ^{k}f^{k}=^{j}^{j}^{k}f^{k} ^{k}f^{k}= 0 if and only if f = 0
(ii) ^{k}f+g^{k}^{}^{k}f^{k}+^{k}g^{k}

for all fg in ^{V} and in ^{C}. The function f ^{!} ^{k}f^{k} is called a norm and the value of ^{k}f^{k}
is said to be the norm of f.

Many important examples of inner product and normed spaces recur throughout Fourier analysis.

(1.9) Examples. (i) ^{R}^{n} with the usual Euclidean inner product
(

### x

### y

) =^{X}

^{n}

i^{=1}xiyi (

### x

### y

^{2}

^{R}

^{n}) (ii)

^{C}

^{n}with the standard inner product

(

### z

### w

) =^{X}

^{n}

i^{=1}ziwi (

### z

### w

^{2}

^{C}

^{n})

(iii) more generally, if I is any index set, let `^{2}(I) be the `sequences'

### z

=^{f}z

_{i}

^{g}

_{i}

^{2}

_{I}of all complex numbers for which

^{P}

_{i}

^{2}

_{I}

^{j}zi

^{j}

^{2}<

^{1}. Then, since

X

i^{2}I ziwi

X

i^{2}I

jzi^{j}^{2}

1=^{2}^{}^{X}
i^{2}I

jwi^{j}^{2}

1=^{2}

the function

(

### z

### w

) =^{X}

i^{2}I ziwi ^{;}

### z

### w

^{2}`

^{2}(I)

^{}

de
nes an inner product on `^{2}(I). The example of ^{C}^{n} is the special case when I =

f12:::n^{g}. We shall usually write`^{2}_{n} instead of ^{C}^{n} and`^{2} instead of `^{2}(^{Z}).

(iii) Replacing the exponent 2 by anyp, 1^{}p^{}^{1}, we obtain normed spaces. Denote by

`^{p}(I) the vector space of sequences

### z

=^{f}zi

^{g}i

^{2}I of complex numbers zi for which

X

i^{2}I

jzi^{j}p (p^{6}=^{1}) resp. sup_{i}

2I ^{j}zi^{j} (p=^{1})
4

are nite. Then under the respective norms

k

### z

^{k}`

^{p}=

X

i^{2}I

jzi^{j}p^{}^{1}=p

(p^{6}=^{1}) ^{k}

### z

^{k}`

^{1}= sup

_{i}

2I ^{j}zi^{j} (p=^{1})
the space `^{p}(I) is a normed space. The inequality

X

i^{2}I

jzi+wi^{j}p^{}^{1}=p

X

i^{2}I

jzi^{j}p^{}^{1}=p

+

X

i^{2}z

jwi^{j}p^{}^{1}=p

which establishes the triangle inequality

k

### z

+### w

^{k}`

^{p}

^{}

^{k}

### z

^{k}`

^{p}+

^{k}

### w

^{k}`

^{p}

for `^{p}(I) is usually known as Minkowski's inequality. Unless p = 2, `^{p}(I) is not an inner
product space, but there is a very useful weak substitute for the Cauchy-Schwarz inequality:

if 1=p+ 1=q = 1, then

(1:10) ^{}^{}^{}^{X}

i^{2}I ziwi

k

### z

^{k}`

^{pk}

### w

^{k}`

^{q}: This is the so-called Holder's inequality for sequence spaces.

Through the use of a norm function the standard de nitions for convergence of se- quences and in nite series of real numbers can be carried over verbatim to any normed space replacing the absolute value of a real number by the norm of an element in the normed space.

(1.11) Definition. Let ^{V} be a normed space. A sequence ^{f}f_{n}^{g} in ^{V} is said to be a
Cauchy sequence if to each " >0 there corresponds N so that

kfm^{;}fn^{k}< " (mn > N)

while^{f}f_{n}^{g} is said to be a convergent sequence and have limit lim_{n}^{!1}f_{n} =f,f ^{2}^{V}, if to
each " > 0 there corresponds N so that

kf_{n}^{;}f^{k}< " (n > N) :
5

Every convergent sequence is automatically a Cauchy sequence, but there exist normed
spaces containing Cauchy sequences which are not convergent sequences. For example,
denote by ^{L}^{p}(^{R}) the space of continuous compactly supported functions f = f(x) on ^{R}
with norm

kf^{k}_{p} =

Z

1

;1

jf(x)^{j}^{p}dx^{}^{1}^{=p} :

It is not hard to construct a sequence of continuous compactly supported functions ^{f}fn^{g}

such that

(i) ^{f}fn^{g} is a Cauchy sequence in ^{L}^{p}(^{R}),

(ii) f_{n} converges to the characteristic function _{=};

1

2^{1}^{2}^{)} of the interval ^{;}^{1}^{2}^{1}^{2}) in
the sense that

nlim^{!1}

Z

1

;1

jfn(x)^{;}_{(}x)^{j}^{p}dx

1=p

= 0 :

Since is not continuous, ^{L}^{p}(^{R}) thus contains Cauchy sequences that are not con-
vergent sequences. On the contrary, every Cauchy sequence of real numbers converges to
a real number | the completeness property of ^{R} | a fact which is fundamental to all of
analysis. The corresponding property for a normed space is equally fundamental.

(1.12) Definition. A normed spaced ^{X} is said to be a complete normed space or, fre-
quently, a Banach space when every Cauchy sequence in ^{X} is a convergent sequence.

A complete inner product space is usually called a Hilbert space (and denoted here by

H^{H}^{0}^{H}^{1}:::).

The completeness property of ^{R} ensures that every `^{p}(I)-space is a Banach space,
while `^{2}(I) is a Hilbert space. On the other hand, after replacing Riemann integrals by
Lebesgue integrals in our de
nition of ^{L}^{p}-spaces we obtain the corresponding family of
Banach spaces on ^{R}. Thus, let L^{p}(^{R}), (1 ^{} p < ^{1}) be the usual space of (equivalence
classes) of Lebesgue measurable functions f on ^{R} for which

(1:13)(i) ^{k}f^{k}L^{p} =

Z

1

;1

jf(x)^{j}^{p}dx

1=p

6

is
nite. ThenL^{p}(^{R}) is a Banach space containing^{L}^{p}(^{R}) as a dense subspace, i.e., to each
f in L^{p}(^{R}) and each " > 0 there corresponds g in ^{L}^{p}(^{R}) so that

(1:13)(ii) ^{k}f^{;}g^{k}L^{p} =

Z

1

;1

jf(x)^{;}g(x)^{j}^{p}dx

1=p

< " :

We shall refer to this property as the ^{L}^{p}(^{R})-density property of L^{p}(^{R}). Most crucially of
all, the Lebesgue L^{2}(^{R})-space is a Hilbert space with respect to the inner product

(1:14) (fg) =

Z

1

;1

f(x)g(x)dx ^{;}fg^{2}L^{2}(^{R})^{}

andL^{2}(^{R}) contains ^{L}^{2}(^{R}) as a dense subspace. The Cauchy-Schwarz inequality for L^{2}(^{R})
ensures that

(1:15)(i) ^{}^{}^{}^{Z} ^{1}

;1

f(x)g(x)dx^{}^{}^{}^{}^{Z} ^{1}

;1

jf(x)^{j}^{2}dx^{}^{1}^{=}^{2}^{Z} ^{1}

;1

jg(x)^{j}^{2}dx^{}^{1}^{=}^{2} :

The substitute for Lebesgue L^{p}(^{R})-spaces is an integral form of Holder's inequality: when
1=p+ 1=q= 1, the inequality

(1:15)(ii) ^{}^{}^{}

Z

1

;1

f(x)g(x)dx^{}^{}^{}^{}

Z

1

;1

jf(x)^{j}^{p}dx

1=p^{Z} ^{1}

;1

jg(x)^{j}^{q}dx

1=q

holds for all f in L^{p}(^{R}) and g in L^{q}(^{R}).

Two properties of Hilbert spaces that make essential use of the inner product structure and the completeness property are natural analogues of familiar geometric results in 3- space. Let

### a

= (a^{1}a

^{2}a

^{3})

### b

= (b^{1}b

^{2}b

^{3})

### c

= (c^{1}c

^{2}c

^{3})

be vectors in ^{R}^{3}. Then ^{f}

### a

### b

### c

^{g}is a linearly independent family, hence a basis for

^{R}

^{3}, if and only if

### a

:(### b

^{}

### c

)^{6}= 0 or, equivalently,

det

2

6

4

a^{1} a^{2} a^{3}
b^{1} b^{2} b^{3}
c^{1} c^{2} c^{3}

3

7

5 6= 0

geometrically,

### a

:(### b

^{}

### c

) is the volume of the parallelepiped having### a

### b

and### c

as adjacent edges. Cramer's rule says, essentially, that every### v

= (v^{1}v

^{2}v

^{3}) in

^{R}

^{3}can be written uniquely as

(1:16)

### v

=### a

+### b

+### c

7

with =

### v

:(### b

^{}

### c

)### a

:(### b

^{}

### c

) =### v

:(### c

^{}

### a

)### a

:(### b

^{}

### c

) =### v

:(### a

^{}

### b

)### a

:(### b

^{}

### c

)whenever

### a

:(### b

^{}

### c

)^{6}= 0. Now let us consider the special case when

### c

=### a

^{}

### b

so that### v

=### a

+### b

+### v

:(### a

^{}

### b

)k

### a

^{}

### b

^{k}

^{2}

(

### a

^{}

### b

) =### u

+### w

where

### u

lies in the plane containing### a

### b

and### w

is perpendicular to this plane. Geomet- rically,^{k}

### w

^{k}is the shortest distance from the point

### v

= (v^{1}v

^{2}v

^{3}) in

^{R}

^{3}to the plane containing

### a

### b

. (Where do these properties use the completeness property of^{R}?) To extend such geometric ideas to an arbitrary Hilbert space

^{H}, let

^{V}be a closed subspace of

H and f an element of ^{H} not lying in ^{V}, and set

d = inf^{fk}f ^{;}g^{k}^{H} :g^{2}^{V}^{g}
the value of d might well be called the distance fromf to ^{V}.

(1.17) Theorem. There exist unique elements gh in ^{H} such that
(i) f =g+h (ii) g^{2}^{V}

(iii) (hv) = 0 (v^{2}^{V}) (iv) ^{k}h^{k}=d :
Proof. To construct g set

Cn =^{n}v^{2}^{V} : ^{k}f^{;}v^{k}^{H} ^{}d+ 1n

o :

In view of the de
nition of d, each Cn is non-empty and Cn ^{} Cn^{+1}. We use the par-
allelogram law and the completeness property to show that ^{T}_{n}Cn contains precisely one
element | the element g.

For arbitraryuv in Cn

ku^{;}v^{k}^{2} =^{k}(u^{;}f) + (f ^{;}v)^{k}^{2}

= 2(^{k}u^{;}f^{k}^{2}+^{k}v^{;}f^{k}^{2})^{;}^{k}(u^{;}f)^{;}(f ^{;}v)^{k}^{2}

= 2(^{k}u^{;}f^{k}^{2}+^{k}v^{;}f^{k}^{2})^{;}4^{k}^{1}^{2}(u+v)^{;}f^{k}^{2}
by the parallelogram law. But

ku^{;}f^{k}^{2}+^{k}v^{;}f^{k}^{2} ^{}2^{}d+ 1n

2 (uv ^{2}C_{n})
8

while

kf ^{;} ^{1}^{2}(u+v)^{k}^{}d :
Consequently,

ku^{;}v^{k}^{2} ^{} n^{8} ^{+ 4}n^{2} ^{(}uv ^{2}Cn)
and so

(^{}) _{n}lim

!1

diameter (C_{n}) = 0 :

Now choose any g_{n} in C_{n}. Since C_{n} ^{} C_{n}^{+1} ^{} ^{}^{}^{}, property (^{}) ensures that ^{f}g_{n}^{g} is a
Cauchy sequence, which must therefore converge to some g in ^{V}. On the other hand, if

fg^{0}_{n}^{g} is any other choice of sequence, property (^{}) ensures also that

nlim^{!1}^{k}gn^{;}g_{n}^{0}^{k}= 0:
Hence ^{T}_{n}Cn =^{f}g^{g}.

Set h=f ^{;}g. Then ^{k}h^{k}=d, since

d^{}^{k}f ^{;}g^{k}^{}^{k}f ^{;}gn^{k}+^{k}g^{;}gn^{k}^{}d+ 1n ^{+}^{k}g^{;}gn^{k}^{!}d

establishing properties (i), (ii) and (iv). To establish (iii), it is sucient to show that
(^{}) (hv) = 0 (v^{2}^{V} ^{k}v^{k}= 1) :

But, for any v in ^{V} with^{k}v^{k}= 1,

d^{2} ^{}^{k}f^{;}^{;}g+ (hv)v^{}^{k}^{2} =^{k}h^{;}(hv)v^{k}^{2}

=^{k}h^{k}^{2}^{;}^{j}(hv)^{j}^{2}^{}d^{2}^{;}^{j}(hv)^{j}^{2}

by the same argument as we used in the proof of (1.7)(i). Hence (hv) = 0, establishing
(^{}).

To complete the proof, therefore, we have only to show that gand h are unique. But
if f =g^{0}+h^{0} is a second decomposition of f satisfying (ii) and (iv), then g^{0} ^{2}^{T}_{n}Cn and
so g=g^{0}. It follows automatically that h =h^{0}, completing the proof.

Just as non-zero vectors

### u

### v

in 3-space are orthogonal if and only if### u

:### v

= 0, so we say that non-zero elements fg in an inner product space are orthogonal when (fg) = 0.With this terminology we obtain a very useful corollary to the previous result.

9

(1.18) Corollary. Each closed subspace ^{V} of a Hilbert space ^{H} admits an orthogonal
complement

V

? = u^{2}^{H}: (uv) = 0 v^{2}^{V}^{}

in the sense that ^{H}=^{V}^{?}^{
}^{V}, i.e., every f in ^{H} can be written uniquely in the form
f =u+v (u ^{2}^{V}^{?} v^{2}^{V}) :

We turn now to the theory associated with particular families of elements in a Hilbert space, the rst of which is the natural abstraction of the vectors

### i

= (100)### j

= (010)### k

= (001)in^{R}^{3}. Geometrically, these are mutually orthogonal vectors in^{R}^{3} having unit length since
they have the properties

k

### i

^{k}=

^{k}

### j

^{k}=

^{k}

### k

^{k}= 1

### i

:### j

=### j

:### k

=### k

:### i

= 0 with respect to the inner product (1.2) on 3-space.(1.19) Definition. A subset S =^{f}

### v

^{1}

### v

^{2}:::

^{g}of an inner product space

^{V}is said to be an orthonormal family when

k

### v

j^{k}= 1 (

### v

j### v

k) = 0 (j^{6}=k) i.e., S consists of mutually orthogonal elements having length 1.

For example, the set ^{f}

### e

i: i^{2}I

^{g}where

### e

i = (00:::010:::)is the sequence having 0 entries except at the i^{th} place, is orthonormal in `^{2}(I). As fre-
quently happens, however, a subset S of an inner product space ^{V} can consist of linearly
independent elements which are not mutually orthogonal. In such a case there are sev-
eral ways of constructing an orthonormal family which retains a particular geometric or
algebraic property that S has. One method is very familiar.

10

LetS =^{f}

### a

### b

### c

^{g}be the linearly independent set of vectors in 3-space in (1.16). Then

### a

and### b

are not parallel, so there is a unique plane in which### a

### b

lie, and the vector### c

cannot lie in the plane (why?). Now normalize

### a

by setting =### a

=^{k}

### a

^{k}, and de ne by =

^{;}

### b

^{;}(

### b

:)^{}

^{.}

^{k}

### b

^{;}(

### b

:)^{k}:

Then is a unit vector in the plane containing

### a

### b

, and is orthogonal to since :=^{;}

### b

:^{;}(

### b

:)^{k}

^{k}

^{2}

^{}

^{.}

^{k}

### b

^{;}(

### b

:)^{k}= 0 :

Finally, to construct a unit vector perpendicular to the plane containing

### a

### b

, and hence to the plane containing , we can take for the normalized projection of### c

onto the vector normal to this plane. As### c

cannot lie in this plane, its projection onto the normal must be non-zero. In detail: =^{;}

### c

^{;}(

### c

:)^{;}(

### c

:)^{}

^{.}

^{k}

### c

^{;}(

### c

:)^{;}(

### c

:)^{k}:

An explicit computation shows that : = : = 0. Hence ^{f}^{g} is an orthonormal
family in 3-space. This construction carries over almost immediately to any inner product
space it is known as the Gram-Schmidt orthogonalization process.

(1.20) Theorem. (Gram-Schmidt process). If ^{f}^{1}^{2}:::^{g} is a possibly innite set of
linearly independent elements in an inner product space ^{V}, then

^{1} =^{1}=^{k}^{1}^{k}
k =

k^{;}k^{X}^{;1}

j^{=1}(k
j)
j

k^{;}k^{X}^{;1}

j^{=1}(k
j)
j

(k >1)
denes a family S =^{f}
^{1}
^{2}:::^{g} which is orthonormal in ^{V}.

Let us now return to the case of a general orthonormal familyS =^{f}
^{1}
^{2}:::^{g} in an
inner product space ^{V} having inner product (::). To each f in ^{V} there corresponds an
orthonormal series

Sf] =^{X}

k (f
_{k})
_{k} (
^{2}^{V})
11

and the n^{th} partial sum of this series is given by
S_{n}f] =^{X}^{n}

k^{=1}(f
_{k})
_{k} (
^{2}^{V}) :

The coecients (f
k) of such a partial sum have a very special signi
cance in approx-
imation theory indicating why they and the associated orthonormal series will play such
an important role in the future. Let a^{1}:::an be arbitrary scalars. Then, because of the
orthonormality of the
k,

(1:21)

f^{;}^{X}^{n}

k^{=1}ak
k

2 =

f ^{;}^{X}^{n}

j^{=1}aj
j f ^{;}^{X}^{n}

k^{=1}ak
k

=^{k}f^{k}^{2}^{;}^{X}^{n}

j^{=1}aj(
jf)^{;}^{X}^{n}

k^{=1}ak(f
k) +^{X}^{n}

k^{=1}

jak^{j}^{2}

=^{k}f^{k}^{2}^{;}^{X}^{n}

k^{=1}

j(f
k)^{j}^{2}+^{X}^{n}

k^{=1}

jak^{;}(f
k)^{j}^{2}
using the symmetry and linearity properties of the inner product on ^{V}. Thus

(1:22) ^{}^{}^{}f ^{;}^{X}^{n}

k^{=1}a_{k}
_{k}^{}^{}^{}

is minimized by ^{P}^{n}_{k}^{=1}^{j}(f
_{k})^{j}^{2} when a_{k} = (f
_{k}) in fact, any other choice of the a_{k}
would give a strictly larger value to (1.22). Hence the n^{th} partial sum ^{P}^{n}_{k}^{=1}(f
k)
k

provides the best approximation tof by a linear combination of
^{1}:::
n, distance being
measured in terms of the length ^{k}:^{k}. In technical terms, the n^{th} partial sum Snf] is said
to provide the least squares approximation to f by linear combinations of
^{1}:::
_{n}.

(1.23) Theorem. Let ^{f}
^{1}
^{2}:::^{g} be an orthonormal family in an inner product space

V. Then

(i) (Bessel's equality)

f ^{;}^{X}^{n}

k^{=1}(f
_{k})
_{k}^{}^{}^{}^{2} =^{k}f^{k}^{2}^{;}^{X}^{n}

k^{=1}

j(f
_{k})^{j}^{2}
12

(ii) (Bessel's inequality)

^{X}n
k^{=1}

j(f
k)^{j}^{2}

1=^{2}

kf^{k}
hold for each n

Bessel's equality follows from (1.21), and Bessel's inequality then follows immediately
from his equality since^{k}:^{k}is always non-negative. This leads us to another crucial point in
our development of Hilbert spaces. Suppose^{V} has
nite dimension, say dimensionn. Then

V contains a linearly independent set ^{f}
^{1}:::
n^{g} which because of (1.20) can be taken
to be orthonormal, and by general vector space theory each f has a unique expansion

f =a^{1}f^{1}+a^{2}f^{2}+^{}^{}^{}+a_{n}f_{n} :
In this case, the orthonormality of the
k ensures that

(f
_{k}) =^{X}^{n}

j^{=1}a_{j}(f
_{k}) =a_{k} (1^{}k ^{}n) :
Hence

(1:24)(i) f = (f
^{1})
^{1}+ (f
^{2})
^{2}+^{}^{}^{}+ (f
n)
n
and so by Bessel's equality,

(1:24)(ii)

^{X}n
k^{=1}

j(f
k)^{j}^{2}

1=^{2}

=^{k}f^{k} (f ^{2}^{V})

in particular, ^{k}f^{k} coincides with the length in `^{2}_{n} of the n-tuple ((f
^{1}):::(f
n)) of
coecients of f. Conversely, to each

### a

= (a^{1}:::a

_{n}) in `

^{2}

_{n}there corresponds an element (1:25)(i) f =a

^{1}

^{1}+a

^{2}

^{2}+

^{}

^{}

^{}+an n

such that

(1:25)(ii) ak = (f
k) ^{k}f^{k}=^{k}

### a

^{k}=

^{X}n
k^{=1}

jak^{j}^{2}

1=^{2}

:

Thus, through (1.24) and (1.25), f ^{$}

### a

de nes a length-preserving isomorphism betweenV and `^{2}_{n} consequently, when ^{V} has dimension n, the orthonormal family ^{f}
^{1}:::
_{n}^{g}
behaves in every respect like the standard family ^{f}

### i

### j

### k

^{g}in 3-space. Now suppose

^{V}has in nite dimension and let

^{f}

^{1}

^{2}:::

^{g}be an in nite orthonormal family in

^{V}. Bessel's inequality still holds for every n consequently, after letting n

^{!}

^{1}we obtain the in nite version of (1.23)(ii).

13

(1.26) Corollary. (Bessel's inequality) If^{f}
^{1}
^{2}:::^{g}is an innite orthonormal family,

then ^{}

1

X

k^{=1}

j(f
k)^{j}^{2}

1=^{2}

kf^{k}

is valid for all f in ^{V}
in particular, the sequence ^{f}(f
k)^{g}^{1}_{k}^{=1} of coecients is always in

`^{2}.

But, conversely, given any sequence

### a

=^{f}a

_{k}

^{g}in`

^{2}, what can be said about theinnite series

^{P}

^{1}

_{k}

^{=1}ak k? Can sense be made of the convergence of it? If so, is the sum an element of

^{V}? As with theorem (1.17), the completeness of the inner product space is essential in answering these questions. Suppose rst that

^{f}ak

^{g}

^{1}k

^{=1}is an arbitrary element of `

^{2}, and set gn=

^{P}

^{n}

_{k}

^{=1}ak k. Then to each " >0 there corresponds an integer N such that

jam^{+1}^{j}^{2}+^{}^{}^{}+^{j}an^{j}^{2} = ^{X}^{n}

k^{=}m^{+1}

jak^{j}^{2} < "

for all mn > N consequently, by Bessel's inequality,

kg_{n}^{;}g_{m}^{k}^{2} =^{}^{}^{} ^{X}^{n}

k^{=}m^{+1}a_{k}
_{k}^{}^{}^{}^{2} = ^{X}^{n}

k^{=}m^{+1}

ja_{k}^{j}^{2} < "

for all mn > N. Thus ^{f}gn^{g} is a Cauchy sequence in ^{V}. If ^{V} is assumed to be complete,
i.e., a Hilbert space rather than just an inner product space, this Cauchy sequence ^{f}gn^{g}

will converge to some g in ^{V}, i.e.,

g= lim_{n}

!1

n

X

k^{=1}ak
k
furthermore,

(g
j) = lim_{n}

!1

^{X}n

k^{=1}ak
k
j

=aj :

Consequently, if^{f}
k^{g}is an orthonormal family in a Hilbert space, then^{P}_{k}ak
k converges
to some g such that

g=^{X}

k ak
k =^{X}

k (g k) k

for each element ^{f}a_{k}^{g} in `^{2}. The crucial question now is whether every element of the
Hilbert space can be written as ^{P}_{k}a_{k}
_{k} for some ^{f}a_{k}^{g} in `^{2}.

14

(1.27) Definition. An innite family ^{f}
^{1}
^{2}:::^{g} in a (separable) Hilbert space ^{H} is
said to be a complete orthonormal family if for each f in ^{H}

nlim^{!1}

f^{;}^{X}^{n}

k^{=1}(f
k)
k

= 0 :

In such a case the orthonormal series ^{P}^{1}_{k}^{=1}(f
k)
k is said to converge in mean to f.
The most basic consequence of the existence of such complete orthonormal families is
contained in the next result.

(1.28) Theorem. For an innite family^{f}
^{1}
^{2}:::^{g}in a (separable) Hilbert space^{H} the
following assertions are equivalent:

(i) ^{f}
^{1}
^{2}:::^{g} is a complete orthonormal family in ^{H},
(ii) (Plancherel) ^{k}f^{k}^{2} =^{X}^{1}

k^{=1}

j(f
k)^{j}^{2} (f ^{2}^{H}) ,
(iii) (Parseval) (fg) = ^{X}^{1}

k^{=1}(f
k)(g
k) (fg ^{2}^{H})
(iv) if (f
k) = 0 for all k= 12:::, then f =

### 0

.Proof. (i) ^{)} (ii) In view of Bessel's inequality, the complete orthonormal family prop-
erty ensures that

nlim^{!1}

kf^{k}^{2}^{;}^{X}^{n}

k^{=1}

j(f
_{k})^{j}^{2}

= lim_{n}

!1

f ^{;}^{X}^{n}

k^{=1}(f
_{k})
_{k}^{}^{}^{}^{2} = 0 :
Thus the equality

kf^{k}^{2} =^{X}^{1}

k^{=1}

j(f
_{k})^{j}^{2}
holds for each f ^{2}^{H}.

(ii) ^{)} (iii) We apply the polarization identity (1.7)(iv). For then by Plancherel's
theorem,

(fg) = ^{1}^{4} ^{k}f+g^{k}^{2}^{;}^{k}f ^{;}g^{k}^{2}+i^{k}f +ig^{k}^{2}^{;}i^{k}f ^{;}ig^{k}^{2}^{}

= ^{1}^{4}^{X}

k

j(f+g
k)^{j}^{2}^{;}^{j}(f^{;}g
k)^{j}^{2}+i^{j}(f +ig
k)^{j}^{2}^{;}i(f^{;}ig
k)^{j}^{2}^{}

= ^{1}^{4}^{X}

k

j(f
k) + (g
k)^{j}^{2}^{;}^{j}(f
k)^{;}(g
k)^{j}^{2}+i^{j}(f
k) +i(g
k)^{j}^{2}^{;}i^{j}(f
k)^{;}i(g
k)^{j}^{2}^{}

=^{X}

k (f
_{k})(g
_{k})

15

applying the polarization identity to the Hilbert space ^{C} also.

(iii) ^{)} (iv) If (f
k) = 0 for all k = 12:::, then
(ff) =^{X}

k

j(f
k)^{j}^{2} = 0
i.e., ^{k}f^{k}= 0. SoF = 0.

(iv)^{)}(i) Letgbe an arbitrary element of^{H}. Then by (1.26) the sequence^{f}g
k)^{g}k

of coecients is in `^{2}, and so by the discussion prior to de
nition (1.27), the series

Pk(g
_{k})
_{k} converges to some h in ^{H} such that (h
_{k}) = (g
_{k}). Now let's apply
condition (iv) to f =g^{;}h thus g=h, i.e., g=^{P}^{1}_{k}^{=1}(g
_{k})
_{k}. But then

0 = lim_{n}

!1

kh^{;}gn^{k}= lim_{h}

!1

kg^{;}gn^{k}

showing that ^{f}
^{1}
^{2}:::^{g} is a complete orthonormal family in ^{H}.

Consequently, if ^{f}
_{i}^{g}_{i}^{2}_{I} is a complete orthonormal family in a Hilbert space ^{H},
then every f has an orthonormal series decomposition f = ^{P}_{i}(f
i)
i such that ^{k}f^{k} =

Pi^{j}(f
i)^{j}^{2}, and this is the only possible decomposition off as a sum of the
i. But in de-
veloping the theory of wavelets one often starts with a family that is not orthonormal. Now
of course, the Gram-Schmidt process could be applied to produce an orthonormal family,
but this destroys the time-scale properties that are basic to wavelet decompositions. So
the notion of orthonormality has to be weakened while still preserving the unique decom-
position property. In the next section we shall weaken the notion still further, eliminating
even the unique decomposition requirement.

(1.29) Definition. A family^{f}
i^{g}i^{2}I in a Hilbert space ^{H}is said to be stable when there
exist positive constants AB so that the inequalities

A

X

i

ji^{j}^{2}

1=^{2}

X

i i i

H

B

X

i

ji^{j}^{2}

1=^{2}

hold for all = ^{f}i^{g} in `^{2}(I). The constants AB will be called ^{stability} ^{constants}.
Since ^{k}x^{k}^{H} = ^{j}^{j}^{k}x^{k}^{H}, ^{2} ^{C}, the stable property is equivalent to the existence of
constants AB so that

(2:19)^{0} 0< A^{}^{}^{}^{}^{X}

i i i

H

B ^{}^{k}^{k}= 1^{}
16

for all norm one =^{f}_{i}^{g} in `^{2}(I), i.e., all in the unit sphere of `^{2}(I).

Important examples of stable families will arise in the theory of wavelets, but there are examples of a more general nature.

(1.30) Examples. (i) Every orthonormal family is stable with stability constants A = B= 1.

(ii) Every
nite linearly independent set is stable. Indeed, for any
nite set
^{1}:::
_{n}
in a Hilbert space ^{H}, de
ne f :^{C}^{n} ^{!}0^{1}) by

f() =^{k}^{1}
^{1} +^{}^{}^{}+n
n^{k} (^{2}^{C}^{n}) :

Then f is continuous and sof is bounded above and below on the unit sphere in ^{C}^{n}, i.e.

0^{}A^{}^{}^{}^{}^{X}^{n}

i^{=1}i
i

B (^{k}^{k}= 1)
where

A= min

k^{k=1}f() B= max

k^{k=1}f() :

But, if ^{f}
^{1}:::
n^{g} is linearly independent, f() vanishes only at = 0, ensuring that
A >0. Hence every
nite set of linearly independent elements in a Hilbert space is a stable
family.

(iii) Not every in
nite set is stable but there are in
nite non-orthonormal stable fam-
ilies. For if ^{f}
^{1}
^{2}:::^{g} is orthonormal, de
ne ^{f}^{1}^{2}:::^{g}by

^{1} = 1^{p}2(
^{1}+
^{2}) ^{2} = 1^{p}2(
^{2}+
^{3})

^{3} = 1^{p}2(
^{4}+
^{5}) ^{4} = 1^{p}2(
^{5}+
^{6})

... :

Then

kj^{k}= 1 (^{2}j^{;1}^{2}j) = ^{1}^{2} (all j)
while (jk) = 0 for all other choices ofjk. Thus

1

X

i^{=1}ii

2 =^{X}^{1}

i^{=1}

ji^{j}^{2} + ^{1}^{2}^{X}^{1}

i^{=1}(^{2}i^{;1}^{2}i+ ^{2}i^{;1}^{2}i)
17