The aspect of Euclidean geometry that sets it apart from geometries which share most of its other features is perpendicularity and its consequences. To set up a notion of perpendicularity in settings other than the familiar Euclidean plane or three dimensional space is to try to copy the Euclidean properties that go with it.
Perpendicularity becomes operationally useful, especially for applications, when it’s linked to measure-ment, i.e., to length. This link is the Pythagorean theorem. 12 Perpendicularity becomes austere when mathematicians start referring to it as orthogonality, but that’s what I’m used to and it’s another term you can throw around to impress your friends.
Vectors To fix ideas, I want to remind you briefly of vectors and geometry in Euclidean space. We write vectors in Rnas n-tuples of real numbers:
v = (v1, v2, . . . , vn) The vi are called the components of v. The length, or norm of v is
kvk = (v12+ v22+ · · · + vn2)1/2.
12How do you lay out a big rectangular field of specified dimensions? You use the Pythagorean theorem. I had an encounter with this a few summers ago when I volunteered to help lay out soccer fields. I was only asked to assist, because evidently I could not be trusted with the details. Put two stakes in the ground to determine one side of the field. That’s one leg of what is to become a right triangle — half the field. I hooked a tape measure on one stake and walked off in a direction generally perpendicular to the first leg, stopping when I had gone the regulation distance for that side of the field, or when I needed rest. The chief of the crew hooked another tape measure on the other stake and walked approximately along the diagonal of the field — the hypotenuse. We adjusted our positions — but not the length we had walked off — to meet up, so that the Pythagorean theorem was satisfied; he had a chart showing what this distance should be. Hence at our meeting point the leg I determined must be perpendicular to the first leg we laid out. This was my first practical use of the Pythagorean theorem, and so began my transition from a pure mathematician to an engineer.
1.9 Orthogonality 27
The distance between two vectors v and w is kv − wk.
How does the Pythagorean theorem look in terms of vectors? Let’s just work in R2. Let u = (u1, u2), v = (v1, v2), and w = u + v = (u1+ v1, u2+ v2). If u, v, and w form a right triangle with w the hypotenuse, then
kwk2 = ku + vk2= kuk2+ kvk2
(u1+ v1)2+ (u2+ v2)2 = (u21+ u22) + (v12+ v22) (u21+ 2u1v1+ v21) + (u22+ 2u2v2+ v22) = u21+ u22+ v12+ v22 The squared terms cancel and we conclude that
u1v1+ u2v2= 0
is a necessary and sufficient condition for u and v to be perpendicular.
And so we introduce the (algebraic) definition of the inner product or dot product of two vectors. We give this in Rn:
If v = (v1, v2, . . . , vn) and w = (w1, w2, . . . , wn) then the inner product is v · w = v1w1+ v2w2+ · · · + vnwn
Other notations for the inner product are (v, w) (just parentheses; we’ll be using this notation) and hv, wi (angle brackets for those who think parentheses are not fancy enough; the use of angle brackets is especially common in physics where it’s also used to denote more general pairings of vectors that produce real or complex numbers.)
Notice that
(v, v) = v21+ v22+ · · · + vn2 = kvk2. Thus
kvk = (v, v)1/2.
There is also a geometric approach to the inner product, which leads to the formula (v, w) = ||v|| ||w|| cos θ
where θ is the angle between v and w. This is sometimes taken as an alternate definition of the inner product, though we’ll stick with the algebraic definition. For a few comments on this see Section 1.10.
We see that (v, w) = 0 if and only if v and w are orthogonal. This was the point, after all, and it is a truly helpful result, especially because it’s so easy to verify when the vectors are given in coordinates. The inner product does more than identify orthogonal vectors, however. When it’s nonzero it tells you how much of one vector is in the direction of another. That is, the vector
(v, w)
||w||
w
||w|| also written as (v, w) (w, w)w ,
is the projection of v onto the unit vector w/||w||, or, if you prefer, (v, w)/||w|| is the (scalar) component of v in the direction of w. I think of the inner product as measuring how much one vector “knows” another;
two orthogonal vectors don’t know each other.
Finally, I want to list the main algebraic properties of the inner product. I won’t give the proofs; they are straightforward verifications. We’ll see these properties again — modified slightly to allow for complex numbers — a little later.
1. (v, v) ≥ 0 and (v, v) = 0 if and only if v = 0 (positive definiteness) 2. (v, w) = (w, v) (symmetry)
3. (αv, w) = α(v, w) for any scalar α (homogeneity) 4. (v + w, u) = (v, u) + (w, u) (additivity)
In fact, these are exactly the properties that ordinary multiplication has.
Orthonormal basis The natural basis for Rnare the vectors of length 1 in the n “coordinate directions”:
e1 = (1, 0, . . . , 0) , e2 = (0, 1, . . . , 0) , . . . , en= (0, 0, . . . , 1).
These vectors are called the “natural” basis because a vector v = (v1, v2, . . . , vn) is expressed “naturally”
in terms of its components as
v = v1e1+ v2e2+ · · · + vnen.
One says that the natural basis e1, e2, . . . , en are an orthonormal basis for Rn, meaning (ei, ej) = δij,
where δij is the Kronecker delta defined by
δij =
(1 i = j 0 i 6= j Notice that
(v, ek) = vk, and hence that
v = Xn k=1
(v, ek)ek. In words:
When v is decomposed as a sum of vectors in the directions of the orthonormal basis vectors, the components are given by the inner product of v with the basis vectors.
Since the ek have length 1, the inner products (v, ek) are the projections of v onto the basis vectors.13
13Put that the other way I like so much, the inner product (v, ek) is how much v and ek know each other.
1.9 Orthogonality 29
Functions All of what we’ve just done can be carried over to L2([0, 1]), including the same motivation for orthogonality and defining the inner product. When are two functions “perpendicular”? Answer: when the Pythagorean theorem is satisfied. Thus if we are to have
kf + gk2= kf k2+ kgk2 then
Z 1 0
(f (t) + g(t))2dt = Z 1
0
f (t)2dt + Z 1
0
g(t)2dt Z 1
0
(f (t)2+ 2f (t)g(t) + g(t)2) dt = Z 1
0
f (t)2dt + Z 1
0
g(t)2dt Z 1
0
f (t)2dt + 2 Z 1
0
f (t)g(t) dt + Z 1
0
g(t)2dt = Z 1
0
f (t)2dt + Z 1
0
g(t)2dt
If you buy the premise, you have to buy the conclusion — we conclude that the condition to adopt to define when two functions are perpendicular (or as we’ll now say, orthogonal) is
Z 1 0
f (t)g(t) dt = 0 .
So we define the inner product of two functions in L2([0, 1]) to be.
(f, g) = Z 1
0
f (t)g(t) dt .
(See Section 1.10 for a discussion of why f (t)g(t) is integrable if f (t) and g(t) are each square integrable.) This inner product has all of the algebraic properties of the dot product of vectors. We list them, again.
1. (f, f ) ≥ 0 and (f, f ) = 0 if and only if f = 0.
2. (f, g) = (g, f )
3. (f + g, h) = (f, h) + (g, h) 4. (αf, g) = α(f, g)
In particular, we have
(f, f ) = Z 1
0
f (t)2dt = kf k2.
Now, let me relieve you of a burden that you may feel you must carry. There is no reason on earth why you should have any pictorial intuition for the inner product of two functions, and for when two functions are orthogonal. How can you picture the condition (f, g) = 0? In terms of the graphs of f and g? I don’t think so. And if (f, g) is not zero, how are you to picture how much f and g know each other? Don’t be silly.
We’re working by analogy here. It’s a very strong analogy, but that’s not to say that the two settings — functions and geometric vectors — are identical. They aren’t. As I have said before, what you should do is draw pictures in R2 and R3, see, somehow, what algebraic or geometric idea may be called for, and using the same words make the attempt to carry that over to L2([0, 1]). It’s surprising how often and how well this works.
There’s a catch There’s always a catch. In the preceding discussion we’ve been working with the real vector space Rn, as motivation, and with real-valued functions in L2([0, 1]). But, of course, the definition of the Fourier coefficients involves complex functions in the form of the complex exponential, and the Fourier series is a sum of complex terms. We could avoid this catch by writing everything in terms of sine and cosine, a procedure you may have followed in an earlier course. However, we don’t want to sacrifice the algebraic dexterity we can show by working with the complex form of the Fourier sums, and a more effective and encompassing choice is to consider complex-valued square integrable functions and the complex inner product.
Here are the definitions. For the definition of L2([0, 1]) we assume again that Z 1
0
|f (t)|2dt < ∞ .
The definition looks the same as before, but |f (t)|2 is now the magnitude of the (possibly) complex number f (t).
The inner product of complex-valued functions f (t) and g(t) in L2([0, 1]) is defined to be (f, g) =
Z 1 0
f (t)g(t) dt .
The complex conjugate in the second slot causes a few changes in the algebraic properties. To wit:
1. (f, g) = (g, f ) (Hermitian symmetry)
2. (f, f ) ≥ 0 and (f, f ) = 0 if and only if f = 0 (positive definiteness — same as before)
3. (αf, g) = α(f, g), (f, αg) = α(f, g) (homogeneity — same as before in the first slot, conjugate scalar comes out if it’s in the second slot)
4. (f + g, h) = (f, h) + (g, h), (f, g + h) = (f, g) + (f, h) (additivity — same as before, no difference between additivity in first or second slot)
I’ll say more about the reason for the definition in Appendix 2. As before, (f, f ) =
Z 1 0
f (t)f (t) dt = Z 1
0
|f (t)|2dt = kf k2.
From now on, when we talk about L2([0, 1]) and the inner product on L2([0, 1]) we will always assume the complex inner product. If the functions happen to be real-valued then this reduces to the earlier definition.
The complex exponentials are an orthonormal basis Number two in our list of the greatest hits of the theory of Fourier series says that the complex exponentials form a basis for L2([0, 1]). This is not a trivial statement. In many ways it’s the whole ball game, for in establishing this fact one sees why L2([0, 1]) is the natural space to work with, and why convergence in L2([0, 1]) is the right thing to ask for in asking for the convergence of the partial sums of Fourier series.14 But it’s too much for us to do.
Instead, we’ll be content with the news that, just like the natural basis of Rn, the complex exponentials are orthonormal. Here’s the calculation; in fact, it’s the same calculation we did when we first solved for the Fourier coefficients. Write
en(t) = e2πint.
14An important point in this development is understanding what happens to the usual kind of pointwise convergence vis `a vis L2([0, 1]) convergence when the functions are smooth enough.
1.9 Orthogonality 31
The inner product of two of them, en(t) and em(t), when n 6= m is (en, em) =
Z 1 0
e2πinte2πimtdt = Z 1
0
e2πinte−2πimtdt = Z 1
0
e2πi(n−m)tdt
= 1
2πi(n − m)e2πi(n−m)t i1
0 = 1
2πi(n − m) e2πi(n−m)− e0
= 1
2πi(n − m)(1 − 1) = 0 . They are orthogonal. And when n = m
(en, en) = Z 1
0
e2πinte2πintdt = Z 1
0
e2πinte−2πintdt = Z 1
0
e2πi(n−n)tdt = Z 1
0
1 dt = 1 . Therefore the functions en(t) are orthonormal :
(en, em) = δnm =
(1 n = m 0 n 6= m
What is the component of a function f (t) “in the direction” en(t)? By analogy to the Euclidean case, it is given by the inner product
(f, en) = Z 1
0
f (t)en(t) dt = Z 1
0
f (t)e−2πintdt ,
precisely the n-th Fourier coefficient ˆf (n). (Note that en really does have to be in the second slot here.) Thus writing the Fourier series
f = X∞ n=−∞
f (n)eˆ 2πint,
as we did earlier, is exactly like the decomposition in terms of an orthonormal basis and associated inner product:
f = X∞ n=−∞
(f, en)en.
What we haven’t done is to show that this really works — that the complex exponentials are a basis as well as being orthonormal. We would be required to show that
lim
N →∞
f −
XN n=−N
(f, en)en
= 0 .
We’re not going to do that. It’s hard.
What if the period isn’t 1? Remember how we modified the Fourier series when the period is T rather than 1. We were led to the expansion
f (t) = X∞ n=−∞
cne2πint/T.
where
cn= 1 T
Z T 0
e−2πint/Tf (t) dt .
The whole setup we’ve just been through can be easily modified to cover this case. We work in the space L2([0, T ]) of square integrable functions on the interval [0, T ]. The (complex) inner product is
(f, g) = Z T
0
f (t)g(t) dt .
What happens with the T -periodic complex exponentials e2πint/T? If n 6= m then, much as before, (e2πint/T, e2πimt/T) =
Z T 0
e2πint/Te2πimt/T dt = Z T
0
e2πint/Te−2πimt/Tdt
= Z T
0
e2πi(n−m)t/Tdt = 1
2πi(n − m)/T e2πi(n−m)t/TiT 0
= 1
2πi(n − m)/T (e2πi(n−m)− e0) = 1
2πi(n − m)/T (1 − 1) = 0 And when n = m:
(e2πint/T, e2πint/T) = Z T
0
e2πint/Te2πint/Tdt
= Z T
0
e2πint/Te−2πint/T dt = Z T
0
1 dt = T .
Aha — it’s not 1, it’s T . The complex exponentials with period T are orthogonal but not orthonormal.
To get the latter property we scale the complex exponentials to en(t) = 1
√
Te2πint/T, for then
(en, em) =
(1 n = m 0 n 6= m This is where the factor 1/
√
T comes from, the factor mentioned earlier in this chapter. The inner product of f with en is
(f, en) = 1
√ T
Z T 0
f (t)e−2πint/Tdt . Then
X∞ n=−∞
(f, en)en= X∞ n=−∞
1
√ T
Z T 0
f (s)e−2πins/Tds
1
√
Te2πint/T = X∞ n=−∞
cne2πint/T,
where
cn= 1 T
Z T 0
e−2πint/Tf (t) dt , as above. We’re back to our earlier formula.
Rayleigh’s identity As a last application of these ideas, let’s derive Rayleigh’s identity, which states
that Z 1
0
|f (t)|2dt = X∞ n=−∞
| ˆf (n)|2.