大學線性代數初步

大學線性代數初步

大學數學

數學 大 學 線性代數 , 大

數 .

大 學 線性代數學 ,

學 線性代數 . 大學線性代數

. 學 大 大 ( ) 線性

代數 , . 大學 線性代數 , 大

學 數學 . 大 大 數學

, 學 , 線性代數

. , 線性代數

數學 . , .

, .

學 , 數學 學 , 線性

代數 . 線性代數 .

, . 學

. , (Question).

, 大

. , 線性代數 . ,

學 線性代數 學 線性代數 , .

, ,

代. , .

, . , 性

, . , .

v

Chapter 7

Inner Product Space

R^m dot product 性 , vector space. vector

space, dot product inner product . vector space V

v, w∈ V ⟨v,w⟩ ∈ R Proposition 1.4.2 性 ,

(1) ⟨v,w⟩ = ⟨w,v⟩, ∀v,w ∈ V.

(2) ⟨v,v⟩ ≥ 0 ⟨v,v⟩ = 0 v = O.

(3) r∈ R ⟨rv,v⟩ = ⟨v,rw⟩ = r⟨v,w⟩.

(4) ⟨u,v + w⟩ = ⟨u,v⟩ + ⟨u,w⟩, ∀u,v,w ∈ V.

V inner product space. inner product space 性 .

性 inner product 性 , finite dimensional

inner product space 性 . 大 inner product

, 大 R^m . ,

, v, w∈ R^m, v· w v, w ,

, ⟨v,w⟩ v, w . , v, w column vectors

, ⟨v,w⟩ = v^Tw.

7.1. Projection

R² linear transformation projection. ,

w∈ R², v∈ R² w Proj_w(v) w (

Span(w)), v− Proj_w(v) w , Span(w) . ,

R^m subspace W . , W R^m

subspace, v∈ R^m, v W projection W w, v− w

W . .

Definition 7.1.1. W R^m subspace.

W^⊥={v ∈ R^m| ⟨v,w⟩ = 0, ∀w ∈ W}.

143

W^⊥ R^m W . W^⊥ orthogonal complement of W .

orthogonal complement projection .

Definition 7.1.2. W R^m subspace. v∈ R^m, w∈ W v−w ∈ W^⊥, w the projection of v on W .

v∈ R^m, the projection of v on W , .

Proj_W(v) . 性, 性,

R^m w Proj_W(v), w∈ W v− w ∈ W^⊥ .

orthogonal complement 性 . W R^m subspace, v, v^′∈ W^⊥ r, s∈ R. 性 (3)(4), w∈ W, ⟨rv+sv^′, w⟩ = r⟨v,w⟩ + s⟨v^′, w⟩. v, v^′∈ W^⊥, ⟨v,w⟩ = ⟨v^′, w⟩ = 0, ⟨rv + sv^′, w⟩ = 0.

v, v^′∈ W^⊥ r, s∈ R, rv + sv^′∈ W^⊥, W^⊥ R^m subspace.

orthogonal complement complement . W orthogonal

complement W^⊥ W . W∩W^⊥ . W∩W^⊥

subspace, O . .

Lemma 7.1.3. W R^m subspace. W∩W^⊥={O}.

Proof. W∩W^⊥ subspace, O∈ W ∩W^⊥. w∈ W ∩W^⊥. w∈ W^⊥,

w^′∈ W ⟨w,w^′⟩ = 0. w∈ W, ⟨w,w⟩ = 0. 性 (2) w = O,

W∩W^⊥={O}. 

projection 性.

Proposition 7.1.4. W R^m subspace. v∈ R^m, v W projection .

Proof. w, w^′∈ W v W projection. w, w^′ v−w ∈ W^⊥ v− w^′∈ W^⊥. W^⊥ subspace, (v− w) − (v − w^′)∈ W^⊥, w− w^′∈ W^⊥.

W subspace, w− w^′∈ W. Lemma 7.1.3 w− w^′= O, w = w^′,

性. 

projection 性. W^⊥ subspace, 步

dimension. W^⊥ W , dim(W^⊥) dim(W ) .

dim(W ) = n w₁, . . . , w_n W basis. A =





— w1 — ...

— wn —



,

A i-th row w_i ( row vector). w_i∈ R^m, A n× m matrix.

v∈ W^⊥, w_i∈ W, ⟨wi, v⟩ = 0, ∀i = 1,...,n. Av = O,

W^⊥ A nullspace. , v∈ R^m A nullspace , Av = O

7.1. Projection 145

⟨wi, v⟩ = 0, ∀i = 1,...,n. w∈ W, w₁, . . . , w_n W basis, c1, . . . , c_n∈ R w = c₁w₁+··· + cnw_n.

⟨w,v⟩ = ⟨c1w₁+··· + cnw_n, v⟩ = c1⟨w1, v⟩ + ··· + cn⟨wn, v⟩ = 0,

A nullspace W^⊥. 性 .

Lemma 7.1.5. W R^m subspace. w₁, . . . , w_n W basis. A w₁, . . . , w_n ( row vector) row vector n× m matrix. W^⊥ A nullspace.

Example 7.1.6. W = Span(





 1 2 2 1





,





 3 4 2 3





) W orthogonal complement W^⊥.

A =

[ 1 2 2 1 3 4 2 3

]

. Lemma 7.1.5, A nullspace{x ∈ R⁴| Ax = O} W^⊥. elementary row operation A 1-st row −3 2-nd row

[ 1 2 2 1

0 −2 −4 0

]

. 2-nd row −1/2

[ 1 2 2 1 0 1 2 0

] .







−1 0 0 1





,





 2

−2 1 0





 A

nullspace, W^⊥ basis.

Question 7.1. Lemma 7.1.5 w₁, . . . , w_n Span(w₁, . . . , w_n) = W linearly

independent, W^⊥ A nullspace?

R^m column vector, Lemma 7.1.5 w₁, . . . , wn

column vector A column vector ( A m×n matrix), Lemma

7.1.5 .

Corollary 7.1.7. W R^m subspace. w₁, . . . , w_n W basis. A w₁, . . . , wn column vector m× n matrix. W^⊥ A^T nullspace.

Corollary 7.1.7 column space W = C(A), Corollary 7.1.7 .

Corollary 7.1.8. A m× n matrix. C(A)^⊥= N(A^T).

dim(W^⊥) dim(W ) . W R^m

subspace dim(W ) = n. W basis row vectors n× m matrix A,

Lemma 7.1.5 W^⊥ A nullspace. dim(W^⊥) A nullspace

( A nullity). A rank A row space ( A row vectors space W ) , rank(A) = dim(W ) = n. Theorem 4.4.13 A nullity A column 數 m rank(A), null(A) = m− rank(A) = m − n.

性 .

Theorem 7.1.9. W R^m subspace. dim(W^⊥) = m− dim(W).

Theorem 7.1.9 .

Corollary 7.1.10. W R^m subspace. (W^⊥)^⊥= W .

Proof. Theorem 7.1.9 dim((W^⊥)^⊥) = m− dim(W^⊥) = m− (m − dim(W)) = dim(W ). w∈ W, w^′∈ W^⊥, ⟨w,w^′⟩ = 0, w∈ (W^⊥)^⊥.

W⊆ (W^⊥)^⊥. dim(W ) = dim((W^⊥)^⊥) Corollary 4.3.6 (W^⊥)^⊥= W . 

, Corollary 7.1.10 dimension . W

infinite dimensional vector space subspace , (W^⊥)^⊥= W .

W⊆ (W^⊥)^⊥, (W^⊥)^⊥⊆ W . finite dimensional , dimension , Corollary 4.3.6 (W^⊥)^⊥= W .

Question 7.2. A m× n matrix. C(A) = N(A^T)^⊥.

Theorem 7.1.9 W R^m subspace w₁, . . . , wn W basis, dim(W^⊥) = m− n, W^⊥ basis u₁, . . . , u_m−n.

w₁, . . . , wn, u1, . . . , um−n linearly independent, .

Corollary 7.1.11. W R^m subspace w₁, . . . , w_n W basis u₁, . . . , um−n W^⊥ basis. w₁, . . . , wm, u1, . . . , um−n R^m basis.

Proof. w₁, . . . , w_n, u₁, . . . , u_m−n linearly independent. , c1, . . . , cn, cn+1, . . . , cm∈ R 0 c1w₁+··· + cnw_n+ cn+1u₁+··· + cmu_m−n= O.

w = c₁w₁+··· + cnw_n, W vector space, w∈ W. w =−(cn+1u₁+

··· + cmu_m−n) W^⊥ vector space, w∈ W^⊥. 言 , w∈ W ∩W^⊥.

Lemma 7.1.3 w = O. w₁, . . . , wn linearly independent c1w₁+··· + cnw_n= w = O c₁ =··· = cn = 0. c_n+1 =··· = cm= 0. c₁, . . . , c_n, c_n+1, . . . , c_m

0 , w₁, . . . , wn, u1, . . . , um−n linearly independent.

dim(R^m) = m, w₁, . . . , w_n, u₁, . . . , u_m−n R^m m linearly independent vectors,

w₁, . . . , w_n, u₁, . . . , u_m−n R^m basis. 

W basis{w1, . . . , wn} W^⊥ basis{u1, . . . , um−n} R^m basis, v∈ R^m, c₁, . . . , c_n, d₁, . . . , d_m−n∈ R

v = c₁w₁+··· + cnw_n+ d1u₁+··· + dm−nu_m−n.

w = c₁w₁+··· + cnw_n, w∈ W v− w = d1u₁+··· + dm−nu_m−n∈ W^⊥.

w the projection of v on W . projection 性. ,

.

7.1. Projection 147

Theorem 7.1.12. W R^m subspace w₁, . . . , w_n W basis u₁, . . . , u_m−n W^⊥ basis. v∈ R^m, v = c₁w₁+··· + cnw_n+ d₁u₁+··· + dm−nu_m−n, v W projection

Proj_W(v) = c1w₁+··· + cnw_n.

Example 7.1.13. Example 7.1.6 v =





 2 0 1 4





 W = Span(





 1 2 2 1





,





 3 4 2 3





)

projection.







−1 0 0 1





,





 2

−2 1 0





 W^⊥ basis,





 2 0 1 4











 1 2 2 1





,





 3 4 2 3





,







−1 0 0 1





,





 2

−2 1 0





 linear combination





 2 0 1 4





 = −





 1 2 2 1





 +





 3 4 2 3





 + 2







−1 0 0 1





 +





 2

−2 1 0





.

v W projection −





 1 2 2 1





 +





 3 4 2 3





 =





 2 2 0 2





.

R^m subspace W , v∈ R^m, Proj_W(v) ,

projection on W 數. Proj_W R^m→ R^m 數 v∈ R^m

Proj_W(v). Proj_W:R^m→ R^m linear transformation.

v, v^′∈ R^m, w =Proj_W(v) w^′= Proj_W(v^′).

w, w^′∈ W v− w ∈ W^⊥ v^′− w^′∈ W^⊥. Proj_W(v + v^′) = Proj_W(v) + Proj_W(v^′) = w + w^′. 言 , w + w^′∈ W (v + v^′)− (w + w^′)∈ W^⊥. W,W^⊥ vector space, w, w^′∈ W v− w,v^′− w^′∈ W^⊥ w + w^′∈ W

(v + v^′)− (w + w^′) = (v− w) + (v^′− w^′)∈ W^⊥. r∈ R,

Proj_W(rv) = rProj_W(v) = rw. rw∈ W rv− rw ∈ W^⊥. W,W^⊥

vector space, w∈ W,v − w ∈ W^⊥, .

Proposition 7.1.14. W R^m subspace, Proj_W :R^m→ R^m linear trans- formation.

Question 7.3. Proj_W:R^m→ R^m image kernel.

Theorem 7.1.12 v W . W

basis w₁, . . . , w_n W^⊥ basis u₁, . . . , u_m−n, v w₁, . . . , w_n, u₁, . . . , u_m−n

線性 , Proj_W(v). 步 . A w₁, . . . , wn column

vectors m× n matrix, W A column space C(A). Proj_W(v) ,

w∈ W v− w ∈ W^⊥, w Proj_W(v) . w∈ W ? column space , w∈ W x∈ Rⁿ Ax = w. v−w ∈ W^⊥= (C(A))^⊥

, Corollary 7.1.8 v− w = v − Ax ∈ N(A^T). x∈ Rⁿ

A^T(v−Ax) = O. 性 , x∈ Rⁿ (A^TA)x = A^Tv.

Proj_W(v) , x∈ Rⁿ Ax = Proj_W(v), x

v− Ax ∈ W^⊥, (A^TA)x = A^Tv . (A^TA)x = A^Tv, x

Ax = Proj_W(v) . (A^TA)x = A^Tv ? .

Lemma 7.1.15. A∈ Mm×n rank(A) = n, A^TA n× n invertible matrix.

Proof. A∈ Mm×n A^T∈ Mn×m, A^TA∈ Mn×n. Theorem 3.5.9, A^TA invertible. (A^TA)x = O nontrivial solution

A^TA invertible matrix. x = u (A^TA)x = O , (A^TA)u = O.

(A^TA)u = A^T(Au), Au∈ N(A^T). Au∈ C(A), Au∈ C(A) ∩ N(A^T) =

C(A)∩C(A)^⊥. Lemma 7.1.3 Au = O. rank(A) = n, Corollary

2.3.5 Ax = O nontrivial solution, u = O. (A^TA)x = O

nontrivial solution, A^TA n× n invertible matrix. 

前 A W basis column vector m× n matrix,

rank(A) = dim(W ) = n. Lemma 7.1.15 A^TA invertible.

(A^TA)x = A^Tv A^TA inverse, x = (A^TA)⁻¹A^Tv.

(A^TA)x = A^Tv , A Proj_W(v). .

Theorem 7.1.16. W R^m subspace w₁, . . . , wn W basis. A w₁, . . . , w_n column vector m× n matrix. v∈ R^m, v W projection

Proj_W(v) = A(A^TA)⁻¹A^Tv.

Example 7.1.17. Theorem 7.1.16 Example 7.1.13 ,

v =





 2 0 1 4





 W = Span(





 1 2 2 1





,





 3 4 2 3





) . A =





 1 3 2 4 2 2 1 3





,

A^T=

[ 1 2 2 1 3 4 2 3

]

, A^TA =

[ 10 18 18 38

]

inverse (A^TA)⁻¹= (1/28)

[ 19 −9

−9 5

] . Theorem 7.1.16

Proj_W(v) = 1 28





 1 3 2 4 2 2 1 3







[ 19 −9

−9 5

][ 1 2 2 1 3 4 2 3

]



 2 0 1 4





 =





 2 2 0 2





.

Example 7.1.13 .

Theorem 7.1.16 W R^m subspace, W =R^m,

dim(W ) = n m. W =R^m , W projection ,

7.2. Inconsistent Systems 149

W^⊥={O}, R^m W =R^m .

dim(W ) = n < m . , W basis column vector

A, m× n matrix . A A^T invertible.

(A^TA)⁻¹ A⁻¹(A^T)⁻¹. Theorem 7.1.16 A(A^TA)⁻¹A^T A(A⁻¹(A^T)⁻¹)A^T, identity matrix.

Theorem 7.1.16 Theorem 7.1.12 projection . W

basis , W^⊥ basis. A(A^TA)⁻¹A^T R^m v,

Proj_W(v). A(A^TA)⁻¹A^T PW , W projection

matrix. , PW Proj_W :R^m→ R^m linear transformation

standard matrix representation. linear transformation standard matrix

representation , W basis column vectors m× n

matrix A , A projection matrix .

Question 7.4. Example 7.1.17 W projective matrix ?

Theorem 7.1.16 projection , projection matrix

(A^TA)⁻¹ .

projection .

7.2. Inconsistent Systems

projection .

linear system inconsistent , .

大 學 線.

性 (2) O , v∈ V ⟨v,v⟩ > 0,

∥v∥ =√

⟨v,v⟩. R³ .

. 性 inner product

space projection .

Proposition 7.2.1. W R^m subspace v∈ R^m. w v W , w^′∈ W w^′̸= w, ∥v − w^′∥ > ∥v − w∥.

Proof. v− w^′= v− w + w − w^′. w =Proj_W(v), v− w ∈ W^⊥. W vector space, w− w^′∈ W.

∥v−w^′∥²=⟨v−w^′, v−w^′⟩ = ⟨v−w+w−w^′, v−w+w−w^′⟩ = ⟨v−w,v−w⟩+⟨w−w^′, w−w^′⟩.

∥v − w^′∥²=∥v − w∥²+∥w − w^′∥². w̸= w^′ ∥w − w^′∥ > 0. ∥v − w^′∥²>

∥v − w∥², ∥v − w^′∥ > ∥v − w∥. 

A∈ Mm×n, Ax = b , b∈ C(A). ,

, Ax = b inconsistent , b

, b₀ Ax = b^′ b^′ b

. b₀ Ax = b₀ , .

Ax = b^′ b^′ C(A) subspace, Proposition 7.2.1

b^′ b b₀ b C(A) projection. projection

, b− b0∈ C(A)^⊥= N(A^T). A^T(b− b0) = O,

A^Tb₀= A^Tb. (7.1)

Ax = b0 代

A^TAx = A^Tb. (7.2)

(7.2) (7.1) , b₀ Ax = b₀,

A^TAx = A^Tb. (7.2) .

x = x₀ A^TAx = A^Tb , b₀= Ax₀ b₀∈ C(A) b₀−b ∈ N(A^T) = C(A)^⊥. b₀ b C(A) projection, Proposition 7.2.1 b₀

Ax = b^′ b^′ b . A^TAx = A^Tb ,

. (7.2) ?

Proposition 7.2.2. A∈ Mm×n b∈ R^m. A^TAx = A^Tb .

rank(A) = n, .

Proof. b₀ b C(A) projection, b₀∈ C(A) b− b0∈ C(A)^⊥ . b₀∈ C(A) x₀∈ Rⁿ Ax₀= b₀. b− b0∈ C(A)^⊥ C(A)^⊥= N(A^T) (Corollary 7.1.8)

O = A^T(b− b0) = A^Tb− A^Tb₀= A^Tb− A^T(Ax₀),

x₀ A^TAx0= A^Tb, A^TAx = A^Tb .

rank(A) = n, Lemma 7.1.15 A^TA invertible, A^TAx = A^Tb

. 

rank(A) < n , A^TAx = A^Tb

Ax = Proj_C(A)(b) , A m×n matrix Theorem 3.4.6 (

). .

Definition 7.2.3. A∈ Mm×n b∈ R^m. Ax = b.

A^TAx = A^Tb Ax = b normal equations. normal equations , Ax = b least squares solution.

, rank(A) = n , A^TAx = A^Tb x = (A^TA)⁻¹A^Tb, Ax = b least squares solution.

, Ax = b least squares solution, normal equation

, b C(A) projection. least squares solution ,

A . Ax = b consistent, b∈ C(A), b

C(A) projection b . Ax = b normal equations A^TAx = A^Tb

7.2. Inconsistent Systems 151

. Ax = b least squares solution ,

Ax = b , normal equations Ax = b .

Example 7.2.4. Ax = b A =





 1 3 2 4 2 2 1 3





 b =





 2 0 1 4





,

least squares solution. A^TA =

[ 10 18 18 38

]

A^Tb = [8

20 ]

, Ax = b

normal equations

10x1+ 18x₂ = 8 18x1+ 38x2 = 20 . [x1

x₂ ]

= [−1

1 ]

Ax = b least squares solution. rank(A) = 2, A^TA invertible inverse (A^TA)⁻¹= (1/28)

[ 19 −9

−9 5

] . [x1

x2

]

= (A^TA)⁻¹A^Tb = 1 28

[ 19 −9

−9 5

][8 20

]

= [−1

1 ]

.





 1 3 2 4 2 2 1 3





 [−1

1 ]

=





 2 2 0 2





 b C(A) projection ( Example 7.1.17).

Question 7.5. W R^m subspace W = Span(w₁, . . . , w_n). A w₁, . . . , w_n column vector m× n matrix. v∈ R^m x = x₀∈ Rⁿ Ax = v normal equations A^TAx = A^Tv , v W projection Ax₀. w₁, . . . , w_n

linearly independent , normal equations , Proj_W(v)

性 ? W projection matrix PW A(A^TA)⁻¹A^T ?

least squares solution , 線.

(x₁, y₁), . . . , (x_m, y_m), 線 y = ax + b, (x_i, y_i) 線. x₁, . . . , x_m y₁, . . . , y_m 數,

x, y 線 a y b. ,

a, b

ax1+ b = y1

ax2+ b = y2

... axm+ b = ym.

Ax = b, A =





 x1 1 x₂ 1 ... ... xm 1





, x = [a

b ]

, b =





 y1

y₂ ... ym





. ,

(x1, y1), . . . , (xm, ym) Ax = b , Ax = b least squares solution, A^TAx = A^Tb. ,

x_i, y_i , x₁, . . . , x_m ( (x₁, y₁), . . . , (x_m, y_m)

線 , ), rank(A) = 2, Proposition 7.2.2

normal equations A^TAx = A^Tb . , 線

?

a, b∈ R, i = 1, . . . , m, y^′_i = ax_i+ b. (x_i, y_i) 線 y = ax + b

. b^′=





 y^′₁ y^′₂ ... y^′_m





, b^′=





 x₁ 1 x2 1 ... ... xm 1





 [a

b ]

∈ C(A). εi= y^′_i− yi,

線 y = ax + b 代 x_i y^′_i yi . 代 xi

, εi , , ,

ε1²+··· +εm² . ε1²+··· +εm² b^′ b ,

∥b − b^′∥². , Ax = b least squares solution x = [a

b ]

, a, b

線 y = ax + b b^′ b C(A) , ∥b − b^′∥

. ε1²+··· +εm² . least squares solution

線, least squares line ( 線, 線), 學

line of regression ( 線).

Example 7.2.5. (−1,0),(1,1),(2,3) least square line y = ax + b.

−1a + b = 0 1a + b = 1 2a + b = 3,



 −1 1 1 1 2 1



[ a b ]

=



0 1 3



. normal equations

[ −1 1 2 1 1 1

] −1 1 1 1 2 1



[ a b ]

=

[ −1 1 2 1 1 1

]0 1 3



,

6a + 2b = 7 2a + 3b = 4,

least squares solution a = 13/14, b = 5/7 least squares line y =13

14x +5 7. least squares line 性 .

Proposition 7.2.6. (x1, y1), . . . , (xm, ym). y = ax + b least squares line i = 1, . . . , m y^′_i= ax_i+ b. 性 :

(1) y₁+··· + ym= y^′₁+··· + y^′m.

7.3. Orthogonal Basis and Gram-Schmidt Process 153

(2) x₁y₁+··· + xmy_m= x₁y^′₁+··· + xmy^′_m.

(3) x y x1, . . . , xm y1, . . . , ym 數, x = (x1+··· + xm)/m, y = (y₁+··· + ym)/m. (x, y) 線 y = ax + b , y = ax + b.

Proof. b =





 y₁ y₂ ... y_m





, b^′=





 y^′₁ y^′₂ ... y^′_m





 A =





 x₁ 1 x₂ 1 ... ... x_m 1





, x = [a

b ]

Ax = b

least squares solution b^′= A [a

b ]

b C(A) projection. b− b^′∈ C(A)^⊥= N(A^T),

A^T(b− b^′) =

[ x1 ··· xm

1 ··· 1 ]





 y₁− y^′₁ y₂− y^′₂

... y_m− y^′m





=

[x1(y₁− y^′₁) +··· + xm(y_m− y^′_m) (y1− y^′₁) +··· + (ym− y^′_m)

]

= [0

0 ]

.

(1), (2).

y^′= (y^′₁+··· + y^′_m)/m, (1) y = y^′. i = 1, . . . , m, y^′_i= ax_i+ b,

y^′= ax + b. (3), y = ax + b. 

, 線,

線. 線 ,

線 y = ax²+ bx + c, 代 a, b, c 數 ,

least squares solution. 線

線 . , .

7.3. Orthogonal Basis and Gram-Schmidt Process

R^m v subspace W projection .

. , Theorem 7.1.12, W basis

W^⊥ basis, v basis R^m basis linear combination,

W^⊥ , v W projection . w

basis linear combination, .

linear combination , . , Theorem

7.1.16, W basis column vectors A, W

projection matrix. , A^TA inverse.

W basis. basis, .

basis, ?

, basis.

W Rⁿ subspace w₁, . . . , wn basis ⟨wi, wj⟩ = 0,∀i ̸= j.

w∈ W, w₁, . . . , w_n W basis, c₁, . . . , c_n∈ R w = c₁w₁+···+cnw_n.

c₁, . . . , c_n, w_i

, ci. i = 1, . . . , n, ⟨w,wi⟩.

⟨w,wi⟩ = ⟨c1w₁+··· + cnw_n, wi⟩ = c1⟨w1, wi⟩ + ··· + cn⟨wn, wi⟩ = ci⟨w1, wi⟩.

w_i̸= O, ⟨wi, w_i⟩ = ∥wi∥²̸= 0, c_i=⟨w,wi⟩/⟨wi, w_i⟩ = ⟨w,wi⟩/∥wi∥². ,

∥wi∥ = 1, ci=⟨w,wi⟩. .

Proposition 7.3.1. W Rⁿ subspace w₁, . . . , w_n basis ⟨wi, w_j⟩ = 0,∀i ̸= j. w∈ W,

w = ⟨w,w1⟩

∥w1∥² w₁+··· +⟨w,wi⟩

∥wi∥² w_i+··· +⟨w,wn⟩

∥wn∥² w_n.

basis, linear combination ,

.

Definition 7.3.2. W R^m subspace w₁, . . . , w_n W basis i̸= j ⟨wi, w_j⟩ = 0. w₁, . . . , w_n W orthogonal basis.

⟨wi, wi⟩ = 1, ∀i = 1,...,n, w₁, . . . , wn W orthonormal basis.

, w₁, . . . , wn W orthogonal basis, i = 1, . . . , n, u_i = w_i/∥wi∥, u₁, . . . , u_n W orthonormal basis.

⟨ui, uj⟩ = ⟨ 1

∥wi∥w_i, 1

∥wj∥w_j⟩ = 1

∥wi∥ 1

∥wj∥⟨wi, wj⟩.

i̸= j, ⟨ui, u_j⟩ = ⟨wi, w_j⟩ = 0; i = j, ⟨ui, u_i⟩ = ⟨wi, w_i⟩/∥wi∥²= 1.

前 , W orthogonal basis w₁, . . . , w_n,

W w₁, . . . , wn 線性 . 前 Theorem 7.1.12

. , W orthogonal basis w₁, . . . , w_n,

v∈ R^m W projection . w =Proj_W(v) w = c₁w₁+···+cnw_n. v− w ∈ W^⊥, i = 1, . . . , n, ⟨v,wi⟩ − ⟨w,wi⟩ = ⟨v − w,wi⟩ = 0.

⟨v,wi⟩ = ⟨w,wi⟩ = ci⟨wi, w_i⟩, c_i=⟨v,wi⟩/⟨wi, w_i⟩. Theorem

7.1.12 projection .

Theorem 7.3.3. W R^m subspace w₁, . . . , w_n W orthogonal basis.

v∈ R^m, v W projection

Proj_W(v) =⟨v,w1⟩

∥w1∥²w₁+··· +⟨v,wn⟩

∥wn∥²w_n. w₁, . . . , w_n W orthonormal basis,

Proj_W(v) =⟨v,w1⟩w1+··· + ⟨v,wn⟩wn.

Theorem 7.3.3, W orthogonal basis,

W^⊥ basis, . , W

7.3. Orthogonal Basis and Gram-Schmidt Process 155

orthogonal basis w₁, . . . , w_n W projection matrix. A w₁, . . . , w_n column vectors m× n matrix,

A^TA =







— w^T₁ —

— w^T₂ — ...

— w^T_n —









    w₁ w₂ ··· wn





(i, j)-th entry w^T_iw_j=⟨wi, w_j⟩, w₁, . . . , w_n orthogonal, A^TA diagonal

matrix, (A^TA)⁻¹,

A^TA =







∥w1∥²

∥w2∥² . ..

∥wn∥²





, (A^TA)⁻¹=









∥w11∥²

∥w12∥²

. ..

∥w1n∥²







.

Theorem 7.1.16 projection , .

Theorem 7.3.4. W R^m subspace w₁, . . . , w_n W orthogonal basis.

v∈ R^m, Proj_W(v) = P_Wv, PW m× m matrix P_W = 1

∥w1∥²w₁w^T₁+ 1

∥w2∥²w₂w^T₂+··· + 1

∥wn∥²w_nw^T_n.

Proof. w_i m× 1 matrix, w_iw^T_i m× m matrix.

A w₁, . . . , w_n column vectors m× n matrix. Theorem 7.1.16 PW = A(A^TA)⁻¹A^T. w₁, . . . , w_n orthogonal, 前

A(A^TA)⁻¹A^T=



    w ₁ w₂ ··· wn













∥w11∥²

∥w12∥²

. ..

∥w1n∥²















— w^T₁ —

— w^T₂ — ...

— w^T_n —





 (7.3)

m× m matrix 1

∥w1∥²w₁w^T₁+ 1

∥w2∥²w₂w^T₂+··· + 1

∥wn∥²w_nw^T_n (7.4)

(7.3) m× m matrix .

i = 1, . . . , m, e_i∈ R^m, i-th entry 1 entry 0 vector.

m× m matrix e_i matrix i-th column. (7.3) (7.4)

e_i , (7.3) (7.4) , .

(7.3) e_i



    w ₁ w₂ ··· wn













∥w11∥²

∥w12∥²

. ..

∥w1n∥²















⟨w1, ei⟩

⟨w2, ei⟩ ...

⟨wn, ei⟩





=



    w ₁ w₂ ··· wn













⟨w1,ei⟩

∥w1∥²

⟨w2,ei⟩

∥w2∥²

...

⟨wn,ei⟩

∥wn∥²









(7.4) e_i

⟨w1, e_i⟩

∥w1∥² w₁+⟨w2, e_i⟩

∥w2∥² w₂+··· +⟨wn, e_i⟩

∥wn∥² w_n.

 Question 7.6. Proj_W:R^m→ R^m linear transformation linear transformation

standard matrix representation P_W, Theorem 7.3.3 Theorem 7.3.4.

W orthogonal basis, v∈ R^m W

projection. R^m nonzero subspace, orthogonal

basis, R^m nonzero subspace orthogonal basis (

orthonormal basis). Gram-Schmidt process.

R^m nonzero subspace W , dim(W ) = n. w₁, . . . , w_k∈ W ⟨wi, wj⟩ = 0, ∀i ̸= j, w₁, . . . , wk linearly independent.

linearly independent c₁, . . . , c_k ∈ R 0 c₁w₁+··· + ckw_k = O.

i = 1, . . . , k 0 =⟨c1w₁+··· + ckw_k, w_i⟩ = ci∥wi∥². ∥wi∥ ̸= 0, ci= 0. c1, . . . , c_k 0 , w₁, . . . , w_k linearly independent.

W orthogonal basis, W w₁, . . . , w_n ⟨wi, w_j⟩ = 0,

∀i ̸= j , linearly independent dim(W ) = n, Corollary 4.3.5

W basis. W nonzero vectors.

W̸= {O}, W nonzero vector v₁. w₁= v₁

W₁= Span(v₁) = Span(w₁). dim(W ) = 1, W = W₁ w₁ W orthogonal basis. dim(W ) > 1, W1( W, nonzero vector v2∈ W v₂̸∈ W1.

v₂, w₂ ∈ W w₂ ̸= O ⟨w1, w₂⟩ = 0. ,

w₂= v2− Proj_W1(v2), w₂ ∈ W₁^⊥, W1 = Span(w1), ⟨w1, w2⟩ = 0.

w₂ ̸= O. w₂= O, v₂ = Proj_W1(v₂)∈ W1, 初

v₂ ̸∈ W1 . W = Span(w₁), Proposition 1.4.9

Proj_W1(v2) = (⟨v2, w1⟩/∥w1∥²)w1,

w₂= v2−⟨v2, w1⟩

∥w1∥² w₁.

Span(w1, w2) = Span(v1, v2), w₁, w2 , w₁, w2∈ Span(v₁, v₂), Span(w₁, w₂)⊆ Span(v1, v₂). v₁, v₂ linearly independent

w₁, w2 linearly independent, dim(Span(w1, w2)) = dim(Span(v1, v2)) = 2 Span(w₁, w₂) = Span(v₁, v₂). , W₂= Span(w₁, w₂) = Span(v₁, v₂).

dim(W ) = 2, W = W2, w₁, w₂ W orthogonal basis. dim(W ) > 2, W₂( W, nonzero vector v₃∈ W v₃̸∈ W2. v₃, w₃∈ W

w₃̸= O ⟨w1, w₃⟩ = ⟨w2, w₃⟩ = 0. 前, w₃= v₃− Proj_W1(v₂),

w₃∈ W₂^⊥, W2= Span(w1, w2), ⟨w1, w3⟩ = ⟨w2, w3⟩ = O. v₃̸∈ W2, 前 w₃̸= O. w₁, w₂ W₂ orthogonal basis, Theorem 7.3.3