## Advanced Calculus (II)

W^{EN}-C^{HING} L^{IEN}

Department of Mathematics National Cheng Kung University

2009

**Ch11: Differentiability on R**

^{n}### 11.7: Optimization

Definition (11.50)

**Let V be open in R**^{n}**, let a**∈*V , and suppose that*
*f* :*V* →**R.**

*(i) f*(a)*is called a local minimum of f if and only if there is*
*an r* >*0 such that f*(a) ≤*f*(x)**for all x**∈*B**r*(a).

*(ii) f*(a)*is called a local maximum of f if and only if there*
*is an r* >*0 such that f*(a) ≥*f*(x)**for all x**∈*B**r*(a).

*(iii) f*(a)*is called a local extremum of f if and only if f*(a)is
*a local maximum or a local minimum of f .*

Remark (11.51)

* If the first-order partial derivatives of f exist at a, and f*(a)

*is a local extremum of f , then*∇f(a) =

**0.**

Remark (11.52)

There exist continuously differentiable functions that
satisfy∇f(a) =* 0 such that f*(a)is neither a local
maximum nor a local minimum.

Definition (11.53)

**Let V be open in R**^{n}**, let a**∈*V , and let f* :*V* →**R be**
**differentiable at a. Then a is call a saddle point of f if**

∇f(a) =* 0 and there is a r*0>0 such that given any
0< ρ <

*r*0

**there are points x,y**∈

*B*

_{ρ}(a)that satisfy

*f*(x) <*f*(a) < *f*(y).

Example (11.54)

Find the maximum and minimum of

*f*(x,*y*) =*x*^{2}−*x* +*y*^{2}−*2y on H* =*B*1(0,0).

Lemma (11.55)

**Let V be open in R**^{n}* , a*∈

*V , and f*:

*V*→

**R. If all second-**

**order partial derivatives of f exist at a and D**^{(2)}

*f*(a;

**h) >**0

*6=*

**for all h***>*

**0, then there is an m***0 such that*

(33) *D*^{(2)}*f*(a;**x) ≥***mkxk*^{2}
* for all x* ∈

**R**

^{n}*.*

Proof.

*Set H* = {x∈**R*** ^{n}* : kxk =1}and consider the function

*g(x) :*=

*D*

^{(2)}

*f*(a;

**x) :**=

*n*

X

*j=1*
*n*

X

*k=1*

∂^{2}*f*

∂x*k*∂x*j*

(a)x*j**x**k*, **x**∈**R*** ^{n}*.

Lemma (11.55)

**Let V be open in R**^{n}* , a*∈

*V , and f*:

*V*→

**R. If all second-**

**order partial derivatives of f exist at a and D**^{(2)}

*f*(a;

**h) >**0

*6=*

**for all h***>*

**0, then there is an m***0 such that*

(33) *D*^{(2)}*f*(a;**x) ≥***mkxk*^{2}
* for all x* ∈

**R**

^{n}*.*

Proof.

*Set H* = {x∈**R*** ^{n}* : kxk =1}and consider the function

*g(x) :* =*D*^{(2)}*f*(a;**x) : =**

*n*

X

*j=1*
*n*

X

*k=1*

∂^{2}*f*

∂x*k*∂x*j*

(a)x*j**x**k*, **x**∈**R*** ^{n}*.

Lemma (11.55)

**Let V be open in R**^{n}* , a*∈

*V , and f*:

*V*→

**R. If all second-**

**order partial derivatives of f exist at a and D**^{(2)}

*f*(a;

**h) >**0

*6=*

**for all h***>*

**0, then there is an m***0 such that*

(33) *D*^{(2)}*f*(a;**x) ≥***mkxk*^{2}
* for all x* ∈

**R**

^{n}*.*

Proof.

*Set H* = {x∈**R*** ^{n}* : kxk =1}and consider the function

*g(x) : =* *D*^{(2)}*f*(a;**x) :** =

*n*

X

*j=1*
*n*

X

*k=1*

∂^{2}*f*

∂x*k*∂x*j*

(a)x*j**x**k*, **x**∈**R*** ^{n}*.

Lemma (11.55)

**Let V be open in R**^{n}* , a*∈

*V , and f*:

*V*→

**R. If all second-**

**order partial derivatives of f exist at a and D**^{(2)}

*f*(a;

**h) >**0

*6=*

**for all h***>*

**0, then there is an m***0 such that*

(33) *D*^{(2)}*f*(a;**x) ≥***mkxk*^{2}
* for all x* ∈

**R**

^{n}*.*

Proof.

*Set H* = {x∈**R*** ^{n}* : kxk =1}and consider the function

*g(x) : =* *D*^{(2)}*f*(a;**x) : =**

*n*

X

*j=1*
*n*

X

*k=1*

∂^{2}*f*

∂x*k*∂x*j*

(a)x*j**x**k*, **x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact,*it follows from the

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H.*

*Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact,*it follows from the

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0.If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0,**then _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0.If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x)**= *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0,**then _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x)**= *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Proof.

**By hypothesis, g is continuous and positive on R*** ^{n}*\{0},

*hence on H. Since H is compact, it follows from the*

*Extreme Value Theorem that g has a positive minimum m*

*on H.*

**Clearly, (33) holds for x**=**0. If x**6=**0, then** _{kxk}** ^{x}** ∈

*H, and it*

*follows from the choice of g and m that*

*D*^{(2)}*f*(a;**x) =** *g(x)*

kxk^{2}kxk^{2}=*g*

**x**
kxk

kxk^{2}≥*mkxk*^{2}.
**We conclude that (33) holds for all x**∈**R*** ^{n}*.

Theorem (11.56 Second Derivative Test)

**Let V be open in R**^{n}* , a*∈

*V , and suppose that f*:

*V*→

**R**

*satisfies*∇f(a) =

**0. Suppose further that the second-***order total differential of f exists on V and is continuous at*

**a.***(i) If D*^{(2)}*f*(a;**h) >*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*minimum of f .*

*(ii) If D*^{(2)}*f*(a;**h) <*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*maximum of f .*

*(iii) If D*^{(2)}*f*(a;**h)***takes on both positive and negative*
* values for h*∈

**R**

^{n}

**, then a is a saddle point of f .**Theorem (11.56 Second Derivative Test)

**Let V be open in R**^{n}* , a*∈

*V , and suppose that f*:

*V*→

**R**

*satisfies*∇f(a) =

**0. Suppose further that the second-***order total differential of f exists on V and is continuous at*

**a.***(i) If D*^{(2)}*f*(a;**h) >*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*minimum of f .*

*(ii) If D*^{(2)}*f*(a;**h) <*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*maximum of f .*

*(iii) If D*^{(2)}*f*(a;**h)***takes on both positive and negative*
* values for h*∈

**R**

^{n}

**, then a is a saddle point of f .**Theorem (11.56 Second Derivative Test)

**Let V be open in R**^{n}* , a*∈

*V , and suppose that f*:

*V*→

**R**

*satisfies*∇f(a) =

**0. Suppose further that the second-***order total differential of f exists on V and is continuous at*

**a.***(i) If D*^{(2)}*f*(a;**h) >*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*minimum of f .*

*(ii) If D*^{(2)}*f*(a;**h) <*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*maximum of f .*

*(iii) If D*^{(2)}*f*(a;**h)***takes on both positive and negative*
* values for h*∈

**R**

^{n}

**, then a is a saddle point of f .**Theorem (11.56 Second Derivative Test)

**Let V be open in R**^{n}* , a*∈

*V , and suppose that f*:

*V*→

**R**

*satisfies*∇f(a) =

**0. Suppose further that the second-***order total differential of f exists on V and is continuous at*

**a.***(i) If D*^{(2)}*f*(a;**h) >*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*minimum of f .*

*(ii) If D*^{(2)}*f*(a;**h) <*** 0 for all h*6=

*(a)*

**0, then f***is a local*

*maximum of f .*

*(iii) If D*^{(2)}*f*(a;**h)***takes on both positive and negative*
* values for h*∈

**R**

^{n}

**, then a is a saddle point of f .**Remark (11.57)

*If D*^{(2)}*f*(a;**h) ≥***0, then f*(a)**can be a local minimum or a**
can be a saddle point.

Lemma (11.58)

*Let A,B,C* ∈* R, D*=

*B*

^{2}−

*AC, and*φ(h,

*k*) =

*Ah*

^{2}+

*2Bhk*+Ck

^{2}.

*(i) If D* <*0, then A and*φ(h,*k*)*have the same sign for all*
(h,*k) 6= (0,*0).

*(ii) If D* >*0, then*φ(h,*k*)*takes on both positive and*
*negative values as*(h,*k*)**varies over R**^{2}*.*

Lemma (11.58)

*Let A,B,C* ∈* R, D*=

*B*

^{2}−

*AC, and*φ(h,

*k*) =

*Ah*

^{2}+

*2Bhk*+Ck

^{2}.

*(i) If D* <*0, then A and*φ(h,*k*)*have the same sign for all*
(h,*k) 6= (0,*0).

*(ii) If D* >*0, then*φ(h,*k*)*takes on both positive and*
*negative values as*(h,*k*)**varies over R**^{2}*.*

Lemma (11.58)

*Let A,B,C* ∈* R, D*=

*B*

^{2}−

*AC, and*φ(h,

*k*) =

*Ah*

^{2}+

*2Bhk*+Ck

^{2}.

*(i) If D* <*0, then A and*φ(h,*k*)*have the same sign for all*
(h,*k) 6= (0,*0).

*(ii) If D* >*0, then*φ(h,*k*)*takes on both positive and*
*negative values as*(h,*k*)**varies over R**^{2}*.*

Theorem (11.59)

**Let V be open in R**^{2}*,*(a,*b) ∈V , and suppose that*

*f* :*V* →* R satisfies*∇f(a,

*b) =*

**0. Suppose further that the***second-order total differential of f exists on V and is*

*continuous at*(a,

*b), and set*

*D*=*f*_{xy}^{2}(a,*b) −f**xx*(a,*b)f**yy*(a,*b).*

*(i) If D* <*0 and f**xx*(a,*b) >0, then f*(a,*b)is a local*
*minimum.*

*(ii) If D* <*0 and f**xx*(a,*b) <0, then f*(a,*b)is a local*
*maximum.*

*(iii) If D* >*0, then*(a,*b)is a saddle point.*

Theorem (11.59)

**Let V be open in R**^{2}*,*(a,*b) ∈V , and suppose that*

*f* :*V* →* R satisfies*∇f(a,

*b) =*

**0. Suppose further that the***second-order total differential of f exists on V and is*

*continuous at*(a,

*b), and set*

*D*=*f*_{xy}^{2}(a,*b) −f**xx*(a,*b)f**yy*(a,*b).*

*(i) If D* <*0 and f**xx*(a,*b) >0, then f*(a,*b)is a local*
*minimum.*

*(ii) If D* <*0 and f**xx*(a,*b) <0, then f*(a,*b)is a local*
*maximum.*

*(iii) If D* >*0, then*(a,*b)is a saddle point.*

Theorem (11.59)

**Let V be open in R**^{2}*,*(a,*b) ∈V , and suppose that*

*f* :*V* →* R satisfies*∇f(a,

*b) =*

**0. Suppose further that the***second-order total differential of f exists on V and is*

*continuous at*(a,

*b), and set*

*D*=*f*_{xy}^{2}(a,*b) −f**xx*(a,*b)f**yy*(a,*b).*

*(i) If D* <*0 and f**xx*(a,*b) >0, then f*(a,*b)is a local*
*minimum.*

*(ii) If D* <*0 and f**xx*(a,*b) <0, then f*(a,*b)is a local*
*maximum.*

*(iii) If D* >*0, then*(a,*b)is a saddle point.*

Theorem (11.59)

**Let V be open in R**^{2}*,*(a,*b) ∈V , and suppose that*

*f* :*V* →* R satisfies*∇f(a,

*b) =*

**0. Suppose further that the***second-order total differential of f exists on V and is*

*continuous at*(a,

*b), and set*

*D*=*f*_{xy}^{2}(a,*b) −f**xx*(a,*b)f**yy*(a,*b).*

*(i) If D* <*0 and f**xx*(a,*b) >0, then f*(a,*b)is a local*
*minimum.*

*(ii) If D* <*0 and f**xx*(a,*b) <0, then f*(a,*b)is a local*
*maximum.*

*(iii) If D* >*0, then*(a,*b)is a saddle point.*

Remark (11.60)

*If the discriminant D* =*0, f*(a,*b)*may be a local maximum,
a local minimum, or (a,*b)*may be a saddle point.

Definition (11.61)

**Let V be open in R**^{n}**, a**∈*V , and f*,*g**j* :*V* →**R for**
*j* =1,2, . . . ,*m.*

*(i) f*(a)*is called a local minimum of f subject to*

*constraints g**j*(a) =*0, j* =1, . . . ,*m, if and only if there is a*
ρ >**0 such that x**∈*B*ρ(a)*and g**j*(x) =*0 for all j* =1, . . . ,*m*
*imply f*(x) ≥*f*(a).

*(ii) f*(a)*is called a local maximum of f subject to*

*constraints g**j*(a) =*0, j* =1, . . . ,*m, if and only if there is a*
ρ >**0 such that x**∈*B*ρ(a)*and g**j*(x) =*0 for all j* =1, . . . ,*m*
*imply f*(x) ≤*f*(a).

Theorem (11.63 Lagrange Multipliers)

*Let m* <**n, V be open in R**^{n}*and f*,*g**j* :*V* →* R be*C

^{1}

*on V*

*for j*=1,2, . . . ,

*∈*

**m. Suppose that there is an a***V such*

*that*∂(g1, . . . ,

*g*

*m*)

∂(x1, . . . ,*x**m*)(a) 6=0.

*If f*(a)*is a local extremum of f subject to constraints*
*g**k*(a) =*0, k* =1, . . . ,*m, then there exist scalars*
λ1, λ2, . . . , λ*m* *such that*

(36) ∇f(a)+

*m*

X

*k=1*

λ*k*∇g*k*(a) =**0.**

Example (11.64)

*Find all extrema of x*^{2}+*y*^{2}+*z*^{2}subject to the constraints
*x* −*y* =*1 and y*^{2}−*z*^{2}=1.