f is differentiable at p ∈ U if there exists a linear function L : Rn → Rm such that x→plim kf (x

Differentiation in Rⁿ

Let U be an open subset of Rⁿ and let f : U → R^m be a map defined on U with values in R^m. 1. f is differentiable at p ∈ U if there exists a linear function L : Rⁿ → R^m such that

x→plim

kf (x) − f (p) − L(x − p)k

kx − pk = 0.

2. If f is differentiable at p, then it is (locally) Lipschitz near p.

3. If the linear function exists at p, then it is unique.

4. If f is differentiable at p, then each of the partial derivatives ∂f_i

∂x_j(p), 1 ≤ i ≤ m and 1 ≤ j ≤ n, exists and

∂f_i

∂x_j(p) = the i-th compnent of L(e_j) ∀ 1 ≤ i ≤ m, 1 ≤ j ≤ n.

5. If f is differentiable at p and if u is any element of Rⁿ, then the partial derivative D_uf (p) of f at p with respect to u exists. Moreover,

D_uf (p) = Df (p)(u) = Df (p)(

n

X

i=1

u_ie_i) =

n

X

i=1

u_iDf (p)(e_i)

and it is called the directional derivative of f at p in the direction of u provided that u is a unit vector of Rⁿ.

Properties of the Derivative

Let U be an open subset of Rⁿ, let f, g : U → R^m be maps defined on U with values in R^m, and let ϕ : U → R a function defined on U.

1. If f and g are differentiable at p ∈ U, and if α, β ∈ R, then h = αf + βg is differentiable at p and

Dh(p) = αDf (p) + βDg(p).

2. If ϕ and f are differentiable at p ∈ U, then k = ϕf : E → R^m is differentiable at p and Dk(p)(u) =Dϕ(p)(u)f (p) + ϕ(p)Df (p)(u) for u ∈ Rⁿ.

3. If f and g are differentiable at p ∈ U, then h = hf, gi =

m

X

i=1

figi : E → R is differentiable at p and

Dh(p) = g(p) · Df (p) + f (p) · Dg(p) =

m

X

i=1

Df_i(p)g_i(p) +

m

X

i=1

f_i(p)Dg_i(p).

Chain Rule Let f have domain E ⊆ Rⁿ and range in R^m, and let g have domain B ⊆ R^m and range in R^`. Suppose that f is differentiable at p and that g is differentiable at q = f (p). Then the composition h = g ◦ f is differentiable at p and

Dh(p) = Dg(q) ◦ Df (p).

Alternatively, we write

Mean Value Theorems

1. Let f be defined on an open subset U of Rⁿ and have values in R. Suppose that the set U contains the points a, b and the line segment S joining them and that f is differentiable at every point of this segment. Then there exists a point c on S such that

f (b) − f (a) = Df (c)(b − a).

2. Let U ⊆ Rⁿ be an open set and let f : U → R^m. Suppose that the set U contains the points a, b and the line segment S joining them and that f is differentiable at every point of this segment. Then there exists a point c on S such that

kf (b) − f (a)k ≤ kDf (c)(b − a)k.

Higher Derivatives

Let U be an open subset in Rⁿ and let f : U → R be a differentiable function on U.

1. If Df : U → Rⁿ is differentiable, we denote the derivative of Df by D²f : U → R^n×n, and we shall refer the n² functions (entries) of D²f as the second partial derivatives of f denoted by

D_ijf or ∂²f

∂xi∂xj

, i, j = 1, 2, . . . , n.

Similarly, if D²f : U → Rⁿ² is differentiable, we denote the derivative of D²f by D³f : U → Rⁿ³, and we shall refer the n³ functions (entries) of D³f as the third partial derivatives of f denoted by

D_ijkf or ∂³f

∂x_i∂x_j∂x_k, i, j, k = 1, 2, . . . , n.

2. Note that if f is differentiable at a point p ∈ U, then Df (p) is a linear map on Rⁿ to R defined by

Df (p)(z) =

n

X

i=1

D_if (p)z_i =

n

X

i=1

∂f

∂x_i(p)z_i for z = (z₁, . . . , z_n) ∈ Rⁿ.

If Df is differentiable at a point p ∈ U, then D²f (p) is a linear map on Rⁿ² to R defined by

D²f (p)(y, z) =

n

X

i, j=1

D_ijf (p)y_jz_i =

n

X

i, j=1

∂²f

∂x_i∂x_j(p)y_jz_i for y, z ∈ Rⁿ.

Similarly, if D²f is differentiable at a point p ∈ U, then D³f (p) is a linear map on Rⁿ³ to R defined by

D³f (p)(y, z, w) =

n

X

i, j, k=1

D_ijkf (p)y_kz_jw_i =

n

X

i, j, k=1

∂³f

∂x_i∂x_j∂x_k(p)y_kz_jw_i for y, z, w ∈ Rⁿ.

3. Taylor’s Theorem Suppose that f has continuous partial derivatives of order m in a neighborhood of every point of a line segment S joining two point a, b = a + u in U. Then there exists a point c on S such that

f (a + u) = f (a) +

m−1

X

k=1

1

k!D^kf (a)(u)^k+ 1

m!D^mf (c)(u)^m where D^kf (p)(u)^k= D^kf (p)(u, u, . . . , u) for k = 1, . . . , m.

The Class C¹(U )

Let U be an open subset in Rⁿ, let f : U → R^m be differentiable on U and let D_jf_i = ∂f_i

∂xj

. 1. A function L : Rⁿ → R^m is said to be linear if

L(ax + by) = aL(x) + bL(y) for all a, b ∈ R and x, y ∈ Rⁿ. Note that if L : Rⁿ→ R^m is linear, then L(0) = 0 ∈ R^m.

2. Let L (Rⁿ, R^m) = {L | L : Rⁿ → R^m is linear}. Define (∗) kLknm= sup

z∈Rⁿ, kzk≤1

kL(z)k for each L ∈L (Rⁿ, R^m).

Since L ∈L (Rⁿ, R^m) is linear,

kLk_nm = sup

z∈Rⁿ, kzk≤1

kL(z)k = sup

z∈Rⁿ, kzk=1

kL(z)k.

If L ∈L (Rⁿ, R^m) is represented by an m × n matrix, then kLk_nm= sup

z∈Rⁿ, kzk=1

kL(z)k = the maximum length of column vectors of L and the derivative of L equals itself, i.e. DL = L.

3. f ∈ C¹(U ) if the derivative Df (x) exists and is continuous under the norm (∗) for all x ∈ U, i.e.

lim

y∈U, y→xkDf (y) − Df (x)k_nm= 0 for all x ∈ U.

4. For all x, y ∈ U,

kDf (x) − Df (y)k_nm ≤ ( _m

X

i=1 n

X

j=1

|D_jf_i(x) − D_jf_i(y)|² )1/2

≤√

mkDf (x) − Df (y)k_nm.

5. f ∈ C¹(U ) if and only if D_jf_i is continuous on U for each 1 ≤ i ≤ m, 1 ≤ j ≤ n.

6. If f ∈ C¹(U ), then det Df = det[D_jf_i] : U → R is continuous on U.

7. Suppose that U contains the points a, b and the line segment S joining them, and let p ∈ U.

Then we have

kf (b) − f (a) − Df (p)(b − a)k ≤ kb − ak sup{kDf (x) − Df (p)k }.

8. If f ∈ C¹(U ), p ∈ U and ε > 0, then there exists δ(ε) > 0 such that if kx_k− pk ≤ δ(ε), k = 1, 2, then x_k ∈ U and

kf (x₁) − f (x₂) − Df (p)(x₁− x₂)k ≤ ε kx₁− x₂k.

9. If L = Df (p) is an injection for some p ∈ U, then there exists r > 0 such that (†) rkuk ≤ kDf (p)(u)k = kL(u)k for all u ∈ Rⁿ.

10. If f ∈ C¹(U ) and if L = Df (p) is an injection for some p ∈ U, then there exists a δ > 0 such that

(††)¹₂r kx₁− x₂k ≤ kf (x₁) − f (x₂)k for all x₁, x₂ ∈ ¯B_δ(p) ⊂ U.

This prove that the restriction of f to ¯B_δ(p) is an injection; hence this restriction has an inverse function which we shall denote by g = (fB¯_δ(p))⁻¹. Note that g is uniformly continuous on f ( ¯B_δ(p)) by using (††).

11. If f ∈ C¹(U ) and if L = Df (p) is a surjection for some p ∈ U, then there exists a bounded linear function M : R^m → Rⁿ such that L ◦ M (y) = y for all y ∈ R^m.

12. If f ∈ C¹(U ) and if L = Df (p) is a surjection for some p ∈ U, then there exists numbers d > 0 and α > 0 such that if y ∈ ¯B_α/2d(f (p)) ⊂ R^m, then there exists an x ∈ U ∩ ¯B_α(p) such that f (x) = y.

13. If f ∈ C¹(U ) and if L = Df (p) is a surjection for all p ∈ U, then f (U ) is open in R^m. 14. If m = n, f ∈ C¹(U ) and if L = Df (p) is a bijection for some p ∈ U, then there exist

open neighborhoods V and W of p and f (p), respectively, such that f : V → W has a differentiable inverse g = f⁻¹ : W → V such that

Df⁻¹(y) = Dg(y) = [Df (g(y))]⁻¹ = [Df (f⁻¹(y))]⁻¹ for y ∈ W.

15. Let W be an open subset of R^n+m, f : W → R^m ∈ C¹(W ) and let (a, b) ∈ W be a point at which f (a, b) = 0. Suppose that a ∈ Rⁿ, b ∈ R^m and the m × m matrix

Dyf |_(a,b)= ∂f_i

∂y_j(a, b)



1≤i,j≤m

is invertible.

Then there exist open neighborhoods A ⊂ Rⁿ and B ⊂ R^m of a and b, respectively, and a unique g : A → R^m ∈ C¹(A) such that

b = g(a) and F (x, g(x)) = 0 for all x ∈ A and

{(x, y) ∈ A × B ⊂ W | f (x, y) = 0}

= {(x, g(x)) | x ∈ A}

= the graph of g over A.

Extremum problems

1. Let U ⊆ Rⁿ be open and let f : U → R have continuous second partial derivatives on U. If p ∈ U is a point of relative minimum [respectively, maximum] of f, then

D²f (p)(w)² = w^t  ∂f_i

∂xj

(p)



1≤i,j≤n

w =

n

X

i, j=1

D_ijf (p)w_iw_j ≥ 0 for all w ∈ Rⁿ

[respectively, D²f (p)(w)² =

n

X

i, j=1

D_ijf (p)w_iw_j ≤ 0 for all w ∈ Rⁿ].

2. Lagrange’s Theorem Let U ⊆ Rⁿ be open and suppose that f and g : U → R are real- valued functions in C¹(U ). Suppose p ∈ U is such that g(p) = 0 and that there exists an r > 0 such that

f (x) ≤ f (p) or f (x) ≥ f (p) for all x ∈ B_r(p) ⊂ U satisfying g(x) = 0.

Then there exist real numbers µ, λ, not both zero, such that µDf (p) = λDg(p).

Moreover, if Dg(p) 6= 0, we can take µ = 1.

Riemann Integrability

1. (Riemann Criterion for Integrability) f is Riemann integrable over K, i.e.

Z

K

f = Z

K

f, if and only if for each ε > 0, there exists a partition P_ε of K such that

|U (P_ε, f ) − L(P_ε, f )| < ε.

2. (Cauchy Criterion for Integrability) f is Riemann integrable over K if and only if for each ε > 0, there exists a partition P_ε of K such that if P and Q are refinements of P_ε and S_P(f, K) and S_Q(f, K) are any corresponding Riemann sums, then

|S_P(f, K) − S_Q(f, K)| < ε.

Convergence and Differentiation

1. Let f_n: [a, b] → C be a sequence of continuously differentiable functions defined on I = [a, b].

If f_n(x₀) converges for some x₀ ∈ [a, b] and if f_n⁰ : [a, b] → C converges uniformly on [a, b], then fn converges uniformly on [a, b] to a function f.

2. Term-by-Term Differentiation For each n ∈ N, let fn be a real-valued function on K = [a, b] which has a derivative f_n⁰ on K. Suppose that

(i) P f_n converges at x₀ ∈ K, (ii) P f_n⁰ converges uniformly on K,

there exists a real-valued function f on K such that (a) P f_n converges uniformly on K to f,

(b) f is differentiable on K and f⁰ =X

f_n⁰ on K.

Convergence and Integration

1. Let {f_n} be a sequence of integrable functions that converges uniformly on a closed cell K ⊂ R^p to a function f. Then f is integrable and

Z

K

f = lim

n→∞

Z

K

f_n.

2. Term-by-Term Integration For each n ∈ N, let fn be a real-valued integrable function on K = [a, b]. Suppose that the seriesP f_n converges to f uniformly on K.

Then f is integrable on K and

Z

K

f =

∞

X

j=1

Z

K

f_n.

3. Bounded Convergence Theorem Let {f_n} be a sequence of integrable functions on a closed cell K ⊂ R^p. Suppose that there exists B > 0 such that kf_n(x)k ≤ B for all n ∈ N, x ∈ K. If the function f (x) = lim

n→∞fn(x), x ∈ K, exists and is integrable, then Z

K

f = lim

n→∞

Z

K

f_n.

4. Monotone Convergence Theorem Let {f_n} be a monotone sequence of integrable func- tions on a closed cell K ⊂ R^p. If the function f (x) = lim f_n(x), x ∈ K, exists and is integrable, then

Z

K

f = lim

n→∞

Z

K

f_n.