Basic definitions
An m × n matrix consists of mn real numbers arranged in m rows and n columns.
The entry in row i and column j of the matrix A is denoted by ai j. An m × 1 matrix is called a column vector of order m; similarly, a 1 × n matrix is a row vector of order n. An m × n matrix is called a square matrix if m = n.
Operations of matrix addition, scalar multiplication and matrix multiplication are basic and will not be recalled here. The transpose of the m × n matrix A is denoted by A0.
A diagonal matrix is a square matrix A such that ai j= 0, i 6= j. We denote the diagonal matrix
λ1 0 · · · 0 0 λ2· · · 0 ... ... . .. ... 0 0 · · · λn
by diag(λ1, . . . , λn). When λi= 1 for all i, this matrix reduces to the identity matrix of order n, which we denote by Inor often simply by I if the order is clear from the context. The matrix A is upper triangular if ai j= 0, i > j. The transpose of an upper triangular matrix is lower triangular.
1
Trace and determinant
Let A be a square matrix of order n. The entries a11, . . . , annare said to constitute the (main) diagonal of A. The trace of A is defined as
trace A = a11+ · · · + ann.
It follows from this definition that if A, B are matrices such that both AB and BA are defined, then
trace AB = trace BA.
The determinant of an n × n matrix A, denoted by det A, is defined as det A =
∑
σ
sgn(σ )a1σ (1)· · · anσ (n),
where the summation is over all permutations σ (1), . . . , σ (n) of 1, . . . , n, and sgn(σ ) is 1 or −1 according as σ is even or odd. We assume familiarity with the basic properties of determinant.
Vector spaces associated with a matrix
Let IR denote the set of real numbers. Consider the set of all column vectors of order n (n × 1 matrices) and the set of all row vectors of order n (1 × n matrices).
Both of these sets will be denoted by IRn. We will write the elements of IRneither as column vectors or as row vectors, depending upon whichever is convenient in a given situation. Recall that IRnis a vector space with the operations matrix addition and scalar multiplication.
Let A be an m × n matrix. The subspace of IRmspanned by the column vectors of Ais called the column space or the column span of A. Similarly the subspace of IRn spanned by the row vectors of A is called the row space of A.
According to the fundamental theorem of linear algebra, the dimension of the column space of a matrix equals the dimension of the row space, and the common value is called the rank of the matrix. We denote the rank of the matrix A by rank A.
For any matrix A, rank A = rank A0. If A and B are matrices of the same order, then rank(A + B) ≤ rank A + rank B. If A and B are matrices such that AB is defined, then rank AB ≤ min{rank A, rank B}.
Let A be an m × n matrix. The set of all vectors x ∈ IRnsuch that Ax = 0 is easily seen to be a subspace of IRn. This subspace is called the null space of A, and we denote it byN (A). The dimension of N (A) is called the nullity of A. Let A be an m× n matrix. Then the nullity of A equals n − rank A.
Minors
Let A be an m × n matrix. If S ⊂ {1, . . . , m}, T ⊂ {1, . . . , n}, then A[S|T ] will denote the submatrix of A determined by the rows corresponding to S and the columns cor-responding to T. The submatrix obtained by deleting the rows in S and the columns
1.1 Matrices 3 in T will be denoted by A(S|T ). Thus, A(S|T ) = A[Sc|Tc], where the superscript c denotes complement. Often, we tacitly assume that S and T are such that these ma-trices are not vacuous. When S = {i}, T = { j} are singletons, then A(S|T ) is denoted A(i| j).
Nonsingular matrices
A matrix A of order n × n is said to be nonsingular if rank A = n; otherwise the matrix is singular. If A is nonsingular, then there is a unique n × n matrix A−1, called the inverse of A, such that AA−1= A−1A= I. A matrix is nonsingular if and only if det A is nonzero.
The cofactor of ai jis defined as (−1)i+ jdet A(i| j). The adjoint of A is the n × n matrix whose (i, j)th entry is the cofactor of aji. We recall that if A is nonsingular, then A−1is given by 1
det Atimes the adjoint of A.
A matrix is said to have full column rank if its rank equals the number of columns, or equivalently, the columns are linearly independent. Similarly, a matrix has full row rank if its rows are linearly independent. If B has full column rank, then it admits a left inverse, that is, a matrix X such that X B = I. Similarly, if C has full row rank, then it has a right inverse, that is, a matrix Y such that CY = I.
If A is an m × n matrix of rank r then we can write A = BC, where B is m × r of full column rank and C is r × n of full row rank. This is called a rank factorization of A. There exist nonsingular matrices P and Q of order m × m and n × n, respectively, such that
A= P Ir 0 0 0
Q.
This is the rank canonical form of A.
Orthogonality
Vectors x, y in IRnare said to be orthogonal, or perpendicular, if x0y= 0. A set of vectors {x1, . . . , xm} in IRnis said to form an orthonormal basis for the vector space S if the set is a basis for S, and furthermore xi0xj is 0 if i 6= j, and 1 if i = j. The n× n matrix P is said to be orthogonal if PP0= P0P= I. One can verify that if P is orthogonal then P0is orthogonal.
If x1, . . . , xkare linearly independent vectors then by the Gram–Schmidt orthog-onalization process we may construct orthonormal vectors y1, . . . , yksuch that yiis a linear combination of x1, . . . , xi; i = 1, . . . , k.
Schur complement
Let A be an n × n matrix partitioned as
A= A11A12 A21A22
, (1.1)
where A11 and A22 are square matrices. If A11is nonsingular then the Schur com-plementof A11in A is defined to be the matrix A22− A21A−111A12. Similarly, if A22is nonsingular then the Schur complement of A22in A is A11− A12A−122A21.
The following identity is easily verified:
I 0
The following useful fact can be easily proved using (1.2):
det A = (det A11) det(A22− A21A−111A12). (1.3) We will refer to (1.3) as the Schur complement formula, or the Schur formula, for the determinant.
Inverse of a partitioned matrix
Let A be an n × n nonsingular matrix partitioned as in (1.1). Suppose A11is square and nonsingular and let A/A11= A22− A21A−111A12be the Schur complement of A11.
Note that if A and A11are nonsingular, then A/A11must be nonsingular. Equiva-lent formulae may be given in terms of the Schur complement of A22.
Cauchy–Binet formula
Let A and B be matrices of order m × n and n × m respectively, where m ≤ n. Then det(AB) =
∑
det A[{1, . . . , m}|S] det B[S|{1, . . . , m}],where the summation is over all m-element subsets of {1, . . . , n}.
To illustrate by an example, let
A= 2 3 −1