Convex functions
3.5 Log-concave and log-convex functions
3.5.1 Definition
A function f : Rn → R is logarithmically concave or log-concave if f(x) > 0 for all x ∈ dom f and log f is concave. It is said to be logarithmically convex or convex if log f is convex. Thus f is convex if and only if 1/f is log-concave. It is convenient to allow f to take on the value zero, in which case we take log f (x) = −∞. In this case we say f is log-concave if the extended-value function log f is concave.
We can express log-concavity directly, without logarithms: a function f : Rn→ R, with convex domain and f (x) > 0 for all x∈ dom f, is log-concave if and only if for all x, y∈ dom f and 0 ≤ θ ≤ 1, we have
f (θx + (1− θ)y) ≥ f(x)θf (y)1−θ.
In particular, the value of a log-concave function at the average of two points is at least the geometric mean of the values at the two points.
From the composition rules we know that eh is convex if h is convex, so a log-convex function is log-convex. Similarly, a nonnegative concave function is log-concave.
It is also clear that a log-convex function is quasiconvex and a log-concave function is quasiconcave, since the logarithm is monotone increasing.
Example 3.39 Some simple examples of log-concave and log-convex functions.
• Affine function. f(x) = aTx + b is log-concave on{x | aTx + b > 0}.
• Powers. f(x) = xa, on R++, is log-convex for a≤ 0, and log-concave for a ≥ 0.
• Exponentials. f(x) = eaxis log-convex and log-concave.
• The cumulative distribution function of a Gaussian density, Φ(x) = 1
√2π Z x
−∞
e−u2/2du,
is log-concave (see exercise 3.54).
• Gamma function. The Gamma function, Γ(x) =
Z ∞ 0
ux−1e−udu, is log-convex for x≥ 1 (see exercise 3.52).
• Determinant. det X is log concave on Sn++.
• Determinant over trace. det X/ tr X is log concave on Sn++(see exercise 3.49).
Example 3.40 Log-concave density functions. Many common probability density functions are log-concave. Two examples are the multivariate normal distribution,
f (x) = 1
p(2π)ndet Σe−12(x−¯x)TΣ−1(x−¯x)
(where ¯x∈ R and Σ∈ S++), and the exponential distribution on R+,
(where λ 0). Another example is the uniform distribution over a convex set C, f (x) =
½ 1/α x∈ C 0 x6∈ C
where α = vol(C) is the volume (Lebesgue measure) of C. In this case log f takes on the value−∞ outside C, and − log α on C, hence is concave.
As a more exotic example consider the Wishart distribution, defined as follows. Let x1, . . . , xp ∈ Rn be independent Gaussian random vectors with zero mean and co-variance Σ∈ Sn, with p > n. The random matrix X =Pp
i=1xixTi has the Wishart density
f (X) = a (det X)(p−n−1)/2e−12tr(Σ−1X),
with dom f = Sn++, and a is a positive constant. The Wishart density is log-concave, since
log f (X) = log a +p− n − 1
2 log det X−1
2tr(Σ−1X), which is a concave function of X.
3.5.2 Properties
Twice differentiable log-convex/concave functions Suppose f is twice differentiable, with dom f convex, so
∇2log f (x) = 1
f (x)∇2f (x)− 1
f (x)2∇f(x)∇f(x)T. We conclude that f is log-convex if and only if for all x∈ dom f,
f (x)∇2f (x)º ∇f(x)∇f(x)T, and log-concave if and only if for all x∈ dom f,
f (x)∇2f (x)¹ ∇f(x)∇f(x)T. Multiplication, addition, and integration
Log-convexity and log-concavity are closed under multiplication and positive scal-ing. For example, if f and g are log-concave, then so is the pointwise product h(x) = f (x)g(x), since log h(x) = log f (x) + log g(x), and log f (x) and log g(x) are concave functions of x.
Simple examples show that the sum of log-concave functions is not, in general, concave. Log-convexity, however, is preserved under sums. Let f and g be log-convex functions, i.e., F = log f and G = log g are log-convex. From the composition rules for convex functions, it follows that
log (exp F + exp G) = log(f + g)
is convex. Therefore the sum of two log-convex functions is log-convex.
More generally, if f (x, y) is log-convex in x for each y∈ C then g(x) =
Z
C
f (x, y) dy is log-convex.
Example 3.41 Laplace transform of a nonnegative function and the moment and cumulant generating functions. Suppose p : Rn→ R satisfies p(x) ≥ 0 for all x. The Laplace transform of p,
P (z) = Z
p(x)e−zTxdx,
is log-convex on Rn. (Here dom P is, naturally,{z | P (z) < ∞}.) Now suppose p is a density, i.e., satisfiesR
p(x) dx = 1. The function M (z) = P (−z) is called the moment generating function of the density. It gets its name from the fact that the moments of the density can be found from the derivatives of the moment generating function, evaluated at z = 0, e.g.,
∇M(0) = E v, ∇2M (0) = E vvT, where v is a random variable with density p.
The function log M (z), which is convex, is called the cumulant generating function for p, since its derivatives give the cumulants of the density. For example, the first and second derivatives of the cumulant generating function, evaluated at zero, are the mean and covariance of the associated random variable:
∇ log M(0) = E v, ∇2log M (0) = E(v− E v)(v − E v)T.
Integration of log-concave functions
In some special cases log-concavity is preserved by integration. If f : Rn×Rm→ R is log-concave, then
g(x) = Z
f (x, y) dy
is a log-concave function of x (on Rn). (The integration here is over Rm.) A proof of this result is not simple; see the references.
This result has many important consequences, some of which we describe in the rest of this section. It implies, for example, that marginal distributions of log-concave probability densities are log-log-concave. It also implies that log-concavity is closed under convolution, i.e., if f and g are log-concave on Rn, then so is the convolution
(f∗ g)(x) = Z
f (x− y)g(y) dy.
(To see this, note that g(y) and f (x−y) are log-concave in (x, y), hence the product f (x− y)g(y) is; then the integration result applies.)
Suppose C ⊆ R is a convex set and w is a random vector in R with log-concave probability density p. Then the function
f (x) = prob(x + w∈ C) is log-concave in x. To see this, express f as
f (x) = (which is log-concave) and apply the integration result.
Example 3.42 The cumulative distribution function of a probability density function f : Rn→ R is defined as
where w is a random variable with density f . If f is concave, then F is log-concave. We have already encountered a special case: the cumulative distribution function of a Gaussian random variable,
f (x) = 1
√2π Z x
−∞
e−t2/2dt,
is log-concave. (See example 3.39 and exercise 3.54.)
Example 3.43 Yield function. Let x∈ Rn denote the nominal or target value of a set of parameters of a product that is manufactured. Variation in the manufacturing process causes the parameters of the product, when manufactured, to have the value x + w, where w ∈ Rn is a random vector that represents manufacturing variation, and is usually assumed to have zero mean. The yield of the manufacturing process, as a function of the nominal parameter values, is given by
Y (x) = prob(x + w∈ S),
where S⊆ Rndenotes the set of acceptable parameter values for the product, i.e., the product specifications.
If the density of the manufacturing error w is log-concave (for example, Gaussian) and the set S of product specifications is convex, then the yield function Y is log-concave.
This implies that the α-yield region, defined as the set of nominal parameters for which the yield exceeds α, is convex. For example, the 95% yield region
{x | Y (x) ≥ 0.95} = {x | log Y (x) ≥ log 0.95}
is convex, since it is a superlevel set of the concave function log Y .
Example 3.44 Volume of polyhedron. Let A∈ Rm×n. Define Pu={x ∈ Rn| Ax ¹ u}.
Then its volume vol Puis a log-concave function of u.
To prove this, note that the function Ψ(x, u) =
½ 1 Ax¹ u 0 otherwise, is log-concave. By the integration result, we conclude that
Z
Ψ(x, u) dx = vol Pu
is log-concave.