Backgrounds

One of the core purposes in information theory is to protect the information when compressing and transmitting. In Shannon's seminal work [9], it was shown that the reliable communication over a channel is possible, provided that the transmission rate is below the channel capacity C, and an arbitrary large coding scheme is given. On the other hand, Sleipian and Wolf [10] studied a source compression scenario with an assistance of the side information. Let X denote the random variable

of the source and Y be that of the side information. They showed that the perfect source recovery is feasible as long as the compression rate is above the conditional entropy H(X|Y ) and an arbitrary large block code is provided.

Therefore, investigating the interplay between the compression/transmission rate, coding block-length and the probability of error is one of the fundamental problems in information theory. Based on dierent ranges of the error probability, analysis of the information processing performance roughly falls into the following three categories: (i) large error probability or non-vanishing error probability regime; (ii) medium error probability regime; and (iii) small error probability regime.

In the non-vanishing error probability regime, the largest code rate, given a coding length n and an error probability no more than , is one of the main research focuses. Strassen [11] rst demonstrated that the maximum size of an n-blocklength code through a discrete memoryless channel (DMC) W, denoted by M^∗(Wⁿ, ), yields an asymptotic expansion to the order √

n, and hence this is called second-order analysis:

log M^∗(Wⁿ, ) = nC +√

nV Φ⁻¹() + O(log n), (1.1)

where the quantities C and V denote the capacity [9] and the dispersion [12] of the channel, and Φ is the cumulative distribution function of a standard normal random variable. Equivalently, Eq. (1.1) yields the following relationship between the optimal decoding error with blocklength n and rate C − A/√

n for any constant A:

n→+∞lim ^∗ n, C − A/√ n
= Φ

 A

√ V



. (1.2)

Strassen's result relied on the Gaussian approximation or the central limit theorem (CLT), and is also called the small deviation regime. His work was latter rened by Hayashi [13], Polyanskiy et al. [12], and extended to quantum channels [14,15,16,7]. The results for higher-order asymptotics are referred to Refs. [17,18,19].

In the small error probability regime, Shannon [20] introduced the reliability function E(R) as the optimal error exponent:

n→+∞lim −1

nlog ^∗(n, R) = E(R), (1.3)

for rate R below the channel capacity¹ C. The quantity E(R) then provides a measure of how rapidly the error probability approaches zero with an increase in blocklength. This characterization of the reliability function is hence called the reliability function analysis or the error exponent analysis. This seminal work entails the analysis of a broad class of channels [22,21,23,24,25,26]. The exponential decay of the error probability in Eq. (1.3) is a consequence of the large deviation principle (LDP) [27].

In summary, the errors in Eqs. (1.2) and (1.3), respectively, fall into the CLT regime and large-deviation regime.

Given a classical channel, lower bounds for the reliability function (termed achievability), can be established by random coding arguments [28, 22, 29, 21]. However, upper bounds (also called

1To the best of our knowledge, the reliability function E(R) is only known in the high rate regime, i.e. at rates above a critical rate (see e.g. [21, p. 160]).

optimality) require dierent techniques since the code-dependent bounds on the error probability need to be optimized over all codebooks. The rst resultthe sphere-packing bound E(R) ≤ E_sp(R)was developed by Shannon, Gallager, and Berlekamp [30]. The sphere-packing exponent Esp(R) is dened as

E_sp(R) := sup

s≥0



maxP E0(s, P ) − sR



, (1.4)

where P is maximized over all probability distributions on the input alphabet, and E0(s, P ) is the auxiliary function or Gallager's function [29]. Unlike Shannon-Gallager-Berlekamp's technique which relates channel coding to binary hypothesis testing, Haroutunian [31, 24] employed a combinatorial method and obtained an upper bound for the reliability function in terms of the following expression

Ee_sp(R) := max

P min

W¯ D ¯WkW|P
: I(P, ¯W) ≤ R , (1.5) where ¯W is minimized over all dummy channels with the same output alphabet as W, D( ¯WkW|P ) is the conditional relative entropy between the dummy channel ¯W and the true channel W, and I(P, ¯W) is the mutual information of the channel ¯W (the detailed denitions are given in Chapter 3). It was later realized that the two quantities in Eqs. (1.4) and (1.5) are equivalent: they are related by convex program duality [32, 33, 25]. Therefore, these two expressions, Eqs. (1.4) or (1.5), are both called sphere-packing exponents.

Error exponent analysis in classical-quantum (c-q) channels is more challenging because of the noncommutative nature of quantum mechanics. Burnashev and Holevo [34] introduced a quantum version of the auxiliary function [35,36] and initialized the study of reliability functions in c-q channels.

However, the random coding bound (i.e. achievability) for c-q channels is still unsolved. Winter [37]

derived a sphere-packing bound (i.e. optimality) for c-q channels in the form of Ee_sp(R) in Eq. (1.5), generalizing Haroutunian's idea [31]. Dalai [38] employed Shannon-Gallager-Berlekamp's approach [30]

to establish a sphere-packing bound with Gallager's exponent in Eq. (1.4). In the follow-up work [39], Dalai and Winter pointed out that these two exponents are not equal in c-q channels. We remark that both Dalai and Winter's results are asymptotic and not nite blocklength.

The Slepian-Wolf coding with quantum side information (QSI) was studied by Devetak and Winter [40]. They generalized Slepian and Wolf's result [10] to the quantum case: the optimal probability of error asymptotically vanishes as the compression rate is above the quantum conditional entropy H(X|B)ρ, where B denotes the quantum system. Similar to the role of channel capacity in channel coding, we term H(X|B)ρthe Slepian-Wolf limit. The non-vanishing error probability regime was later studied by Renes and Renner [41], and Tomamichel and Hayashi [14]. A second-order asymptotics similar Eq. (1.1) was established.

The most paragraph of this thesis will focus on the error exponent analysis for both Slepian-Wolf coding with QSI and classical-quantum channel coding. We especially focus on the nite blocklegnth characterizations of the optimal error probability. In Chapters6and 7, we establish nite blocklength bounds for Slepian-Wolf coding with QSI. In Chapters 10 and 11, we review the best-to-date achiev-ability bound for c-q channel coding, and prove a tight sphere-packing bound in nite blocklengths.

The study of the medium error probability regime was pioneered by Altu§ and Wagner [42, 43].

verges to capacity suciently slowly. Specically, they studied under which conditions the error is asymptotically equal to²

^∗(n, C − an) ∼ Φ √nan

√v



∼ e^−na22vn, (1.6)

where the sequence of positive numbers (an)_n∈N satises (i) lim

n→+∞an= 0;

(ii) lim

n→+∞an

√n = +∞. (1.7)

Evidently, the transmission rate in Eq. (1.6) approaches capacity slower than 1/√

n. A DMC with errors satisfying Eq. (1.6) possesses a moderate deviation property (MDP) [27, Section 3.7], and hence it is also called the moderate deviation regime. The constant v in Eq. (1.6) equals the channel dispersion V when both the limit in Eq. (1.2) and MDP hold [44, Theorem 1]. We refer the interested readers to Refs. [44,45,46,47,43] for further results in classical channel coding.

As an application of our established error exponent bounds, we extend our techniques to the moderate deviation regime. In Chapters 8and 12, we demonstrate that the optimal error probability of the both two QIP tasks vanishes when the compression rate approaches the Slepian-Wolf limit and the transmission rate approaches the channel capacity, respectively. Specically, we show that

n→+∞lim

log ^∗(n, H(X|B)ρ+ an)

na²_n = − 1

2V; (1.8)

n→+∞lim

log ^∗(n, C − a_n)

na²_n = − 1

2V, (1.9)

where (an)_n∈N is any sequence satisfying Eq. (1.7).

We remark that these error probability regime described above(i), (ii), and (iii)all have the-oretical signicance and practical value. The non-vanishing error probability regime, (i), has been relatively well studied in the quantum scenario, while the small and medium error probability, (ii) and (iii), are rarely explored, which is the ultimate goal and purpose of this thesis. We summarize the error behaviors in these three regimes in Table 1.1.

Our methodology contains a varieties of matrix inequalities and matrix calculus. Moreover, we employ the sharp concentration inequalitiesBahadur-Ranga Rao's concentration inequality [48] and Chaganty-Sethuraman's concentration inequality [49]in strong large deviation theory to establish our

nite blocklength bounds. We collect the mathematical tools of matrix analysis and large deviation theory in Chapter2.