Properties of the Von Neumann entropy

Properties of the Von Neumann entropy

1. Purity. A pure state ρ = |ϕihϕ| has S(ρ) = 0.

2. Invariance. The entropy is unchanged by a unitary change of basis

S(UρU^†) = S(ρ),

because the entropy depends only on the eigenvalues of the density matrix.

3. Maximum. If ρ has D non-vanishing eigen- values, then

S(ρ) ≤ log D,

with equality when all nonzero eigenvalues are equal (maximum randomness).

1

4. Concavity. For λ_i ≥ 0 and ^P_i λ_i = 1, S(^X

i

λ_iρ_i) ≥ ^X

i

λ_iS(ρ_i).

That is, the Von Neumann entropy is larger if we know less about how the state was prepared.

5. Entropy of measurement. If we measure A = ^P_y a_y|a_yiha_y| in ρ, then outcome a_y oc- curs with probability p(a_y) = ha_y|ρ|a_yi. The Shannon entropy for the ensemble of mea- surements outcomes Y = {a_y, p(a_y)} satis- fies

H(Y ) ≥ S(ρ),

with equality when A and ρ commute. By measuring a non-commuting observable the results would be less predictable.

6. Entropy of preparation. For ρ = ^P_x p_x|ϕ_xihϕ_x| and X = {|ϕ_xi, p_x},

H(X) ≥ S(ρ),

with equality when the |ϕ_xi’s are mutually orthogonal. When the different states are not orthogonal then information received would be less then when different charac- ters are fully distinguishable.

7. Subadditivity. For a bipartite system AB in the state ρ_AB,

S(ρ_AB) ≤ S(ρ_A) + S(ρ_B),

with equality when ρ_AB = ρ_A⊗ρ_B. Entropy is additive for independent subsystems, but for correlated subsystems total entropy is less than the sum of the entropy of the subsystems. Similarly H(X, Y ) ≤ H(X) + H(Y ).

8. Strong subadditivity. For any state ρ_ABC of a tripartite system,

S(ρ_ABC) + S(ρ_B) ≤ S(ρ_AB) + S(ρ_BC).

When B is one dimensional this property reduces to subadditivity. This property may be viewed as the fact that the sum of the entropies of two systems’ union and inter- section does not exceed the sum of the entropies of the two systems.

9. Triangle inequality (Araki-Lieb inequal- ity). For a bipartite system

S(ρ_AB) ≥ |S(ρ_A) − S(ρ_B)|, in contrast to Shannon entropy

H(X, Y ) ≥ H(X), H(Y ) or

H(X|Y ), H(Y |X) ≥ 0.

There exists more information in the whole classical system than any part of it. But for quantum systems and Von Neumann entropy, we could have S(ρ_A) = S(ρ_B) and S(ρ_AB) = 0 in the case of a bipartite pure state. That is, for the whole system the state is completely known, yet consider- ing only one of the subsystems the mea- surement result could be complete random.

This is the consequence of quantum entan- glement.

If we could somehow define a conditional Von Neumann entropy, then negative en- tropies should result, leading to insights into quantum entanglement and measure- ment.

Quantum Data Compression

Consider a message composed of n letters, each chosen at random from the ensemble of pure states {|ϕ_xi, p_x}, where the states may not be orthogonal. Then each letter is described by the density matrix

ρ = ^X

x

p_x|ϕ_xihϕ_x|, and the entire message by

ρⁿ = ρ ⊗ ρ ⊗ · · · ⊗ ρ.

The message can be compressed to a Hilbert space of nS(ρ) dimensions, without decreasing the fidelity of the message.

So the Von Neumann entropy can be seen as the number of qubits of quantum information carried per letter by the message. Analogous to the classical case, when ρ = ¹₂1, the (com- pletely random) message could not be com- pressed.

2

Schumacher encoding

Similar to classical compression in which we only consider typical sequences, typical sub- spaces are considered in quantum messages.

That is, we can represent a given quantum message in the typical subspace of its Hilbert space, and throw away the orthogonal compo- nent.

Consider a quantum message ρⁿ = ρ⊗ρ⊗· · ·⊗ρ, where ρ = ^P_x p_x|ϕ_xihϕ_x|. In the orthonormal basis that diagonalizes ρ, the message can be seen as a classical source in which each letter is chosen from ρ’s eigenstates, with probabil- ity given by the eigenvalues. Then the typi- cal sequence of ρ eigenstates appearing in the message ρⁿ forms a typical subspace. That is, we need only consider the typical eigenstates of ρⁿ. Specifically, the eigenstates with eigen- value λ satisfying

2^{−n(S(ρ)−δ)} ≥ λ ≥ 2^{−n(S(ρ)+δ)}.

3

Each eigenstate of ρⁿ is a sequence of eigen- states of ρ, with eigenvalues given by the prod- uct of the corresponding eigenvalues of ρ.

There are 2^nS(ρ) typical sequences, each with probability (eigenvalue) λ satisfying (for a spec- ified δ)

2^{−n(S(ρ)−δ)} ≥ λ ≥ 2^{−n(S(ρ)+δ)}.

For any δ and  > 0 sufficiently large, the sum of the above typical eigenvalues satisfies tr (ρⁿE) > 1 − , (where E is the projection onto the typical subspace spanned by the typi- cal eigenstates of ρⁿ) and the dimension of the typical subspace Λ satisfies

2^nS(ρ)+δ ≥ dimΛ ≥ 2^{n(S(ρ)−δ)}.

The coding strategy is to send messages in the typical subspace faithfully. First the sender performs a unitary transformation that rotates the typical eigenstates of the message to the form U|Ψ_typi = |Ψ_compi|0_resti, where |Ψ_compi is a state of n(S(ρ) + δ) qubits, and |0_resti repre- sent |0i’s for all remaining qubits. The |Ψ_compi is send, and the receiver appends |0_resti and apply U⁻¹ to recover the original message.