Optimal Solution Based on Dynamic Programming

Although the proposed Lagrangian-based method is efficient and theoretically sound, it optimizes the upper bound of the true objective function. In this section, we propose another optimal bit allocation method based on dynamic programming (DP). Although the proposed method uses more memory and requires more computation time than the Lagrangian-based method, it can find the optimal bit allocation for the true objective function.

To solve the bit-allocation problem with the DP-based method, we represent the problem as an acyclic directed graph G= (N,A), called a DP graph for short, where N is the set of nodes and the members of A are arcs. The arc from node i_k to node i_l is represented by i_k→ il where i_k and i_l are the source node and the sink node of the arc respectively. A path can be represented as a concatenation of arcs. For example, i₀→ i1→ i2··· → iK is a path, where the arc i_k→ ik+1is concatenated to the end of the arc i_k−1→ ik for 1≤ k ≤ K − 1.

Let seq be a bijection mapping from the subbands to the integer set from 0 to

|W|−1, where |W| is the number of subbands; and let seq⁻¹be its inverse mapping from

an integer to a subband. To construct the DP graph for the problem (P), we arrange the subbands z= (xy,mn) as a sequence seq(z) ∈ {0,··· ,|W|−1}, and divide bv−1into M=

^bv_ε⁻¹ components. For convenience, we set the bit constraint br with r= 0,··· ,v − 2 as an integer multiple of ε. Then, we introduce the function q, which maps a node in the DP graph to its corresponding quality resolution.

The nodes in the DP graph are {(seq(z),k)| k = 0,1,··· ,M,z ∈ W}. The arcs are constructed as follows. If we let(a,b) be the source node of the arc (a,b) → (seq(z),k) to the sink node(seq(z),k), then

(a,b) ∈

⎧⎪

⎪⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎪⎪

⎩

{(seq(z) − 1,i)|i ≤ k} ∪ {(seq(z),k − 1)} if seq(z) ≥ 1 and k ≥ 1,

{(0,k − 1)} if seq(z) = 0 and k ≥ 1,

{(seq(z) − 1,0)} if seq(z) ≥ 1 and k = 0.

(3.36)

Figure 3.3 shows a constructed DP graph. The x-axis indicates the subbands and the y-axis indicates the rate. For a node (x,k), the corresponding subband is seq⁻¹(x) and the corresponding quality resolution is q(k) = ri+1. We can calculate the num-ber of bits assigned to the subband associated with the sink node of each arc in the DP graph as well as the weighted distortion of the subband. For example, if the arc is (u,ku) → (v,kv), then the number of bits assigned to the subband seq⁻¹(v) is (kv−ku)ε^.

We can also calculate the number of bits assigned to each subband on any path in the DP graph. Let us use the path {(0,k0) → (1,k1)··· → (m,km)} as an example. We can derive that k₀ε bits are assigned to subband seq⁻¹(0), (k1− k0)ε bits are assigned to subband seq⁻¹(1), ···, and (km− km−1)ε bits are assigned to subband seq⁻¹(m).

Recall that if l bits are assigned to subband z, we represent it as β[z] = l. There-fore, for the path in the above example, we have β[seq⁻¹(0)] = k0ε,β[seq⁻¹(1)] = (k1− k0)ε,··· ,β[seq⁻¹(m)] = (km− km−1)ε.

Figure 3.3: Construction of a DP Graph.

The DP approach uses|W| passes to solve the problem (P) with the constraint that at the end of pass i, the optimal path to each node in the DP graph indexed by(i,.) (ex:

(i,3) represents the third node in the i-th pass) is found and recorded. At the first pass 0,

we find the number of bits allocated to any node(0, j) with j ∈ {0,··· ,M − 1}. Then, based on that result, at pass 1, we find and record the optimal bit allocation path (the path with the smallest weighted distortion among all the paths that end at the node) to each node(1,.). Based on the result of the previous pass i−1, we can repeat the process to derive the optimal bit allocation path to any node(i,.) and record the path to the node.

Note that, in practice, we do not record the path; instead, we only record the source node of the last arc along the optimal path to a node. After the pass|W|−1, the bit allocation corresponding to the optimal path from (0,0) to (|W| − 1,kl) with klε = br for some b_r is the optimal bit allocation to the quality resolution r; i.e., the optimal solution for

problem (P_r) is derived.

The DP approach ensures that there is only one optimal path beginning at(0,0) to any node in the DP graph. The optimal path to node(i,k) is the path with the minimum distortion among all the paths leading to the node(i,k). Let Dp(p) denote the distortion of the path p. In addition, let p(i,k) be the path that leads to node (i,k), and let Γ(i,k) denote the set of all paths that lead to node(i,k). Then, the optimal path to (i,k) is the path that minimizes:

p(i,k)∈Γ(i,k)min Dp(p(i,k)). (3.37)

Let the set of source nodes on the arcs with the sink node (i,k) be represented by N (i,k); i.e., any path leading to (i,k) must pass through a node in N (i,k). Recall that

N (i,k) =

⎧⎪

⎪⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎪⎪

⎩

{(i − 1, j)| j ≤ k} ∪ {(i,k − 1)} if i,k ≥ 1,

{(0,k − 1)} if i= 0 and k ≥ 1,

{(i − 1,0)} if i≥ 1 and k = 0.

(3.38)

The distortion of a path to (i,k) can be derived recursively from the distortion of the optimal path to any node in N (i,k) and the distortion from that node to node (i,k).

The distortion calculation is based on Equations (3.12) and (3.10). If a new node with a corresponding subband z and l bits are added to the current path, then, based on Equa-tion (3.10), the term w^s,t_z D¯^w_z(l) is added to the current Θ(s,t,βr) in order to obtain the newΘ(s,t,βr). The distortion of the new path is calculated from the new Θ(s,t,βr) by Equation (3.12), as shown by the following example. Let the current path be(0,k0) → (1,k1) → ··· → (i,ki), and let q(ki) = r (the last node that belongs to the quality resolu-tion r). The bit allocaresolu-tion profile for the path is [βseq⁻¹(0)(k0ε),βseq⁻¹(1)((k1− k0)ε),

··· ,βseq⁻¹(i)((ki− ki−1)ε), βseq⁻¹(i+1)(0),··· ,βseq⁻¹(|W|−1)(0)]. From the profile, we

can derive the current Θ(s,t,βr) and the weighted distortion corresponding to the new path obtained by appending a new node (i + 1,ki+1) to the end of the current path.

Without loss of generality, we assume that the new node belongs to the quality resolu-tion r; then the bit allocaresolu-tion profile for the new path is[βseq⁻¹(0)(k0ε),βseq⁻¹(1)((k1− k₀)ε),··· ,βseq⁻¹(i)((ki−ki−1)ε),βseq⁻¹(i)((ki+1−ki)ε)βseq⁻¹(i+2)(0),··· ,βseq⁻¹(|W|−1)(0)].

We can derive the newΘ(s,t,βr) and the weighted distortion of the new path from the profile.

To verify that the DP approach solves the problem (P) optimally, we need to show that the respective optimal solution paths of (P_r) and (P_r−1) satisfy βr[z] ≥βr−1[z] for all z∈ W. Let the optimal solution paths of (Pr−1) and (Pr) be (0,k0) → (1,k1) →

··· → (|W|−1,k_|W|−1) and (0,l0) → (1,l1) → ··· → (|W|−1,l_|W|−1) respectively. Then,

k₀,k1,··· ,k_|W|−1and l₀,l1,··· ,l_|W|−1are non-decreasing sequences. We will show that ki≥ li for all i. If we assume that this is not true, then ki₀ < li₀ for some i0. Because k_|W|−1> l_|W|−1, the two non-decreasing sequences must intersect at some j> i0. This corresponds to the two optimal solution paths intersecting at the node( j,kj) with kj= lj, which is impossible because each node in the DP graph can only have one optimal path from(0,0) passing through it. Thus, we conclude that ki≥ lifor all i, as shown in Figure 3.4. This proves thatβr[z] ≥βr−1[z] for all z ∈ W; therefore, the DP method solves the problem (P) optimally.

At each pass of the DP method, we need to update the distortion of M nodes; and for each node, we need to compare at most M paths leading to it. Thus, the complexity of the algorithm isO(M²|W|). If our wavelet codec has p spatial decompositions and q temporal decompositions, then|W| is equal to 2^q(3p + 1). Meanwhile, M is inversely

Figure 3.4: Optimal Rate-Distortion Path to Quality Resolution ri.

propositional toε, so the value ofεrepresents a trade-off between the granularity of the bit allocation procedure and the computational complexity.