2.3 Lagrangian Optimization
2.3.2 Optimization Problems with Single Constraint
Allocated Bits Water Line
Figure 2.5: Illustration of water …lling principle.
That is, the optimal solution includes the following steps:
1. Ingore the inequality constraints and bring constant slope into practice.
2. If all constraint are satis…ed then the optimal solution is achieved. Otherwise, adjusting the violated constraints by moveing them to the boundary values and no more optimization operations for them.
3. Update the equality constraint.
4. Repeat step 1, 2, 3 until the optimal solution is achieved or there is no suitable solution for current constrained problem.
A graphic illustration of the solution is shown in Figure 2.5, and it is commonly called a water …lling principle.
2.3.2 Optimization Problems with Single Constraint
In [6], a theorem is proposed to solve the budget constrained problem that for any real positive number , the Lagrange multiplier, if the mapping x (i) for i = 1; 2; :::; n minimizes
Xn i=1
dix(i)+ rix(i) then it is also the optimal solution to the problem
min Xn
i=1
dix(i), s:t:
Xn i=1
rix(i) Rt
Chapter 2. Constrained Optimization Problems : Principles and Applications
Figure 2.6: Illustration of optimization solution by minimal Lagrangian cost.
where
That is, referring to as the slope (Figure 2.6), for a …xed , we can obtain the best possible solution that meets the budget constraint Rt = Rtotal. And the is needed to iteratively change by bisection search algorithm [6][10] until we …nd the multiplier , such that the total number of used bits meets the original budget constraint, R( ) = Rtotal, within a convex hull approximation.
Therefore, we can transform the original constrained problem to unconstrained problem, and the solution by this constant slope algorithm is optimal for rate distortion trade-o¤. For example, the typical rate control problem is de…ned to minimize the total distortion
Pn i=1
Di(Q1; Q2; :::; Qi), where Qi 2 fq1; q2; :::; qNg, subject to the total rate/budget constraint Rtotal as follows:
min
and the optimal solution is equal to …nding Q , and to minimize
J (Q; ) =
Owing to that current coding unit can reference previous coded units to reduce
tem-Chapter 2. Constrained Optimization Problems : Principles and Applications
2 3
1 J1(2)
J1(1) J1(3)
R1 D1
J2(1,2) J2(2,2) J2(3,2)
R2 D2
Figure 2.7: Di¤erent quantizer choice for frame 1 leads to di¤erent R-D curve of frame 2, also the solution of minimal lagrangian cost to dependent problem.
poral redundancy in video coding, the quality of previous coded units will impacts the coding e¢ ciency of the following coding units. That is, the sum of minimal Lagrangian cost at each individual stage will not always result in the optimal solution, as Figure 2.7 [7]. Song et al: assumed the dependency relationship between sub-GOPs and propose a real-time system to optimize low-bitrate constrained problem with frame skipping, where the Lagrange multiplier and frames to be encoded for each sub-GOPs are pre-decided [11]. Schuster et al: also developed an MINMAX distortion criterion based on Lagrangian method to solve the minimum rate subject to each source distortion constrained dependent problem [9].
In order to solve this dependent problem, the Viterbi algorithm with Lagrangian cost is applied and we will introduce it in section 2.3.2.1.
2.3.2.1 Viterbi Algorithm for Dependent Problems
Viterbi algorithm [2] is a trellis-based forward dynamic programming procedure which iteratively determines possible shortest paths and prunes out non-optimal paths stage by stage.
For each stage, a node is an operating point of a quantizer, and a growing branch is connected from node at the previous stage to node at the current stage with corre-sponding Lagrangian cost J = D + R. The optimal solution is the path of minimal Lagrangian cost from the beginning stage to the end stage for a speci…c . When increases, the optimal path is tend to smaller the total coding bits, and vice versa.
Still, we have to iteratively …nd by bisection search until R( ) = Rtotal.
Chapter 2. Constrained Optimization Problems : Principles and Applications
Figure 2.8: Illustration of Viterbi algorithm with minimal Lagrangian cost path in IBBP case.
Ramchandran et al: applied VA algorithm with Lagrangian cost and propose prun-ing rules based on monotonicity property, as Figure 2.8 [7]. Assume the quantizer grades ordered from …nest to coarsest, for any 0, there exists monotonicity prop-erty that for i i0
J2(i; j) < J2(i0; j)
where quantizer j of frame 2 is dependent on quantizer i of frame 1. Afterwards, the pruning conditions based on monotonicity property are used to eliminate suboptimal operating points:
According to monotonicity property and pruning conditions, if J1(i) < J1(i0) for i < i0, then state node i0 can be pruned (for I frames).
In [4], Liu et al: also improved VA by considering frame skipping situation, as Figure 2.9 [4], and assumed monotonicity property with frame skipping brings into being for
Chapter 2. Constrained Optimization Problems : Principles and Applications
Figure 2.9: VA with skip nodes
any 0,
J (i; sij; j) J (i0; si0j; j); if i i0 J (i; sij; j) J (i; sij0; j0); if j j0
J (i; sij; j) J (i0; si0j0; j0); if i i0; j j0
where sij represents skipped frame reconstructed from forward coding frame with quantizer i and backward coding frame with quantizer j. Also, the new pruning rules are: can be pruned out.
2.3.2.2 Viterbi Algorithm for Independent Problems
Though dynamic programming can apply to …nding the optimal solution to depen-dent problems, it takes too much computation considering frame-to-frame dependency.
In order to reduce complexity, solution to dependent problem usually reduced to in-dependent problem; that is, take rate constrained problem for example, the problem formulation becomes
Chapter 2. Constrained Optimization Problems : Principles and Applications
The solution to independent problem only focuses on …nding the minimal La-grangian cost at current stage despite of the e¤ect of other stages.