, where Vbottomand Vtopare the set of bottom and top boundary vertices respectively.
Equation 3.17 and 3.18 can be done by substituting the value to the linear system. However, to make it simple to implement, the two equations can be rewritten as a soft constraint with large weights: If the weight of this constraint is large enough, then the solution of those boundary vertices will be very close to the boundary.
3.4.4 Total Energy Function
The total energy function of this work can be simply combined as the weighted sum of the four energy functions described previously.
E(W0) = wcEc(W0) + wgEg(W0) + wsEs(W0) + wbEb(W0), (3.20) where wc, wg, wsand wbare weights.
The above equation can be viewed as a linear function of W0 since V0 is a subset of W0, and it can be represented in the matrix form:
E(W0) = kQW0 − bk2, (3.21)
where Q is a large sparse matrix.
3.5 Optimization and Foldover Prevention
Minimizing the objective function of Equation 3.21 can be viewed as a linear least-squares problem because it is a quadratic function of x. Then we solve this problem using conjugate gradient method.
3.5 Optimization and Foldover Prevention 20
We observed that the deformed mesh we obtained might have foldover problem, which is common to see in most warping methods. Since the strength of structure preservation in each region is distinct, this situation occurred seems reasonable. If there are foldover triangles on the deformed mesh, the result obviously has visible distortion since it causes discontinuity in the mesh.
To deal with this problem, we detect on the deformed mesh and record the foldover triangles.
Then we add constraints on those triangles to prevent the foldover triangles.
Given an original mesh and a deformed mesh, the deformation matrix of each triangle on the mesh can be easily determined by solving Equation 3.15 on each triangle. Because the deforma-tion matrix of a triangle is an affine matrix, it can be factored into a rotadeforma-tion part and a scale part using polar decomposition. There are several ways to compute a polar decomposition. One way is to use the results of Singular Value Decomposition (SVD). However, using SVD to compute a polar decomposition is expensive. An alternative way is solved by a Newton algorithm [23].
What is more, there is a relatively simple 2D approximation of polar decomposition [24]. The rotation part R of an affine matrix Macan be computed as follow:
Given an affine matrix scaled by a factor that makes column unit vectors.
When the rotation matrix is extracted, the scale part can be easily determined as S = MaR−1.
It should be noted that the extraction of rotation part might include a reflection. If the rotation part is not a pure rotation, the reflection should be extracted. To detect the reflection, just check the diagonal elements of the rotation matrix since the rotation matrix is in the form
−b aa b. Ideally, it should be the same sign of the diagonal since the two diagonal elements
3.5 Optimization and Foldover Prevention 21
(a) (b)
(c) (d)
Figure 3.6: An example of foldover. Green triangles represent foldover triangles. (a) Resized image with foldover. (b) Resized mesh with foldover. (c) Resized image without foldover. (b) Resized mesh without foldover.
3.5 Optimization and Foldover Prevention 22
should be equal. If one diagonal element is positive and the other one is negative, it means the rotation matrix includes a reflection. Then we extract it and multiply it to the scaling matrix.
By observing the elements in the scaling matrix, it can determine whether the triangle is flipped and determine the flipping direction. If the first diagonal element is negative, it means the triangle flips in x-direction. Likewise, flipping in y-direction can be observed in the second diagonal element.
To encourage foldover triangles not to flip, a desired transformation matrix Mdesirei for each flipping triangle i is defined. We wish the foldover triangles to be deformed as what we ex-pected.
The desired transformation matrix Mdesirei of triangle i can be determined by modifying the original transformation matrix Mi. The matrix Mi has been divided into a rotation matrix Ri
and a scaling matrix Si. Flipping means the scale factor of x-axis or y-axis is negative. Thus, to discourage the triangle flipping, it can be done by encouraging the scale factor of the triangle to be a positive value. A threshold of scale factor ts can be defined by the user. When the scale factor of x-axis or y-axis is negative, we modify it to be ts. Then Mdesirei can be computed as the product of rotation matrix Ri with the modified scaling matrix.
Some triangles might be extremely squeezed but no flipping. For those triangles, it is desir-able to constrain their scale factors to be the threshold value ts. Thus, besides flipping triangles, we also detect the over-squeezed triangles whose scale factors are less than tsand treat them as foldover triangles.
Therefore, the energy function on the foldover triangles can be defined as:
Ef(V0) = X
i∈Tm
kMdesirei − M0ik2, (3.22)
where Tm is the set of foldover triangles, Mdesirei is the expected transformation matrix of triangle i and M0i is the actual transformation matrix of triangle i. The above energy function measures the total error of the transformation matrix and the desired matrix of all foldover triangles.
Since the foldover constraint is defined, the overall energy function in Equation 3.20 can be
3.5 Optimization and Foldover Prevention 23
modified by including Ef:
E(W0) = wcEc(W0) + wgEg(W0) + wsEs(W0) + wbEb(W0) + wfEf(W0), (3.23) where wc, wg, ws, wband wf are weights.
Equation 3.23 can be expressed in matrix form:
E(W0) = kAW0 − bk2. (3.24)
For the whole resizing process, we use a set Tm to record the triangles which need modi-fying. Initially, the set is empty. Each time we resize the image, it detects the triangles which should restrict the scale as tsand puts those triangles into the set Tm. Besides, for the triangles which are already in the set Tm, it is required to check whether a triangle in Tm still needs modifying. A way to detect these triangles is to check current scaling matrix of these triangles.
If the scale factors of both axis are greater than the scaling threshold ts, it means the triangle desires larger scale than the scaling threshold tsbut it is restricted to be near the threshold. In other words, the foldover constraint on the triangle is no longer needed. Then the triangle is removed from the set Tm.
Algorithm 1 summarizes the iterative optimization process of our resizing framework.
3.5 Optimization and Foldover Prevention 24
Algorithm 1 Resize Image
Input: x0: initial solution; m0: new width; n0: new height;
Output: optimal x∗
1: iter ⇐ 0
2: Set system matrix A
3: Set right-hand side matrix b
4: Solve Ax = b (Equation 3.24) using conjugate gradient method
5: xlast ⇐ x∗
6: Detect foldover triangles
7: while state of foldover triangles is changed ∨iter > itermax do
8: Reset system matrix A
9: Set right-hand side matrix b
10: Solve Ax = b (Equation 3.24) using conjugate gradient method
11: if xlast ' x∗then
12: return x∗
13: end if
14: xlast ⇐ x∗
15: Detect foldover triangles
16: iter ⇐ iter + 1
17: end while
18: return x∗
C H A P T E R 4
Results
In this chapter, we present the experiment results produced by our method and compare with several state-of-art methods. Moreover, we make some discussions with this method.