Chapter 5. Experiment and Results
5.2 Results
In this section, we present the results of our algorithm. To demonstrate the effects of each component, we also show some other results without the functional terms we add to our algorithm. Discussion about the results is also presented in this section.
Our Results
The results for our algorithm are in Figure 5.1-1~Figure 5.1-7. We apply our methods on cartoon images. Parts of the characters are from famous comics and some cartoon images are retrieved on internet. These cartoon images have various painting styles. Another type of painting we tried is the portrait painting of ancient art work. The results are shown in Figure 5.2-1~Figure 5.2-5. We also tried our method on sketch paintings, in Figure 5.3-1~Figure 5.3-5.
The outcomes in Figure 5.1-1~5.3-5 show satisfying photorealistic results, especially for cartoon image and sketch paintings. The results for portrait paintings such as images in Figure 5.2-1~Figure 5.2-5 is not as surprising as the other types of painting because the portrait painting is realistic in the first place. However, comparing to the original portrait paintings, our results shows more details of human skin and facial texture. For the sketch paintings, we can even use gray level intensity for our data term and use 3 channel color intensity for the smooth term. In this way, the similarity and smoothness can be kept and the results can be colored instead of monochrome.
- 41 -
(a) (b) (c)
Figure 5.4 (a) The input portrait painting (b) The halfway result using Poisson image editing
(c) The results using Poisson image editingPoisson Image Editing
We have tried to use Poisson image editing [5] to solve gaps between patch boundaries.
However, this method is easily influenced by large illumination changes and does not match our expectation of the result. Figure 5.4 shows the progressive results for using Poisson image cloning. Image (a) is the original input painting. After we seamlessly cloned a few patches, the results seams fine but not very clear, such as in image (b). For image (c), it adds all the patches, but the result is blurred due to the inherent problem of Poisson Editing. We believe the reason is that we had many small patches and they are irregular shaped. Each time one patch is stitched with Poisson Editing, the boundary of the patch is smoothed for one time.
After merging more patches, the image becomes seriously blurred. Therefore, instead of using Poisson image editing, we propose using gain compensation to lessen the color differences between adjacent patches.
Edge Term
Figure 5.5 shows the results for the penalty term. Image (c) is the results with edge
- 42 -
penalty term and image (d) show the results without edge penalty term. From image (a) the original input sketch paint, we can see that the lower lip has shining reflections on it. On the edge maps also show the edge of the reflections. Therefore our result finds shining patches for the lower lip based on the edge. If the edge term is turned off, with only the intensity similarity, the shining reflection is not shown in the results.
(a) (b) (c) (d)
Figure 5.5 (a) The input sketch painting (b) The edge from Canny edge detector
(c) Result with edge penalty term (d) Without edge penalty term(a) (b)
Figure 5.6 (a) With edge term (b) Without edge term
- 43 -
Figure 4.9 (a) in the previous chapter, shows the result of the discontinuity of the lip if edge term is not added. Figure 5.6 also shows the discontinuity on the lip and eyes. The lip for Figure 5.6 (b) does not seem smooth and continuous as in image (a). The eye for Figure 5.6 (b) appears a segment of outer corner of the eye, which is should appears.
Smooth term
The variable
determines the size of the patches. Here we show some results of different
. Figure 5.7 shows when is small the patches are smaller with the face is similar to the input. However, if the patches are too small it sometimes may affect the result for outlier pixels. When is larger, the patch size is bigger and it is more smooth. However, it
lost the similarity to the input. So how to decide the value of is an interesting question. It depends on user’s requirement. In our cases, most of the image use 0.1~0.3 as the smoothness term weight. In most of the cases, 0.2 return satisfying results, though sometimes other values return better result.=0.1 =0.2 =0.3 Figure 5.7 The effects of different (*
3)(*
3)The upper row is piece-up image without seamless stitching. The lower row is the result after seamless
stitching from upper row.
- 44 -
(a) (b)
Figure 5.8 (a) Result without symmetry term (b) Result with symmetry term
(a) (b) (c)
Figure 5.9 (a) Without option eyes and mouth completeness (b) Without option eyes and
mouth completeness (c) The result with complete eyes and blended mouth(a) (b) (c)
Figure 5.10 (a) The input image for (b) and (c)
(b) Without completeness (c) With completeness- 45 -
Symmetry term
In Figure 5.8, image (a) is the results without the symmetry term and image (b) is with the symmetry term. If there is no limitation, there is a high probability that the eyes and eyebrows may choose from different sources. But human eyes are sensitive to symmetry of faces, so symmetry term is added to our energy function.
Identical source of eyes or mouth
There is a large rage of color in eyes, so occasionally blending methods makes the eyes unreal. We provide an option for the user to choose whether they want the whole eyes or mouth is extracted from the same source. We call it the completeness constraint. The completeness is for the eye and mouth.
For Figure 5.9, image (a) is the results without eyes and mouth completeness and image (b) is the results with completeness for both eyes and mouth. It is obvious that the eyes in image (a) after blending looks more similar to the input image but not realistic and image (b) has more realistic eyes. So the user can choose the option for eyes completeness for better result. For the mouth in image (b), even though it is completely from the same source but it is not as good as the result for mouth in image (a). The mouth in image (a) is more realistic and natural. So in this case, the user can choose not to limit the completeness for the mouth.
Figure 5.10 (b) and (c) give another example for this case. The eyes without completeness are more similar to the input image, and the result is also satisfying. However, it lacks the reality of human eyes.
Ground truth
Image (b) is the result of our algorithm as the sketch painting of image (a) as the input
- 46 -
image. Figure 5.11 (a) is the sketch painting that the artist draw from photograph (c), even though the drawing is not exactly the same as the photograph, we can still seem photograph (c) as our ground truth to compare with. Our result and input sketch painting is quite similar.
However, the similarity between our result and ground truth is adequate. The reason for this is because the sketch painting and ground truth is not exactly the same. If the similarity between sketch painting and ground truth is higher, our result can be more similar to the ground truth photograph.
(a) (b) (c)
Figure 5.11 (a) Input image (sketch painting) (b) Our Result
(c) The photograph (ground truth)(a) (b)
Figure 5.12 (a) Pocahontas (b) Our Result
- 47 -
(a) (b)
Figure 5.13 (a) Simple line drawing (b) Our Result
Special Case
Figure 5.12 (a) is the input cartoon image and (b) is the result we generated. This case is special because though the result looks satisfying but it is not really similar to Pocahontas.
The reason is we do not have Indian photographs in our database.
If we use an input image that has only very simple and easy line drawing, we can get the result as in Figure 5.13 (b). When very simple line drawing are used, it means that we have very little information even lesser than the cartoon images. In this case, our result still gets satisfactory image appearance quality. However, the geometry of our result is limited to the original input image’s geometry. The geometry of our result is similar to human or not is depending on the input geometry.