HDDR Depth Map Rendering of Three Depth of Field

In previous section, even though we use 8 times smaller f-number to capturing the elemental images, the rendering of depth map is still not restricted by the shallow depth of field and according to the experimental result, our HDDR depth map is almost identical to the rendered depth with larger f-number. So in this section, we use one additional depth of field to exceed range of large f-number (F/22).

Figure 4-12 Elemental images of three focal positions and captured under F/2.8 (a)(b)(c) focus at first object (d)(e)(f) focus at middle object (g)(h)(i) focus at the last object from left, central and right perspective respectively

In this section, we place an additional object, little butterfly, at 35 cm to examine the feasibility in the nearer region and the positions of the other two objects are similar in the

67

previous experiment. They are set at 76 and 152 cm respectively. First of all, same f-number, f/2.8, is applied to capture nine elemental images containing three focal positions as shown in Figure 4-12. Subsequently, every three of them are inputted into DERS to render a depth map. Figure 4-13 illustrates the idea again that the object will be well-defined in the depth map as long as it is focused. According the result in Figure 4-14, we could further verify that blur is one of the factors that govern the accuracy of depth map rendering. From the tendency of the degradation of contours, it is clear that when the object is distant from the depth of field, say the last object while focusing at the first object, the result becomes worse because it blurs more. Likewise, the first object is ill-defined especially when we focus at the last object. Due to the fact that depth of field is a function of object distance, so it shrinks when the objects are placed closer to the camera. This phenomenon particular benefits our HDDR system in near field because f-number has its upper limit for conventional cameras.

Figure 4-13 Three rendered depth maps (a) focus at first object (b) focus at middle object (c ) focus at the last object

By the same steps of fusion elaborated in chapter 3, because we have three depth of field, three representative focal points should be detected as shown in Figure 4-15. Once the corresponding gray levels are found, threshold value can be calculated by averaging two of them. Owing the noise, the thresholding will bring about imperfection combination as illustrated in Figure 4-16. There are many voids lying along the boundaries. Besides, some isolated regions remain after thresholding such as the blue and cyan spots in the magnified image. Unfortunately these redundant spots cannot be eliminated in the following process

68

because they should have been cut out while thresholding. Hence, when we reconstruct the HDDR depth map, they will leave the darker spots around the first object.

Figure 4-14 Details of objects in rendered depth maps (a)(d)(g) focus at the first object (b)(e)(h) focus at the middle object (c)(f)(i) focus at the last object

69

Figure 4-15 Experiment images of finding three representative focal points

As shown in Figure 4-17 (a), the surrounding points of the first object in HDDR depth map are worse than that in its origin depth map owing to two reasons. One is described in the previous paragraph. The other is that the lighter regions are the noise of the second depth map.

Because the evaluation of edge is judged by the contrast, the object will look ill-outlined if its background is in a mess.

70

Figure 4-16 Experiment results of fusion and its details

Figure 4-17 Depth maps rendered of (a) HDDR system (b) large f-number (f/22)

Figure 4-18 Comparison of the first object in color image and depth map of (a)(c) HDDR system (b)(d) large f-number (f/22)

71

Therefore, when we compare the HDDR depth map with the rendered depth map shown in Figure 4-17 (b), the discrepancy of two depth maps is lessened. However, even for the largest f-number of our camera, the scene still cannot be captured all in focus. As shown in Figure 4-18 (b), the veined wings is actually blurred, so the first object in the rendered depth map is slightly fuzzy. And this result can prove that our HDDR system not only surpasses the dynamic range of the capturing with large f-number, but also can be extended to the case of stacking more depth of field so as to render even higher dynamic depth range.

To quantify the working range of different focal design, we utilize Figure 4-4 to judge the range with acceptable degree of blur. We set focal plane at 150 cm and measure the variance of a black-and-white edge. Figure 4-19 shows the concept of point spread function that an ideal point image diverges as it is distant from the focal plane and the smaller f-number, the faster it diverges. Because we hope the error rate of rendered depth map can be less than 10%, the upper bound of variance is around 10 pixels. Accordingly, the working range can be decided as shown in Figure 4-20.

Figure 4-19 Variance of different F/# versus depth

The working range of HDDR system is counted from the first object to the terminal wall (200 cm). Apparently, the working range increase with larger f-number, but working range of

72

HDDR using small f-number (F/2.8) is even wider than that of the largest f-number (F/22) of our camera. Furthermore, the exposure time is also minimized as illustrated in Figure 4-21.

Around 21 times shorter exposure time will benefit the capturing of the instantaneous moments. If spatial HDDR system is implemented, the exposure time will be reduced more and kept the same while stacking more depth of field.

Figure 4-20 Working range of different focal designs

Figure 4-21 Exposure time of different focal designs

73

To conclude, as long as the render depth map is less vulnerable to noise, the performance of HDDR depth map will be better. However, compared to largest f-number of our camera (F/22), HDDR system not only extends to the wider working range 165 cm (>150 cm), but also minimizes the exposure time by around 21 times shorter. And in the following section, we will change the pitch to a reasonable quantity, the size of lens, to verify the feasibility of applying lens array. Moreover, undersigned background is utilized in the scene to meet with more general situation in real word.