Optics

Combinations of the two techniques have also been developed (Wallace, Cohen, and Green-berg 1987), as well as more general light transport techniques for simulating effects such as the caustics cast by rippling water.

The basic ray tracing algorithm associates a light ray with each pixel in the camera im-age and finds its intersection with the nearest surface. A primary contribution can then be computed using the simple shading equations presented previously (e.g., Equation (2.93)) for all light sources that are visible for that surface element. (An alternative technique for computing which surfaces are illuminated by a light source is to compute a shadow map, or shadow buffer, i.e., a rendering of the scene from the light source’s perspective, and then compare the depth of pixels being rendered with the map (Williams 1983;Akenine-M¨oller and Haines 2002).) Additional secondary rays can then be cast along the specular direction towards other objects in the scene, keeping track of any attenuation or color change that the specular reflection induces.

Radiosity works by associating lightness values with rectangular surface areas in the scene (including area light sources). The amount of light interchanged between any two (mutually visible) areas in the scene can be captured as a form factor, which depends on their relative orientation and surface reflectance properties, as well as the1/r²fall-off as light is distributed over a larger effective sphere the further away it is (Cohen and Wallace 1993; Sillion and Puech 1994;Glassner 1995). A large linear system can then be set up to solve for the final lightness of each area patch, using the light sources as the forcing function (right hand side).

Once the system has been solved, the scene can be rendered from any desired point of view.

Under certain circumstances, it is possible to recover the global illumination in a scene from photographs using computer vision techniques (Yu, Debevec, Malik et al. 1999).

The basic radiosity algorithm does not take into account certain near field effects, such as the darkening inside corners and scratches, or the limited ambient illumination caused by partial shadowing from other surfaces. Such effects have been exploited in a number of computer vision algorithms (Nayar, Ikeuchi, and Kanade 1991;Langer and Zucker 1994).

While all of these global illumination effects can have a strong effect on the appearance of a scene, and hence its 3D interpretation, they are not covered in more detail in this book.

(But see Section12.7.1for a discussion of recovering BRDFs from real scenes and objects.)

aber-2.2 Photometric image formation 69

zi=102 mm f = 100 mm

W=35mm

zo=5 m f.o.v.

c Δzi

P d

Figure 2.19 A thin lens of focal lengthf focuses the light from a plane a distance zoin front of the lens at a distancezi behind the lens, where_z¹

o +_z¹

i = _f¹. If the focal plane (vertical gray line next toc) is moved forward, the images are no longer in focus and the circle of confusionc (small thick line segments) depends on the distance of the image plane motion

∆zi relative to the lens aperture diameterd. The field of view (f.o.v.) depends on the ratio between the sensor widthW and the focal length f (or, more precisely, the focusing distance zi, which is usually quite close tof ).

ration, we need to develop a more sophisticated model, which is where the study of optics comes in (M¨oller 1988;Hecht 2001;Ray 2002).

Figure2.19shows a diagram of the most basic lens model, i.e., the thin lens composed of a single piece of glass with very low, equal curvature on both sides. According to the lens law(which can be derived using simple geometric arguments on light ray refraction), the relationship between the distance to an objectzoand the distance behind the lens at which a focused image is formedzican be expressed as

1 zo

+ 1 zi

= 1

f, (2.96)

wheref is called the focal length of the lens. If we let zo→ ∞, i.e., we adjust the lens (move the image plane) so that objects at infinity are in focus, we getzi= f, which is why we can think of a lens of focal lengthf as being equivalent (to a first approximation) to a pinhole a distancef from the focal plane (Figure2.10), whose field of view is given by (2.60).

If the focal plane is moved away from its proper in-focus setting ofzi(e.g., by twisting the focus ring on the lens), objects atzoare no longer in focus, as shown by the gray plane in Figure2.19. The amount of mis-focus is measured by the circle of confusionc (shown as short thick blue line segments on the gray plane).⁷ The equation for the circle of confusion can be derived using similar triangles; it depends on the distance of travel in the focal plane∆zi

relative to the original focus distanceziand the diameter of the apertured (see Exercise2.4).

7If the aperture is not completely circular, e.g., if it is caused by a hexagonal diaphragm, it is sometimes possible to see this effect in the actual blur function (Levin, Fergus, Durand et al. 2007;Joshi, Szeliski, and Kriegman 2008) or in the “glints” that are seen when shooting into the sun.

(a) (b) Figure 2.20 Regular and zoom lens depth of field indicators.

The allowable depth variation in the scene that limits the circle of confusion to an accept-able number is commonly called the depth of field and is a function of both the focus distance and the aperture, as shown diagrammatically by many lens markings (Figure2.20). Since this depth of field depends on the aperture diameterd, we also have to know how this varies with the commonly displayed f-number, which is usually denoted asf /# or N and is defined as

f /# = N = f

d, (2.97)

where the focal length f and the aperture diameter d are measured in the same unit (say, millimeters).

The usual way to write the f-number is to replace the# in f/# with the actual number, i.e.,f /1.4, f /2, f /2.8, . . . , f /22. (Alternatively, we can say N = 1.4, etc.) An easy way to interpret these numbers is to notice that dividing the focal length by the f-number gives us the diameterd, so these are just formulas for the aperture diameter.⁸

Notice that the usual progression for f-numbers is in full stops, which are multiples of√2, since this corresponds to doubling the area of the entrance pupil each time a smaller f-number is selected. (This doubling is also called changing the exposure by one exposure value or EV.

It has the same effect on the amount of light reaching the sensor as doubling the exposure duration, e.g., from ¹/125to¹/250, see Exercise2.5.)

Now that you know how to convert between f-numbers and aperture diameters, you can construct your own plots for the depth of field as a function of focal length f , circle of confusionc, and focus distance zo, as explained in Exercise2.4and see how well these match what you observe on actual lenses, such as those shown in Figure2.20.

Of course, real lenses are not infinitely thin and therefore suffer from geometric aber-rations, unless compound elements are used to correct for them. The classic five Seidel aberrations, which arise when using third-order optics, include spherical aberration, coma, astigmatism, curvature of field, and distortion (M¨oller 1988;Hecht 2001;Ray 2002).

8This also explains why, with zoom lenses, the f-number varies with the current zoom (focal length) setting.

2.2 Photometric image formation 71

zi’=103mm f’ = 101mm

zo=5m

P d

c

Figure 2.21 In a lens subject to chromatic aberration, light at different wavelengths (e.g., the red and blur arrows) is focused with a different focal lengthf⁰and hence a different depth z_i⁰, resulting in both a geometric (in-plane) displacement and a loss of focus.

Chromatic aberration

Because the index of refraction for glass varies slightly as a function of wavelength, sim-ple lenses suffer from chromatic aberration, which is the tendency for light of different colors to focus at slightly different distances (and hence also with slightly different mag-nification factors), as shown in Figure2.21. The wavelength-dependent magnification fac-tor, i.e., the transverse chromatic aberration, can be modeled as a per-color radial distortion (Section2.1.6) and, hence, calibrated using the techniques described in Section6.3.5. The wavelength-dependent blur caused by longitudinal chromatic aberration can be calibrated using techniques described in Section10.1.4. Unfortunately, the blur induced by longitudinal aberration can be harder to undo, as higher frequencies can get strongly attenuated and hence hard to recover.

In order to reduce chromatic and other kinds of aberrations, most photographic lenses today are compound lenses made of different glass elements (with different coatings). Such lenses can no longer be modeled as having a single nodal pointP through which all of the rays must pass (when approximating the lens with a pinhole model). Instead, these lenses have both a front nodal point, through which the rays enter the lens, and a rear nodal point, through which they leave on their way to the sensor. In practice, only the location of the front nodal point is of interest when performing careful camera calibration, e.g., when determining the point around which to rotate to capture a parallax-free panorama (see Section9.1.3).

Not all lenses, however, can be modeled as having a single nodal point. In particular, very wide-angle lenses such as fisheye lenses (Section2.1.6) and certain catadioptric imaging systems consisting of lenses and curved mirrors (Baker and Nayar 1999) do not have a single point through which all of the acquired light rays pass. In such cases, it is preferable to explicitly construct a mapping function (look-up table) between pixel coordinates and 3D rays in space (Gremban, Thorpe, and Kanade 1988;Champleboux, Lavall´ee, Sautot et al.

zi=102mm f = 100mm

zo=5m δi

d

δo

α

α

α J P

I

O

Q

ro

Figure 2.22 The amount of light hitting a pixel of surface areaδi depends on the square of the ratio of the aperture diameterd to the focal length f , as well as the fourth power of the off-axis angleα cosine, cos⁴α.

1992;Grossberg and Nayar 2001;Sturm and Ramalingam 2004;Tardif, Sturm, Trudeau et al.2009), as mentioned in Section2.1.6.

Vignetting

Another property of real-world lenses is vignetting, which is the tendency for the brightness of the image to fall off towards the edge of the image.

Two kinds of phenomena usually contribute to this effect (Ray 2002). The first is called natural vignettingand is due to the foreshortening in the object surface, projected pixel, and lens aperture, as shown in Figure2.22. Consider the light leaving the object surface patch of sizeδo located at an off-axis angle α. Because this patch is foreshortened with respect to the camera lens, the amount of light reaching the lens is reduced by a factorcos α. The amount of light reaching the lens is also subject to the usual1/r² fall-off; in this case, the distance ro = zo/ cos α. The actual area of the aperture through which the light passes is foreshortened by an additional factorcos α, i.e., the aperture as seen from point O is an ellipse of dimensionsd× d cos α. Putting all of these factors together, we see that the amount of light leavingO and passing through the aperture on its way to the image pixel located at I is proportional to

δo cos α r_o² π

d 2

²

cos α = δoπ 4

d²

z²_ocos⁴α. (2.98)

Since triangles∆OP Q and ∆IP J are similar, the projected areas of of the object surface δo and image pixelδi are in the same (squared) ratio as zo: zi,

δo δi = z_o²

z_i². (2.99)

Putting these together, we obtain the final relationship between the amount of light reaching