comm dpreview.cFrom

(1)

Cameras

Digital Visual Effects g Yung-Yu Chuang

with slides by Fredo Durand, Brian Curless, Steve Seitz and Alexei Efros

(2)

Camera trial #1

scene film

Put a piece of film in front of an object

Put a piece of film in front of an object.

(3)

Pinhole camera

pinhole camera pinhole camera

scene barrier film

Add a barrier to block off most of the rays.

• It reduces blurring

• The pinhole is known as the aperture

• The image is inverted

(4)

Shrinking the aperture

Why not making the aperture as small as possible?

• Less light gets through

• Diffraction effect

(5)

Shrinking the aperture

(6)

High-end commercial pinhole cameras

$200 $700

$200~$700

(7)

Adding a lens

scene lens film

(8)

Lenses

Thin lens equation:

(9)

Thin lens formula

Similar triangles everywhere!

y’/y = D’/D

D’ D f D’ D f

y y’

Frédo Durand’s slide

(10)

Thin lens formula

y’/y = D’/D

Similar triangles everywhere!

D’ D

y’/y = (D’-f)/f f

D’ D f

y y’

(11)

Thin lens formula D’ D 1 1 1

+ = f

The focal length f determines the lens’s ability to bend (refract)

D’ D

D D f

light. It is a function of the shape ^y ⁽ ⁾ and index of refraction of the lens.

f D’ D f

(12)

Adding a lens

“circle of confusion”

scene lens film

confusion”

A lens focuses light onto the film

Th i ifi di t t hi h bj t “i f ”

• There is a specific distance at which objects are “in focus”

• other points project to a “circle of confusion” in the image

• Thin lens applet:

http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html

(13)

Zoom lens

200mm

28mm

simplified zoom lens Nikon 28-200mm zoom lens. simplified zoom lens

in operation From wikipedia

(14)

Field of view vs focal length

i o

Scene

w α Sensor

f

1 1

1  

Gaussian Lens Formula:

f

f o

i  

Gaussian Lens Formula:

α = 2arctan(w/(2i))

Field of View: ( ( )) ≈ 2arctan(w/(2f)) Example: w = 30mm, f = 50mm => α ≈ 33.4º

( ( ))

Slides from Li Zhang

(15)

Focal length in practice

24mm

50mm

135 135mm

(16)

Distortion

No distortion Pin cushion Barrel No distortion Pin cushion Barrel

• Radial distortion of the image

– Caused by imperfect lenses

– Deviations are most noticeable for rays that pass through the edge of the lens

(17)

Correcting radial distortion

from Helmut Dersch

(18)

Vignetting g g

Vignetting

L1

L2

L3 B

A A

more light from A than B !

(19)

Vignetting g g

Vignetting

L1

L2

L3 B

A A

more light from A than B !

original corrected

Goldman & Chen ICCV 2005

(20)

Chromatic Aberration

Lens has different refractive indices for different wavelengths.g

http://www.dpreview.com/learn/?/Glossary/Optical/chromatic_aberration_0 1.htm

Special lens systems using two or more pieces of glass with different refractive indexes can of glass with different refractive indexes can

reduce or eliminate this problem. Slides from Li Zhang

(21)

Exposure = aperture + shutter speed

F

• Aperture of diameter D restricts the range of rays (aperture may be on either side of the lens)

(aperture may be on either side of the lens)

• Shutter speed is the amount of time that light is allowed to pass through the aperturep g p

(22)

Exposure

• Two main parameters:

Aperture (in f stop) – Aperture (in f stop)

– Shutter speed (in fraction of a second)

(23)

Effects of shutter speeds

• Slower shutter speed => more light, but more motion blur

• Faster shutter speed freezes motion From Photography, London et al.

Walking people Running people Car Fast train

1/125 1/250 1/500 1/1000

(24)

Aperture

• Aperture is the diameter of the lens opening,

usually specified by f-stop, f/D, a fraction of the usually specified by f stop, f/D, a fraction of the focal length.

– f/2.0 on a 50mm means that the aperture is 25mmf/2.0 on a 50mm means that the aperture is 25mm – f/2.0 on a 100mm means that the aperture is 50mm

• When a change in f-stop

• When a change in f stop occurs, the light is either doubled or cut in half.

doubled or cut in half.

• Lower f-stop, more light (larger lens opening)

(larger lens opening)

• Higher f-stop, less light

(smaller lens opening)

(25)

Depth of field

Changing the aperture size affects depth of field.

A smaller aperture increases the range in which A smaller aperture increases the range in which the object is approximately in focus

Di h

Diaphragm

Point in focus lens

sensor Object with texture

(26)

Depth of field

Changing the aperture size affects depth of field.

A smaller aperture increases the range in which A smaller aperture increases the range in which the object is approximately in focus

Di h

Diaphragm

Point in focus lens

sensor Object with texture

(27)

Depth of field

From Photography, London et al.

(28)

Exposure

• Two main parameters:

A t (i f t ) – Aperture (in f stop)

– Shutter speed (in fraction of a second)

• Reciprocity

The same exposure is obtained with

i l d

an exposure twice as long and an aperture area half as big

H t f t g i f

– Hence square root of two progression of f stops vs. power of two progression of shutter speed

shutter speed

– Reciprocity can fail for very long exposuresp

From Photography, London et al.

(29)

Reciprocity

• Assume we know how much light we need We have the choice of an infinity of shutter

• We have the choice of an infinity of shutter speed/aperture pairs

Wh t ill id h i f h tt d?

• What will guide our choice of a shutter speed?

– Freeze motion vs. motion blur, camera shake

Wh ill id h i f ?

• What will guide our choice of an aperture?

– Depth of field, diffraction limit

f

• Often we must compromise

– Open more to enable faster speed (but shallow DoF)

(30)

Exposure & metering

• The camera metering system measures how bright the scene is

bright the scene is

• In Aperture priority mode, the photographer sets

th t th t th h tt d

the aperture, the camera sets the shutter speed

• In Shutter-speed priority mode, photographers sets the shutter speed and the camera deduces the aperture

• In Program mode, the camera decides both

exposure and shutter speed (middle value more p p ( or less)

• In Manual mode, the user decides everything In Manual mode, the user decides everything

(but can get feedback)

(31)

Pros and cons of various modes

• Aperture priority

– Direct depth of field controlDirect depth of field control

– Cons: can require impossible shutter speed (e.g. with f/1.4 for a bright scene)g )

• Shutter speed priority

– Direct motion blur control

– Cons: can require impossible aperture (e.g. when requesting a 1/1000 speed for a dark scene)

• Note that aperture is somewhat more restricted

• Program

– Almost no control, but no need for neurons

• Manual

– Full control, but takes more time and thinking

(32)

Sensitivity (ISO)

• Third variable for exposure

• Linear effect (200 ISO needs half the light as 100 ISO)

• Film photography: trade sensitivity for grain

Di it l h t h t d iti it f i

• Digital photography: trade sensitivity for noise

comm dpreview.cFrom

(33)

Demo

See http://www.photonhead.com/simcam/

(34)

Film camera

aperture

& shutter

scene lens & film

motor

(35)

Digital camera

aperture

& shutter

scene lens & sensor

array motor

A digital camera replaces film with a sensor array

• A digital camera replaces film with a sensor array

• Each cell in the array is a light-sensitive diode that converts photons to electrons

converts photons to electrons

(36)

CCD v.s. CMOS

• CCD is less susceptible to noise (special process, higher fill factor))

• CMOS is more flexible, less expensive (standard process), less power consumption

CCD CMOS

(37)

Sensor noise

• Blooming Diff i

• Diffusion

• Dark current

• Photon shot noise

• Amplifier readout noise

(38)

SLR (Single-Lens Reflex) ( )

• Reflex (R in SLR) means that we see through the same lens used to take the image

the same lens used to take the image.

• Not the case for compact cameras

(39)

SLR view finder

Prism

Your eye

Mirror Mirror (flipped for exposure)

Film/sensor

Mirror

(when viewing) Light from scene

lens

(40)

Color

So far, we’ve only talked about monochrome

sensors Color imaging has been implemented in a sensors. Color imaging has been implemented in a number of ways:

Fi ld ti l

• Field sequential

• Multi-chip

• Color filter array

• X3 sensor

(41)

Field sequential

(42)

Field sequential

(43)

Field sequential

(44)

Prokudin-Gorskii (early 1900’s) ( )

L

Lantern projector

http://www.loc.gov/exhibits/empire/

(45)

Prokudin-Gorskii (early 1990’s) ( )

(46)

Multi-chip

wavelength

dependent

(47)

Embedded color filters

Color filters can be manufactured directly onto th h t d t t

the photodetectors.

(48)

Color filter array

Kodak DCS620x

Color filter arrays (CFAs)/color filter mosaics

CMY

Color filter arrays (CFAs)/color filter mosaics

(49)

Why CMY CFA might be better

(50)

Color filter array

Bayer pattern

Color filter arrays (CFAs)/color filter mosaics

(51)

Bayer’s pattern

(52)

Demosaicking CFA’s

bilinear interpolation

i i l i t li i t l ti

original input linear interpolation

(53)

Demosaicking CFA’s

Constant hue-based interpolation (Cok) interpolation (Cok)

Hue:

Interpolate G first

(54)

Demosaicking CFA’s

Median-based interpolation (Freeman)

(Freeman)

1. Linear interpolation 2. Median filter on color

differences

(55)

Demosaicking CFA’s

Median-based interpolation (Freeman)

original input linear interpolation

color difference median filter Reconstruction (e.g. G-R) (kernel size 5) (G=R+filtered

difference)

(56)

Demosaicking CFA’s

Gradient-based interpolation (LaRoche-Prescott)

(LaRoche Prescott)

1. Interpolation on G

(57)

Demosaicking CFA’s

Gradient-based interpolation (LaRoche-Prescott)

(LaRoche Prescott)

2. Interpolation of color differences

differences

(58)

Demosaicking CFA’s

bili C k F L R h

bilinear Cok Freeman LaRoche

(59)

Demosaicking CFA’s

G ll F ’ i h b i ll f l i

Generally, Freeman’s is the best, especially for natural images.

(60)

Foveon X3 sensor

• light penetrates to different depths for different wavelengths

wavelengths

• multilayer CMOS sensor gets 3 different spectral iti iti

sensitivities

(61)

Color filter array

red green blue output

(62)

X3 technology

red green blue output

(63)

Foveon X3 sensor

X3 sensor

Bayer CFA

(64)

Cameras with X3

Sigma SD10, SD9 Polaroid X530

(65)

Sigma SD9 vs Canon D30

(66)

Color processing

• After color values are recorded, more color processing usually happens:

processing usually happens:

– White balance

N li it t i t fil t h

– Non-linearity to approximate film response or match TV monitor gamma

(67)

White Balance

automatic white balance warmer +3

(68)

Manual white balance

white balance with white balance with the white book the red book

(69)

Autofocus

• Active

S

– Sonar – Infrared

• Passive

(70)

Digital camera review website

• A cool video of digital camera illustration

h // d i /

• http://www.dpreview.com/

(71)

Camcorder

(72)

Interlacing

with interlacing without interlacing

(73)

Deinterlacing

bl d

blend weave

(74)

Deinterlacing

Discard

(even field only or Progressive scan

odd filed only)

(75)

Hard cases

(76)

References

• http://www.howstuffworks.com/digital-camera.htm

• http://electronics howstuffworks com/autofocus htm

• http://electronics.howstuffworks.com/autofocus.htm

• Ramanath, Snyder, Bilbro, and Sander. Demosaicking Methods for Bayer Color Arrays Journal of Electronic Methods for Bayer Color Arrays, Journal of Electronic Imaging, 11(3), pp306-315.

• Rajeev Ramanath, Wesley E. Snyder, Youngjun Yoo, Rajeev Ramanath, Wesley E. Snyder, Youngjun Yoo,

Mark S. Drew, Color Image Processing Pipeline in Digital Still Cameras, IEEE Signal Processing Magazine Special Issue on Color Image Processing, vol. 22, no. 1, pp. 34- 43, 2005.

htt // ld t / h t / hit b l /i d

• http://www.worldatwar.org/photos/whitebalance/ind ex.mhtml

http://www 100fps com/

• http://www.100fps.com/