# From dpreview.com

(1)

## Cameras

Digital Visual Effects Yung-Yu Chuang

with slides by Fredo Durand, Brian Curless, Steve Seitz and Alexei Efros

### Camera trial #1

scene film

Put a piece of film in front of an object.

### Pinhole camera

scene film

Add a barrier to block off most of the rays.

• It reduces blurring

• The pinhole is known as the aperture

• The image is inverted

barrier

pinhole camera

### Shrinking the aperture

Why not making the aperture as small as possible?

• Less light gets through

• Diffraction effect

(2)

scene lens film

(3)

### fD’D

Similar triangles everywhere!

### y’/y = D’/D

Frédo Durand’s slide

### y’/y = D’/D y’/y = (D’-f)/f

Frédo Durand’s slide

Similar triangles everywhere!

### + = f

The focal length f determines the lens’s ability to bend (refract) light. It is a function of the shape and index of refraction of the lens.

Frédo Durand’s slide

scene lens film

“circle of confusion”

### A lens focuses light onto the film

• There is a specific distance at which objects are “in focus”

• other points project to a “circle of confusion” in the image

• Thin lens applet:

http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html

(4)

### Zoom lens

Nikon 28-200mm zoom lens.

200mm

28mm

simplified zoom lens in operation From wikipedia

### Field of view vs focal length

i o

f o i

1 1 1

Gaussian Lens Formula:

Scene

Sensor

f α

α = 2arctan(w/(2i)) w

Field of View:

Example: w = 30mm, f = 50mm => α ≈ 33.4º

≈ 2arctan(w/(2f))

Slides from Li Zhang

24mm

50mm

135mm

### • Radial distortion of the image

– Caused by imperfect lenses

– Deviations are most noticeable for rays that pass through the edge of the lens

No distortion Pin cushion Barrel

(5)

from Helmut Dersch

### Vignetting

Vignetting

L1

L2

L3 B

A

more light from A than B !

Slides from Li Zhang

### Vignetting

Vignetting

L1

L2

L3 B

A

more light from A than B !

original corrected

Goldman & Chen ICCV 2005

Slides from Li Zhang

### Chromatic Aberration

Lens has different refractive indices for different wavelengths.

Special lens systems using two or more pieces of glass with different refractive indexes can

reduce or eliminate this problem.

http://www.dpreview.com/learn/?/Glossary/Optical/chromatic_aberration_0 1.htm

Slides from Li Zhang

(6)

### Exposure = aperture + shutter speed

• Aperture of diameter D restricts the range of rays (aperture may be on either side of the lens)

• Shutter speed is the amount of time that light is allowed to pass through the aperture

F

### • Two main parameters:

– Aperture (in f stop)

– Shutter speed (in fraction of a second)

• Slower shutter speed => more light, but more motion blur

• Faster shutter speed freezes motion

### Effects of shutter speeds

1/125 1/250 1/500 1/1000

Walking people Running people Car Fast train

From Photography, London et al.

### Aperture

• Aperture is the diameter of the lens opening, usually specified by f-stop, f/D, a fraction of the focal length.

– f/2.0 on a 50mm means that the aperture is 25mm – f/2.0 on a 100mm means that the aperture is 50mm

• When a change in f-stop occurs, the light is either doubled or cut in half.

• Lower f-stop, more light (larger lens opening)

• Higher f-stop, less light (smaller lens opening)

(7)

### A smaller aperture increases the range in which the object is approximately in focus

lens sensor

Point in focus

Object with texture Diaphragm

### A smaller aperture increases the range in which the object is approximately in focus

lens sensor

Point in focus

Object with texture Diaphragm

### Depth of field

From Photography, London et al.

### • Two main parameters:

– Aperture (in f stop)

– Shutter speed (in fraction of a second)

### • Reciprocity

The same exposure is obtained with an exposure twice as long and an aperture area half as big

– Hence square root of two progression of f stops vs. power of two progression of shutter speed

– Reciprocity can fail for very long exposures

From Photography, London et al.

(8)

### • What will guide our choice of a shutter speed?

– Freeze motion vs. motion blur, camera shake

### • What will guide our choice of an aperture?

– Depth of field, diffraction limit

### • Often we must compromise

– Open more to enable faster speed (but shallow DoF)

### • Aperture priority

– Direct depth of field control

– Cons: can require impossible shutter speed (e.g. with f/1.4 for a bright scene)

### • Shutter speed priority

– Direct motion blur control

– Cons: can require impossible aperture (e.g. when requesting a 1/1000 speed for a dark scene)

• Note that aperture is somewhat more restricted

### • Program

– Almost no control, but no need for neurons

### • Manual

– Full control, but takes more time and thinking

### Sensitivity (ISO)

• Third variable for exposure

• Linear effect (200 ISO needs half the light as 100 ISO)

• Film photography: trade sensitivity for grain

• Digital photography: trade sensitivity for noise

From dpreview.com

(9)

### Summary in a picture

source hamburgerfotospots.de

### Film camera

scene lens & film

motor

aperture

& shutter

### Digital camera

scene sensor

array lens &

motor

aperture

& shutter

• A digital camera replaces film with a sensor array

• Each cell in the array is a light-sensitive diode that converts photons to electrons

(10)

### CCD v.s. CMOS

• CCD is less susceptible to noise (special process, higher fill factor)

• CMOS is more flexible, less expensive (standard process), less power consumption

### SLR view finder

lens

Mirror

(when viewing) Mirror

(flipped for exposure)

Film/sensor

Light from scene

(11)

(12)

(13)

CMY

### Color filter arrays (CFAs)/color filter mosaics Bayer pattern

Pattern Design

Demosaicking

Color filter array (CFA) Sensor array

Bayer CFA

(14)

### bilinear interpolation

original input linear interpolation

(15)

### Median-based interpolation (Freeman)

original input linear interpolation

color difference (e.g. G-R)

median filter (kernel size 5)

Reconstruction (G=R+filtered difference)

### Demosaicking CFA’s

bilinear Cok Freeman LaRoche

(16)

### Demosaicking CFA’s

Generally, Freeman’s is the best, especially for natural images.

### Interpolation

Observations

mosaicked image full-color image

### Convolution

Deep Learned

CNN

mosaicked image full-color image

### ……… ൅

ReLU

Demosaicking (learning residual)

### CNN-based demosaicking

(17)

Algorithm

Kodak (12 photos) McM (18 photos) Kodak + McM (30 photos)

PSNR CPSNR PSNR

CPSNR PSNR

CPSNR

R G B R G B R G B

SA 39.8 43.31 39.5 40.54 32.73 34.73 32.1 32.98 35.56 38.16 35.06 36.01 SSD 38.83 40.51 39.08 39.4 35.02 38.27 33.8 35.23 36.54 39.16 35.91 36.9 NLS 42.34 45.68 41.57 42.85 36.02 38.81 34.71 36.15 38.55 41.56 37.46 38.83

CS 41.01 44.17 40.12 41.43 35.56 38.84 34.58 35.92 37.74 40.97 36.8 38.12 ECC 39.87 42.17 39.00 40.14 36.67 39.99 35.31 36.78 37.95 40.86 36.79 38.12 RI 39.64 42.17 38.87 39.99 36.07 39.99 35.35 36.48 37.5 40.86 36.76 37.88 MLRI 40.59 42.97 39.86 40.94 36.35 39.9 35.36 36.62 38.04 41.13 37.16 38.35 ARI 40.81 43.66 40.21 41.31 37.41 40.72 36.05 37.52 38.77 41.9 37.72 39.03 PAMD 41.88 45.21 41.23 42.44 34.12 36.88 33.31 34.48 37.22 40.21 36.48 37.66 AICC 42.04 44.51 40.57 42.07 35.66 39.21 34.34 35.86 38.21 41.33 36.83 38.34 DMCNN 39.86 42.97 39.18 40.37 36.50 39.34 35.21 36.62 37.85 40.79 36.79 38.12 DMCNN-DR 42.43 45.66 41.55 42.86 39.37 42.24 37.45 39.14 40.59 43.61 39.09 40.63

### evaluation

ground truth

ARI RTF

DMCNN-DR DMCNN-DR-Tr

### Evaluation with different patterns

Algorithms Patern

Kodak (12 photos) McM (18 photos) Kodak + McM (30 photos) PSNR

CPSNR

PSNR

CPSNR

PSNR

CPSNR

R G B R G B R G B

NLS Bayer 42.34 45.68 41.57 42.85 36.02 38.81 34.71 36.15 38.55 41.56 37.46 38.83 ARI Bayer 40.75 43.59 40.16 41.25 37.39 40.68 36.03 37.49 38.73 41.84 37.68 39.00 DMCNN-DR Bayer 42.43 45.66 41.55 42.86 39.37 42.24 37.45 39.14 40.59 43.61 39.09 40.63 DMCNN-DR Diagonal Stripe 42.00 42.47 41.36 41.91 39.70 39.5 38.02 38.87 40.62 40.69 39.36 40.08 DMCNN-DR CYGM 41.16 46.00 41.80 42.48 38.64 41.98 38.44 39.36 39.65 43.59 39.78 40.60 DMCNN-DR Hirakawa 43.20 44.95 42.53 43.43 39.59 40.52 38.42 39.38 41.03 42.29 40.06 41.00 Condat Hirakawa 41.99 43.18 41.53 42.16 33.93 34.83 33.44 33.94 37.15 38.17 36.68 37.23 Condat Condat 41.68 42.7 41.27 41.83 34.05 35.08 33.57 34.1 37.1 38.13 36.65 37.19

Condat pattern

Diagonal Stripe CYGM Hirakawa

### ……… ൅

ReLU

Pattern Design Demosaicking (Learning Residual)

Color Reconstruction

͵ ൈ ͵ ൌ ͻ ͻ

͵ ൈ ͵ kernel

(18)

### Learned pattern

Without non-negative constraints

With non-negative constraints

### Evaluation with the learned pattern

Algorithm

Kodak (12 photos) McM (18 photos) Kodak + McM (30 photos)

PSNR CPSNR PSNR

CPSNR PSNR

CPSNR

R G B R G B R G B

SA 39.80 43.31 39.50 40.54 32.73 34.73 32.10 32.98 35.56 38.16 35.06 36.01 SSD 38.83 40.51 39.08 39.40 35.02 38.27 33.80 35.23 36.54 39.16 35.91 36.90 NLS 42.34 45.68 41.57 42.85 36.02 38.81 34.71 36.15 38.55 41.56 37.46 38.83 CS 41.01 44.17 40.12 41.43 35.56 38.84 34.58 35.92 37.74 40.97 36.80 38.12 ECC 39.87 42.17 39.00 40.14 36.67 39.99 35.31 36.78 37.95 40.86 36.79 38.12 RI 39.64 42.17 38.87 39.99 36.07 39.99 35.35 36.48 37.50 40.86 36.76 37.88 MLRI 40.59 42.97 39.86 40.94 36.35 39.9 35.36 36.62 38.04 41.13 37.16 38.35 ARI 40.81 43.66 40.21 41.31 37.41 40.72 36.05 37.52 38.77 41.9 37.72 39.03 PAMD 41.88 45.21 41.23 42.44 34.12 36.88 33.31 34.48 37.22 40.21 36.48 37.66 AICC 42.04 44.51 40.57 42.07 35.66 39.21 34.34 35.86 38.21 41.33 36.83 38.34 DMCNN 39.86 42.97 39.18 40.37 36.50 39.34 35.21 36.62 37.85 40.79 36.79 38.12 DMCNN-DR 42.43 45.66 41.55 42.86 39.37 42.24 37.45 39.14 40.59 43.61 39.09 40.63 DMCNN-Pa 43.06 43.76 42.13 42.92 40.63 40.14 38.74 39.68 41.60 41.59 40.01 40.98

### Visual Comparisons

71

original image

ground truth

CS NLS ARI

DMCNN-DR DMCNN-DR-Pa

(19)

### Cameras with X3

Sigma SD10, SD9 Polaroid X530

(20)

### • After color values are recorded, more color processing usually happens:

– White balance

– Non-linearity to approximate film response or match TV monitor gamma

### White Balance

automatic white balance warmer +3

(21)

(22)

### • Active

– Sonar – Infrared

(23)

### CamcorderInterlacing

with interlacing without interlacing

(24)

### More emerging camerasReferences

• http://www.howstuffworks.com/digital-camera.htm

• http://electronics.howstuffworks.com/autofocus.htm

• Ramanath, Snyder, Bilbro, and Sander. Demosaicking Methods for Bayer Color Arrays, Journal of Electronic Imaging, 11(3), pp306-315.

• Rajeev Ramanath, Wesley E. Snyder, Youngjun Yoo, Mark S. Drew, Color Image Processing Pipeline in Digital Still Cameras, IEEE Signal Processing Magazine Special Issue on Color Image Processing, vol. 22, no. 1, pp. 34- 43, 2005.

• http://www.worldatwar.org/photos/whitebalance/ind ex.mhtml

• http://www.100fps.com/

Updating...

## References

Related subjects :