High dynamic range imaging
Digital Visual Effects, Spring 2006 Yung-Yu Chuang
2006/3/8
with slides by Fedro Durand, Brian Curless, Steve Seitz and Alexei Efros
Announcements
• Room change to 102?
• Assignment #1 is online (due on 3/25 midnight)
Camera pipeline
12 bits 8 bits
Real-world response functions
High dynamic range image Short exposure
10
-610
610
-610
6Real world radiance
Picture intensity
dynamic range
Pixel value 0 to 255
Long exposure
10
-610
610
-610
6Real world radiance
Picture intensity
dynamic range
Pixel value 0 to 255
Camera is not a photometer
• Limited dynamic range
⇒ Perhaps use multiple exposures?
• Unknown, nonlinear response
⇒ Not possible to convert pixel values to radiance
• Solution:
– Recover response curve from multiple exposures, then reconstruct the radiance map
•
• Limited dynamic rangeLimited dynamic range
⇒⇒Perhaps use multiple exposures?Perhaps use multiple exposures?
•• Unknown, nonlinear response Unknown, nonlinear response
⇒
⇒Not possible to convert pixel values to radianceNot possible to convert pixel values to radiance
•• Solution:Solution:
–
– Recover response curve from multiple exposures, Recover response curve from multiple exposures, then reconstruct the
then reconstruct the radiance mapradiance map
Varying exposure
• Ways to change exposure
– Shutter speed – Aperture
– Natural density filters
Shutter speed
• Note: shutter times usually obey a power series – each “stop” is a factor of 2
• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec
Usually really is:
¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec
• Note: shutter times usually obey a power series – each “stop” is a factor of 2
• ¼, 1/8, 1/15, 1/30, 1/60, 1/125, 1/250, 1/500, 1/1000 sec
Usually really is:
¼, 1/8, 1/16, 1/32, 1/64, 1/128, 1/256, 1/512, 1/1024 sec
Varying shutter speeds Math for recovering response curve
Idea behind the math Idea behind the math
Idea behind the math Recovering response curve
• The solution can be only up to a scale, add a constraint
• Add a hat weighting function
Recovering response curve
• We want
If P=11, N~25 (typically 50 is used)
• We want selected pixels well distributed and sampled from constant region. They pick points by hand.
• It is an overdetermined system of linear equations and can be solved using SVD
Matlab code
Matlab code Matlab code
Sparse linear system
⎥ ⎥
⎥ ⎥
⎥ ⎥
⎥ ⎥
⎥ ⎥
⎥
⎦
⎤
⎢ ⎢
⎢ ⎢
⎢ ⎢
⎢ ⎢
⎢ ⎢
⎢
⎣
⎡
=
⎥ ⎥
⎥ ⎥
⎦
⎤
⎢ ⎢
⎢ ⎢
⎣
⎡
⎥ ⎥
⎥ ⎥
⎥ ⎥
⎥ ⎥
⎥ ⎥
⎥
⎦
⎤
⎢ ⎢
⎢ ⎢
⎢ ⎢
⎢ ⎢
⎢ ⎢
⎢
⎣
⎡
Ax=b
256 n
np
1 254
g(0)
g(255) lnE1
lnEn :
: :
Recovered response function
Constructing HDR radiance map
combine pixels to reduce noise and obtain a more reliable estimation
Reconstructed radiance map
What is this for?
• Human perception
• Vision/graphics applications
Easier HDR reconstruction
raw image =
12-bit CCD snapshot
Easier HDR reconstruction
Yij=Ei* ΔΔttjj
Exposure (Y)
Δ Δtt
• 12 bytes per pixel, 4 for each channel
sign exponent mantissa
PF 768 512 1
<binary image data>
Floating Point TIFF similar
Text header similar to Jeff Poskanzer’s .ppm image format:
Portable floatMap (.pfm)
(145, 215, 87, 149) = (145, 215, 87) * 2^(149-128) =
(1190000, 1760000, 713000)
Red Green Blue Exponent
Red Green Blue Exponent
32 bits / pixel 32 bits / pixel
(145, 215, 87, 103) = (145, 215, 87) * 2^(103-128) = (0.00000432, 0.00000641, 0.00000259)
Ward, Greg. "Real Pixels," in Graphics Gems IV, edited by James Arvo, Academic Press, 1994
Radiance format (.pic, .hdr, .rad) ILM’s OpenEXR (.exr)
• 6 bytes per pixel, 2 for each channel, compressed
sign exponent mantissa
• Several lossless compression options, 2:1 typical
• Compatible with the “half” datatype in NVidia's Cg
• Supported natively on GeForce FX and Quadro FX
• Available at http://www.openexr.net/
Radiometric self calibration
• Assume that any response function can be modeled as a high-order polynomial
Space of response curves
Space of response curves Assorted pixel
Assorted pixel Assorted pixel
Assignment #1 HDR image assemble
• Work in teams of two
• Taking pictures
• Assemble HDR images and optionally the response curve.
• Develop your HDR using tone mapping
Taking pictures
• Use a tripod to take multiple photos with different shutter speeds. Try to fix anything else. Smaller images are probably good enough.
• There are two sets of test images available on the web.
• We have tripods and a Canon PowerShot G2 for lending.
• Try not touching the camera during capturing.
But, how?
1. Taking pictures
• Use a laptop and a remote capturing program.
– PSRemote – AHDRIA
• PSRemote
– Manual – Not free
– Supports both jpg and raw
– Support most Canon’s PowerShot cameras
• AHDRIA
– Automatic – Free
– Only supports jpg – Support less models
AHDRIA/AHDRIC/HDRI_Helper
Image registration
• Two programs can be used to correct small drifts.
– ImageAlignment from RASCAL – Photomatix
• Photomatix is recommended.
2. HDR assembling
• Write a program to convert the captured images into a radiance map and optionally to output the response curve.
• We provide image I/O library, gil, which support many traditional image formats such as .jpg and .png, and float-point images such as .hdr and .exr.
• Paul Debevec’s method. You will need a linear solver for this method. (No Matlab!)
• Recover from CCD snapshots. You will need dcraw.c.
3. Tone mapping
• Apply some tone mapping operation to develop your photograph.
– Reinhard’s algorithm (HDRShop plugin) – Photomatix
– LogView
– Fast Bilateral (.exr Linux only) – PFStmo (Linux only)
pfsin a.hdr | pfs_fattal02 | pfsout o.hdr
Bells and Whistles
• Other methods for HDR assembling algorithms
• Implement tone mapping algorithms
• Others
Submission
• You have to turn in your complete source, the executable, a html report, pictures you have taken, HDR image, and an artifact (tone- mapped image).
• Report page contains:
description of the project, what do you learn, algorithm, implementation details, results, bells and whistles…
• The class will have vote on artifacts.
• Submission mechanism will be announced later.
References
• Paul E. Debevec, Jitendra Malik, Recovering High Dynamic Range Radiance Maps from Photographs, SIGGRAPH 1997.
• Tomoo Mitsunaga, Shree Nayar, Radiometric Self Calibration, CVPR 1999.
• Michael Grossberg, Shree Nayar, Modeling the Space of Camera Response Functions, PAMI 2004
