where S(x, y) is the saliency value at pixel (x,y) of the saliency map, max is the maximum of the saliency map, and min is the minimum of the saliency map.
4.2 Tone mapping function adjustment
We first convert an input HDRI from the RGB color space to XYZ space in order to get the luminance of the HDRI. We then calculate the Weber contrast [45] of every HDR pixel defined as follows,
C(x, y) = L(x, y) − Lw Lw
, (4.2)
where L(x, y) is the luminance of the pixel (x,y) and Lw = exp( 1
N
X
∀x,y
log(L(x, y)). (4.3)
For the simplicity of notation, we will omit (x, y) of C(x, y) and L(x, y) in the following text.
We adopt the NaKa-Rushton function (Equation 4.4), which is used in many tone mapping methods [30][10][46], as the base model of our tone mapping function.
R = a1RmaxCα
Cα+ a2C50α , (4.4)
where R is the neuron response, C is the contrast of stimulus, Rmax is the maximal firing rate of population, α is the slope of the contrast response function, and C50 is the contrast required to produce 50% of the neuron’s maximum response.
Pestilli et al. [40] indicate that adaptation changes the contrast gain by modulating it through the variable a2and transient attention changes the response gain via the variable a1.
As mentioned in Section 3.1, transient attention influences the contrast sensitivity function.
It increases contrast sensitivity when we pay attention to some locations and decreases sensitiv-ity when we do not keep an eye on them. We apply the findings of Pestilli et al. [40] to adjust our tone mapping function, which takes advantage of the transient attention effects. Equation 4.5,which we call attention function, lists our tone mapping function under transient attention.
4.2 Tone mapping function adjustment 23
T A(C) = β · (C + 1)
2 + C , (4.5)
where β is used to adjust the tone mapping function according to the value of attention map.
Specifically,we use Equation 4.6 to adjust β,
β = 0.2 ∗ A(x, y) + 1 (4.6)
Figure 4.2 illustrates the behavior of our tone mapping function due to transient attention.
We set β = 1 as the baseline, which corresponds to the contrast sensitivity function in neutral condition in Figure 3.1. The tone mapping function will shift toward the red curve when β gradually increases. This simulates the curve in the valid condition in Figure 3.4. If β gradually decreases, the function will move toward the blue curve. This equation behaves like the experi-mental result in Pestilli et al. [40]. To avoid over or under exposure, we restrict 0.8 ≤ β ≤ 1.2.
Figure 4.2: Plot of Equation 4.5. We treat β = 1 as the baseline. The red and blue curve corresponds to β = 1.2 and β = 0.8, respectively.
4.2 Tone mapping function adjustment 24
Sustained attention will enhance contrast sensitivity like transient attention does for a short period of time, but it will impair perception like adaptation does for a long duration. We consider the effect of sustained attention as the mixture of transient attention and adaptation. The effect of adaptation dominates the effect of sustained attention, so we approximate the effect of sustained attention as the effect of adaptation. We use Equation 4.7 to model the function of adaptation.
SA(C) = C + 1
δ + C, (4.7)
where δ is used to adjust the tone mapping function based on the adaptation effect. We call Equation 4.7 the adaptation function. As we consider adaptation a steady response, which does not vary a lot in a short period, we fix δ at 13 in our approach. Figure 4.3 shows the function used in our model. δ = 2 and δ = 13 denotes the curve in the neutral and adaptation condition, respectively.
Figure 4.3: Plot of Equation 4.7. We set δ = 2 as the baseline curve that represents the neutral condition. δ = 13 denotes the curve in adaptation condition
4.2 Tone mapping function adjustment 25
Because transient attention and adaptation act independently and change contrast sensitivity function simultaneously [1], we use a weighting function to combine transient attention and adaptation(Equation 4.8). Adaptation is more important as it affects contrast sensitivity func-tion longer than transient attenfunc-tion, which is short-lived. Adaptafunc-tion should weight more than transient attention in HDRI. Therefore, we set a higher weight for the adaptation term
R(C) = 1
3T A(C) + 2
3SA(C) (4.8)
After we obtain R, the intensity value of each color channel in an HDRI is mapped to that of an LRDI as follows, the corresponding pixel in the HDRI.
Our tone mapping algorithm can also be applied to an HDR video by modifying two com-ponents First, we adopt the video saliency map [43], which includes flicker feature map and motion feature map to compute the attention map in an HDR video. Second, we modify the weighting function to address the change of contrast sensitivity due to the temporal property of videos
R(C) = f (A) · T A(C) + (1 − f (A)) · SA(C), (4.10) where
f (A) = A + 3
6 . (4.11)
Equation 4.11 is inspired by the study of Pestilli et al. [1] who found transient attention plays a more important role than adaptation when people watch a video since transient attention would alter the contrast sensitivity function, which had been optimized by adaptation. The weighting function f (A) depends on the time course of transient attention.
C H A P T E R 5
Experimental Results
5.1 Validation of our approach
We validate the necessity of the attention function and adaptation function used in our tone mapping approach via three experiments. First, we show what would happen if we just use one of our tone mapping functions. As shown in Figure 5.1, the function of attention and adaptation are complementary. The regions that are worse in the result of transient attention can be com-plemented by the regions in the result of adaptation and vise versa. This also demonstrates that attention and adaptation can both optimize visual performance [1]. Second, Figure 5.2 shows the results of our tone mapping function when different δ values are used in the adaptation function. As δ increases, the results becomes better. Third, we change the attention function by varying the range of β. As the range of β increases, one cna observe that the results become darker in Figure 5.3
26
5.1 Validation of our approach 27
(a) Transient Attention (b) Adaptation
Figure 5.1: Memorial Church in Stanford University. Randiance map courtesy of Paul Debevec.
(a) is produced only by the transient attention function T A(C) (b)is generated only by the adaptation function SA(C).
5.1 Validation of our approach 28
(a) δ = 2 (b) δ = 12
(c) δ = 30 (d) δ = 50
Figure 5.2: Memorial Church in Stanford University. Randiance map courtesy of Paul Debevec.
These results are produced by tone mapping method with different δ.
5.1 Validation of our approach 29
(a) 0.8 < β < 1.2 (b) 0.4 < β < 1.6
(c) 0.01 < β < 2 (d) 0.001 < β < 3
Figure 5.3: Memorial Church in Stanford University. Randiance map courtesy of Paul Debevec.
These results are produced by tone mapping method with different range of β.