Summary

Backlight scaling is by far the most effective technique for reducing power consumption in a transmissive display. This technique scales down the backlight luminous intensity to save power consumption at the cost of reduced image luminance, which results in lower visual quality. To compensate for the visual quality loss due to reduced luminance, proper image enhancement is necessary.

(a) (b)

(c) (d)

Figure 2-1. Night Watch (a) Rembrandt (b) Choi (c) Iranli (d) Cheng.

As mentioned before (2.1-2.3), different image enhancement algorithms for backlight

scaling have been proposed in the past years. Choi et al. proposed a technique that increases the pixel values (t) to recover the original luminance (L) [7]. However, the transmittance t in Equation 1-1 is bounded by 0 and 1. When t needs to be greater than 1, the original luminance can not be recovered and image distortion occurs. Choi’s algorithm can preserve the luminance of the dark regions, but the bright regions will be over-saturated. In their work, the number of over-saturated pixels was chosen to evaluate the image quality loss. Apparently the approach of preserving luminance L is too conservative, so preserving alternative objectives, such as a transformation on the luminance, L^*=f(L), have been proposed by the other groups [13]. In addition, a series of studies were proposed in [14][15][16][17]. All of their algorithm used the same image-enhancement method for preserving luminance and restrict to image quality loss by the different conditions. They implemented their algorithm by a server and network.

Iranli et al. proposed using histogram equalization, an image processing algorithm that balances the number of pixels on each graylevel, to perform the image enhancement [12]. The transformation is the cumulative distribution function of the histogram. Histogram equalization can reproduce each graylevel distinctly without over-saturation or under-saturation. However, the proportional difference between bright and dark regions, i.e., tonality, will be distorted greatly when the original histogram tends to be irregular. Figure 2-1 shows such an example.

While the other researchers treat the backlight scaling problem in the brightness domain (pixel values), the same authors proposed another work that emphasizes on the mapping between luminance and brightness, commonly referred as gamma correction [18]. Gamma correction is one of the standard features for modern displays such that the end users can adjust their monitors for different viewing condition, user preferences, manufacturing variations, and luminance/color degradation as aging.

Cheng et al. proposed an algorithm to compensate for the luminance loss by increasing

the contrast [10]. The following linear transformation was used:

Although Cheng’s algorithm is a compromise between preserving the brightness and preserving the contrast, it does preserve the original tonality. The relationship between brightness and contrast, however, was employed without substantial support.

The visual effects of different image enhancement algorithms are shown in Figure 2-1.

The original image with 100% backlighting is shown in Figure 2-1a, in which most pixels are either very bright or very dark. Figure 2-1 show Choi’s, Iranli’s, and Cheng’s results with backlight scaled to 50%.

The minimal perceivable radiant flux from a display was calculated from the aspect of human factors by Zhong et al. in [19]. The authors concluded that the comfortable reading luminance is seven orders of magnitude larger than the just-perceptible threshold. However, such pure radiometric calculation is oversimplified without considering the other dominating human vision factors such as light adaptation and dark adaptation [20]. For example, for the classical just-noticeable difference (JND) data to be valid, the ambient light cannot be ignored, since the adaptation mechanism of human eyes can change visual sensitivity by five orders of magnitude.

In a nutshell, the principle of backlight scaling is to trade perceived image quality for power savings. In our work, we employ the results and methods that were well established in vision study, color science, psychophysical experiments to design our proposed backlight scaling algorithm.

Chapter 3

Principle: Assess Visual Quality

The research of backlight scaling involves photometry, colorimetry, psychophysics, etc.

The key concepts and terminologies are reviewed in this section.

3.1 Photometric Definitions

Though light is a form of electromagnetic radiation, measurement of luminous intensity from a light source requires extra information about the relative sensitivity of the human eye to different wavelengths. Photometry is the science of measurement of the intensity of visible light and its illuminating power according to the sensitivity of the human eye. The following photometric quantities are illustrated around a TFT-LCD display in Figure 3-1.

Figure 3-1. Illustration of LED backlight and photometric terms.

Luminous flux (lumen): is the emission rate of light energy corrected for the standardized spectral response of human vision.

Luminous intensity (candela): is defined as cd=lm/sr, one lumen of luminous flux per steradiam (unit of solid angle). Luminous intensity can be used to characterize the optical power emitted from a spot light source, such as a light bulb.

Illuminance (lux): is defined as one lumen of luminous flux per area (lm/m²). Illuminance can be used to characterize the luminous power emitted from a surface. Most light meters (e.g., for photographic purpose) measure the illuminance quantity. The luminous flux may not travel in parallel after passing the surface, so that the light intensity decreases as the travel distance increases.

Luminance (nit) is physical measure, and defined as lumen per area per steradiam (lm/m² /sr or cd/ m²) [10][20].

Luminance is used to rate the maximum brightness of CRT or LCD monitors. We use backlight factor to express the percentage of the backlight illumination, and transmissivity to express the transparency of the TFT-LCD. The backlight factor and TFT-LCD transmissivity determine the perceived luminance from the TFT-LCD display.

3.2 Colorimetry

The human eyes have three types of cones perceiving light at different wavelengths. The L-, M-, and S-cones respond to, roughly, the wavelengths of red, green, and blue – the primaries. Mixing the three primaries in different ratios generates different colors. The colors can be ordered in different ways and be represented by different color spaces. Computers use the RGB color space, which is convenient for the graphics adaptor and displays. Printers use the CMY color space or its variation. In color science, a color space usually consists of the one-dimensional luminance and the two-dimensional chromaticity. For a color, luminance indicates its magnitude, while chromaticity indicates the ratio between red, green and blue.

Colorimetry is the discipline of determining and specifying colors of objects by standardizing

the observer, illuminator, viewing geometry, etc. Most de facto color spaces were defined by CIE, the International Commission on Illumination. The CIE standard observer was defined by standardizing the color matching functions, the response function of red (x), green (y), and blue (z) as shown in Figure 3-2.

Figure 3-2. The CIE color matching functions for the 1931 Standard Colorimetric Observer.

Once the standard observer is defined, the CIEXYZ color space can be obtained by defining the X, Y, and Z values (in uppercase) as the product of spectral power distribution of the object Φ and the color matching function (x,y,z) across wavelength λ:

,

To separate the luminance component from chromaticity, the x, y, z values are defined as Z,

Since there are only two dimensions of information in chromaticity coordinates, the third

chromaticity coordinate can always be obtained from the other two by noting that the three always sum to unity. Thus z can be calculated from x and y using Equation (3-3.

y x

z= 01. − − _(3-3)

The x, y, and z value represent chromaticity, the ratio between red, green, and blue, which is independent of luminance. The z value is redundant because it can be obtained from x and y. The two-dimensional chromaticity diagram of CIEXYZ is shown in Figure 3-3. The horseshoe shape depicts the range of visible colors. The Euclidean distance between two different colors can be used to measure their color difference. However, CIEXYZ is not an ideal color space because the color difference is not perceived uniformly. For example, for the same color difference, a pair of blue colors is perceived more differently than a pair of green colors.

Figure 3-3. The CIE x,y chromaticity diagram.

A more uniform color space, CIELAB, is defined by ,

The CIELAB ∆E_ab^* color difference is defined by . ) ( ^{*2 *2 *2}

* L a b

E_ab = ∆ +∆ +∆

∆ (3-5)

The ∆E_ab^* color difference is considered as a uniform metric and has been widely used and implemented in commercial colorimeters.

Figure 3-4. CIE-Lab chromaticity diagram.

3.3 Color Appearance Terminology

The following definitions introduced in this chapter have been culled from the International Lighting Vocabulary [20].

Brightness: attribute of a visual sensation according to which an area appears to emit amore or less light.

Lightness: the brightness of an area judged relative to the brightness of a similarly illuminated area that appears to be white or highly transmitting.

Besides, another very important concept is contrast. The following definitions are adopted from Fairchild [22]. There are two different definitions for contrast. One definition for contrast, which is used in tone reproduction, is the rate of change of the relative luminance of image elements of a reproduction as a function of the relative luminance of the same image elements of the original image. On log-log coordinates, the contrast is the slope of the relationship between the reproduction and original. The contrast defined in this way is an

attribute of the system transfer function.

Another definition for contrast, which is used in visual science, is the difference between minimum and maximum luminance in an image. This contrast is an attribute of the image.

The perceived image contrast is the perceived lightness difference between dark part and the light part of an image.

3.4 Psychophysics

Psychophysics is the scientific study of the relationships between the physical measurements of stimuli and the sensations and perceptions that those stimuli evoke.

Psychophysics can be considered a discipline of science similar to the more traditional disciplines such as physics, chemistry, and biology.

The tools of psychophysics are used to derive quantitative measures of perceptual phenomena that are often considered subjective. It is important to note that the results of properly designed psychophysical experiments are just as objective and quantitative as the measurement of length with a ruler or any other physical measurement. One important difference is that the uncertainties associated with psychophysical measurements tend to be significantly larger than those of physical measurements. However, the results are equally useful and meaningful as long as those uncertainties are considered as they always should be for physical measurements as well. Psychophysics is used to study all dimensions of human perception.

In our work, two threshold techniques proposed by Fechner were used [22][23]. The threshold experiments were designed to determine the just-perceptible change in a stimulus, sometimes referred to as a just-noticeable difference, which is used to measure the observers’

sensitivity to changes in a given stimulus. Absolute thresholds are defined as the just-perceptible difference for a change from no stimulus, while difference thresholds

represent the just-perceptible difference from a particular stimulus level greater than zero.

3.4.1 Method of Adjustment

The method of adjustment is the simplest and most straightforward technique for deriving threshold data. In this method, the observer controls the stimulus magnitude and adjusts it to a point that is just visible. The threshold is taken to be the average setting across a number of trials by one or more observers.

3.4.2 Method of Limits

The method of limits is only slightly more complex than the method of adjustment. With this method, the experimenter presents the stimuli at predefined discrete intensity levels in either ascending or descending series. In the ascending series, the experimenter presents a stimulus, beginning with one that is certain to be imperceptible, and asks the observers to respond ‘yes’ if they perceive it and ‘no’ if they do not. If they respond ‘no’, the experimenter increases the stimulus intensity. The descending series begins with a stimulus intensity that is clearly perceptible and continues until the observers respond ‘no’, they cannot perceive the stimulus. The threshold is taken to be the average stimulus intensity at which the transition from ‘no’ to ‘yes’ responses occurs for a number of ascending and descending series.

3.5 Stevens Effect

According to the classic psychophysical brightness scaling experiment from Stevens, perceived brightness could be expressed as power function of physical luminance [24].

ψ = k ( L − L

)

where k is a constant, β is brightness (psychological magnitude), L is luminance, L0 is the absolute threshold value. All the parameters, k, L0, and β change systematically with light

adaptation. The exponent β increases from 0.33 for the dark-adapted eye to 0.44 for the eye adapted to 1 lambert.

Furthermore, in J. C. Stevens and S. S. Stevens’ study, observers were asked to perform magnitude estimations on the brightness of stimuli across various adapting conditions. The results illustrated that the relationship between perceived brightness and measured luminance tended to follow a power function.

Figure 3-5. Results of determining the brightness functions under different states of adaptation [24].

A relationship that follows a power function when plotted on linear coordinates becomes a straight line on log-log coordinates, and shows in Figure 3-6. The Stevens effect indicates that, as the luminance level increases, dark colors will appear darker and light colors will appear lighter. While Stevens effect was demonstrated by viewing an image at high and low luminance levels, a black-and-white image is particularly effective for this demonstration. At a low luminance level, the image will appear of rather low contrast. White areas will not appear very bright, and the dark areas will not appear very dark. If the image is then moved to a significantly higher level of illumination, white areas appear substantially brighter and dark areas darker, it meant the perceived contrast has increased.

Figure 3-6. Change in lightness contrast as function of adapting luminance according to the Stevens effect [25].

3.6 Bartleson-Breneman Effect

Bartleson and Breneman conducted psychophysical experiments to investigate the perceived contrast of elements in complex stimuli (images) and how it varied with luminance level and surround [26]. Their experimental results were similar to those described by the Stevens effect with respect to luminance changes. They also observed some interesting results with respect to changes in the relative luminance of an image’s surround.

Figure 3-7. Changes in lightness contrast as a function of surround relative luminance according to the results of Bartleson and Breneman [22][29].

Their experimental results, obtained through matching and scaling experiments, showed that the perceived contrast of images increased when the image surround was changed from dark to dim to light. This effect occurs because the dark surround of an image causes areas to appear lighter while having little effect on light areas. Thus, since there is more of a perceived change in the dark areas of an image than in the light areas, there is a resultant change in perceived contrast.

Chapter 4

Psychophysical Experiments

4.1 Objective and Background

In Chapter 3, two well-known visual phenomena are introduced. They both are associated with contrast. The Stevens’ effect predicts that the perceived contrast of a simple stimulus (e.g. color) increases with the surround illumination [24]. The Bartleson-Breneman effect predicts that the perceived contrast of complex stimuli (e.g. an image) increases with the surround illumination [26]. Recently, the relationship between image contrast and surround illumination was studied by Liu [27]. However, the interaction between brightness and apparent contrast of complex stimuli has not been investigated yet.

The goal of this study is to establish the relation between perceived brightness and apparent contrast of natural images, and apply the results to display designs such as power minimization of transmissive TFT-LCDs. Our objective is to answer the following question:

When the brightness of an image is reduced, can we compensate for it by increasing its contrast? If the answer is positive, then this concept can be used in low-power applications.

In this study, psychophysical experiments were employed to find the answer of the question. The psychophysical experiment consists of two steps. In the first step, the sinusoidal grating pattern was equipped for validating the interaction between perceived contrast and brightness. Furthermore, the complex stimuli were used to explore the same approach.

4.2 Perceived Brightness versus Contrast in Sinusoidal Grating Pattern

The psychophysical experiments were conducted in a dedicated darkroom. The experimental setup and apparatus are shown in Figure 4-1. A 17” CRT monitor (Viewsonic

E71f ) was used to display the sinusoidal grating patterns, which were generated by MATLAB.

The resolution of the CRT is 1024×768, and the sinusoidal grating pattern in center of the display is 256×256. The distance between the observer and the monitor is 150cm.

Figure 4-1. The experimental apparatus.

The sinusoidal grating pattern, which had 1.6 circles per degree (CPD), was vertically divided into two parts as left-hand side and right-hand side. The pattern of left-hand side presented the original pattern, which had reduced brightness; the right-hand side one presented the contrast-enhanced pattern. However, the patterns of both sides can be switched randomly for the purpose of psychophysical experiments, which are shown in Figure 4-2.

Figure 4-2. The sinusoidal grating pattern with varied apparent brightness and contrast.

We used method of adjustment, a classical psychophysical method [23]. The original pattern was reduced brightness from 100% to 60%. The experiment was divided into eight trials. Each trial consists of a series of differently contrast-enhanced patterns according to reduced brightness of the original pattern. If the variation of contrast enhanced patterns was

contrast-enhanced pattern was limited to 200%. Each observer was asked to compare the original pattern against a series of differently contrast-enhanced patterns, and to find the most resemble one. In order to avoid the observer guessing the resemble pattern, the patterns of right-hand side and left-hand side were switched randomly. It meant the original pattern and contrast-enhanced pattern were not fixed on the same side.

Four observers were enlisted as subjects in the experiments. They were Asian male, aged from 23 to 29, and have normal vision after lens correction. Before starting the experiments, the subjects were asked to adapt the surround illumination for 30 seconds. Figure 4-3 shows the experimental results of four subjects as contrast vs. related brightness in the eight trials.

Each curve represents an individual observer. All curves show the same tendency. This tendency points out the subjects agreed on the same isoluminance point under increasing contrast while the brightness was reduced. However, we were specifically interested in the region of brightness between 0.7 and 0.9. The experimental results show the relation between the contrast-enhancement and related brightness is close to linear in this region. The results indicate the subjects had similar perception while they viewed the two sinusoidal patterns -- one without adjustment and the other with reduced brightness and enhanced contrast -- in a specific region. Thus, the interaction between the apparent contrast and brightness is validated.

0.6 0.7 0.8 0.9 1.0 0.6

0.8 1.0 1.2 1.4 1.6 1.8 2.0

Contrast

Related Brightness

Corey Geoff Andy May

Figure 4-3. Contrast versus related luminance.

4.3 Perceived Brightness versus Perceived Contrast in Complex Stimuli

In this study, we investigated the relationship between perceived image brightness and contrast by conducting psychophysical experiments. Thirty-seven observers were enlisted to perform visual experiments of determining the optimal contrast enhancement with reduced brightness.

4.3.1 Darkroom and Apparatus

The experiments were conducted in a light-proof darkroom, in which the measurable illumination is less than 0.03 lx (cf. Figure 4-4).

Figure 4-4. The setup of darkroom.

A set of six identical LED lights were installed for controlling the surround illumination.

Each LED light contains three channels -- red, green, and blue. The luminance and chromaticity of the six LED lights can be controlled freely (cf. Figure 4-5). The surround

Each LED light contains three channels -- red, green, and blue. The luminance and chromaticity of the six LED lights can be controlled freely (cf. Figure 4-5). The surround