The objective of this thesis is to develop color optimization method to improve the color accuracy, maintain image details, and reduce power consumption on the current green display systems. This thesis is organized as follows: The prior arts of high dynamic range display and field sequential color display are presented in Chapter 2. The proposed color optimization method for colored-backlight in HDR-LCD will be described and demonstrated in Chapter 3.
Four FSC methods, optimized RGBWmin, optimized RGBD, RGBWw, and RGBDw, will be illustrated and discussed in Chapter 4. Finally, the conclusion and future work are given in Chapter 5.
9
Chapter 2
Prior Arts in HDR-LCDs & FSC-LCDs
The concept and prior arts of high dynamic range LCDs and field sequential color LCDs will be introduced in this chapter.
2.1 High Dynamic Range LCDs with Local Backlight Control
The contrast ratio and power consumption will be enhanced by using local dimming backlight. The local dimming technology comprises the intensity control methods and color control methods.
2.1.1 Hardware Structure
In conventional LCDs, luminance levels of emitted lights at each pixel are controlled by polarization states of liquid crystal. The light leakage at the dark state reduces a contrast ratio results from defective LC polarizer. Therefore the HDR-LCDs combine dimming backlight and liquid crystal panel to yield the full-color image to improve a higher contrast ratio, as shown in 2-1.
BL Module LC Panel
Optics
Fig. 2-1 HDR-LCD structure
10
2.1.2 Algorithm for HDR-LCD
High dynamic range display consists of LED backlight panel with backlight divisions and LC panel. A complete rendering algorithm of HDRD is illustrated in Fig. 2-2 [2]. In the beginning, the algorithm evaluates square roots of original HDR image with intensity I (step1) for improving luminance. The resulted image (step 2) derives the target intensity (IL) for each individual LED (step 2a). The image samples to resolutions of LED array and the intensity required LC signals is calculated according to the inverse of the panel’s response function r2
(step 6).
To evaluate values of LCD signals, taking overlaps of PSF into account. The solution can be approximated by single Gauss-Seidel iteration over neighboring LED pixels, since the PSF of a LED affects the whole backlight intensity. This approach to compensate for differences between the LED values and the target image relies on the LCD panel. Therefore, the forward-simulated low-frequency image (step 4) generated by the LED panel is to derive the LCD pixel values. The LED image is low-pass filtered.
The HDR display with local backlight dimming could reach higher contrast ratio than that of the conventional display. Moreover, the illumination plays an essential role in visual perception from literature.
11
(a)
(b) (c)
(d) (e)
(f) (g)
Fig. 2-3 (a)Target image (Robot) : convolution results of backlight signals determined by the intensity control methods: (b)Average, (c)Maximum, (d)Square root, (e)IMF methods; the color control methods: (f)DCA, (g)SCC methods, respectively
Fig. 2-2 Conventional HDR display algorithm
12
In addition, the determination of backlight signals is one of dominant factors for output image of HDR-LCDs. An appropriate backlight determination will result in high contrast ratio, efficient brightness, and less image distortion. The backlight determinations have been widely investigated in the last years. These methods could be divided into two parts: intensity control and color control. These methods are as follows.
Intensity Control Methods:
The intensity control method assessed the intensity of the target image to determine the backlight signals in local black-and-white illumination.
1. Average :
The backlight signal in each backlight region is determined by taking the average gray-level of all maximum sub-pixel values in this method, as shown in Fig. 2.3 (b). The contrast ratio could be enhanced well with the dark illumination. However, the output image may also be dark due to limitation of compensated LC transmittance (T<100%).
2. Max :
The backlight signal in each backlight division is determined according to the maximum gray-level of the maximum sub-pixel values in this method, as shown in Fig. 2.3 (c). The backlight illumination could be improved much more. However, the dark state may be bright to result in low contrast ratio.
3. Root:
The root method, proposed by Brightside [2], is to calculate the average value of each BL division by taking the square-root operation on normalized BL signals from the average gray-level values, as shown in Fig. 2.3(d). This method could enhance the whole backlight illumination. The image details for dark state can be maintained. However, the bright state reaches insufficient luminance.
13
4. Inverse of a Mapping Function (IMF) :
The IMF method, proposed by F.C. Lin, et al. [15], is to calculate maximum of the maximum sun-pixel values for each backlight zone. By using the modulation curve established according to the information of each frame, the backlight signals are optimized, as shown in Fig. 2.3(e). The IMF method not only keeps high contrast ratio but also maintains the maximum luminance well.
The intensity control methods provide adequate intensity backlight and acceptable image quality. However, the power dissipation can be reduced more by assessing the image contents in color control backlight.
Color Control Methods:
The color control method pondered the image information in three dimensions to modulate the backlight signals of RGB channels individually. These methods could produce colorful backlight distribution close to the image contents.
1. Delta-Color Adjustment (DCA):
The DCA method is to optimize the backlight image by modulating three dimensions (R, G, and B) backlight signals based on the adjusted results of intensity control, as shown in Fig.
2.3(f) [18].
2. Segment Color Control (SCC):
In the SCC method, each block of three dimensions image is decided to map different segments by taking the average algorithm individually. The optimized segment method is processed by various algorithms, such as average, root, and max method, as shown in Fig.
2.3(g) [19].
The color control backlight optimizes the backlight signals in three dimensions according to the image information and has unique feature in high contrast ratio and low power consumption. However, the complicated backlight increases the color distortion more than that of intensity backlight control.
14
2.1.3 Compensation of Liquid Crystal for Intensity Control Backlight
The dimming backlight decreases the luminance of the yielded image in HDR-LCD system. The transmittance of LC cell must be modulated to compensate the luminance decrease [16][17]. Pondering the algorithm of HDRD mentioned in the previous section, the light spread function (LSF) of each LED groups is measured first, as shown in Fig. 2-4. The real backlight distribution is simulated by convolving the backlight signal with the LSF. The compensational pixel values are derived in the second panel-LC panel. The procedure of convolution operation is shown in Fig. 2-5. The light distribution of dimming LED backlight could be simulated.
Fig. 2-4 Light spread function (LSF)
Fig. 2-5 Backlight distribution convoluted by the light spread function according to the backlight signals
15
2.1.4 Summary of HDR-LCD
The concept of high dynamic range display (HDRD) and backlight determination have been studied recently. In the traditional intensity backlight system, the “intensity” model has been proposed to compensate LC signals for maintaining the brightness as the target image, as shown in Fig. 2-6 [16]. The compensation signals of liquid crystal (GLHDR) were obtained from simulated distribution of backlight illumination (BLHDR) and original signals (GLFull), as shown in Eq. (2-1). The gamma effect ( ) of the display device was evaluated to keep brightness.
Furthermore, the color control backlight has been proposed to improve power consumption, color saturation, and contrast ratio. However, the traditional LC compensation in intensity domain is insufficient for colorful backlight distribution. By applying optimization method in LC signals, the optimized HDR image achieved accurate colorimetric color reproduction, as shown in Fig. 2-7 [28]. The color shift phenomena were suppressed with small CIEDE2000 color-difference value (∆E00) [29]. Due to accurate color reproduction, image details were maintained as well. Therefore, the objective of this thesis is to develop a
“color optimization model” for redistributing the LC signals to yield high color accuracy on HDR-LCD [30][31][32].
Fig. 2-6 Flowchart of a conventional HDR image
Fig. 2-7 Flowchart of an optimized HDR image
Target
16
2.2 Field Sequential Color Liquid Crystal Display
The mechanism of eye movement, such as smooth pursuit and saccade, will affect the image quality, especially for FSC-LCDs. The CBU phenomenon is classified into two parts:
static and dynamic CBU based on the eye movement. Several CBU suppression methods have proposed to improve the image quality recently.
2.2.1 Human Color Vision
The human eye is a complicated visual system, as shown in Fig. 2-8 [24]. The optical image formed by the eye is projected onto the retina. The retina incorporated the visual system’s photosensitive cells, such as photoreceptions, and initial signal processing and transmission circuitry, as shown in Fig. 2-9 (a). The photoreceptors, rods and cones, sever to transducer the information present in the optical image, as shown in Fig. 2-9 (b). The rods serve luminous vision at low luminance levels while the cones serve color vision at high luminance.
Fig. 2-8 Schematic diagram of the human eye
17
Three types of cones are referred to as L, M, and S cones named according to the long-wavelength, middle-wavelength, and short-wavelength, as shown in Fig. 2-10 (a). The outputs of all three cone types are summed (L+M+S) to produce an achromatic response that matches the CIE V(λ) as long as the summation is taken in proportion to the relative populations of the three cone types. Differencing of the cone signals allows construction of red-green (L-M+S) and yellow-blue (L+M-S) opponent signals. The transformation from LMS signals to the opponent signals serves to decorrelate the color information carried in the three channels, thus allowing more efficient signal transmission, as shown in Fig. 2-10 (b).
The three opponent pathways also have distinct spatial and temporal characteristics that are important for predicting color appearance for human perception.
(a) (b)
Fig. 2-9 (a) Schematic diagram of the neurons in the human retina, and (b) Rod and cone photoreceptors
18
2.2.2 Physiology of Eye Movement
The observers could perceive optical image by converting light into electrochemical signals and transporting the signals along the optic neurons in the brain. Human eyes can move to track the object clearly. Two kinds of eye movement, saccade and smooth pursuit, will be discussed as follows [33].
For the saccade phenomena, human eye will move rapidly around the target to focus on the fovea for gathering correct visual information [34][35][36][37]. The example of the saccade movement on an image is shown Fig 2-11 in which the white line is the movement of the eyes. The saccade is a spontaneous phenomenon, and the velocity of movement is up to 200 degree/sec [38].
The other type of eye movement is smooth pursuit. Human eye will follow the object at the same velocity to focus on the fovea to perceive clear image while recognizing the dynamic target. The pursuit is much slower and the velocity of movement is about 90 degree/sec [39].
A R-G
Y-B
(a) (b)
Fig. 2-10 (a) Illustration of the encoding of cone signals into opponent-colors, and (b) Opponent-color spectrums
19
2.2.3 Mechanism of Color Break-up (CBU)
The FSC-LCD displays at least three color fields (R, G, and B field) to constitute a full color image. The frequency of the three color fields should be higher than the time resolution of eye. The yielded images will be projected onto the retina in the same position, thus human eyes will perceive a complete full color image. However, the relative velocity between the target and human eye results in color break-up (CBU) artifact (or rainbow effect). The mechanism of CBU phenomena are as follows.
While perceiving a stationary image, human eye performs saccadic movement for receiving CBU along the motion direction. For example, the image with three static white bars and the gray path of a saccade is shown in Fig 2-12 (a). While human eye moves from the left to right, the static CBU image will be yielded, shown in Fig 2-12 (b) [40].
Fig. 2-11 An example of saccadic eye movement
20
The perceived image is separated into three color fields respectively due to a rapid saccade. The color sequential operating frequency of the LCDs will have influence on the amount of static CBU. Therefore, some FSC methods increase the field rate to suppress CBU issue.
The other type of CBU is dynamic CBU while tracking a dynamic image, as shown in Fig 2-13 (a). While the white image moves from left to right, human eye will pursue the moving image. In the meantime the FSC-LCDs will display multiple color fields sequentially.
However, different color fields can be perceived separately on the edge by smooth pursuit eye motion and temporal integration in the visual system. The schematic diagram for illustrating the dynamic CBU issue is shown in Fig 2-13 (b) [41].
(a) (b)
Fig. 2-12 (a) Static white image in black background and the path of eye movement, and (b) Generation of static CBU
21
2.2.4 Prior Solutions in CBU Suppression
In recent years, there has been a dramatic proliferation of research for CBU suppression in FSC-LCDs. The field rate increasing method, such as RGBRGB (360Hz), and RGBKKK (360Hz), could decrease CBU width so that human eye will be less sensitive to the CBU phenomenon. Since the liquid crystal response and TFT scanning time are limited, it is extremely difficult to implement these above methods on FSC-LCDs.
(a)
(b)
Fig. 2-13 (a) Temporally displayed color fields integrated on separate position, and (b) Dynamic CBU in visual perception
22
Furthermore, the multi-primer color fields proposed by Tatsuo Uchida research group in Tohoku University inserted multi-primary color fields to suppress CBU [42]. The Stencil, RGBWmin, and RGBD method, proposed by National Chiao Tung University, have been implemented in 15.4” and 32” FSC-LCDs, as shown in Fig. 2-14. The perceived CBU image on RGBWmin method and RGBD method are illustrated in Fig. 2-15. Since the RGBD method takes the image contents as consideration to determine the dominant color field (D-field), the CBU artifact is suppressed more.
Fig. 2-14 Conventional RGB, RGBWmin, and RGBD method Target Image
Method Light
Source 1st Field 2nd Field 3rd Field 4th Field
RGB 180Hz
3-in-1 LED
RGBWmin 240Hz
3-in-1 LED
RGBD 240Hz
3-in-1 LED
23
RGB RGBWmin
Space robot Airplane Baboon Lena Tiffany Avg.
80 90 100 110 120 130 140
Relative Power Ratio (%) RGBD
(a) (b)
(c) (d) (e)
Fig. 2-15 (a) Static target image and (b) details of (a); and the CBU images in: (c) RGB method, (d) RGBWmin method, and (e) RGBD method
Fig. 2-16 Power consumption compared with RGB, RGBWmin, and RGBD methods
24
However, the yielded image on the FSC-LCDs can’t achieve accurate color reproduction, as shown in Fig. 2-15. Therefore, the color model will redistribute the liquid crystal signals in optimized RGBW/RGBD methods to achieve accurate color reproduction. Moreover, comparing to power consumption of original RGB method, the above two methods increase the power consumption, especially in RGBWmin method, as shown in Fig. 2-16. Novel RGBWw and RGBDw methods were proposed to reduce power dissipation by using RGBW LED backlight.
2.2.5 Color-Difference Equation CIEDE2000 (Delta E
00)
To describe the “color” we received in the visual system, the color matching function was created. The color-matching experiment is contrasted in which red, green, and blue primaries are projected onto a screen, as shown in Fig 2-17. The intensity of each primary can be adjusted by the observer. In the bottom test field, a test light is also projected (shown as cyan) along with a second set of identical red, green, and blue primaries. The observers will adjust the reference field until the fields are distinguishable.
During the 1920s, two experiments that were performed in England measured the color-matching functions of a small number of color-normal observers. Guild (1931) [43]
measured seven observers and Wright (1928, 1929) [44] measured ten observers. Both of the experiments employed the same viewing conditions, a bipartite field subtending a 2 degree visual angle that was surrounded by darkness. In 1931, the Colorimetry Committee of CIE selected three primaries (435.8 nm, 546.1 nm, and 700nm) and 17 color-normal observers with 2 degree visual angle in the color-matching experiment [45]. The average result of color-matching function is illustrated in Fig 2.18 (a).
25
However, the λ, λ, and λ color-matching functions all have both positive and negative tristimulus values, such the devices would have to have six channels, greatly increasing their complexity and cost. Therefore, the transformed system is called X, Y, Z system with color-matching functions of λ, λ, and λ, as shown in Fig 2-18 (b) [46][47]. This system is often referred to as the 1931 standard observer (or the 2 degree observer). The transformation represents the color matching results of the average of the human population having normal color vision. The color is affected by three components, illuminant (P), object’s reflectance factor (R), and the tristimulus values of human eye ( , and ) as shown in Eq. 2-2, the CIE tristimulus could be defined by XYZ tristimulus by those components.
Fig. 2-17 Setup of the color-matching experiment
(a) (b) Fig. 2-18 (a) Original color-matching function, and (b) Transformed color-matching function
26
However, the CIE XYZ color space is not uniform to describe color difference. Several color spaces have been studied to quantify the amount of color difference for examining color quality until now. The CIELAB color space has been intended for equal perceptual differences for equal changes as the uniform color space close to human opponent vision most.
The non-linear transformations from CIE XYZ to CIELAB are described in Eq. 2-3 [48]. The coordinates L*, a* and b* represented the lightness (L*), color component of red-green (a*), and color component of yellow-blue (b*), as shown in Fig 2-19. The CIELAB provided a uniform chromaticity diagram so that most of the color-difference equations were established based on CIELAB.
Fig. 2-19 3D views of CIELAB color space
L*
b* a*
27
Furthermore, The CIE committee created the CIEDE2000 color-difference formula in 2000 [20]. The CIEDE2000 color-difference equation is also developed based on CIELAB color space. The ∆L*, ∆C*, and ∆H* are the CIELAB metric lightness (L*), chroma (C*), and hue (H*) differences, respectively, calculated between the standard and sample in a pair. ∆R is an interactive term between chroma and hue differences, as shown in Eq. 2-4. The SL, SC, and SH are the weighting functions for the lightness, chroma, and hue components individually.
The weighting values vary according to the positions of the sample pair being evaluated in CIELAB color space. The kL, kC, and kH values are the parametric factors to be adjusted according to different viewing parameters such as textures, backgrounds, separations, etc., for the lightness, chroma, and hue components, respectively.
CIEDE2000 color-difference formula includes lightness, chroma, and hue weighting functions. An interactive term between chroma and hue differences improves the performance for blue colors. A scaling factor for CIELAB a* scale enhances the performance for gray colors.
Four reliable color discrimination datasets based on object colors were accumulated and combined. The equation was tested together with the other advanced CIELAB based equations using the combined dataset and each individual dataset. It outperformed other color-difference equations (CIELAB, CIELUV, CIE94, CMC, and BFD). The CIEDE2000 (or ∆E00) could be applied to examine the color accuracy on wide color gamut display and
28
phenomenon by the ΔE00 index between the CBU image and the original image to verify the proposed methods.
2.2.6 Summary of FSC-LCD
In previous discussion, the RGBW and RGBWmin methods suppressed the CBU.
However, the yielded image on the FSC-LCDs suffered from color distortion, as shown in Fig.
2-15. Therefore, by using RGB scanning backlight modeling, the liquid crystal signals of each color field were redistributed in optimized RGBW/RGBD methods.
In recent years, the image quality about color appearance and power consumption are concerned by observers. The original RGBW/RGBD also increased power dissipation due to four color sequence driving. Therefore, the RGBWw and RGBDw methods were proposed to reduce power consumption in this thesis. RGBW LEDs are implemented as light source due to powered W LEDs provide most luminance in the white (or dominant) field instead of mixing color from RGB LEDs. The RGBWw method performs R, G, B, and W fields and turns on R, G, B, and W LEDs simultaneously. The RGBDw displays the R, G, B, and dominant fields (Dw-field) in which the W LEDs provide luminance. To achieve accurate color reproduction in RGBWw and RGBDw methods, the color optimization model was useful to maintain the color accuracy.