Chapter 3 Principle: Assess Visual Quality
3.6 Bartleson-Breneman Effect
Bartleson and Breneman conducted psychophysical experiments to investigate the perceived contrast of elements in complex stimuli (images) and how it varied with luminance level and surround [26]. Their experimental results were similar to those described by the Stevens effect with respect to luminance changes. They also observed some interesting results with respect to changes in the relative luminance of an image’s surround.
Figure 3-7. Changes in lightness contrast as a function of surround relative luminance according to the results of Bartleson and Breneman [22][29].
Their experimental results, obtained through matching and scaling experiments, showed that the perceived contrast of images increased when the image surround was changed from dark to dim to light. This effect occurs because the dark surround of an image causes areas to appear lighter while having little effect on light areas. Thus, since there is more of a perceived change in the dark areas of an image than in the light areas, there is a resultant change in perceived contrast.
Chapter 4
Psychophysical Experiments
4.1 Objective and Background
In Chapter 3, two well-known visual phenomena are introduced. They both are associated with contrast. The Stevens’ effect predicts that the perceived contrast of a simple stimulus (e.g. color) increases with the surround illumination [24]. The Bartleson-Breneman effect predicts that the perceived contrast of complex stimuli (e.g. an image) increases with the surround illumination [26]. Recently, the relationship between image contrast and surround illumination was studied by Liu [27]. However, the interaction between brightness and apparent contrast of complex stimuli has not been investigated yet.
The goal of this study is to establish the relation between perceived brightness and apparent contrast of natural images, and apply the results to display designs such as power minimization of transmissive TFT-LCDs. Our objective is to answer the following question:
When the brightness of an image is reduced, can we compensate for it by increasing its contrast? If the answer is positive, then this concept can be used in low-power applications.
In this study, psychophysical experiments were employed to find the answer of the question. The psychophysical experiment consists of two steps. In the first step, the sinusoidal grating pattern was equipped for validating the interaction between perceived contrast and brightness. Furthermore, the complex stimuli were used to explore the same approach.
4.2 Perceived Brightness versus Contrast in Sinusoidal Grating Pattern
The psychophysical experiments were conducted in a dedicated darkroom. The experimental setup and apparatus are shown in Figure 4-1. A 17” CRT monitor (Viewsonic
E71f ) was used to display the sinusoidal grating patterns, which were generated by MATLAB.
The resolution of the CRT is 1024×768, and the sinusoidal grating pattern in center of the display is 256×256. The distance between the observer and the monitor is 150cm.
Figure 4-1. The experimental apparatus.
The sinusoidal grating pattern, which had 1.6 circles per degree (CPD), was vertically divided into two parts as left-hand side and right-hand side. The pattern of left-hand side presented the original pattern, which had reduced brightness; the right-hand side one presented the contrast-enhanced pattern. However, the patterns of both sides can be switched randomly for the purpose of psychophysical experiments, which are shown in Figure 4-2.
Figure 4-2. The sinusoidal grating pattern with varied apparent brightness and contrast.
We used method of adjustment, a classical psychophysical method [23]. The original pattern was reduced brightness from 100% to 60%. The experiment was divided into eight trials. Each trial consists of a series of differently contrast-enhanced patterns according to reduced brightness of the original pattern. If the variation of contrast enhanced patterns was
contrast-enhanced pattern was limited to 200%. Each observer was asked to compare the original pattern against a series of differently contrast-enhanced patterns, and to find the most resemble one. In order to avoid the observer guessing the resemble pattern, the patterns of right-hand side and left-hand side were switched randomly. It meant the original pattern and contrast-enhanced pattern were not fixed on the same side.
Four observers were enlisted as subjects in the experiments. They were Asian male, aged from 23 to 29, and have normal vision after lens correction. Before starting the experiments, the subjects were asked to adapt the surround illumination for 30 seconds. Figure 4-3 shows the experimental results of four subjects as contrast vs. related brightness in the eight trials.
Each curve represents an individual observer. All curves show the same tendency. This tendency points out the subjects agreed on the same isoluminance point under increasing contrast while the brightness was reduced. However, we were specifically interested in the region of brightness between 0.7 and 0.9. The experimental results show the relation between the contrast-enhancement and related brightness is close to linear in this region. The results indicate the subjects had similar perception while they viewed the two sinusoidal patterns -- one without adjustment and the other with reduced brightness and enhanced contrast -- in a specific region. Thus, the interaction between the apparent contrast and brightness is validated.
0.6 0.7 0.8 0.9 1.0 0.6
0.8 1.0 1.2 1.4 1.6 1.8 2.0
Contrast
Related Brightness
Corey Geoff Andy May
Figure 4-3. Contrast versus related luminance.
4.3 Perceived Brightness versus Perceived Contrast in Complex Stimuli
In this study, we investigated the relationship between perceived image brightness and contrast by conducting psychophysical experiments. Thirty-seven observers were enlisted to perform visual experiments of determining the optimal contrast enhancement with reduced brightness.
4.3.1 Darkroom and Apparatus
The experiments were conducted in a light-proof darkroom, in which the measurable illumination is less than 0.03 lx (cf. Figure 4-4).
Figure 4-4. The setup of darkroom.
A set of six identical LED lights were installed for controlling the surround illumination.
Each LED light contains three channels -- red, green, and blue. The luminance and chromaticity of the six LED lights can be controlled freely (cf. Figure 4-5). The surround illumination was projected on a white screen as the field of view. A pair of identical transmissive 19” TFT-LCD monitors (Viewsonic VX912) was used to display the test images (cf. Figure 4-6). They were driven by the same computer, and color-corrected by a GretagMacBeth Eye-One XT spectrophotometer (cf. Figure 4-7).
(a) LED light. (b) The LED lights project white surround.
Figure 4-5. The LED lights.
Figure 4-6. TFT-LCD monitors used in the experiments.
a b
Figure 4-7. GretagMacBeth Eye-One XT.
4.3.2 Procedure
The psychophysical experiments were performed for three different levels of surround – 0%, 50%, and 100% of the full luminance of the monitors. The test image was chosen from a popular set of benchmark images from the HDR research community (www.devevec.org).
The original softcopy was displayed by the left-hand-side monitor with 100% of backlighting.
The contrast-adjusted softcopy was displayed by the right-hand-side monitor with 70% of backlighting. The adjusted image was displayed on the right with 70% of backlight intensity after the following contrast enhancement
255
Most of the thirty-seven observers were Asian male, aged from 22 to 28, and have normal vision after lens correction. The subjects were screened by a simple Ishihara Color
Deficiency Test, and two of them were excluded based on the test results.
Figure 4-8. Matching the original softcopy (left) with the contrast-adjusted one (right).
The experiment started with a 30 seconds period for the subject to adapt to the surround illumination while the monitors displayed gray fields of the same luminance. The subject was asked to match the original softcopy with the contrast-adjusted one (cf. Figure 4-8). The classical psychophysical method, Method of Limits [23], was used. A series of differently contrast-enhanced images in ascending/descending order was displayed sequentially (cf.
Figure 4-9).
Figure 4-9. The image contrast was modulated between high (left) and low (right).
The subject had to decide if the point of isoluminance was reached. The corresponding contrast adjustment at the point of isoluminance was recorded for analysis. The experiments were repeated for three times with surround at 0%, 50%, and 100%.
4.3.3 Experimental Results
Figure 4-10 shows the experimental data as subject count vs. contrast adjustment in the ascending and descending trials. The top/bottom curve represents the descending/ascending
data. Each point indicates the number of subjects who agreed on the same isoluminance point.
Both ascending and descending distributions are near Gaussian with a mean value around 120% contrast-adjustment. It indicates that most subjects agreed on this point of isoluminance. The mean of the ascending trial is greater than that of the descending trial. The reason is the persistence of visual perception, which is common in experiments using Method of Limits.
Figure 4-10. Subject count vs. contrast adjustment for ascending (bottom) and descending (top) trials.
4.4 Summary
According to the experimental results, decreased brightness can be compensated by increasing luminance contrast. The experimental results can be applied to low-power display design. In a transmissive TFT-LCD, reducing the backlight intensity to save power consumption, at the cost of degraded image quality, is a common practice. Based on our findings, one can increase the contrast to recover the image quality.
To compare our psychophysical results with [10], we calculated the image quality of the
same image. The relation between image quality and contrast enhancement is shown in Figure 4-11.
Figure 4-11. Contrast fidelity vs. contrast.
We found high resemblance between Figure 4-10 and Figure 4-11. In other words, for the given image, our psychophysical findings are in accordance with the analytical model in [10]. Although further experiments are undergoing for more concrete conclusions, in this work we adopted the CBCS algorithm.
Chapter 5
Proposed Algorithm
We propose a backlight scaling algorithm for an LED backlit display. The proposed algorithm consists of two phases. In the first phase, chromaticity scaling, the image is chromatically scaled subject to the perceivable colorfulness difference. The optimal ratio of RGB backlights is found to achieve maximum power savings at the same luminance. In the second phase, luminance scaling, the luminance is scaled down to achieve maximum power savings subject to the perceivable lightness difference.
5.1 Chromaticity Scaling
Figure 5-1. Photo and R, G, and B histogram.
The motivation of chromaticity scaling is that most images have different distributions in their RGB histograms. To achieve more power savings, the RGB channels can be scaled differently. For example, in Figure 5-1, in average the red channel has higher luminance than the blue channel does. This observation implies that the blue backlight can be scaled down more than the red backlight. However, the side effect of scaling RGB channels differently is color shift. Therefore, we use the color difference defined by Equation (3-5) to govern the chromaticity scaling. We define the following notation to represent an image after
chromaticity scaling.
respectively, the red, green, and blue sub-pixel will be clipped by bR, bG, and bB respectively.
We use ∆E(I1, I2) to indicate the average color difference between the original image I1 and chromaticity-scaled image I2. Given Q as the color shift constraint, the following algorithm finds the optimal chromaticity scaling factors that save the most power consumption with the least color shift. The procedure of finding backlight factor shows in Figure 5-2.
Chromaticity_Scaling:
goto Chromaticity_Scaling;
else
return bR, bG, and bB
Figure 5-2. Procedure of finding backlight factor.
5.2Luminance Scaling
The original CBCS algorithm [10] is used in the second phase. The affined linear transformation of a pixel pL can be parameterized by
,
The image quality is defined by
),
0, 0
and PDF(x) is the probability density function of pixel value x. The RGB backlights are scaled simultaneously by a factor bL such that no more color shift will be introduced. Since the ratio between the RGB is fixed, this phase is called luminance scaling. The following LED power equation is used.
PR(bLbR)+PG(bLbG)+PB(bLbB) (5-6)
The objective function is to find the optimal gl, gu, and bL that minimize Equation (5-6) subject to the given image quality constraint FC.
Sample transfer functions of chromaticity and luminance scaling are shown in Figure 5-3.
Figure 5-3. Transfer functions of chromaticity and luminance scaling.
5.3 Summary
In our algorithm, the chromaticity scaling is employed to find the maximum power consumption which can be saved in an image while the color difference ∆E is determined. We reduce the backlight intensity by using the results of chromaticity scaling.
Luminance scaling uses CBCS algorithm. The transfer function changes the pixel data for compensating the luminance loss while the backlight intensity is reduced. The algorithm is the foundation that constructing a platform of the low power display.
0 x
Chapter 6
Prototype
6.1 Framework
For implementing our algorithms of backlight scaling, a prototype of experimental platform is necessary. Unfortunately, it is too difficult to get an appropriate hardware for matching the portable electronic device. Furthermore, the structure and optical mechanism of the portable electronic device can be easily shifted or damaged after the panel refitted.
Therefore, it is critical to determine a prototype to overcome these difficulties.
Figure 6-1. Backlight scaling of portable electronic device.
The design of our prototype is changed from the portable electronic device system (small size panel) to the laptop computer system (large size panel) through the support of CPT Corporation. Because our purpose is to reduce the power consumption of the TFT-LCD backlight, this switch has no influence on our implementation. There are several benefits in this switch. A ready hardware (FPGA board) from the support of CPT Corporation can be used to design this prototype. Refitting the large size panel is easier than refitting the small ones for us. Moreover, for a panel manufacturer, they can apply the technology of large size
panel to the small size panel readily. The power consumption of the LCD backlight can be measured more conveniently.
In our framework, the backlight scaling algorithm is realized in a Field Programmable Gate Array (FPGA) board. The FPGA board is used for image processing and backlight controller, which connects the TFT-LCD and the personal computer. The TFT-LCD displays the images before/after the backlight scaling process with the FPGA, while we measure the power consumption of TFT-LCD backlight.
Figure 6-2. Our notion of backlight scaling.
6.1.1 Image Processing Flow
In image data processing flow of the personal computer, first, the image data is stored in the frame buffer by the CPU, and then graphic card fetches this image data and generates appropriate analog or digital image data to the video interface. The image data is carried by VGA or DVI interface and delivered to the scaler board.
According to the panel characteristic, the scaler board automatically scales the image data to appropriate frame resolution and frame rate. Moreover, some scaler chips include extra functionality, such as video decoding, video processing and 3D graphics accelerator, etc.
Finally, the image data is translated into the signal format of TTL or Low Voltage Differential Signaling (LVDS) by scaler board and is delivered to panel controller board.
Figure 6-3. Block diagram of image processing flow.
6.2 Block Diagram of Prototype
Our goals are to reduce the power consumption of the LCD backlight and to maintain the image brightness with our backlight scaling algorithms. This consideration of fetching and processing image data from the PC will be used to design the experimental platform. There are different methods available to fetch the image data; one is to fetch the image data as it is delivered by graphic card, and another is to fetch the image data as it passes the scaler board.
It is more complex to process image data with the first method. Furthermore, some scaler boards include the functionality of image processing. While the image data passes the scaler board, the image data is probably adjusted or changed by scaler board. Base on the above reasons, we chose the second method. Unfortunately, because of the higher resolution TFT-LCD panel, the image data is translated into LVDS signal format by the scaler board while the image data passes the scaler board. LVDS signal format will complicate the image
fetching and processing.
Figure 6-4. Fetching the image data as the image data is delivered by graphic card.
Figure 6-5. Fetching the image data as the image data passes the scaler board.
6.2.1 Low Voltage Differential Signaling
The signal format of LVDS is a technology addressing the needs of today’s high
performance data transmission applications. The LVDS standard is becoming the most popular differential data transmission standard in the industry.
In the TFT-LCD field, the LVDS standard is usually used in the signal transmission of the higher-resolution LCD because it requires more data flow than low-resolution LCD.
LVDS delivers high data rates while consuming significantly less power than competing technologies. In addition, it brings many other benefits, which include:
• Low-voltage power supply compatibility
• Low noise generation
• High noise rejection
• Robust transmission signals
• Ability to be integrated into system level ICs
LVDS technology allows products to address high data rates ranging from 100’s of Mbps to greater than 2 Gbps. For all of the above reasons, it has been deployed across many market segments wherever the need of speed and low power exists.
Figure 6-6. Simplified diagram of LVDS driver and receiver.
LVDS is a low swing, differential signaling technology, which allows single channel data transmission at hundreds or even thousands of megabits per second (Mbps). Its low swing and
a wide range of frequencies.
LVDS outputs consist of a current source (nominal 3.5 mA) that drives the differential pair lines. The basic receiver has a high DC input impedance, so the majority of driver current flows across the 100Ω termination resistor generating about 350 mV across the receiver inputs. When the driver switches, it changes the direction of current flow across the resistor, thereby creating a valid “one” or “zero” logic state.
6.3 Platform
6.3.1 Experimental Panel
The experimental panel is equipped with a 1280×1024, 19”, 24-bit color, transmissive, color TFT-LCD monitor, ViewSonic VX912.
In our work, LED was used to fabricate backlight module. The backlight scaling panel and backlight was evaluated by a conventional TFT-LCD monitor. The prototype was built on a VX912 monitor. Originally the LCD panel has two side-lit CCFL backlights on the top and bottom. We custom made two LED backlights to replace the CCFL backlights.
Figure 6-7. VX912 monitor and panel.
The display panel manufactured by AUO Corporation includes a major application specific integrated circuit (ASIC) and an SXGA TFT-LCD panel [30]. The image data feeds into the AUO ASIC, which includes the timing controller and RSDS transmitter. The connector receives the LVDS signal which is from the scaler board, and then delivers to the AUO ASIC. The LVDS signal is translated into RSDS by the AUO ASIC, and then delivers to TFT-LCD.
Figure 6-8. Block diagram of the AUO panel.
6.3.2 FPGA Board
In the beginning, we implement the panel controller and backlight controller with an FPGA board which is supplied by CPT Corporation and Xilinx Spartan-3 Starter Kit [31], respectively. The CPT FPGA board includes an LVDS transmitter and receiver, and an FPGA chip. The Xilinx Spartan-3 starter kit houses 200,000-gate Xilinx Spartan-3 XC3S200 FPGA in a 256-ball thin ball grid array package (XC3S200FT256), 2Mbit Xilinx XCF02S platform flash, in-system programmable configuration PROM, 1M-byte of fast asynchronous SRAM, 3-bit, 8-color VGA display port, 9-pin RS-232 serial port, 50 MHz oscillator, and several I/O ports.
Figure 6-9. Xilinx Spartan-3 Starter Kit Board [31].
The CPT FPGA board includes three major ports; they are LVDS receiver, FPGA chip, and LVDS transmitter. The function of the LVDS receiver is to translate the LVDS signal
The CPT FPGA board includes three major ports; they are LVDS receiver, FPGA chip, and LVDS transmitter. The function of the LVDS receiver is to translate the LVDS signal