AN EFFICIENT BAYESIAN FRAMEWORK FOR IMAGE ENHANCEMENT WITH SPATIAL CONSIDERATION

AN EFFICIENT BAYESIAN FRAMEWORK FOR IMAGE ENHANCEMENT

WITH SPATIAL CONSIDERATION

Tzu-Cheng Jen and Sheng-Jyh Wang

Inst. of Electronics, National Chiao Tung University, HsinChu, Taiwan

E-mail: [email protected]

ABSTRACT

In this paper, a Bayesian framework is proposed for image enhancement. We model the image enhancement problem as a maximum a posteriori (MAP) estimation problem and the posteriori distribution function is formulated based on the local structures and local gradients of the given image. By solving the MAP estimation problem, image contrast gets properly enhanced while image noise gets suppressed at the same time. Moreover, since directly solving an MAP estimation problem is impractical for real-time applications, we further simplify the process to generate an intensity mapping function that achieves comparable performance in image enhancement. Simulation results have demonstrated the applicability of the proposed method in providing a flexible and efficient way for image enhancement.

Index Terms— Image Enhancement, MAP estimation 1. INTRODUCTION

Capturing images under inappropriate illumination may produce images with insufficient contrast. To improve the quality of poorly illuminated images, an effective image enhancement algorithm is usually needed. Due to its simplicity, histogram equalization (HE) [1] is a widely used method for image enhancement. However, since the HE method only considers global statistics but neglects local information, it may cause undesired drawbacks, like over-enhancement, noise amplification, and loss of fine details.

In [2][3], the authors claim that the over-enhancement problem of HE could be prevented by preserving the intensity mean of the image. In [4], the authors formulate the mean-preserving HE problem as a constrained optimization problem and adopt variational calculus to find the optimal solution. Even though these mean-preserving enhancement methods may alleviate the over-enhancement problem, the performance of enhancement is somewhat restrained by the mean-preserving constraint.

Some other authors [5][6] claim that the over-enhancement problem of HE is due to some extremely high peaks in the histogram. Hence, they modify the shape of the histogram before the histogram equalization process is executed. In [5], a nonlinear transformation is applied for histogram modification. In [6], additional prior information is included

for the adjustment of the histogram shape. Even though these histogram modification-based approaches may effective avoid the over-enhancement problem, some drawbacks, like noise amplification and detail loss, still exist in these approaches.

On the other hand, retinex-based algorithms are also popular in image enhancement [7][8]. The key concept of these approaches is to apply different operations for the luminance component and the reflectance component of the given image. These retinex-based approaches usually offer impressive enhancement results. However, halo effects may appear around strong edges and noise component in the dark regions may get amplified after enhancement.

Unlike the aforementioned approaches, a few algorithms [9][10] adopt DCT-based approaches for the sake of compatibility with existing compression standards. The strategy of most DCT-based approaches is to adaptively scale the DC coefficients and AC coefficients for image enhancement. However, these approaches could generate undesired blocking artifacts that greatly degrade the visual quality of the image.

In this paper, we propose a Bayesian framework for image enhancement. The proposed approach takes into account both local structures and local gradients of the image. Consequently, the proposed method can efficiently enhance the contrast of the image but avoid the occurrence of undesired artifacts. Besides, based on the proposed MAP formulation, we further propose a much efficient process to automatically generate an intensity transfer function that can achieve comparable performance in image enhancement. In this paper, the basic concept of the proposed method is first introduced in Section 2 and 3. Simulation results are then presented in Section 4. Finally, conclusions are drawn in Section 5.

2. PROPOSED METHOD

In this paper, a Bayesian methodology is applied for image enhancement. Here, we aim to infer a well illuminated image f from the given image data d by maximizing a posterior probability p(f|d). In mathematics, we have

) | ( max arg * _{p f d} f f . (1) After applying the Bayes’ rule, we have

3285

978-1-4244-7994-8/10/$26.00 ©2010 IEEE ICIP 2010

)} ( ) | ( { max arg * _{p d f p f} f f , (2)

In (2), p(d|f) is the likelihood model that represents the relationship between the expected image f and the observed image d; p(f) is the prior model that represents the statistical properties of an well illuminated image. In the next section, we will introduce the proposed likelihood model and prior model for the task of image enhancement.

2.1. Likelihood model

We construct the likelihood model based on the simple but widely used image formation model: an image f can be expressed as the product of its luminance component L and reflectance component R. If ignoring the cases of over-exposure and under-over-exposure, the main difference among images with different illumination levels is the L component. Moreover, if we assume uniform illumination over the image, L can be modeled as a constant. Based on the concept of linear algebra, we have the following relation:

*f xd*| f*|˜|d*| 0, (3) where f* and d* are the 1-D vector representations of f and d, and “

x

” and “| |” indicate the inner product and l2-norm

operators, respectively. In a real image, since it is reasonable to assume uniform illumination over a local region, we propose the following likelihood model:

). |) 1 1 | | 1 1 | ) 1 1 ( ) 1 1 (( exp( ) | ( 2 , , , , , , , , * * * * * * * * * * * * " " " " " c d d c f f c d d c f f w f d p m i i m i i i m i i m i i   ˜      x   ˜  v

¦

(4) In (4), i denotes the pixel index; d*_i,"is the 1-D vector representation of the pup image data centered at pixel i; *f_i,"

is the 1-D vector representation of the pup desired image data centered at pixel i; fi,m and di,m are the mean values of

the elements in f*_i,"andd*_i,"; 1* is the 1-D vector whose dimension is the same as f*_i," and whose elements are all equal to 1; w_" is a weight that controls the influence of the likelihood information; and c is a constant specially designed for flat regions. In (4), we aim to reconstruct a well-illuminated image f whose local structures are similar to that of the observed image d. Since for flat regions

1 , , * * " im˜ i d

d is zero and p(d| f) will always be 1 for any realization f, we add in the constant c to avoid the convergence into an erroneous reconstruction result.

2.2. Prior model

On the other hand, we assume that the prior probability p(f) in (2) satisfies the Markov random field (MRF) model. That is, we model p(f) as

¦ ¦

 ˜  v i i N i i c p i f f V w f p ' ')}) , ( { exp( ) ( , (5)

where Ni is the collection of 8-connectivity neighbors of the

pixel i; wp is a weigh that controls the influence of the prior

information; and Vc(f) is a clique potential function

depending on fi and fi’. Here we propose the following

simple but effective definition for Vc(f):

¯ ® ­  ˜     ˜ Otherwise, , ) ( | | if , ) ( ) , ( ₂ ' ' 2 ' ' i i e i i i i s i i c f f w Th d d f f w f f V (6)

where fi and di are the desired intensity value and the

original intensity value at pixel i, respectively; we and ws are

positive weighting constants that depend on the enhancement strategy; and Th is a predefined threshold to distinguish noise variation from image detail. In (6), if the magnitude of the local gradient |di-di’| is smaller than Th, a

smoothness prior ( )2 'i

i f

f  is applied for noise suppression. On the contrary, if the local gradient magnitude is larger than Th, an enhancement prior 2

')

(fifi

 is adopted to enlarge gradient magnitude.

2.3. Bayesian restoration

Based on the aforementioned likelihood model and prior model, we get the following optimization formula for image enhancement:

¦ ¦

¦

 ˜    ˜      x   ˜ { i i N i i c p m i i m i i i m i i m i i f i f f V w c d d c f f c d d c f f w f C f C f ' ' 2 , , , , , , , , ) , ( |) 1 1 | | 1 1 | ) 1 1 ( ) 1 1 (( ) ( with )} ( { min arg ˆ * * * * * * * * * * * * " " " " " (7)

In (7), we aim to find an enhanced image f whose local structures are similar to the original one but with adjusted local gradients. By properly choose the setting of the controlling parameters, we may manipulate the gradients of the image to achieve improvement of image quality.

3. ALGORITHM SIMPLIFICATION

However, solving (7) is time-consuming. In this section, we present the way to alleviate the computational complexity of the proposed MAP approach. Here, we first reformulate the optimization problem (7) by assuming that the relationship between fi and di can be approximately expressed by an

intensity transfer function T. Under such an assumption, the number of unknown variables in the optimization process can be greatly reduced. Besides, in order to efficiently obtain a transfer function T for image enhancement, we also propose a simplified procedure for the finding of T. More details about the simplification of the proposed MAP approach will be explained in the following sections. 3.1. Reformulation of the optimization problem

In order to reduce the computational complexity, we assume that the relationship between fi and di can be expressed by a

monotonically increasing intensity transfer function T: ) ( i i T d f and (8) 0 ) ( ' di t T . (9) Strictly speaking, the transfer function T is a continuous function. However, in a typical 8-bit imaging system, what

we really care about is the output intensity values of T at each discrete input intensity level L. That is, we focus only on the finding of T(0), T(1), …,T(255). Under such a condition, the set of unknown variables are reduced to 7 { {T(0), T(1), …,T(255)} and the computational complexity of the optimization process can be greatly reduced. Moreover, the monotonically increasing constraint in (9) can be expressed as a system of linear inequality equations:

} 255 ..., 2 , 1 { where , 0 ) 1 ( ) (L T L t L T . (10)

Combining (7) and (10), together with the discrete transfer function assumption, we can get the following constrained optimization formula:

¦ ¦

¦

 ˜    ˜      x   ˜ { i i N i i c p m i i T m i T i i m i i T m i T i i d T d T V w c d d c d d c d d c d d w C C ' ' 2 , , , , , , , , , , , , )) ( ), ( ( |) 1 1 | | 1 1 | ) 1 1 ( ) 1 1 (( ) ( with )}, ( { min arg ˆ * * * * * * * * * * * * " " " " " T T T T (11)

subject to: 0dT(di),T(di')d255; and (12) }. 255 ,..., 2 , 1 { where , 0 ) 1 ( ) (L T L t L T (13)

Here, d*_i,",T is a 1-D vector representation of the pup transformed image data centered at pixel i; and d_i,m,Tis the mean of the elements in d*_i,",T. By solving (11)-(13), we can generate a discrete transfer function for enhancement. 3.2. Efficient solver of the optimization problem

In this subsection, we further introduce an efficient process to solve the constrained optimization problem defined in (11)-(13). The proposed procedure includes the followings steps:

1. Let k = 0.

Initialize the set T0_{{T(0)0_{, T(1)}0_{, …,T(255)}0_{} as} T(L)0 _{= L , for L = 0, 1, …, 255.} 2. Update Tk _{{T(0)k_{, T(1)}k_{, …,T(255)}k_{} by} k L T C L T L T k k T T T  w w ˜  )| ) ( ) ( ( ) ( ) ( 1 _P , for L = 0, 1, …, 255. 3. If the set Tk+1 _{violates the constraint in (12) or (13), we}

refine Tk+1 _{as follows:}

3a. Calculate the gradient of T(L)k+1_{, which is defined}

as ¯ ® ­       0 if ; ) ( 5} {1,2,..,25 if ; ) 1 ( ) ( ) ( ₁ 1 1 L L T L L T L T L S _k k k 3b. Project S(L) onto non-negative domain to satisfy

the constraint in (13). Here, we define 0,1,..,255 for 0 ) ( if 0 0 ) ( if ) ( ) ( * ¯ ® ­  t L L S L S L S L S

3c. Rescale the magnitude of S*_{(L) to satisfy the}

constraint in (12). Here, we define . 255 ,..., 1 , 0 for , ) ( ) ( 255 ) ( ₂₅₅ 0 * * * * _˜

¦

S k L L S L S k

3d. Get the refined Tk+1 _{by integrating S}**_(L):

. 255 ,..., 1 , 0 for , ) ( ) ( 0 * * 1

¦

 _{S k L} L T L k k

In the proposed process, C(T) is the cost function defined in (11); Ӵ is the step size; (wC(T)/wT(L))|_{T T}k is the gradient of C(T) with respect to T(L); and T(L)k _{is the}

feasible solution obtained at the kth _{step. The iteration stops}

when the change between two successive steps is sufficiently small. In our procedure, the partial derivative of C(T) with respect to T(L), (wC(T)/wT(L))|_TTk, can be obtained by , ] [ ) | ) ( ( | ) ) ( ) ( ( _{( )} d L f f C L T C i i i f d k k ˜  w w w w

_¦

_G T T T T ₍₁₄₎ where ¯ ® ­ z0 , 0 0 , 1 ] [ n n n G and ( ) |_{f (d)} i k f f C T w w _{is the}

partial derivative of C(f) with respect to the pixel value fi.

Due to the limited space, we omit the lengthy expression of

) ( | ) ( d f i k f f C T w

w _{, which can be obtained via straightforward} deduction. On the other hand, Step 3b represents an Euclidean projection of S(L) onto the constraint set (13) [11]. This operation guarantees the monotonically increasing property of the transfer function T. Besides, the dynamic range constraint in (12) is restated as

. 0 ) 1 ( with , 255 ) ) 1 ( ) ( ( 255 0   

¦

T L T L T L (15) The combination of Step 3c and Step 3d can be viewed as a projection of S*_{(L)onto the constraint set (15) while keeping}

the constraint (13) unchanged.

4. SIMULATION RESULTS

In our simulations, each RGB-valued test image is converted into the HSI color space. Only the I-component is processed for image enhancement. The values of p, w , w_" p,

ws, we and th are set to be 3, 10, 2, 1, 0.5 and 6, respectively.

Figures 1 and 2 show the results of the proposed method, in comparison with the HE method, the method in [3], the method in [10], and the MSR (Multi-Scale Retinex) method in [7]. We can find that the proposed method provides effective and natural-looking enhancement results. In comparison, the results provided by [3] are restricted due to the mean-preserving constraint; the results provided by [10] are polluted by some blocking artifacts, like the face region in Fig. 1(d); and the results provided by MSR [7] possess more apparent noise inference, like over the background region in Fig.2 (e). The MSR method may also cause undesired color shifts, like the color tone in Fig 1(e) and 2(e) becomes bluish after the enhancement. We also compare the Matlab implementation of the proposed MAP-based method (7) (Method-1) and the simplified method (11)-(13) (Method-2) defined in Section 3.2. Since Method-1 is basically a local processing, we can find that the enhancement results of Method-1 in Fig.1(f) and 2(f) are

somewhat better than that of Method-2 in Fig.1(g)-1(i) and Fig.2(g)-2(i)). However, since the local structure and local gradients have been taken into account in Method-2 to deduce the global intensity transfer function, the performance of Method-2 is still much better than the HE method. Moreover, if Method-2 is iteratively performed over the processed image, the enhancement performance gets improved. The result in Fig. 1(g) looks somewhat over-enhanced. This is because in the first iteration the information mainly comes from the prior model due to the initial settings. As the iteration proceeds, the likelihood model contributes more and the enhancement result gets improved. We also list in Table 1 the comparison of the execution time between Method-1 and Method-2. This indicates the feasibility of Method-2 in real-time applications.

5. CONCLUSION

In this paper, we propose an effective Bayesian framework for image enhancement. The proposed posteriori distribution function merges both local structure and local gradient information into the enhancement process. With proper simplification, we deduce an efficient method to generate an intensity transfer function that may achieve similar enhancement performance with much lower computational complexity. Simulation results have demonstrated the feasibility and effectiveness of the simplified MAP-based method for image enhancement.

6. REFERENCES

[1] R.C. Gonzalez and R. E. Woods, Digital image processing, Addison-Wesley, 2002.

[2] Y.T. Kim, “Contrast Enhancement Using Brightness Preserving Bi-Histogram Equalization,” IEEE Transactions on Consumer

Electronics, Vol. 43, No. 1, pp.1-8, Feb. 1997.

[3] S. Chen and A.R. Ramli, “Minimum mean brightness error bi- histogram equalization in contrast enhancement, ”IEEE Trans. On

Consumer Electronics, Vol. 49, No. 4, pp. 1310-1319, Nov. 2003.

[4] C. Wang and Z.F. Ye, “Brightness preserving histogram equalization with maximum entropy: A variational perspective,”

IEEE Trans. on Consumer Electronics, Vol. 51, No. 4, pp. 1326-

1334, Nov. 2005.

[5] O. Wang and R.K. Ward, “Fast image/video contrast

enhancement based on weighted threshold histogram equalization, ” IEEE Trans. on Consumer Electronics, Vol. 53, No. 2, pp. 757- 764, May 2007.

[6] T. Arici, S. Dikbas and Yucel Altunbasak, “A histogram modification framework and its application for image contrast enhancement,” IEEE Trans. on Image Processing, Vol. 18, No. 9, pp. 1921-1935, Sep. 2009.

[7] D.J. Jobson, Z. Rahman and G.A. Woodell, “A multiscale retinex for bridging the gap between color images and the human observation of scenes,” IEEE Trans. on Image Processing, Vol. 6, Issue 7, pp. 965-976, July 1997.

[8] Li Tao, R. Tompkins and V.K. Asari, “An illuminance- reflectance model for nonlinear enhancement of color images,”

IEEE CVPR, 2005

[9] S. Lee, “An efficient content based image enhancement in the compressed domain using retinex theory,” IEEE Trans. on

Circuits and System for Video Technology, Vol. 17, No. 2, pp.

199-213, Feb. 2007.

[10] J. Mukherjee and S.K. Mitra, “Enhancement of color images by scaling the DCT coefficients,” IEEE Trans. on Image

Processing, Vol. 17, No. 10, pp. 1783-1794, Oct. 2008.

[11] S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004.

Table 1 Comparison of Execution time

Image in Fig. 1 Image in Fig.2

Method-1 (using fmincon in

Matlab)

657.14 seconds 2202.65 seconds

Method-2

(1 iteration) 0.93 seconds 0.97 seconds

Method-2

(3 iterations) 0.98 seconds 1.17 seconds

Method-2

(5 iterations) 1.02 seconds 1.24 seconds

(a) Original image (b) HE (c) Method [3]

(d) Method [10] (e) MSR [7] (f) Method-1

(g) Method-2;

1 iteration (h) Method-2; 3 iterations (i) Method-2; 5 iterations

Figure 1. Comparison of enhancement results

(a) Original image (b) HE (c) Method [3]

(d) Method [10] (e) MSR [7] (f) Method-1

(g) Method-2;

1 iteration (h) Method-2; 3 iterations (i) Method-2; 5 iterations

Figure 2. Comparison of enhancement results