Least Mean Squares

2.2 Least Mean Squares

A commonly used bias-gain linear model for an FPA sensor is given by:

( ) ( ) ( ) ( )

where ^aij

( )

^{n and b nij}

( )

are the gain and the offset of the ij-th detector at frame n,

( )

Xij n is the real incident infrared radiation collected by the respective detector, and

( )

Y nij is the measured output signal. The main idea of the NUC scene-based methods relies on estimating the gain and the offset parameters of each detector on the NIR FPA with only the readout data Y nij

( )

. The algorithm has the ability of adapting sensor’s parameters over time under a frame by frame basis. To understand how the neural network based approach proposed, Eq. (2.9) must be reordered as following:

( ) ( ) ( ) ( )

ij ij ij ij

X n =g n ×Y n +o n (2.10)

where the new parameters gij

( )

n and o nij

( )

are related to the real gain and offset parameters of the detectors, as expressed in the following expression:

( )

¹

( )

( ) ( )

^ij

( )

ij ij

E n =T n −X n (2.12)

The unknown parameters are estimated by using the neural network method, and the desired target value can be calculated as the local spatial average (mean filter) of the output data
^Xij

( )

^{n .}

Then, we can get gradients relative to each parameter in Eq. (2.14).

^ij 2 _{ij ij}

The steepest descent algorithm is a good way to solve this Least Mean Squares (LMS) optimization problem. In this gradient-based search algorithm, the parameters to be estimated are recursively updated with a portion of each respective error gradient. The parameter learning procedure is finally described as following:

ij

(

¹

)

ij

( )

ij

( )

ij

( )

g n+ =g n + η×E n ×Y n

ɵ^ij

(

¹

)

^ɵij

( )

ij

( )

o n+ =o n + η×E n (2.15)

where η is a fixed parameter known as the learning rate.

Basic LMS method for NUC of sensor array pixels is described above. Then we use three improvements on LMS method [4], which include regularization, momentum term and adaptive learning rate as explained below.

2.2.1. Regularization

To test the algorithm, we note that the gain is usually much larger than the offset, and gain value is usually around 1. When the gain value is much larger than the offset value, it is difficult to select the appropriate learning rate. In the LMS algorithm, when the learning rate is very small, the numerical convergence is slow; when the learning rate is large, the iteration would not converge. This makes the algorithm encounter an obstacle in practical applications. According to above analysis, the gain value was normalized to improve the calibration of artificial neural network algorithm.

When the gain and offset are adjusted, you can get the same adjustment in the same order of magnitude. the issues of the difficulty in selecting the learning rate are resolved, and that also eliminates the obstacles of the LMS algorithm in practical applications. Thus, the purpose of regularization is to eliminate the difference in the magnitude of the gain and offset adjustment. Parameter adjustment is as following:

( )

( )

^{ij( )
ij}

( ) ( )

Another possible enhancement to the steepest descent algorithm is the well-known momentum. The gradient descent can be very slow of if the learning constant eta is small and can oscillate widely if eta is too large. This problem essentially results from error-surface valleys with steep sides but a shallow slope along the valley floor. Other efficient and commonly used method that allows larger learning constant without divergent oscillations occurring is the addition of a momentum term to the normal gradient-descent method. The idea is to give each weight some inertia or momentum so that it tends to change in the direction of the average downhill force that it feels. This scheme is implemented by giving a contribution from the previous time step to each weight change:

^ij

(

¹

)

^ij

( )

^ij

( )

^ij

( ) (

^ij

( )

^ij

(

¹

) )

g n+ = g n − η×E n ×Y n + α ⋅ g n −g n−

ij

( )

ij

( )

ij

( )

ij

( )

g n E n Y n g n

= − η× × + α ⋅∆ (2.18)

ɵ^ij

( )

^ij

( )

^ɵij

( )

o n E n o n

= − η× + α ⋅∆ (2.19)

where α∈[0,1] is a momentum parameter and a value of 0.9 is often used. For example, Figure 2.1 shows the offset of the correction curve. Note that the trajectory without momentum (the left curve) has larger oscillations than the one with momentum (the right curves). We further observe from the right curves in Fig. 2.1 that the momentum can enhance process toward the target point if the weight update is in the right direction (point A to A' in Fig. 2.1). On the other hand, it can redirect movement in a better direction toward the target point in the case of overshooting (point B to B' in Fig. 2.1). This observation indicates that the momentum term typically helps to speed up the convergence and to achieve an efficient and more reliable learning profile.

The use of momentum could improve the performance of the adaptive algorithm, improving its stability and probably reducing the production of ghosting artifacts, and it can leave some beneficial fluctuation in trajectory.

ɵ^ij

(

¹

)

^ɵij

( )

^ij

( ) (

^ɵij

( )

^ɵij

(

¹

) )

o n+ =o n − η×E n + α ⋅ o n −o n−

Fig. 2.

2.2.3. Adaptive Learning Rate

In the neural network

calculation of the error E_ij n

(

T nij

Ideal T nij

( )

is the output neighborhood averaging neighborhood average, the

weakened. However, in particular,

Fig. 2.2. The offset of the correction curve.

Adaptive Learning Rate

output value of the 2-points correction algorithm method is the image smoothing filter smoothing filter. After doing noise intensity is may become blurred.

Thus, based on the knowledge that the local spatial average is not always a good estimation for the desired target response of an adaptive NUC method, the proposed adaptive learning rate ηij

( )

n showed in Eq. (2.21) is designed to be dependent, and inversely proportional to the local spatial variance of the input image

ij

Therefore, if input image of the working window is smooth enough, the desired averaged target value at the output is more confident, and the learning rate gets larger.

On the other hand, if a given sliding window size of the input image is not smooth enough, the local variance is too high, like in a object border, and the learning rate gets a smaller. To add this adaptive learning rate to the adaptive NUC algorithm, η in equation (2.15) must be replaced by its counterpart ηij

( )

n in Eq. (2.20), where K is a constant that limits the maximum learning rate. The local variance

ij

2

σY(n) can be calculated with any desired window size, and a 3 × 3 window size would be assumed along this paper.

Chapter 3 The Improvement of Bad Pixel

在文檔中 近紅外線影像之非均勻校正與壞點修復 (頁 23-31)