Fuzzy Inference Digital Image Stabilization (FIDIS)

The objective of motion compensation is to achieve the optimal stabilization of shaking images within a specified compensation range (window shifting allowance). Eq. 3.1 can get a good result in a general case. But in the constant motion case, the undesired shaking effect cannot be eliminated by this method (mentioned in 3.2.3). The method proposed in 3.2.3 can overcome this drawback. The inner feedback-loop integrator was combined with a clipper function to reduce the steady-state lag for steady motion, as well as to keep the CMV operating in the appropriate range. Due to the characteristics of the integral, the overshoot will occur in certain back and forth low frequency image panning. Fig. 3.4 shows overshoot in low frequency back and forth image panning. The motion trajectory has obvious overshoot. This overshoot will reduce the effect of stabilization.

0 50 100 150 200

0 50 100 150 200 -150

-100 -50 0 50 100 150

Frame

Displacement(pel)

Original Trajectory Compenseted Trajectory

overshoot

Fig. 3.4. Overshoot in low frequency back and forth image panning.

In this section, the architecture of fuzzy inference applied to DIS will be proposed. The idea is to keep the merits of each motion compensation method, i.e. reduce compensated motion trajectory lag and overshoot, to improve the DIS in different conditions. Fig. 3.5 shows the architecture of fuzzy inference applied to digital image stabilization. The system architecture includes two processing units, the motion estimation unit and the motion compensation unit. The motion estimation unit is the same as 2.2.1. The motion compensation unit includes two motion compensation methods, fuzzy inference system and motion compensation. Due to the differences of environments, conditions and shaking patterns of the captured images, the different motion compensation methods have individual merits separately. Fig. 3.6 shows the architecture of the fuzzy inference system. The pre-process block inputs GMV, CMV#1, CMV#2 are to transform into delta of short term smoothness index(ΔSI n( )) and delta of deviation(Δd) which are inputs of fuzzy inference. The fuzzy inference determines the weighted value to calculate the compensation vector. And then the block of image compensation uses the compensation vector to stabilize the image sequence.

Fig. 3.5. Architecture of fuzzy inference applied to digital image stabilization.

Fig. 3.6. Architecture of fuzzy inference system.

The motion compensation unit uses two motion compensation methods. These are the conventional CMV estimation mentioned in Eq. 3.1 and the proposed CMV estimation mentioned in Eq. 3.3. The results are denoted as CMV#1 and CMV#2, respectively.

The main function of pre-process is to transform external signals into distinguished signals of fuzzy inference input. The major consideration is the smoothness index. To realize the effect of the compensation method in a short term period, the short term smoothness indexSI n is ( ) defined as

WhenΔSI n( ) 0< , the motion trajectory compensated by CMV#1 is smoother than what was compensated by CMV#2 and vice versa. The secondary consideration is the deviation between the original and compensated motion trajectories. The deviation ( ( )d n ) is defined as

( ) _o( ) _c( )

d n = MTraj n −MTraj n (3.9)

where ( )MTraj n is the original motion trajectory and _o MTraj n is the compensated _c( ) motion trajectory.

The large deviation between original and compensated motion trajectories will affect the image stabilization result. Therefore, the delta of deviation (Δd) is defined as a distinguished signal which is expressed as

( ) #1( ) # 2( )

d n d n d n

Δ = − (3.10)

where d#1( )n and # 2( )d n are deviations of motion trajectory compensated by CMV#1

and CMV#2. When Δd n( ) 0< means the deviations of motion trajectory compensated by CMV#1 is better than compensated by CMV#2 and vice versa.

Fig. 3.7 shows the Architecture of internal fuzzy inference. It contains a fuzzifier, a fuzzy rule base, an inference engine and a defuzzifier. The function of the fuzzifier is to transform crisp measured data ΔSI n( )and ( )Δd n into suitable linguistic values. The membership functions of these two signals are shown in Fig. 3.8. The linguistic descriptors are: large negative (FN), small negative (SN), zero (Z), small positive (SP), large positive (LP). Fuzzy rule base is characterized by a collection of fuzzy IF-THEN rules in which the preconditions and consequents involve linguistic variables. The collection of fuzzy inference rules characterizes the simple input-output relation of the system. The Eq. 3.8~3.18 are the statements of fuzzy rules:

Fuzzy

Fig. 3.7. Architecture of internal fuzzy inference.

-5 -4 -3 -2 -1 0 1 2 3 4 5

IF ^ΔSI is Z and ^Δd is Z THEN Output is M (3.15)

Mamdani’s fuzzy implication methodR_c, associated with the min operation, is used in the proposed inference system. The membership function of output signal is shown in Fig. 3.9.

The linguistic descriptors are: large (L), Medium (M), small (S). The output represents the weighted value of motion compensation method#1. Because two methods are included in the system, the final motion compensation vector is calculated by:

( ) ( 1)

compensated vectors estimated by method #1 and #2.

Defuzzifier is a mapping from a space of fuzzy set defined over an output universe of discourse into a space of nonfuzzy set. The method of defuzzification used in the fuzzy inference system is the center of area (COA).

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Fig. 3.9. Membership function of output.

3.3. Summary

In this chapter, we addressed the related research works of motion compensation. First, the motion trajectory of an image sequence was introduced to analyze the problem of motion compensation. From the motion trajectory, it is easy to view the smoothness of processing digital image stabilization and the deviation between the original image sequence and the compensated image sequence. Secondly, the quantitative evaluation method was introduced to evaluate the degree of image stabilization quantitatively. The smoothness index (SI) was proposed to quantitatively evaluate the performance of different DIS algorithms. Then two methods of motion compensation, the CMV estimation with inner feedback-loop integrator and fuzzy inference applied to motion compensation, were proposed. The CMV estimation with inner feedback-loop integrator solved the lag problem in the constant motion condition.

The fuzzy inference DIS (FIDIS) was proposed to adaptively determine better motion compensation methods through two different MCs for various conditions of image sequences.