Chapter 1 Introduction
1.2 Related Work
Fig. 1-2 Four categories of video fire detection
Methods of fire detection used for image processing thesis can be divided into four categories, as illustrated in the Fig.1-2. The first category is based on the color of the fire. Normally, fire will exhibit yellow, yellow-red, and red color. A fire model will be established with different color space according to the color distribution of the fire. Chen et al [1] and [3] noted that fire must comply with RGB color space
3
R G , GB, and RRTdistribution law. In addition, Horng et al [2] further established different color model in the HIS space according to saturation differences.
This method can be adapted to different brightness environments, so that the fire model can be more widely used in various environments. Horng et al [8] combined the RGB model and the HIS model to identify the color of fire distribution. Celik et al [7] proposed that using the YCbCr space flame color model can efficiently separated fire luminance and chroma. This thesis flame color model also refers to this method, and making further improvements.
The second category is judged the fire texture. From a distance the burning flame appears similar, but upon closer look, nearby pixel brightness or chroma shows clear subtle changes.Ebert et al [3] uses preset masks to calculate neighboring pixel’s red, green, and blue value difference of the area being observed. Only if there is enough blue color difference will it be judged to be flame pixel. Toreyin et al [4] uses wavelet analysis on the colors in the flame region. They believe that there must be enough variation in the fire area’s neighboring pixel brightness, and using 2-D wavelet analysis, calculate the absolute value of the high-band and low-band to define the parameters. Only objects with bigger parameters can be considered fire bodies.
Zhao et al [9] mentioned that entropy formula can be used to calculate the texture parameters of the region of interest.
The third category is determining the movement of the flame. The flame is a very special part of the fire. It will usually keep flashing and swaying on its own, or swing specifically due to the interference of the wind. This type of swaying will cause the flame to appear suddenly red and then suddenly returning to the background color in the red channel of the pixel. Gunay et al [5] proposed that the detected fire color moving body’s pixel red channel be accumulated, and then use the wavelet
4
analysis on the time of the accumulated data to find its frequency, and then the Markov model be used to train the optimal threshold value.
The last category is to use the fire shape as the judgment standard. Fire shape is usually changing and rough, and different than the relatively smooth features of other objects. Borges [6] uses the object’s actual circumference length and the circumference length ratio of the convex body nearest to this object to determine if this object is the fire’s exterior.Gunay et al [5], according to different angles of the object, will uses data analysis of the distance from the center of mass of the object to the fire’s boundary, and utilizing wavelet analysis to determine if this object’s appearance is rough. Zhang et al [10] uses Laplacian operator to identify the boundaries of the object and conduct Fourier analysis to determine if it is the fire contour.
5
2 Chapter 2 System Overview
The following diagram is my proposed fire detection system structure, which can be divided into three parts, pre-processing, local block feature, and global feature.
Pre-processing is cutting the image into a fixed position block, and using the block as a unit to find if there is moving foreground in the block. If the side label is the candidate block, continue with the later feature calculation after labeling the candidate block. In this way, the calculation of blocks with no moving foreground can be eliminated. Connecting Component Block is using the same number label for the adjacent candidate blocks, forming it into one body. This part is the pre-processing for specialized variance analysis.
Local block feature is the feature analysis of fire body features when judging for candidate blocks, including fire color, fire source, and temporal difference. When the candidate block matches these three features, then it becomes a strong candidate
Video
6
block. Global features include variance calculation and fire area analysis. Variance calculates the changes in pixel texture in the blame body. Fire area analysis uses strong candidate block as a benchmark to identify a region of interest, including the entire fire candidate object, and conducts a more accurate analysis of this region of the fire area. If the global feature is matched, it will give a single fire alarm signal.
ADU conducts a buffer zone analysis of this alarm signal to avoid alarms caused by external interference, then issues the real final alarm.
7
3 Chapter 3 Fire Detection Algorithm
3.1 Block Processing
Figure is the flow chart of block processing. The input is the gray level image sequence, and the output is candidate blocks with moving property. There are two common methods for obtaining foreground image. One is temporal difference, the other is background subtraction. Temporal difference method is that we subtract frame t-1 from frame t, and the regions with a obvious intensity variation are considered as foreground region. Background subtraction is also in similar way but we use a constructed background image instead of frame t-1.
Generally, cameras are usually set fixedly by considering surveillance monitoring system. Therefore, background subtraction is a better choice for our proposed algorithm. Foreground regions can be found by background subtraction, but they could also include static objects. Next, temporal difference of two successive frames will be calculated. The two methods are combined to further filter out static objects.
Fig. 3-1 Block processing
8
3.1.1 Background Modeling
Gaussian Mixture Model (GMM) [9] is a common and robust method in background construction, and we choose GMM to build the background image. It will be described as follows.
Generally speaking, the intensity of each pixel varies in a small interval except the region of foreground objects. It is proper to use a Gaussian model to construct the background image. But in many surveillance videos, we would observe that there are waving leaves, sparking light, etc. In these situations, some background pixels would vary in several specific intervals. In other words, using two, three or more Gaussian distributions to model a pixel will obtain a better performance. We present the flow chart of GMM background construction in Fig. 3-2.
Fig. 3-2 GMM background model construction
9 time t, and is a Gaussian probability density function shown in Eq. (3.2)
/ 2 1/ 2 1where n is the dimension of data. In order to simplify the computation, it assumed that each channel of data are independent and have the same variance, and then can assume the covariance matrix as Eq. (3.3):
, 2I
k t k
(3.3) Temporal difference is applied to extract the possible background regions, and update pixels inside these regions. Then, we sort Gaussian distributions by the value of / , and choose the first B distributions to be the background model, i.e. shown as Eq. (3.4): this pixel is considered as background. Then, we update weight, mean, variance by Eq.
(3.5), (3.6), (3.7):
, 1 (1 ) , ( , 1)
k t k t Mk t
(3.5)10
1 (1 ) 1
t t Xt
(3.6)2 2
1 1 1 1
1 (1 ) ( t t ) (T t t )
t t X X
(3.7) where is a learning rate, Mk t, 1 is 1 for the model which matched and 0 for remaining models, and Eq. (3.8) shows the second learning rate .
1 , ,
(Xt | k t, k t)
(3.8) Besides, the remaining Gaussians only update the weight. If there is no any distribution is matched, we replace the mean, variance and weight of the last distribution by Xt1, a high variance and a low weight value, respectively. Figure 3-3 shows the constructed background image by GMM. Figure 3-4 shows the foreground image obtained by background subtraction.
(a) Video sequence (b) GMM Background Image Fig. 3-3 Background image construction by GMM
(a) Current image (b) Foreground image Fig. 3-4 Foreground image obtained by background subtraction
11
3.1.2 Candidate Selection
Moving regions come into existence and disappear continuously because of the special particle property during ignition and combustion as shown in Fig.3-5. It is inefficient to track or analyze the target using object-based method. Block-based technique provides a better way to solve this problem. The image will be divided into non-overlapped blocks, and each block has the same size in a same image. First, we will find out the blocks with a gray-level change. The foreground image will be obtained by the GMM approach, and we compute the summation of foreground image for each block as shown in Eq. (3.9)
Fig. 3-5 Moving regions come into existence and disappear continuously
12
Foreground regions can be found by the GMM approach, but they could also include static objects. Next, temporal difference of two successive frames will be calculated. In dynamic image analysis, all pixels in the difference image with value “1”
are considered as moving objects in the scene. As we know, video images usually have a great amount of noises due to intrinsic electronic noises and quantification. So the difference of two successive frames pixel inevitably produces false segmentation.
To reduce the disturbance of noises, we also compute its summation for each block to determine the moving property. The block difference is defined as
2 and T2 is the predefined threshold.In order to reduce the computational cost, only when the value of background subtraction and temporal difference lager than the predefined thresholds will be regarded as candidates containing moving objects by Eq. (3.11).
1, 1
We consider the information of a particular block over time as a “block process” in the following sections. Fig.3-6 illustrates some results produced by block processing.
Fig. 3-6 Results of block processing
13 representation concentration is RGB (255,255,255), and the black color representation is RGB (0,0, 0). Chen et al [1] proposed that the main features of the fire in RGB space, speaking in terms of single pixel, are that R channel value will be greater than the G Channel value. The G Channel value would be greater than the B channel value, and the R value saturation level will be greater than the average R value of the entire screen. But the drawback of the RGB fire color model is that it does not take into account the impact of the camera on the fire in different brightness. That is, this algorithm may vary with the change in intensity, and resulting in a decrease in detection rate.
In order to improve this problem, Hasan Demirel, [3] proposed the fire color model based on YCbCr color space base. Y represents the luminance; Cb and Cr represent the blue-difference and red-difference chroma components. This fire color model also changed from the above described RGB color space algorithm calculation to the YCbCr color space calculation algorithm. In addition, it adds adaptive adjustment to the surrounding brightness, and improved other harmful effects, increasing detection performance. The algorithm we propose also refers to this algorithm, and improved. Because RGB space converting to YCbCr space is linear, we can use the following matrix to turn RGB color space to YCbCr color space:
14 screen’s average luminance value and the concentration of blue and red average color difference chroma components, respectively. K Represent this screen’s total pixel quantity. chroma component, and blue difference chroma component of the coordinates( , )x y . That is to say, the luminosity must be greater than blue difference, and the red
15
difference must be greater than blue difference.
The fire picture statistics of the YCbCr distribution picture is as follows:
Fig. 3-7 distribution of fire color in YCbCr color space
In addition, from various fire pictures some futures can be observed. Pixel with fire usually will have greater luminosity than the average luminosity of the entire picture and the red difference chroma component will be greater than the average difference of the entire picture. The blue difference chroma component will be smaller than the average of the entire picture. The sum of the above-mentioned features is expressed with the following formula:
1, ( , ) & ( , ) & ( , ) chroma component and the blue difference chroma component, as in the following:
1, | ( , ) ( , )
16
adjusted according to different situation. The value suggested in the thesis is 40.
Results retrieved from fire color model will usually include some objects that has similar color with the fire, but is not the fire, such as metal reflecting the fire in the picture. Thus, simply using color for analysis is not enough. Fire color is just a looser feature analysis. The experimental diagram is as follows:
Fig. 3-8 Fire color analysis results
However, we discovered in the experiment that this algorithm only meet the general red and yellow fire color. For some high temperature, closer to white color, high intensity fire, it cannot accurately be detected. Thus we cut the color model into two areas. One is normal intensity fire (Y <220), which we use the above algorithm. For high intensity fires (Y> 200), we adjust the conditions as follows:
( ( , ) 220) between the two values, so it is easy to not comply:
( , ) mean
Cb x y Cb (3.23)
|Cb x y( , )Cr x y( , ) (3.24) The pre and post modification comparison of these two formulas are as follows:
17
Fig. 3-9 High intensity formula modification comparison.
As the above figure shows, the modified algorithm can also detect high intensity fire. Conversely, false detection will increase. But because the color is the first hurdle, later we will propose other algorithms to filter out these errors. When the fire pixels in the block are over the block’s total pixel by 10%, this block matches the fire color features.
3.3 Fire Source Analysis
In general, fire will have a burning fire source. Although the flame will move with the wind, we can observe that the fire source location will not usually move, or will expand with the fire, forming a slow movement. The burnt location will form a leftover concept.
The remnants represent objects that originally do not belong in the screen. After moving by certain methods into the observation screen, it will remain in the same place for a long time. For example, luggage dropped by traveler. Fire will also have similar features. Flames will instantaneously combust after a flammable material passes the ignition point. Flame originally does not belong in the screen, but is suddenly produced in the screen, and the fire source position will not change.
18
As can be seen from the above figure, fire at the fire source contains two features.
First, high intensity feature in the fire source area, and second, the position is fixed.
The fixed location feature just happens to be able to match our use of candidate blocks to judge the fire starting point. First, we must make sure there are fire pixels we are interested in inside the block:
1, ( , ) ( , ) average intensity is greater than the average intensity of the image, then it matches the high intensity feature of the fire starting point. Then calculate this candidate block high intensity state holding time. This high intensity state must be greater than t times to match the fixed position feature. The judgment is as follows:
B ( , ) ,
Fire will move with the wind, but will produce different changes in intensity due to the physical features of the combustion. Intensity change usually is produced randomly, with no set regularity. Even if the fire location area of the flame does not have any substantial changes, the pixel intensity will be different for every frame.
We hope to be able to find a way to calculate this chaotic fire intensity change feature.
Intensity timeline changes have many different analysis methods, such as:
19
intensity timeline’s differential, wavelet analysis. The actual significance is to calculate the difference in pixel intensity within the calculation time interval. The method we used is the Temporal Difference as our analysis method. The reason for using this method is that for the same calculation of the difference in timeline intensity, this calculation method is simpler, and has lower calculation computation compared to other methods. However, it can still calculate the intensity difference of the fire.
Temporal Difference is a simple method for calculating changes in two different time span pictures. Using the t time and t-1 time grayscale image subtraction in order to filter out the noise effect, so the subtracted absolute value must be greater than a threshold value as follows:
1, ( ) ( 1)
THgrayis a preset threshold value which can be obtained via experimental experience.
After calculation, the picture use thresholding method that matches the formula is displayed as shown below:
Fig. 3-10 temporal difference result
20
Because the method we proposed uses block as our observation unit, so we must use the calculated result and extend it to block unit feature analysis. A block must have a certain number of mixed intensity pixel to be considered matching this feature.
count features as analysis. Next, we combine the local feature results to observe the fire’s global feature.
This chapter’s focus is placed on the global feature analysis of the fire. We must use a macroscopic observation angle to analyze the fire features, to make up for the inadequacies of the local features analysis. Global analysis contains three steps, texture variation value analysis, fire area analysis, and decision-making unit. The flow chart is as follows:
Fig. 3-11 Flow chart of global feature Texture
21
3.6 Texture Variance Analysis
Simply relying on fire color for detection cannot completely filter out some objects that are fire like colored, such as red and yellow clothes or orange car. We have to put forward a new feature to filter the above-mentioned objects. And one thing that clearly separate fire from other objects is that, although fire objects are all red and yellow color, but the intensity and red difference chroma of each pixel will have subtle changes. That is to say, fire is not smooth textured, as shown below:
Fig. 3-12 Fire and fire like objects texture
Obviously, fire will have different color information in each pixel, whether it is intensity or red color difference. We must find a way to calculate the color feature differences in the candidate region.
Variance analysis is a common data analysis model. The variance of a set of data describes the degree of dispersion of this set of data. That is, the distance of this set of data in comparison to the expected value (means). Simply put, the variance also represents the average degree of dispersion for that set of data.
The first step in calculating the variance is to calculate this data set’s combined average value:
22
x is our observed feature value, and the feature value we use is based on connecting i
block labeling as the same object inside the fire pixel’s Cr value for calculation. N represents the fire pixel number within this Label. Pay attention here, because not all the label’s pixels are calculated for Cr variance. Only fire pixels that match the above fire color algorithm are listed into calculation. Foreground moving object calculated by block algorithm often includes non-flame objects near the fire position (i.e. smoke), which we must eliminate.
The following is the variance formula:
The following is the variance formula: