There are various extensions and modifications of the original LBP. A good source of references can be found in [1].
Since this research focuses on issues regarding the fuzzy representation of LBP, we will restrict our discussion of related work to this particular subject.
Ahonen et al. [4] introduced soft histograms for the LBP texture descriptor. The basic idea is to replace the hard-limit thresholding operation with two fuzzy membership functions, as depicted in Fig. 4 and defined in Eq. (1). The parameter d controls the degree of fuzziness and is determined empirically.
The histogram becomes ‘soft’ since a single pattern can contribute a fraction to many different bins in the histogram.
The fuzzy representation guarantees that a small change in the input will result in a small change in the output.
() = 0.5 + 0.5
The authors observed that in texture classification experiments, the performance of the fuzzy LBP is better compared to the original LBP in the presence of noise, as can be expected from the new mechanism of the mapping process.
However, the classification errors exceed 45% when Gaussian noise with = 0, = 25.5 is added to the test images. This leaves much room for improvement and motivates us to investigate encoding methodologies which can prove to be more robust to noise.
In [5], the authors proposed to use fuzzy LBP operator to characterize ultrasonic textures. Their formulation of the fuzzy operation is very similar to the approach given in [4]. The only difference lies in the application domain. The best classification accuracy for the ultrasonic images is 84% using a fuzzification parameter (d) of 5 with as polynomial kernel support vector machine. How the classification accuracy changes with the degradation introduced by additive noise is not investigated.
The local fuzzy pattern defined in [6] is basically the same as the fuzzy LBP. However, the authors’ main interest is to combine local fuzzy patterns with spatially constrained fuzzy C-means algorithm for color texture segmentation.
In summary, all the related materials that incorporate the fuzzy logic operation share a common thought: replacing the hard-limit thresholding operator with fuzzy membership functions. For our formulation of fuzzy local ternary patterns, we will follow the same principle. Additionally, we will also exploit some unique properties of the extended local ternary patterns to arrive at alternative methods to bring in the fuzziness concept.
III. FUZZIFICATION OF EXTENDED LOCAL TERNARY
PATTERNS
In this section, we will describe two different approaches to bring in the fuzziness property for the extended local ternary patterns. This first approach is based on the fuzzification of the mapping function. The other takes advantage of the fuzzy clustering operation in the dimensionality reduction procedure required by ELTP.
A. Fuzzification of the Mapping Function
To recap, we briefly describe the process for converting a region into its ELTP representation,
() = 2 if () − (0) > 1 if |() − (0)| ≤
0 if () − (0) < − (2) where P(0) is the intensity of the center pixel, and P(i) is the intensity of the neighboring pixel. Instead of employing a fixed threshold d, however, we propose to assign its value based on the local statistics of the region. Specifically, we use Eq. (3) to compute d:
= max { × , } (0 < ≤ 1) (3) where σ is the standard deviation of the local patch, α is a
Eq. (2) employs a hard-limit thresholding operation to
define the input/output mapping. A direct extension of the fuzzy LBP to fuzzy LTP operator is to introduce new forms of membership functions to cope with three possible classes of output :{0, 1, 2} as depicted in Fig. 5.
Figure 5. Three fuzzy membership functions f0, f1 and f2.
For the fuzzy ternary patterns, the corresponding membership functions are expressed as follows:
() = fuzzification parameter which determines the range of the membership function.
The feature dimension increases drastically when we use the ternary representation. For a neighborhood of size P, there are a total of 3P unique patterns to consider. For example, for P=2, nine fuzzy local ternary patterns need to be evaluated:
(0,0),(0,1),(0,2),(1,0),(1,1),(1,2),(2,0),(2,1),(2,2).
Given a fuzzy ternary string of length P composed of ternary digits , the fulfillment degree of the m-th fuzzy pattern can be calculated according to:
= ! "
In other words, one fuzzy pattern can contribute to more than one bin in the histogram, yet the sum of the contributions should always be 1.
In principle, one fuzzy ternary pattern can contribute to as many as 3P bins as indicated in Eq. (7). However, due to the way we define the membership function (Fig. 5), it is not possible for any element to be a member of both class 0 and class 2 simultaneously. This effectively reduces the number of combinations in the resulting pattern. In practice, the number of bins that a single pattern contributes to is usually much smaller than the maximum value.
In consideration of the high dimensionality issue arisen from ternary representation, we discuss at this point how to tackle this problem when a fuzzy operation is involved. One obvious strategy is to utilize a descriptor with a smaller number of samples. For example, LTP(4,1) will have 34=81 combinations compared to 6561 dimensions for LTP(8,1). To maintain the same number of neighborhood samples, however, some type of dimensionality reduction process is required. In ELTP [1], we have applied the spectral clustering technique to group the original 3P patterns into K clusters of our choice. For the fuzzy ternary operator, we can aggregate the fractional contributions of all patterns belonging to the same cluster to generate a ‘soft histogram’ that is of dimension K, which serves as the basis for subsequent classification tasks. The difference between the ‘crisp’ and the fuzzy ELTP representation is that the latter histogram can contain non-integer values.
Although the inclusion of fuzzy logic will enhance the resistance to noise, we also need to take into account the increased computational complexity. In the most general case, for each fuzzy ternary string of length P, a total of 3P calculations have to been carried out as specified in Eq. (7). For the more restricted, yet practical case of the membership functions shown in Fig. 5, a total of 2P calculations are needed for each pattern, which is still a significant increase even for a small neighborhood size P. The forgoing analysis also applies to fuzzy LBP. Therefore, one should always be cautious about the computational cost of fuzzification when using local fuzzy pattern descriptors.
B. Fuzzification of the Clustering Process
The significant increase in computational complexity for fuzzy ELTP prompts us to investigate alternative means to incorporate fuzzy logic without impacting the computing cost.
We realize that the additional demand arises from the fuzzy mapping functions as defined in Eqs. (4-6). For the LBP operator, fuzzifying the mapping functions is the only plausible choice. For the ELTP descriptor, since a clustering process is involved during the dimensionality reduction step, it may become useful to replace the clustering operation with a fuzzy clustering algorithm so that a single pattern may belong to more than one cluster simultaneously without the need to perform fuzzy mapping in the encoding stage. In this manner, a single pattern may also contribute to more than one bin in the histogram. The degree of contribution is proportional to the membership function obtained from the fuzzy C-means algorithm. The revised dimensionality reduction procedure which incorporates the fuzzy C-means clustering technique is illustrated Fig. 6.
S1. Choose P (sample points) and K (histogram size) S2. Form a 3D× 3D affinity matrix using Hamming distance based similarity measure
S3. Perform a fuzzy K-way partition of the 3D patterns using fuzzy C-means clustering.
S4. Distribute the contribution of a pattern to the clusters it belongs to according to the corresponding membership function.
S5. Use the K-dimensional soft histogram for feature representation.
Figure 6. Fuzzy C-means clustering algorithm for generating FCM-ELTP
The fuzzy clustering (S3) needs to be performed only once.
As a result, the computation complexity is much lower using this representation, which will be denoted as FCM-ELTP to distinguish it from the fuzzy ELTP descriptor discussed previously.
IV. EXPERIMENTAL RESULTS
We present experimental results comparing the performance of ELTP and the newly proposed fuzzy ELTP and FCM-ELTP in terms of texture classification accuracy. The scaling factor α for defining ELTP is set to 0.5 in the experiment. The fuzzification parameter H is set to 20.
We use all 36 texture classes (10 of them are shown in Fig.
7) from Brodatz database to carry out the classification experiment. Each texture image is of size 640x640 and represents a texture class. In the experiments, we partition the original images into overlapping sub-images of size 96x96 and generate 10800 image patches. Half of them are used as training samples. We then add different levels of Gaussian noise to the test images and repeat experiments. Results are summarized in Table 1. We employ support vector machine with radial basis kernel for the classification task in all the experiments. Generally speaking, when the noise becomes severe, the performance degrades accordingly. For texture classification tasks, the proposed fuzzy ELTP descriptors maintain robust performance in all experiment settings. FCM-ELTP also yields satisfactory results when the noise is not too severe.
Figure 7. Sample texture images from the Brodatz database
TABLE I. TEXTURE CLASSIFICATION RESULTS USING DIFFERENT LOCAL PATTERNS
V. CONCLUSIONS
We have presented two approaches to define fuzzy ELTP operators in this paper. Both methods produce soft histograms in the final feature representation. The fuzzy ELTP obtained by incorporating fuzzy mapping functions exhibits better performance in terms of noise resistivity and texture classification. Yet it requires far more computations than the original ELTP. The additional complexity is proportional to the dimension of the feature vector. Therefore, parallel implementation on platform such as CUDA is essential to make this method practical. The FCM-ELTP created by fuzzy C-means algorithm is not as robust as the fuzzy ELTP. However, its performance in low to mediocre noise conditions is still remarkable and can become a viable choice when computation resource is limited.
REFERENCES
[1] M. Pietikäinen, A. Hadid, G. Zhao, and T. Ahonen.
Computer Vision Using Local Binary Patterns, Springer, 2011.
[2] W. Liao, “Region Description Using Extended Local Ternary Patterns”, Proceedings of the 20th International Conference on Pattern Recognition, pp. 1003-1006, 2010.
[3] W. Liao and T. J. Young, “Texture Classification Using Uniform Extended Local Ternary Patterns”, Proceedings of The IEEE International Symposium on Multimedia, pp.191-195, 2010.
[4] Ahonen, T., Pietikäinen, M.: “Soft Histograms for Local Binary Patterns”, In: Proceedings of Finnish Signal Processing Symposium, p. 4 ,2007.
[5] Iakovidis, D.K., Keramidas, E.G., Maroulis, D., "Fuzzy Local Binary Patterns for Ultrasound Texture Characterization", International Conference on Image Analysis and Recognition, LNCS Springer, Vol. 5112, pp.
750-759, Póvoa de Varzim, Portugal, 2008.
[6] P. Gorecki and L. Caponetti, “Color Texture Segmentation with Local Fuzzy Patterns and Spatially Constrained Fuzzy C-Means”, Proceedings of the 7th International Workshop on Fuzzy Logic and Applications:
Applications of Fuzzy Sets Theory, pp.362-369, 2007.
Descriptor Clustering Dim SNR->∞ SNR=44 SNR=35
ELTP spectral 59 99.85 98.88 89.94
ELTP spectral 128 99.9 99.71 92.31
ELTP spectral 256 99.91 99.64 92.91 Fuzzy ELTP spectral 256 99.98 99.98 99.28
ELTP FCM 59 99.74 99.12 94.33