Bio-Inspired Computer Fovea Model Based on
Hexagonal-Type Cellular Neural Network
Chao-Hui Huang, Student Member, IEEE, and Chin-Teng Lin, Fellow, IEEE
Abstract—For decades, numerous scientists have examined the
following questions: “How do humans see the world?” and “How do humans experience vision?” To answer these questions, this study proposes a computer fovea model based on hexagonal-type cellular neural network (hCNN). Certain biological mechanisms of a retina can be simulated using an in-state-of-art architecture named CNN. Those biological mechanisms include the behaviors of the photoreceptors, horizontal cells, ganglions, and bipolar cells, and their co-operations in the retina. Through investigating the model and the abilities of the CNN, various properties of the human vision system can be simulated. The human visual system possesses numerous interesting properties, which provide natural methods of enhancing visual information. Various visual information enhancing algorithms can be developed using these properties and the proposed model. The proposed algorithms include color constancy, image sharpness, and some others. This study also discusses how the proposed model works for video enhancement and demonstrates it experimentally.
Index Terms—Bipolar cell, cellular neural networks (CNNs),
color constancy, fovea, ganglion, hexagonal, horizontal cell, pho-toreceptor, retina, sharpness.
I. INTRODUCTION
A
S A HIGHLY structured neuron network, a retina extracts and processes the stimulation from an image projected upon it by the optical system of the eye [1]–[3]. This process is extremely complex. Consequently, analyzing, modeling, and even simulating the retina have been considered highly chal-lenging tasks. To meet these challenges, numerous studies have been published during the past decade: for example, Shah etal. studied the information processing procedure of the retina
in both the space and time domains [4]. Based on the study of Shah et al., Thiem later proposed a bio-inspired retina model capable of implementing some visual properties in the human vision system [5]. Roska and Bálya et al. even discussed the parallel structure of retina and proposed an implementation on cellular neural networks (CNNs) [6]. Their study demonstrated Manuscript received January 5, 2006; revised September 17, 2006. This work was supported in part by the National Science Council, Taiwan, under Grant 95-2752-E-009-011-PAE, Ministry of Economic Affairs, Taiwan, under Grant 95-EC-17-A-02-S1-032, and MOE ATU Program 95W803E. This paper was recommended by Guest Editor O. Yadid-Pecht.
C.-H. Huang is with the Department of Electrical and Control Engineering, National Chiao-Tung University (NCTU), Hsinchu 300, Taiwan, R.O.C. (e-mail: [email protected]).
C.-T. Lin is with the Department of Electrical and Control Engineering and the Department of Computer Science and the Brain Research Center, National Chiao-Tung University (NCTU), Hsinchu 300, Taiwan, R.O.C. (e-mail: [email protected]).
Digital Object Identifier 10.1109/TCSI.2006.887975
that certain visual information enhancing mechanisms exist in the human vision system. Accordingly, this study focuses on how to simulate the retinal fovea based on hexagonal-type CNN (hCNN). Furthermore, this study examines how to improve im-ages and videos based on the proposed computer fovea model.
First, the biological structure of a retina must be understood. Five major types of neurons exist in the five layers of the retina. Outer nuclear layer contains photoreceptors, Inner nuclear layer contains horizontal cells, amacrine cells and bipolar cells, and ganglion layer contains ganglions. Moreover, the outer plexi-form layer contains the synapse connections among the pho-toreceptors, the horizontal cells and the bipolar cells. Finally, inner plexiform layer contains the synapse connections among the bipolar cells, the amacrine cells and the ganglions [1], [7].
The human visual system contains four kinds of photore-ceptor: L-cone, M-cone, S-cone, and rod cells. These different types of photoreceptors react to different wavelengths of light (Fig. 1). Rod cells can sense light intensity, while L-cone, M-cone, S-cone cells can detect color information. Sometimes the cooperation of these cone cells can also detect the light intensity. We are already aware that the ganglion is usually activated by a set of the photoreceptors via other bipolar, hori-zontal, and other types of cells [2], [8]. This set contains several varieties of photoreceptors. The differences among those dif-ferent types of photoreceptors results in the variation among the ganglions. The two main types of ganglions are center-on/sur-round-off and center-off/surround-on. Fig. 2 illustrates two examples, where Fig. 2(a) shows an example of center-sur-round ganglion. The center of a group of photoreceptors reacts to the stimulation differently to the peripheral part of the group. The ganglions can generally be classified as red-green (RG) ganglion, blue-yellow (BY) ganglion, and black-white (BW) ganglion [see Fig. 2(b) and (c)] and others [1]. Furthermore, based on the different responses of neighboring ganglions, the visual information can be sensed. The visual information included luminance and color. Fig. 3 shows the ganglions’ response to the light simulation. Fig. 3 (a) represents how the center-green-on/surround-red-off ganglions respond on the border of the red and green region. Fig. 3 (b)–(d) shows the responses of the different kinds of ganglions.
The ganglion is important in the human vision system and processes most visual information. Visual information includes light intensity and color information. The mechanism which is used to process this visual information in the human vision system is so-called early vision system, also known as the pre-attentive vision system. The early vision system represents a set of the first stage information processing mechanisms of visual processing. Those mechanisms are operated in parallel across 1057-7122/$25.00 © 2007 IEEE
Fig. 1. Biological structure of the photoreceptors in the fovea of a mammalian. (a) Photoreceptor group in a fovea. (b) Photoreceptors can be classified according to the stimulation they react to [1].
Fig. 2. Example of a ganglion. (a) A ganglion collects information from a set of neighboring photoreceptors. The response of the central part of the set differs from that of the peripheral part of the set. Based on the difference between the types of photoreceptor, ganglion can be classified into RG, BY, and BW types. (b) The RF of a center-green-on/surround-red-off ganglion. (c) The RF of a center-yellow-on/surround-blue-off ganglion.
the visual field, and are believed to be used for detecting certain fundamental visual features [1].
According to Jain et al., the human vision system possesses two fundamental features: first, in some respects the retina acts like a low-pass filter. Generally, the result of the low-pass fil-tering represents an average intensity of light for a specific local area. The result of this operation is termed “the first-order fea-ture.” Second, a difference exists between the intensity of the external light and that projected into the retina. According to some studies, the boundary detection operation in the human vision system is based on this kind of feature, and is related to the zero crossing of a Laplacian of two Gaussians (LoG). Some-times the result of this operation is called “the second-order fea-ture [9].”
The mechanisms of information processing in the retina are rather complex and remain unclear. This study thus mainly aims to design and approximate the receptive fields (RFs) of the cells
on a retinal fovea. From the signal processing perspective, the RFs can be referred to as a finite impulse response of a spatial filter [4], [8], [10]. In this investigation, the well-known CNN is used to realize the spatial filter.
CNN represents the next generation of computational ar-chitecture. Similar to a biological system, each cell in the CNN can communicate only with its immediate neighborhood. Consequently, using the CNN to implement the proposed model makes sense. According to some previous studies, hexagonal image processing (HIP) is much reasonable for image processing, particularly for bio-inspired models (refer to Fig. 1) [11]–[15]. Thus, this study suggests a special type of CNN)—the hCNN. Furthermore, hCNN provides a means of reducing implementation problems without increasing the complexity [12].
Based on the biological investigations and the abilities of the CNN, this work proposes an hCNN-based computer fovea model. The proposed model simulates certain biological mech-anisms of the retina and the fovea, including the photoreceptors, bipolar cells, horizontal cells, ganglions, and the co-operations of those cells in a fovea and a retina. Consequently, some prop-erties of the human vision system can be simulated. The human vision system possesses various interesting properties. Some of those properties can even be used to enhance the visual infor-mation. This study also presents how these properties provide visual enhancement.
This study first briefly introduces the hCNN, and also dis-cusses the stable central linear system for the hCNN and de-velops its implementations. Those implementations are required for the proposed model, including CNN-based Laplace-like op-erators, CNN-based Gaussian-like operators and their inverse operators. Subsequently, the CNN-based computer fovea model is introduced. Building on the above, several experiments are presented, including vision enhancing algorithms based on this model. Finally, conclusions are drawn.
II. METHOD ANDMETHODOLOGY
A. hCNNs
The CNN has already been shown to be a very powerful image processing kernel with artificial-intelligent-like abilities
Fig. 3. Based on the different responses of neighboring ganglions, the visual information can be sensed. The visual information included luminance and color. (a) The center-green-on/surround-red-off ganglions respond on the border of red and green region. (b)–(d) Responses of the different kinds of ganglions. [1].
Fig. 4. Architecture of CNN is shown in (a), and along with the dynamic route of state in CNN.
[16], [17]. Since Chua and Yang first introduced CNN, it has been widely applied in numerous areas. CNN offers an alterna-tive to a fully connected neural network, and has evolved into a paradigm for this type of array. As shown in Fig. 4(a), the dy-namic equation of the CNN can be represented by the following:
(1) where
(2) Here (2) can be represented as Fig. 4(b).
1) CNNs for HIP: The CNN is particularly considered as an
image processor in many areas. Given its flexibility and other abilities, CNN provides excellent computational performance. One area of flexibility is that CNN can be used as a parallel and symmetric structure neural network. Usually, the CNN are
eight-connected structured network. In the fact, CNN is not only being constructed as eight-connected. If we construct the CNN on a flat plane, the cells can be connected as many kinds of sym-metric structure [18]. One possible structure is the six-connected structure, or as the well-known, the hexagonal-type structure [12].
An hCNN can be described by [12]
(3) where
(4) To be consistent with 1-D, 2-D rectangular, and 2-D hexagonal systems, the index is represented using bold characters in this study). Meanwhile, for convenience, this study uses the fol-lowing mathematical symbols to present the templates in the hCNN
(5) where denotes the feedback template, represents the con-trol template, and is the bias. Notably, the indexes in the
tem-plates are the same as HIP designed by Middleton for realizing the hexagonal Fourier transform where necessary [19].
2) Stable Central Linear Systems and Their Inverse Systems:
If the CNN is considered as an image processor then numerous linear properties must be analyzed; for example, the states lo-cated in the non-saturated region. Crounse and Chua mentioned how to analyze the CNN-based image processing in the fre-quency domain [20]. A similar mechanism can be applied to the hCNN [12]. Assuming all of the cells operate in the linear region, that is, , then . Thus, (3) can be refor-mulated as
(6) Let , the linearized templates then can be represented by [20]
otherwise. and
otherwise. (7) The dynamics can then be represented in a convoluted form as
(8) where represents a hexagonal convolution.
The dynamics can be written into [21]
(9) if and only if all of the discrete hexagonal Fourier transforms exist. Regarding the dynamics, assume that infinite time is avail-able and that a discrete hexagonal Fourier transform exists for all , then (8) can be represented as follows [20]:
(10) where
(11) It is already known that if for all , then the central linear system will be stable. Thus, the equilibrium state can be thought of as a version of the input that has been spatially filtered by the hexagonal transfer function [22].
Based on the above, this study develops some operators that are required by the proposed model. These operators include CNN-based Laplace-like operators, CNN-based Gaussian-like operators and their inverse operators. These operators are intro-duced in Sections II-A-II-A–B.
CNN-Based Laplace-Like Operator: A Laplace operator can
be represented as
(12)
The zero-frequency component of the Fourier transform of (12) is zero. This fact implies the inverse operation of (12) is difficult in the discrete environment. Thus, a small positive value is added to (12). Accordingly, the Laplace-like operator is as fol-lows:
(13)
Notably, the value of approaches zero as approaches a Laplace operator. For HIP, the Laplace-like operator is
(14) If and only if the Fourier transform of the Laplace-like operator
exists and is denoted as
(15) Next, map (15) into (11). Since for all are required,
we choose . Thus,
(16) According to (7), the coefficients of the system that has been described in (16) can be represented as
and (17)
Thus, the templates are given by , and , where represents Dirac Delta function. Finally, for the CNN-based Laplace-like operator, the templates are as follows:
and (18)
Meanwhile, for the HIP, the templates are
(19) For convenience, this study denotes the CNN-based Laplace-like operator as follows:
(20) where denotes the input, is a corresponding parameter, and the initial state of all cells is zero.
In fact, for any CNN, if the initial condition , the template , and the bias are zero, then the system is simply a FIR
system and the template B can be considered as the FIR op-erator.
For the inverse Laplace-like operator, the inverse system of (16) can be used. That is
(21)
Notably, since the condition, for all must be sat-isfied, this study sets . Thus, the templates can be obtained as follows:
and (22)
For HIP, the templates are
(23)
For convenience, the CNN-based Inverse Laplace-like Operator is denoted here as follows:
(24) where denotes the input, and represents the corre-sponding parameter in (18) and (19). The initial condition and the bias are zero.
CNN-Based Gaussian-Like Operator: Kobayashi et al.
sug-gested an active resister network for Gaussian-like filtering for the image [20]. The proposed system can be described as fol-lows [13], [22]:
(25) where
(26) The Fourier transforms are denoted as , and
, then the system can be described as
(27) Thus, the templates can be obtained by
and (28)
The templates for different architectures are listed below.
1-D CNN: and (29) 2-D Rectangular-type CNN: and (30) 2-D hCNN: (31)
Shi proposed a CNN-based Gabor-type filtering model based on the study of Kobayashi [22]. His model is based on a Gaussian-like operator similar to (30) However, Shi modified the dynamic equation of the CNN to reduce the complexity of the hardware implementation. The proposed model requires no such modification.
For convenience, this study denotes the CNN-based Gaussian-like operator as follows:
(32) where is the input, is the corresponding parameter, and the initial condition and the bias are zero.
For the inverse Gaussian-like operator, we have
(33)
Since the condition for all must be met, it is concluded that . The templates can be obtained
from and .
The CNN-based inverse Gaussian-like operators for different architectures are listed as follows.
1-D CNN:
and
(34)
2-D Rectangular-type CNN:
2-D hCNN:
(36)
Finally, a CNN-based inverse Gaussian-like operator can be denoted as
(37) where denotes the input, represents the degree of Gaussian function and the initial condition and the bias are zero.
B. Computer Fovea Model
Since a retina is a highly structured network of neurons, it has become a valuable research topic in the field of human computer interaction (HCI). Many studies have studied the structure of the retina and the biological evidence regarding the functions of the human vision system [23], [24]. Notably, some researchers have even analyzed visual information processing in the retina [4], [8]. Building on these previous works, this study constructs] a new computer fovea model.
The retinal fovea is located at the center of the retina, and is the region with the highest visual acuity. The fovea is a 0.2–0.4 mm diameter rod-free area with very thin, densely packed photoreceptors. The photoreceptors in the fovea are arranged in a roughly hexagonal pattern [see Fig. 1(a)] [3], and the average cone spacing (csp) has been estimated at around 2.5 to 2.8 m, where has been considered as the most important area in a retina. The retinal fovea is directed towards the desired object of study. The retinal fovea almost exclusively contains high density cones.
Fig. 5 shows the proposed hCNN-based computer fovea model, where Fig. 5(a) illustrates the top-view of the computer fovea. The computer fovea is constructed using a set of pho-toreceptors, which are hexagonally arranged [see Fig. 5(b)]. Fig. 5(c) is the signal processing system of the cells in the fovea model. Thiem mentioned that because there are direct synaptic connections between bipolar cell and the ganglion, and only few influences by the amacrin cell in the fovea. Hence, in his research, he suggested that the bipolar cell and the amacrin cell are neglected [5]. However, the horizontal cells are already known to exist and connect directly to the bipolar cells [8]. Thus, this study suggests keeping the bipolar cell and the ganglion in the system, and considering the ganglion as a direct synaptic connection.
A simplified version of the proposed model is required to ob-tain the parameters of the proposed architecture. The simpli-fied version of the proposed model ignores the differences be-tween the L-cone, the M-cone, and the S-cone cells. Restated, the system considered for obtaining the parameters is assumed
Fig. 5. Proposed computer fovea model. (a) Computer fovea which has been constructed using hexagonally arranged cells. The width of the computer fovea is approximately 0.4 mm, and is 161 csp. (b) Part of the computer fovea and illustrates the arrangement of the photoreceptors. The diameter of the photore-ceptor is approximately 2.5 um. (c) Signal processing system of each cell in the fovea model. The inputx is the input signal of the center area, while x is the input signal of the surrounding area. For a monotonic input signal,x equals x .
to be monochromatic. Thus, equals . Sections II-B-1)–3), discuss behavioral variation among the cone cells.
1) Ganglion: The ganglions of the mammalian retina have
been characterized into -, -, and -types based on the spa-tiotemporal distribution of the excitation and inhibition. The cells are classified according to their latency from the optic chiasm stimulations [8]. The -type ganglions behave linearly in both the space and time domain and also provide the best spatial resolution, but the most ganglions are considered direct synapse connections. Consequently, in this study, the ganglion has been considered as an amplifier which contains a bias .
Based on the above, can take the form
(38) Some physiological experiments indicated that the RF of the ganglion exhibits a center/surround characteristic. Furthermore, Thiem stated that the RF of ganglions can be modeled as follows [5]:
(39) where represents the Laplace operator, and is a Gaussian function. According to Hubel, under the optimum lighting condition, the central part of RF is about 10 m (4 csp) [1]. Thus, Thiem also recommends a standard deviation of
m (csp) [5].
A combination of the CNN-based Laplace-like operator and the CNN-based Gaussian-like operator can be employed to de-rive the . That is
TABLE I
CORRESPONDING IN(42)FORDEVIATION IN(41)
where can be any extremely small positive value but not zero, and can be obtained by performing GA. Based on the exper-iments performed here, it was concluded that if equals and , then is approximately 0.567700. (see Table I).
2) Photoreceptor: An impulse response of a photoreceptor
can be represented as a Difference of two Gaussians (DoG). In the retinal fovea, the DoG can be described as a Gaussian function, as shown below, in most cases [4], [5], [8]
(41) As mentioned in [8], parameter represents the standard deviation with a range from 1.5 to 12 (csp).
In the proposed approach, a CNN-based Gaussian-like oper-ator is used for the simulation of the Gaussian function. That is
(42) where indicates the diffusion level of the Gaussian-like func-tion. The next question is how to obtain the parameter and make the final state of the CNN approximate that shown in (41). To obtain the corresponding value , genetic algorithm (GA) is used again. Based on the results of the experiments, it was concluded that if equals 1.5, is approximately 0.536040, and if equals 12, then is approximately 0.071233 (see Table I).
Clearly, the RF of the photoreceptor is used to determine the average intensity of a specific area of a visual signal. Generally, it acts like a low-pass filter. The output of the filter, described in (42), is known as the first-order feature.
3) Horizontal Cell: Equation (38) implies that a horizontal
cell can be implemented by the following:
(43) Thus, estimating results in the realization of a hori-zontal cell , if and only if the inverse operator of exists. Thus, can be obtained as
(44) Above it is assumed that the subject system is a monochro-matic system. This means that the input of the bipolar and hor-izontal cells derives from the same set of the photoreceptors. If the inputs of the two types of cells do not derive from the same set of photoreceptors, then the RF of the horizontal cell needs
to be known. Assume the input of a horizontal cell derives from photoreceptor system which is illustrated in Fig. 5(c), then based on (43), can be obtained as follows:
(45) Based on CNN, the horizontal cell can be imple-mented by
(46) Horizontal cell determines the difference between the output signal of the photoreceptors in the center and the surroundings of a RF. This kind of feature is termed the second-order feature. Meanwhile, the final feature involves the determination of the parameters and . From a biological perspective, value rep-resents the related weights of the input signals of the horizontal cells, while denotes the related weight of the ganglion. Unfor-tunately the values are unknown. However, considering the co-operation among the photoreceptor, horizontal cell, bipolar cell, and ganglion, it can be concluded that if an input is nothing but gray, which means there is no stimulation, then the output of the ganglions should be zero. This fact can help us to determine the values of and in a specific environment.
Finally, this study concludes that the photoreceptors, hori-zontal cells, and ganglions can be simulated as follows:
(47)
(48) and
(49) Notably, and are the inputs of the photoreceptors. For a monochromatic system, the ganglion function is
(50)
III. EXPERIMENTS
A. Biology Related Visual Response and Illumination
Figs. 6 and 7 illustrate the impulse response of a horizontal cell in the hCNN-based computer fovea model. Note that
Fig. 6. Example of a center-on/surround-off RF, where, = 0:53604, = 0:5677, " = 0:01 and b = 0:75.
Fig. 7. Example of the model, where (a) is the input and (b) represents the output of the ganglions, while (c) shows the intensity of a section of (b). Notably, the behaviors of this model resembles that of the human vision visual system (see Fig. 3).
Fig. 6(a) can be compared with the well known RF structure (see Fig. 1). Notably, Fig. 7(b) resembles a common illumina-tion in the human vision system.
Fig. 8 shows another example usign the proposed model, where the input is a natural image. Fig. 8(a) is the input, Fig. 8(b) is the output of horizontal cells, Fig. 8(c) is the output of the ganglions, and finally, Fig. 8(d) is the output of the rectification. Notably, to present the hexagonal structure, the resolution of the images in this example is reduced.
B. Simulation of a Retina and Central retinal vein occlusion (CRVO)
Color vision is quite an arbitrary experience, and thus a stan-dard distribution of the sensitivity of the L-cone, the M-cone, and the S-cone cells in the human vision system is used for the simulation purpose in this study [see Fig. 9(a)], and mapped into an image frame. The data of the image frame is described using the specifications of a standard display device. The specifica-tions include the spectral power distribuspecifica-tions of the RGB pri-maries, and a transformation look-up table that describes the nonlinear relationship between the frame-buffer values in the image frame. The intensity of the light emitted by the each of the primaries on the display devices is labeled the display gamma curve (DGC). Fig. 9(b) and (c) shows these data.
Fig. 8. Another example of the proposed model, where the input is a natural image. (a) Input. (b) Output of the horizontal cells. (c) Output of the ganglions. (d) Output following the rectification. Notably, to present the structure of the hexagonal structure, the resolution of the image is reduced in this example.
Thus, a transform operator can be obtained as follows:
(51) where
Here denotes the normalized response of the L-cone, M-cone, and S-cone cells in the human vision system, and represents the radiance data of red, green, and blue phosphor of the stan-dard CRT. Notably, the input data , , and are adjusted using the GDC coefficient , and is a necessary
Fig. 9. (a) Corresponding responses of the L-cone, the M-cone, and the S-cone cells in the different wavelengths. (b) Spectral power distributions of the RGB primaries. (c) Transformation look-up table which describes nonlinear relationship between the frame-buffer values in the image data and the intensity of the light emitted by each of the primaries on the display, named the DGC.
scale value. The related data , , , and then can be obtained as follows:
(52) where , , and represent the related signals of the L-cone, M-cone, and S-cone cells.
To present the image, the experiments use
color space. The diagram is known as the psycho-metric color diagram. The colors of the diagram lie at right an-gles to one another in two directions, and the plane thus created is distributed at right angles to the achromatic axis. The resultant uniform color-space is of course based on the four psycholog-ical basic-colors of red, green, blue, and yellow—first described by Hering in his opponent-theory—which are now known to be transmitted directly to the brain.
In this simulation, the R, G, B, and Y channels can be selected as the inputs. Consequently, the input values of
can be obtained as follows:
and
(53) Based on the definition of , the color channels red, green, and blue can be obtained as follows:
otherwise otherwise and
otherwise. (54)
Fig. 10. This study uses an RG and a BY channel as the input of the center-red-on/surround-green-off and center-blue-on/surround-yellow-off gan-glions, and consequently the new red-to-green and blue-to-yellow chromatic aberrations are obtained. (a) Test image. (b) Reconstructed result. These new chromatic aberrations were used to reconstruct the image, and compare it with the original. The average PSNR is 32.2263 dB. (c) Simulation result of the CRVO.
Finally, the work of (51) and GDC coefficient are reversed. Restated
(55)
where , , and are the final outputs.
On the other hand, the present experiment simulates a center-red-on/surround-green-off and a center-blue-on/sur-round-yellow-off ganglion. The new red-to-green and blue-to-yellow chromatic aberrations are thus obtained.
A result is shown in Fig. 10, where Fig. 10(a) denotes the input, and Fig. 10(b) represents the corresponding output. These new chromatic aberrations are used to reconstruct the image and compare it with the original one. The average peak signal-to-noise rate (PSNR) is 32.2263 dB.
CRVO is a well-known retinal disease. Because a retina needs a lot of oxygen to function, significant blood circulation is necessary. Normally, blood flows into the retina via the cen-tral retinal artery (CRA) and leaves via the cencen-tral retinal vein (CRV). Both of these blood vessels enter the eye through the optic nerve. CRVO is caused by a blood clot in the CRV, which slows or stops blood flow out of the retina. Although initially blood may continue to enter the retina through the CRA, the
Fig. 11. Case of a retinal injury and its visual response. (a) Retina CRVO. The vein on the left side of the fovea is clogged and results retinal injury. (b) The patient will “sense” a piece of gray in the area of the retina that has lost blood circulation.
blockage ultimately stops blood circulation. As a result, blood and fluid are backed up, causing retinal injury and vision loss. Consequently, the brain thus becomes confused and the patient will “sense” a piece of gray in the area of the retina that has lost blood circulation (see Fig. 11).
A simulation result of CRVO is represented in Fig. 10(c). Human vision is a complex system, and cannot be compre-hensively simulated. Even now, only some of the architecture of this system is known. However, some of the features of the present simulation closely resemble those of human vision system. First, the output of the simulator [Fig. 10(b)] closely resembles the input [Fig. 10(a)], even though the color channels are modified. This result implies that the human vision system cannot sense minor differences in color. Second, in the CRVO region, the stimulations of the cells in a specific region are stopped, and the CRVO region can be seen to be gray. This result demonstrates that if there is no signal in a given region, the human vision system will sense a piece of gray in the scene. These conclusions correlate with the results of medical investigations.
C. Image Sharpness Improvement
In the human vision system, the second-order features can provide [the related difference of light intensity between the cells. This is why the human vision system can reliably identify different textures under variable light conditions. On the other hand, a kind of amacrine cell, named biplexiform ganglion cell, is directly connected to the photoreceptors. The biplexform gan-glion cell are depolarising in response to increases in photon catch and as a result, it can provide information regarding am-bient light level, which is useful for controlling pupil diameter and diurnal body rhythm. Thus, the responses of the connected ganglion can be used to implement the improvement in sharp-ness. For the luminance channel of , the following equation can be applied:
(56) where represent the function of the biplexi-forms. Fig. 12 shows an example of image sharpness, where Fig. 12(a) is the original input, and Fig. 12(b) is the output. In fact, this algorithm matches the conclusions of Kotera. Korera
Fig. 12. Example of improvement in image sharpness. (a) Input. (b) Output. The images can be obtained at (http://cnn.cn.nctu.edu.tw/~chhuang/paper/hC-NNCFM/).
previously proposed a sharpness improvement algorithm based on the detection of adaptive edges [25].
D. Color Constancy
According to the gray world hypothesis, in the absence of other colors the world is perceived as gray , [26]. Based on this assumption, the proposed model can be used to estimate the shift in the light. Furthermore, light shifting can be removed via this model.
The stimulation outputs of a horizontal cell are fed back to the connected photoreceptors to reduce the influence of pho-toreceptors on the bipolar cells. This lateral inhibition results in activation of any one photoreceptor reducing that of surrounding photoreceptors. In some cases, the central and peripheral pho-toreceptors react to same kind of the stimulations and thus exert a mutually depressive effect [1]. A monochromatic system pro-vides an example. Thus, this study concludes that a special case of central/surround structure stands as follows:
(57)
where is a small value, , , , and
are the related scale, and and represent the definition of “gray”. Generally, the and are unnecessary. However, in some situations the related environment of the input image is quite extreme and thus the range of the color space is limited. In such cases, and are required to restore the image. Fig. 13 shows an example and the results.
E. Video Auto Adjustment and Enhancement
Since the model parameters are obtained from the input image, the computer fovea model can be used to perform adaptive adjustment and enhancement of sequential images (i.e., video). This experiment applied the proposed visual information from the enhancing algorithm for the each image
Fig. 13. Two results of the proposed image color constancy algorithm. First, a photo is taken under a blue light condition. The photo is shown in (a), and the reconstructed result is shown in (b). Next, a photo is taken of the same scene under yellow light condition, as shown in (c), and the reconstructed result is shown in (d). The chromatic diagrams are shown to the side of the images. The images can be obtained at (http://cnn.cn.nctu.edu.tw/~chhuang/paper/hCNNCFM/).
in the videos. Fig. 14 shows the experiment results, where Fig. 14(a) and (c) are the inputs, while (b) and (d) are the video
outputs. Notably, the algorithm is adaptively adjusted during processing.
Fig. 14. Two experimental results of the integrated video auto adjustment and enhancing algorithm, where(a), (c) are the input videos, (c) and (d) are the recon-structed outputs. The full video can be obtained at (http://cnn.cn.nctu.edu.tw/~chhuang/paper/hCNNCFM/).
IV. CONCLUSION
This work studied the possibilities for implementing an hCNN-based computer fovea model. Although the details of the fovea, retina, and even the whole of the human vision system remain unknown, based on some results of previous researches on both biology and digital image processing, the fovea model can be roughly realized, and consequently some possible applications can be fulfilled. However, some issues re-main open to question. For example, this experiment examines 2 or 3 types of ganglions. In fact, over 20 different structures of ganglions have already been identified. Thus, it is interesting to consider the relationships among these different types of implementations, particularly in cases where the response of a ganglion is related to the time domain. On the other hand, the results of the video processing may differ from those in the proposed model. Roska and Bálya et al. discussed the parallel structure of the mammalian retina and its implementation in CNN [6]. Their study indicated demonstrated the complexity of the retina. Studying the relationship between the structure of the retina and possible applications for image processing offers one of the most interesting topics for future research.
REFERENCES
[1] D. H. Hubel, Eye, Brain, and Vision. New York: Scienceific American, 1988.
[2] D. H. Hubel and T. N. Wiesel, “Receptive fields, binocular interaction, and functional architecture in the cat’s visual cortex,” J. Physiol.
(Lond.), vol. 160, pp. 106–154, 1962.
[3] J. I. Yellot, “Spectral consequences of photoreceptor sampling in the rhesus retina,” Science, vol. 221, pp. 382–385, 1983. [4] S. Shah and M. D. Levine, “Visual information processing in
primate cone pathways— II: Experiments,” IEEE Trans. Syst., Man,
Cybern. B, Cybern., vol. 26, no. 2, pp. 275–289, Feb. 1996.
[5] J. Thiem and G. Hartmann, “Biology-inspired design of digital gabor filters upon a hexagonal sampling scheme,” in Proc. Int.
Conf. Pattern Recogn. (ICPR’00), Barcelona, Spain, 2000.
[6] D. Bálya, B. Roska, T. Roska, and F. S. Werblin, “A CNN framework for modeling parallel processing in a mammalian retina,”
Int. J. Circuit Theory Appl., vol. 30, pp. 363–393, 2002.
[7] B. Roska and F. S. Werblin, “Vertical interactions accross ten paralel, stacked representations in the mammalian retina,” Nature, vol. 410, pp. 583–587, 2001.
[8] S. Shah and M. D. Levine, “Visual information processing in primate cone pathways—Part I: A model,” IEEE Trans. Syst., Man,
Cybern. B, Cybern., vol. 26, no. 2, pp. 259–274, Feb. 1996.
[9] A. K. Jain, “Unsupervised texture segmentation using gabor filters,”
Pattern Recogn., vol. 24, pp. 1167–1186, 1991.
[10] A. Jacobs and F. S. Werblin, “Spatiotemporal patterns at the retinal output,” J. Neurophysiol., vol. 80, pp. 447–451, 1998. [11] B. K. P. Horn, “Determining lightness from an image,” Comput.
Graphics Image Process., pp. 277–299, 1974.
[12] C. H. Huang and C. T. Lin, “Cellular neural networks for hexagonal image processing,” in Proc. Int. Workshop Cellular Neural Networks
Their Applications, , Hsinchu, Taiwan, R.O.C., 2005.
[13] H. Kobayashi, J. L. White, and A. A. Abidi, “An active resister network for Gaussian filtering of images,” IEEE J. Solid-State
Circuits, vol. 26, no. 3, pp. 738–748, Jun. 1991.
[14] I. Her, “A symmetrical coordinate frame on the hexagonal grid for computer graphics and vision,” ASME J. Mech. Design, vol. 115, pp. 447–449, 1993.
[15] S. B. M. Bell, F. C. Holroyd, and D. C. Mason, “A digital grometry for hexagonal pixels,” Image Vision Comput., vol. 7, pp. 194–204, 1989.
[16] L. O. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE
Trans. Circuits Syst., vol. 35, no. 12, pp. 1257–1272, Dec. 1988.
[17] ——, “Cellular neural networks: Applications,” IEEE Trans. Circuits
Syst., vol. 35, no. 12, pp. 1273–1290, Dec. 1988.
[18] G. Seiler and J. A. Nossek, “Symmetry properties of cellular neural networks on square and hexagonal grids,” in Proc. Int.
Workshop Cellular Neural Networks Their Applications, , 1992,
pp. 258–263.
[19] L. Middleton and J. Sivaswamy, “Framework for partical hexagonal-image processing,” J. Electron. Imag., vol. 11, pp. 104–114, Jan. 2002.
[20] K. R. Crounse and L. O. Chua, “Methods for image processing and pattern formation in cellular neural networks,” IEEE Trans.
Circuits Syst. I, Fundam. Theory Appl., vol. 42, no. 4, pp. 583–601,
Apr. 1995.
[21] L. Middleton, J. Sivaswamy, and G. Coghill, “The FFT in a hexagonal-image processing framework,” in Proc. New Zealand
Image Vision Comput. Conf., Dunedin, New Zealand, 2001.
[22] B. E. Shi, “Gabor-type filtering in space and time with cellular neural networks,” IEEE Trans. Circuits Syst. I, Fundam. Theory
Appl., vol. 45, no. 1, pp. 121–132, Jan. 1998.
[23] H. Steklis and J. Erwin, “The primate retina,” Compar. Primate
Biol., Neurosci., vol. 4, pp. 203–278, 1988.
[24] S. Shah and M. D. Levine, “The primate retina: A biological summary,” Ph.D. dissertation, Center for Intelligent Machines, , McGill University, Montreal, QC, Canada, 1992.
[25] H. Kotera, Y. Yamada, and K. Shimo, “Sharpness improvement adaptive to edge strength of color images,” in Proc. Color Imaging
Conf. 2000, Scottsdale, AZ, 2000.
[26] G. Buchsbum, “A spatial processor model for object colour perception,”
Chao-Hui Huang (S’05) received the M.S. degree in
computer science and information engineering from Chung-Hua University (CHU), Taiwan, R.O.C., in 2001. He is currently working toward the Ph.D. degree at National Chiao-Tung University (NCTU), Taiwan, R.O.C.
His research interests are in the areas of artificial intelligence, soft computing, and bio-inspired infor-mation system.
Chin-Teng Lin (F’05) received the B.S. degree from
National Chiao-Tung University (NCTU), Taiwan, R.O.C., in 1986, and the Ph.D. degrees in electrical engineering from Purdue University, West Lafayette, IN, in 1992.
He is currently the Chair Professor of Electrical and Computer Engineering, Dean of Computer Sci-ence College, and Director of Brain Research Center at NCTU. His current research interests are fuzzy neural networks, cellular neural networks, smart vision systems, and computational neuroscience. He is the coauthor of the book Neural Fuzzy Systems—A Neuro-Fuzzy Synergism
to Intelligent Systems (Prentice-Hall, 1996), and the author of Neural Fuzzy Control Systems with Structure and Parameter Learning (World Scientific,
1994).
Dr. Lin is an IEEE Fellow for his contributions to biologically inspired infor-mation systems. He currently serves on the Board of Governors at IEEE Circuits and Systems (CAS) Society (2005–2008), and served for the IEEE Systems, Man, Cybernetics (SMC) Society in 2003–2005. He was the Distinguished Lec-turer of IEEE CAS Society from 2003 to 2005. He was Special Session Co-Chair of ISCAS 2006 in Greece, and the Program Co-Chair of IEEE Proc. Int. Conf. SMC 2006 in Taiwan. He has been the President of Asia Pacific Neural Network Assembly since 2004.
