CHAPTER 5 SIMULATION RESULTS
5.2 Chip Implementation
5.2.4 Testing consideration
The testing methods could be separated in two parts: digital CNN control circuit testing and fully custom ASCNN circuit testing. The package will first test by IMS testing machine to check the function of CNN control circuit. With the design of scan chain in digital circuit, the errors could simply feed the ATPG test pattern which generated by Tetra-max into chip testing pad (itsi) and analysis the output pattern for output pad (oY_tso). The testing pad information is discussed in Appendix A.
Then, testing image generated by Logic Analyzer could feed into LAM by passing on chip 8-bit DAC pad (iV8~1). Wait until all the LAM have loaded the image SAD information, testing pads (oY_mem1_1、oY_mem19_1、oY_mem25_1、oY_mem19_25) which represent the 4 corner LAM storage voltage could be analyzed and check if the memories voltage could raise as prediction.
The CNN threshold control circuit could check the testing pad (oY_biasL) voltage and see if the bias voltage can active after all LAM have loaded information.
The pixel processing array could be check by feeding simple test pattern which only contain a extremely small pixel value and see if the CNN global connect output chain could mark the minimum position.
◎ Digital Part： Add a scan chain by using DFT compiler. The test patterns have 119 sets in total and the fault and test coverage are 98.72 % and 100 %, respectively.
Fig. 59 ATPG reports fromTetra-max.
◎ Analog Part：
1. LAM test: with a flat image pattern loaded by passing 8-bit DAC into LAM as shown in Fig. 60, the testing pad could help to analysis the properties of LAM functions to see if they can charge successfully and check the mismatch due to fabrication.
minimum coordinate.
3. Bias circuit test: feed the input with a flat but large SAD iamge and let all the pixel prcessing units cannot catch the minimuim position. Then test the bias testing point voltage and see if the volgate has reach the predic value. If the bias is out of the predic range due to differnet fabrication environment, the pad iVsh can be tunned to change the VCC input current to the CNN to feed the need of threshold level.
Fig. 60 Digital and analog part testing consideration of ASCNN chip.
6. CHAPTER 6
CONCLUSIONS AND FUTURE WORKS
In this thesis we propose the local motion estimation chip based on CNN architecture for image stabilization and combine the motion compensation method to construct the image stabilization system. To obtain reliable LMVs from the video sequence captured in various conditions and to reduce the computational complexity in finding candidate of motion vectors, these are two challenges for an image stabilization system. According to the simulation results, the proposed technique demonstrates the remarkable performance in both quantitative and qualitative (human vision) evaluations compared with the existence of approaches. The ASCNN chip is implemented for real-time video stabilization applications.
In particular, CNN is used to reduce computational complexity occurring in motion estimation and the ASCNN chip has superior performance compared with DSP processors. An 8-bit D/A converter is designed to transform digital input into analog current and stored in LAM.
LAM is designed to store SAD values and the memory cell is made of MOS capacitance. Various SAD voltage stored in LAM can produce input current of CNN by the VCC circuit. An adaptive and adjustable bias current circuit is combined with CNN to automatically increase the threshold level and detect the global minimal SAD position. The global output connected chains which take the CNN connection properties are used to decode LMV position without wasting more processing cycles in searching address. The proposed CNN-based method can be implemented on VLSI, and CNNUM is able to perform the idea of the proposed design. Results with mixed-signal simulation based on TSMC 0.35µm 2P4M process have demonstrated the superior functionality of the designed circuit.
For the chip design or DIS implementation in the future, several possible improved directions will be considered as follows.
1. The power consumption of CNN array should be reduced to fit the most consumer electronics.
Reduce the supply current of CNN seems to be the best way to solve this problem since
turned on.
2. A programmable CNN template is used instead of a fixed CNN template. The programmable method can reuse CNN architecture and perform more CNN functions. Dual clock design can also be used to speed up processing efficiency. The faster clock is used for CNN array in searching global minimum position. The other clock is used for new LAM which has shorter loading time.
3. To further distinguish the SAD value into fewer pixels, more digits should be used to control bias signals, and a judgment control circuit should also be used to skip the raising clock cycle if the bias current is much lower than input current.
REFERENCES
[1] M. Sekine, T. Kondou, and H. Hirose, “Motion Vector Detecting System for Video Image Stabilizers,” Proc. of IEEE International Conference on Consumer Electronics, pp.
268-269, Jun. 1994.
[2] http://www.personaltechpipeline.com/news/59100088 [3] http://www.canon.com/technology/dv/02.html
[4] D. W. Holder and W. R. Philips, “Electronic Image Stabilization,” U.S. Patent no.4637571, Jan. 20, 1987.
[5] M. Oshima, “VHS camcorder with electronic image stabilizer,” IEEE Trans. on Consumer
Electronics, vol. 35, pp. 749-758, Nov. 1989.
[6] Joon ki Paik, Yong Chul Park, and Dong Wook Kim, “An adaptive motion decision system for digital image stabilizer based on edge pattern matching,” IEEE Transactions on
Consumer Electronics, vol. 38, pp. 607-616, Aug. 1992.
[7] Jyh-Yeong Chang, Wen-Feng Hu, Mu-Huo Cheng, and Bo-Sen Chang, “Digital image translational and rotational motion stabilization using optical flow technique,” IEEE
Transactions on Consumer Electronics, vol. 48, pp. 108-115, Feb. 2002.
[8] Sung-Jea Ko, Sung-Hee Lee, Seung-Won Jeon, and Eui-Sung Kang, “Fast digital image stabilizer based on Gray-coded bit-plane matching,” IEEE Transactions on Consumer
Electronics, vol. 45, pp. 598-603, Aug. 1999.
[9] http://www.itri.org.tw/index.jsp
[10] M. Irani, B. Rousso, and S. Peleg, “Recovery of ego-motion using image stabilization,”
Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp.
454-460, Jun. 1994.
[11] Kwon, R. Chellappa, and C. Morimoto, “Motion compensated sub-band coding of video acquired from a moving platform,” Proc. of IEEE Int. Conf. on Acoustics, Speech, and
Signal Processing, pp. 2185-2188, Jan. 1995.
[12] C. Morimoto, D. DeMenthon, L. Davis, R. Chellappa, and R. Nelson, “Detection of independently moving objects in passive video,” Proc. of Intelligent Vehicles Workshop, pp.
270-275, Sep. 1995.
[13] K. Uornori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Kitamura, “Automatic image stabilizing system by full-digital signal processing,” IEEE Trans. on Consumer Electronics, vol. 36, pp. 510-519, Aug. 1990.
[14] J. K. Paik, Y. C. Park, and D. W. Kim, “An adaptive motion decision system for digital image stabilizer based on edge pattern matching,” IEEE Trans. on Consumer Electronics, vol. 38, pp. 607-615, Aug. 1992.
[15] T. Kinugasa, N. Yamamoto, H. Komatsu, S. Takase, and T. Imaide, “Electronic image stabilizer for video camera use,” IEEE Trans. on Consumer Electronics, vol. 36, pp.
520-525, Aug. 1990.
[17] Y. Egusa, “An application of fuzzy set theory for an electronic video camera image stabilizer,” IEEE Trans. on Fuzzy Systems, vol. 3, pp. 351-356, Aug. 1995.
[18] J. K. Paik, Y. C. Park, and D. W. Kim, “An adaptive motion decision system for digital image stabilizer based on edge pattern matching,” IEEE Trans. Consumer Electronics, vol.
38, pp. 607-616, Aug. 1992.
[19] S. W. Jeon, “Fast digital image stabilizer based on Gray-coded bit-plane matching,” IEEE
Trans. Consumer Electronics, vol. 45, pp. 598-603, Aug. 1999.
[20] L. O. Chua and L. Yang, “Cellular neural networks: theory and application,” IEEE Trans. on
Circuits and Systems, vol. 35, pp. 1257-1290, Oct. 1988.
[21] R. Carmona Galan, A. Rodriguez Vazquez, S. Espejo Meana, R. Dominguez-Castro, T.
Roska, T, Kozek, and L. O. Chua, “An 0.5-µm CMOS analog random access memory chip for TeraOPS speed multimedia video processing,” IEEE Transactions on Multimedia , vol.1, pp. 121-135, Jun. 1999.
[22] L. O. Chua and T. Roska, “The CNN paradigm,” IEEE Transaction on Circuits and System,
I, vol. 40, pp. 147-156, Mar. 1993.
[23] S. Espejo, A. Rodr´ıguez-V´azquez, and R. Dom´ınguez-Castro, “Smartpixel cellular neural networks in analog current-mode CMOS technology,” IEEE J. Solid-State Circuits, vol. 29, pp. 895-905, Aug. 1994.
[24] R. Dom´ınguez-Castro, S. Espejo, A. Rodr´ıguez-V´azquez, R. Carmona, P. Foldesy, A.
Zar´andy, P. Szolgay, T. Sziranyi, and T. Roska, “A 0.8 um CMOS programmable mixed-signal focal-plane array processor with on-chip binary imaging and instructions storage,” IEEE J. Solid-State Circuits, vol. 32, pp. 1013-1026, Jul. 1997.
[25] S. Espejo, R. Dom´ınguez-Castro, G. Li˜n´an, and A. Rodr´ıguez-V´azquez, “64 X 64 CNN universal chip with analog and digital I/O,” in IEEE1998 Inter. Conference Electronic
Circuits and Systems, pp. 203-206, Sep. 1998.
[26] T. Roska and L. O. Chua, “The CNN universal machine: An analogic array computer,”
IEEE Trans. Circuits Syst. II, vol. 40, pp. 163-173, Mar. 1993.
[27] J. E. Varrients, “A current-mode cellular neural network implementation,” IEEE Trans. on
Circuits and Systems-II, vol. 40, pp. 147-155, Mar. 1993.
[28] S. Erturk, “Image sequence stabilization: motion vector integration (MVI) versus frame position smoothing (FPS),” Proc. of the 2nd International Symposium on Image and Signal
Processing and Analysis, pp. 266-271, 2001.
[29] M. K. Gullu and S. Erturk, “Fuzzy image sequence stabilization,” Electronics Letters, vol.
39, no. 16, pp. 1170-1172, Aug. 2003.
[30] M. K. Gullu, E. Yaman, and S. Erturk, “Image sequence stabilization using fuzzy adaptive Kalman filtering,” Electronics Letters, vol. 39, pp. 429-431, Mar. 2003.
[31] M. K. Gullu and S. Erturk, “Membership function adaptive fuzzy filter for image sequence stabilization,” IEEE Trans. Consumer Electronics, vol. 50, pp. 1-7, Feb. 2004.
[32] C. T. Lin, S. C. Hsu, and S. F. Liang, “A robust digital image stabilization technique based on inverse triangle method and background detection,” IEEE Trans. on Circuits and
Systems-I, submitted paper for publication, 2004.
[33] Ealgoo Kim, Daeyun Shim, Seongik Jang, Gyudong Kim, and Wonchan Kim, “A global motion characterization method using one-dimensional characteristic curves for image
processing applications,” IEEE International Symposium on Circuits and Systems, ISCAS
'97, vol. 2, pp. 1393-1396, Jun. 1997.
[34] M. M. Sondhi, “New methods of pitch extraction,” IEEE Trans. Audio Electroacoustics, vol.
AU-16, pp. 262-266, Jun. 1968.
[35] Bing J. Sheu and Joongho Choi, Neural Information Processing and VLSI, KLUWER ACADEMIC PUBLISHERS, Ch.5, pp. 97-149, 1995.
[36] Bing J. Sheu and Joongho Choi, Neural Information Processing and VLSI, KLUWER ACADEMIC PUBLISHERS, Ch. 14, pp. 397-452, 1995.
[37] N. H. Farhat, “Optoelectronic neural networks and learning machines,” IEEE Circuits and
Device Magazine, pp. 32-41, Sep. 1989.
[38] G. A. Carpenter, S. Grossberg, and D. B. Rosen, “ART 2-A: an adaptive resonance algorithm for rapid category learning and recognition,” Neural Networks, vol.4, 1991.
[39] J. S. Kane and M. J. Paquin, “POPART: partial optical implementation of adaptive resonance theory 2,” IEEE Trans. on Neural Networks, vol. 4, pp. 695-702, Jul. 1993.
[40] T. Roska and L. O. Chua, “Cellular neural networks with nonlinear and delay-type template elements,” Proc. IEEE int. Workshop on Cellular Neural Networks and Their Applications, pp. 12-25, 1990.
[41] A. Rodriguez-Vazquez, S. Espejo, R. Dominguez-Castron, Huertas, L. J., and E.
Sanchez-Sinencio, “Current-mode techniques for the implementation of continuous- and discrete-time cellular neural networks,” IEEE Trans. on Circuits and Systems, Part II, vol.40, pp. 132-146, Mar. 1993.
[42] J. E. Varrientos, E. Sanchez-Sinencio, and J. Ramirez-Angulo, “A current-mode cellular neural network implementation,” IEEE Trans. on Circuits and Systems, part II, vol.40, pp.
147-155, Mar.1993.
[43] Hyongsuk Kim, T. Roska, H. Son, and I. Petras, “Analog addition/subtraction on the CNN-UM chip with short-time superimposition of input signals,” IEEE Trans. on Circuits
and systems- I: Fundamental Theory and Applications, vol.50, Mar. 2003.
[44] “CNN software library-version 1.1,” Analogic Computing LTD.
[45] http://www.mxeduc.org.tw/SiliconAwards/2004/5thSiliconAward.htm
[46] C. Y. Wu, P. H. Lu, C. H. Lee, and M. K. Tsai, “New fully differential HF CMOS op amps with efficient common mode feedback,” IEEE International Symposium on Circuits and
Systems, vol. 3, pp. 2076-2079, May, 1989.
[47] C. A. Poynton, A Technical Introduction to Digital Video. New York: Wiley, 1996.
[48] T. Roska and L. O. Chua, “The CNN universal machine: An analogic array computer,”
IEEE Trans. Circuits Syst. II, vol. 40, pp. 163-173, Mar. 1993.
[49] L. O. Chua, T. Roska, T. Kozek, and A. Zarandy, “CNN universal chips crank up the computing power,” IEEE Circuits Devices Mag., vol. 12, pp. 18-28, July 1996.
[50] S. Espejo, R. Dom´ınguez-Castro, G. Li˜n´an, and A. Rodr´ıguez-V´azquez, “64 _ 64 CNN universal chip with analog and digital I/O,” IEEE 1998 Int. Conf. Electronic Circuits and
Systems, pp. 203-206, Sep. 1998.
[51] TSMC 0.35um Mixed-signal 2p4m Polycide 3.3V/5V SPICE Model.
7. APPENDIX
A. Chip Information
Chip Specification Table
Spec. Result
Technology TSMC 0.35µm 2P4M CMOS Mixed Signal Process
Power supply 3.3 Volts
Power dissipation 274mW(fully custom) + 8.3mW(cell based design)
Input level Logical low: 0V, high: 3.3V
Output level Logical low: 0V, high: 3.3V
Operational Frequency (Cell-based) 20MHz
Cell-based design Area 63315
Bias step time 0.1us
LAM accessing time 3.1us
Bias resolution 32-tap (5-bit)
No. of output 18 (fully=5, cell=13)
Finding LMV for each frame Less than 6.2ms
Package 48-pin
Pad Names and Description
Pin No. Name I/O direction Description
47 oY_mem19_25 O Local Analog Memory testing point 19_25
2 oY_mem19_1 O Local Analog Memory testing point 19_1
3 oY_mem1_25 O Local Analog Memory testing point 1_25
6 oY_mem1_1 O Local Analog Memory testing point 1_1
7 VDD_1 Power Analog power 3.3 V
40 oY_biasL O CNN threshold bias output voltage test point
47 oY_biasU I Unity gain buffer voltage test point
8 iV8 I Input Vin[8]
9 iV7 I Input Vin[7]
10 iV6 I Input Vin[6]
11 iV5 I Input Vin[5]
12 iV4 I Input Vin[4]
13 iV3 I Input Vin[3]
14 iV2 I Input Vin[2]
15 iV1 I Input Vin[1]
16 itse I Scan chain enable
17 iVsh I Current voltage converter bias (2.85V)
18 iSTART I Start signal for CNN control circuit
20 itsi I Scan chain input
21 VSS_0 Power Digital power 3.3V
22 iFRST I Reset
23 iFCLK I Clock
24 oY_Fsh O Finish signal
25 oY_LE I Load enable
26 oY_aY0 O axis_Y[0]
27 oY_aY1 O axis_Y[1]
28 oY_aY2 O axis_Y[2]
29 oY_aY3 O axis_Y[3]
30 oY_aY4 O axis_Y[4]
33 VDD_0 power I/O PAD power 3.3V
34 oY_ax0 O axis_X[0]
35 INT_VSS_0 power Digital power 0V
36 oY_ax1 O axis_X[1]
37 oY_ax2 O axis_X[2]
38 INT_VDD_0 power Digital power 3.3V
39 oY_ax3 O axis_X[3]
40 oY_ax4 O axis_X[4]
41 oY_tso O Scan chain output
佈局驗證結果說明
Fully custom design LVS check
Whole CHIP DRC check
DRC error : VERTEX OFFGRID 假錯
! 1000
M3.W.1 為 corner layout 假錯 ! 2 M4.W.1 為 corner layout 假錯 ! 2
M1.R.1 為 metal1 density 不足 30% warning，為 CIC 可忽略假錯 1
M2.R.1 為 metal2 density 不足 30% warning，為 CIC 可忽略假錯 1 M3.R.1 為 metal3 density 不足 30% warning，為 CIC 可忽略假錯 1 M4.R.1 為 metal4 density 不足 30% warning，為 CIC 可忽略假錯 1
共 1008 個可忽略假錯
Whole CHIP LVS check
*** Chip Features CAD Tools ***
CKT name : CHIP_TOP (設計名稱) HSPICE
Technoloy : TSMC 0.35um 2P4M CMOS (使用製程) OPUS
Package : 48 S/B (包裝種類)
Chip Size : 2.375× 3.442 mm2 (晶片面積)
Transistor/Gate Count : 4776 MOS / 6.33K gate count (電晶體/邏輯閘數)
Power Dissipation : ~355mW(最高)/~170mW(平均) (功率消耗；mW)
Max. Frequency : < 20 MHz (最高工作頻率，MHz) (最高工作頻率) Testing Results : □Function work □Partial work □Fail
佈 局 平 面 圖
Side Braze 48-pin lead
Bonding Diagram (打線圖)
B. Optical IS Architecture
Source: Cannon website: http://www.canon.com/technology/dv/02.html [3]
If the lens front moves downward due to camera shake, the light from the subject shifts from the optical axis of the lens, and the image on the film plane is moved downwards. When this occurs, the corrective optical system is moved downwards to refract the light and the image is returned to the center of the film plane. In reality, both vertical and horizontal vibration occurs and the corrective optical system is moved in four directions along a plane perpendicular to the optical axis.