Component and Platform

1.2.1 System Block Diagram

The embedded system mainly includes CMOS image sensors more than one, an interface chip and a FPGA development board with peripherals. The interface chip is named as ‘image combiner and acquisition interface’ or ‘ICAI’. ICAI processes the data from multiple CMOS image sensors and is integrated with the FPGA development board. The FPGA development board is the integrator for all components by hardware solutions and the communicator with the host computer. For example, FPGA development board must implement ICAI controller and VGA controller to drive the operation of ICAI and VGA chip. By the way, a VGA monitor could display image in real-time from VGA port on FPGA development board.

IDE-Link is like the link between a satellite and a server in satellite transmission station. IDE-Link is built in Nios II IDE (Integrated Development Environment) and provides a simple method for downloading data from the FPGA development board to the host computer. The hardware of IDE–Link is with JTAG interface that is normally for debug. Besides, the host computer is the receiver of image data.

3

To demonstrate the imagery system, the study designs a prototype to approach the concept. The architecture of the system is shown as Figure 3. In below articles, these components are introduced in detail.

Fi gure 3: System block diagram

1.2.2 Sensor

The system use CMOS image sensors to capture image and the size of one sensor is 704 pixel × 1 pixel. The geometry about the strip sensor is 6.5um × 1500um. It means that the sensor is a one-dimension structure. To get a full image in the space, the sensor constructs a two-dimensional image by scanning in spatial domain. The full image size is determined by scan steps and the scan rate must match the satellite speed.

These strip sensors are aligned as a ‘liner’ sensor, the resolution becomes (several times of 704) pixel × 1 pixel. After package, the image sensor array is like an interleaved structure and there are overlap areas between neighboring sensors. The practical alignment is shown in Figure 5.

4

Figure 4: Interleaved alignment of sensors and x-y axes definition

1.2.3 ICAI

The other important part of the system is ICAI chip to capture the image of the sensor array. ICAI is a programmable interface between CMOS image sensor chip and host computer. There are three blocks in the ICAI – CIS control logic, CIS image combiner and host interface. The CIS control logic activate CMOS image sensor and acquire image data periodically. The one-dimension image is combined to form a two-dimension image by the CIS image combiner. The host interface receives and parses commands from the host computer. In addition, the host interface wraps the processed image data, and then transmits back to host computer for further image processing. The architecture of ICAI is shown in Figure 5. Due to the CMOS image sensor space arrangement, the image data should be rearranged by CIS image combiner.

The process flow of combining image is as Figure 6. The symbols, A, B, C and D, represent the image from four CMOS image sensor. The suffix in these symbols means their acquisition time index. Due to the acquisition timing, A1, B1, C1 and D1 will be acquired by ICAI at the same time. However, A1, B2, C1 and D2 should be the same line.

Figure 5: ICAI architecture

5

A1 B2 C1 D2

Figure 6: Process flow of combining image

1.2.4 Integrator by FPGA Development Board

In the embedded system, the integrator is FPGA development board. The integrator responds for gathering data from ICAI, transferring image data to VGA monitor in real time and saving image file to host computer. To achieve the functions, the necessary peripherals include VGA port and GPIO port. The VGA port is for image displaying and the GPIO port is used for ICAI controller. We choose the Altera DE2-70 development board to build the image system.

The DE2-70 use Altera Cyclone II 2C70 FPGA as the core chip, and the core chip characteristics: 68,416 LEs (Logic Element), 250 M4K RAM blocks, 1,152,000 total RAM bits, 150 embedded multipliers, 4 PLLs, 622 user I/O pins, and FineLine BGA 896-pin package. The photograph of DE2-70 is shown in Figure 7. The block diagram of DE2-70 is shown in Figure 8.

6

Figure 7: Photograph of DE2-70

Figure 8: Block diagram of DE2-70

7

1.2.5 Host Computer and Image Post-Processing Software

In the overall system, it includes a host computer that is as server in earth. The software on the host computer responds for image post-processing and these image raw data are from the FPGA board. The OS (operation system) of the host computer is

Microsoft Windows.

The host computer must process the raw bitmap to approach the real image. ‘Real’

means that it must eliminate some variation from hardware parts, ex. difference among sensor chips, package mismatch, etc. The other special function is HD Photo conversion. HD Photo is a better image compressible format than JPEG. The higher compressible rate is efficient in space application. In conclusion, the software has two main functions: eliminating hardware variation and HD photo encoding.

1.2.6 Sensor Emulator

To implement the system rapidly, we use the sensor emulator to take a trial-rum.

The sensor emulator is also designed by using DE2-70 development platform. First of all, pattern image, which has resolution of 704 × 704 pixels, has been extended sixteen times to store in the memory of sensor emulator. Because there is only 65,536 bytes in the memory, the pattern image (704 × 704 pixels) is stored in 1 bit per pixel format. Due to the output bandwidth, two dual-port memories are utilized for storing pattern image.

Each dual-port memory has two output port and two address port. The configuration of memory block is shown in Figure 9. Every line is formed by 88 bytes, that is, 704 (88 × 8) pixels per line. However, each line is split equal and stored in two dual-port memories. Because the data is programmed into the sensor emulator while programming FPGA, there is no need to write any data to the memory during emulation.

The control logic in sensor emulator read out pattern image in sequence. However, every pixel and every column are repeated four times. A simple interpolation technique is applied to extend 704 × 704 pixels to 2816 × 2816 pixels. The concept is depicted in Figure .

8

Figure 9: The configuration of two dual-port memory.

704 columns (lines)

704 pixels

704 4 pixels

704 columns (lines)

(a) CIS Image sensor (b) CIS Image sensor

Figure 10: The applied interpolation concept in sensor emulator

9

Chapter 2 Hardware Architecture