Thesis Organization - 利用搭載雙環場攝影機取像系統之自動車作自動建立房間立體空間圖之研究

Chapter 1 Introduction

1.5 Thesis Organization

The remainder of this thesis is organized as follows. In Chapter 2, the configuration of the proposed two-camera omni-directional imaging system and the principle of automatic 3-D house-layout construction are described. In Chapter 3, the proposed techniques for space-mapping calibration of the omni-cameras and for mechanic-error correction are described. In Chapter 4, the proposed methods for mopboard detection and the proposed strategy of vehicle navigation by mopboard following are presented. In Chapter 5, the proposed methods for room space construction and analysis of environment data in different omni-images are described.

Experimental results showing the feasibility of the proposed methods are described in Chapter 6. Conclusions and suggestions for future works are included finally in Chapter 7.

Chapter 2 Principle of Proposed Automatic 3-D House-layout Construction and

System Configuration

2.1 Introduction

For indoor 3-D house-layout construction, the use of a vision-based intelligent autonomous vehicle is a good idea because it can save manpower and the vision sensors on the vehicle can assist navigation and localization. Besides, the vehicle can also gather indoor environment information with its mobility.

In the proposed system, we equip the vehicle with two catadioptric omni-cameras which have larger FOV’s, each covering a half sphere of the 3D space around the camera. The two omni-cameras are connected and vertically-aligned in a bottom-to-bottom fashion, form a two-camera imaging device, and are installed on the vehicle. In such a connected fashion, the imaging system covers the upper and lower semi-spherical FOV’s simultaneously so that the images of the floor and the ceiling can be captured simultaneously. With this imaging system, the vehicle can navigate and collect desired information in indoor rooms. Related control instructions of the vehicle and communication tools are also developed in this study. The entire system configuration, including hardware equipments and software, will be described in Section 2.2.

In order to use the vehicle to carry out the indoor house-layout construction task, it is necessary to have an appropriate strategy to tell the vehicle where to go, when to

turn around, and what kind of information should be collected. When a house agent wants to use the vehicle to obtain the layout of each empty room in a house in advance before getting it ready for sale, it is inconvenient to conduct a learning work to teach the vehicle a navigation path that includes some specific positions as turning spots. Instead, it is desired that the vehicle can navigate and gather environment information in an unlearned indoor room space automatically. We will introduce the main idea of such an automatic 3D house-layout construction process, which we propose in this study, in Section 2.3. Also, the system must have a process to transform the collected data into house-layout structure information. In Section 2.4, we will describe the outline of the proposed process of house-layout construction.

2.2 System Configuration

In the proposed vehicle system, we make use of a Pioneer 3-DX vehicle made by MobileRobots Inc. as a test bed. The vehicle is equipped with an imaging system composed of two omni-directional catadioptric cameras. The imaging system is not only part of the vehicle system but also plays an important role of gathering environment information and locating the vehicle. A diagram illustrating the configuration of this system is shown in Figure 2.1. Because we control the vehicle system remotely, some wireless communication equipments are necessary. The detail of the hardware architecture and the used equipments are described in Section 2.2.1.

In order to develop the vehicle system, the software that provides some commands and control interface is essential for users to control the vehicle and cameras. The software we use or developed in this study is described in Section 2.2.2.

Figure 2.1 Equipment on proposed vehicle system used in this study.

2.2.1 Hardware configuration

The hardware equipments we use in this study include three principal systems --- the vehicle system, the control system, and the imaging system. In the vehicle system, the Pioneer 3-DX vehicle itself has a 44cm×38cm×22cm aluminum body with two 19cm wheels and a caster. It can reach a speed of 1.6 meters per second on flat floors, and climb grades of 25° and sills of 2.5cm. It can carry payloads up to 23 kg. The payloads include additional batteries and all accessories. By three 12V rechargeable lead-acid batteries, the vehicle can run 18-24 hours if the batteries are fully charged initially. An embedded control system in the vehicle allows a user to issue commands to control the vehicle to move forward or backward and turn around. The system can also return some status information to the user. Besides, the vehicle is equipped with an odometer which is used to record the pose and the position of the vehicle. To control the vehicle remotely, a wireless connection between the user and the vehicle is necessary. A WiBox is used to communicate with the vehicle by RS-232, so the user can control the vehicle remotely over a network from anywhere.

The second part is the two-camera imaging device which includes two catadioptric omni-cameras vertically-aligned and connected in a bottom-to-bottom fashion, as shown by Figure 2.2(a). The catadioptric omni-camera used in this study is a combination of a reflective hyperbolic mirror and a CCD camera. Each CCD camera used in the imaging system is shown in Figure 2.2(b), and a detailed specification about it is included in Table 2.1.

(a) (b)

Figure 2.2 Two catadioptric omni-cameras connected in a bottom-to-bottom fashion into a single two-camera imaging device used in this study. (a) The two-camera device. (b) The CCD camera used in the imaging device.

Table 2.1 Specification of the CCD cameras used in the imaging device.

Model Name RYK-2849

Image Device Color camera with SONY 1/3" super had CCD sensor in mini metal case

Picture Elements NTSC:510x492, PAL:500x582

Resolution 420 TVL

Power Supply DC12V±10%

Operating Temp. -10°C ~ 50°C

Video Output 1 Vp-p / 75 Ohms

Audio Output 2 Vp-p / 50 Ohms (option)

Because the output signals of the CCD cameras used in this study are analog, an

AXIS 241QA Video Server installed in it, as shown in Figure 2.3, has a maximum

frame rate of 30 fps. It can convert analog video signals into digital video streams and send them over an IP network to the main control unit (described next). The imaging device and the AXIS Video Server are combined into an imaging system.

(a) (b)

Figure 2.3 The Axis 241QA Video Server used in this study. (a) A front view of the Video Server. (b) A back view.

In the control system, a laptop computer is used to run the program developed in this study. A kernel program can be executed on the laptop to control the vehicle by issuing commands to it and to conduct processing tasks on captured images. With an access point, all information between the user and the vehicle can be delivered through wireless networks (IEEE 802.11b and 802.11g), and captured images can also be transmitted to users at speeds up to 54 Mbit/s. The entire structure of the vehicle system used in the study is shown in Figure 2.4.

Figure 2.4 Structure of proposed system.

2.2.2 Software configuration

The MobileRobots, Inc. provides an ARIA (Advanced Robotics Interface

Application) for use in this study, which is an API (application programming interface)

that assists developers in communicating with the embedded system of the vehicle, through an RS-232 serial port or a TCP/IP network connection. The ARIA is an object-oriented toolkit usable under Linux or Win32 OS by the C++ language.

Therefore, we use the Borland C++ builder as the development tool in our experiments to control the vehicle by the ARIA. The status information of the vehicle can be obtained by means of the ARIA.

About the AXIS 241QA Video Server, the AXIS Company provides a development toolkit called AXIS Media Control SDK. Using the Media ActiveX component from the SDK, we can easily have a preview of the omni-image and capture the current image data. It is helpful for developers to conduct any task with the grabbed image.

2.3 Principle of Proposed Automatic Floor-layout Construction by

Autonomous Vehicle Navigation and Data Collection

For vehicle navigation in an unknown empty room space, we propose a navigation strategy based on mopboard following. After a navigation session is completed, we use the estimated mopboard edge points to construct a floor-layout map. The main process is shown in Figure 2.5.

In this study, we use a space-mapping technique proposed by Jeng and Tsai [7] to compute the location of a concerned object using a so-called pano-mapping table.

When the vehicle starts a navigation session, it uses the detected mopboard edge

points from omni-images to decide whether or not it has to conduct an adjustment of its direction to keep its navigation path parallel to each wall. Then, the mopboard edge points are detected again from the omni-images taken by the imaging system mentioned previously. By a pattern classification technique, the distance between the vehicle and each wall can be estimated more accurately, and each extracted mopboard edge point can be assigned to the wall which contains it. In this way, the vehicle can estimate the distances between the walls and itself, and know whether to move forward or turn around. After the navigation session is completed, a floor-layout map is created according to the collected mopboard edge data by using a global optimization method for mopboard edge line fitting proposed in this study.

2.4 Outline of Proposed Automatic 3-D House-layout Construction Process

Only creating a floor-layout map, as described in Section 2.3, is insufficient for use as a 3-D model of the indoor room space. The objects on walls such as windows and doors must also be detected and be drawn to appear in the desired 3-D room model. We have proposed methods for detecting and recognizing the doors as well as windows on walls in the upper and lower omni-images. The principal steps of the methods are shown in Figure 2.6.

First, we determine a scanning range with two direction angles for each pair of omni-images based on the line equation of the floor layout. Because the lower omni-camera is installed to look downward, it can cover the mopboard on the wall.

We use a pano-mapping table lookup technique to get the scanning radius in the omni-image by transforming the 3-D space points into corresponding image coordinates. With the scanning region for each omni-image, we can retrieve

appropriate 3-D information from different omni-images. Each object which is detected by the scanning region of each omni-image is regard as an individual one.

Some objects on the wall, such as windows and doors, may appear in the pair of omni-images (the upper and the lower ones). Therefore, we have to merge the objects which are detected separately from the upper omni-image and the lower omni-image according to their positions in order to recognize the doors as well as the windows on walls. Then, we can locate them in the 3-D space and draw them in the final 3-D room model in a graphic form.

In summary, the proposed automatic 3-D house-layout construction process includes the following major steps:

1. automatic floor-layout construction by autonomous vehicle navigation and data collection as described in Section 2.4;

2. determine a scanning region for each omni-image according to the floor–layout edges;

3. retrieve information from the scanning region of each omni-image;

4. combine those objects which are detected separately from the upper and lower omni-cameras according to their positions;

5. recognize doors and windows from these combined objects;

6. construct the house-layout model with doors and windows on it in a graphic form.

Figure 2.5 Flowchart of proposed process of automatic floor-layout construction.

Floor-layout Information Determine Scanning Region for

Each Omni-image

Omin-images

Door Detector and Window Detector Combine Objects

Start of 3-D House-layout Construction

For Each Floor-layout Edge

Objects Information

End of 3-D House-layout construction Construct House-layout

Detect Objects Objects Information

in Each Omni-image

Figure 2.6 Flowchart of proposed outline of automatic 3-D house-layout construction.

Chapter 3 Calibration of a Two-camera

Omni-directional Imaging System and Vehicle Odometer

3.1 Introduction

The vehicle used in this study is equipped with two important devices which are a two-camera imaging system and a vehicle odometer. We describe the proposed methods of calibration for these two equipments in this chapter. Before describing the proposed methods, we introduce the definition of the coordinate system used in this study in Section 3.1.1 and the relevant coordinate transformation in Section 3.1.2.

The catadioptric omni-camera used in this study is a combination of a reflective hyperboloidal-shaped mirror and a perspective CCD camera. Both the perspective CCD camera and the mirror are assumed to be properly set up so that the omni-camera becomes to be of a single-viewpoint (SVP) configuration. It is also assumed that the optical axis of the CCD camera coincides with the transverse axis of the hyperboloidal mirror and that the transverse axis is perpendicular to the mirror base plane.

For vehicle navigation by mopboard following, the mopboard positions are essential for vehicle guidance. Besides, these mopboard edge points are very important for a 3-D house-layout construction. In the proposed system, the vehicle estimates the distance information by analyzing images captured from the imaging system. Before the use of the imaging system, a camera calibration procedure is

needed. For this purpose, we use a space-mapping technique proposed by Jeng and Tsai [7] to create a space-mapping table for each omni-camera by finding the relations between specific points in 2-D omni-images and the corresponding points in 3-D space. In this way, the conventional task of calculating the projection matrix for transforming points between 2-D omni-image and 3-D space can be omitted. The detail about camera calibration is described in Section 3.2.

For vehicle navigation in indoor environments, the vehicle position is the most important information which is not only used for guiding the vehicle but also as a local center to transform the estimated positions in the camera coordinate system (CCS) into the global positions in the global coordinate system (GCS). Though, the position of the vehicle provided by the odometer of the vehicle may be imprecise because of the incremental mechanic errors of the odometer. It also results in deviations from a planned navigation path. Therefore, it is desired to conduct a calibration task to eliminate the errors. In Section 3.3, we will review the method for vehicle position calibration which was proposed by Chen and Tsai [10], and a vision-based calibration method for adjusting the vehicle direction during navigation will be described in the following chapter.

3.1.1 Coordinate Systems

Four coordinate systems are utilized in this study to describe the relative locations between the vehicle and the navigation environment. The coordinate systems are illustrated in Figure 3.1. The definitions of all the coordinate systems are described in the following.

(1) Image coordinate system (ICS): denoted as (u, v). The u-v plane coincides with the image plane and the origin I of the ICS is placed at the center of the image

plane.

(2) Global coordinate system (GCS): denoted as (x, y). The origin G of the GCS is a pre-defined point on the ground. In this study, we define G as the starting position of the vehicle navigation by the mopboard following process.

(3) Vehicle coordinate system (VCS): denoted as (V_x, V_y). The V_x-V_y plane is coincident with the ground. And the origin V is placed at the middle of the line segment that connects the two contact points of the two driving wheels with the ground. The Vx-axis of the VCS is parallel to the line segment joining the two driving wheels and through the origin V. The V_y-axis is perpendicular to the

V

x-axis and passes through V.

(4) Camera coordinate system (CCS): denoted as (X, Y, Z). The origin O_m of the CCS is a focal point of the hyperbolic mirror. And the X-Y plane coincides with the image plane and the Z-axis coincides with the optical center inside the lens of the CCD camera.

Figure 3.1 The coordinate systems used in this study. (a) The image coordinate system. (b) The vehicle coordinate system. (c) The global coordinate system. (d) The camera coordinate system.

Figure 3.1 The coordinate systems used in this study. (a) The image coordinate system. (b) The vehicle coordinate system. (c) The global coordinate system. (d) The camera coordinate system. (continued)

3.1.2 Coordinate Transformation

In this study, the GCS is determined when starting a navigation session. The CCS and VCS follow the vehicle during navigation. The relation between the GCS and the VCS is illustrated in Figure 3.2(a). We assume that (xp, yp) represents the coordinates of the vehicle in the GCS, and that the relative rotation angle denoted as θ is the directional angle between the positive direction of the x-axis in the GCS and the positive direction of the V_x-axis in VCS. The coordinate transformation between the VCS and the GCS can be described by the following equations:

x ＝ V

_x × cosθ － V_y × sinθ ＋ x_p; (3.1)

y ＝ V

_x × sinθ ＋ V_y × cosθ ＋ y_p. (3.2) The main concept about the relation between the CCS and the ICS is illustrated in Figure 3.2(b), though the CCS in Figure 3.2(b) is a little different from the CCS in this study. The relation plays an important role for transforming the camera coordinates (X, Y, Z) of a space point P into the image coordinates (u, v) of its

where a and b are two parameters satisfying the equation of the hyperboloidal mirror as follows: omni-camera [17], and by combining with the above equations, we have

2 2 2 2

where θ is the angle of the space point P with respect to the X-axis as well as that of

the corresponding image point p with respect to the u-axis, as shown in Figure 3.2(c).

(a)

(b)

u v

P(X, Y, Z)

p(u, v) θ

omni-image plane

(c)

Figure 3.2 The relations between different coordinate systems in this study. (a) The relation between the GCS and VCS (b) Omni-camera and image coordinate systems [11]. (c) Top view of (b).

3.2 Calibration of Omni-directional Cameras

Before using the imaging system, a calibration procedure for the omni-cameras

is indispensible. However, the conventional calibration method is complicated for calculating intrinsic and extrinsic parameters. An alternative way is to use the

space-mapping technique to estimate the relation between points in the 2-D image

plane and 3-D space and to establish a space-mapping table for it [7]. The detailed process is reviewed in Section 3.2.2. In the process of establishing the space-mapping table, the information about the focal location of the hyperboloidal mirror is important because the focal point is taken to be the origin in the CCS. The process to find the focal point of the hyperboloidal mirror is described in Section 3.2.1.

3.2.1 Proposed Technique for Finding Focal Point of Hyperboloidal Mirror

In order to creating the space-mapping table, it is indispensible to select some pairs of world space points with known positions and the corresponding points in the omni-images. Note that an image point p is formed by any of the world space points which all lie on the incoming ray R, as shown in Figure 3.3, where we suppose that Om is the focal point of the hyperboloidal mirror, Ow is on the transverse axis of the hyperboloidal mirror, and P₁ and P₂ are two space points on the ray R. Besides, we also assume that the corresponding image point is p .Subsequently, we have the corresponding point pairs (P₁, p) and (P₂, p) which then are used to create the table.

However, if we take Ow as the focal point, as a result P1 and P2 will lie on different light rays, though the corresponding image points are still p. In this way, the incorrect pairs will result in an incorrect space-mapping table. To provide accurate pairs, we must find out the position of the focal point of the hyperboloidal mirror.

Figure 3.3 The space points and their corresponding image points.

To find out the focal point of the hyperboloidal mirror, as shown in Figure 3.4, we use two different landmarks L1 and L2 which have the same corresponding image point p with known heights and horizontal distances from the transverse axis of the hyperboloidal mirror. We assume that Ow is at (0, 0, 0). Then, according to the involved geometry shown in Figure 3.4, the position of the focal point can be

在文檔中利用搭載雙環場攝影機取像系統之自動車作自動建立房間立體空間圖之研究 (頁 21-0)