Thesis Organization

The remainder of this thesis is organized as follows. The system configuration of the vehicle used as a test bed and the principles of learning and navigation are described in Chapter 2. The proposed fuzzy-control techniques and the designed fuzzy control system are described in Chapter 3. In Chapter 4, the proposed innovative learning strategy with simple and flexible capabilities, and the proposed path map creation method are described. In Chapter 5, the proposed vehicle navigation method with obstacle avoidance capability, a smooth curve following process for turning sections, and some strategies for navigation accuracy maintenance are described. The application of person following in corridors by tracking the positions of human feet is described. Satisfactory experimental results are shown in Chapter 7. Finally, some conclusions and suggestions for future works are given in Chapter 8.

Chapter 2

System Configuration and Navigation Principles

2.1 Introduction

Recently, the applications of indoor navigation of mobile robots or autonomous vehicles become more and more popular, especially in home services and security patrolling. Hence, the size of the vehicle was designed to be smaller and smaller in order to achieve the purpose of dexterous movement in indoor environments. With this goal, a small vehicle with wireless control and image grabbing capabilities is used as a test bed for our research and development in this study. Because of the dexterous property of this small vehicle, it can be used in many kinds of applications.

In order to navigate in any unknown indoor environment, a learning process is essential for the vehicle. Therefore, a simple and flexible learning strategy which learns user-driving parameters and odometer data is developed in this study. Through the learning process, desired navigation paths can be learned by the vehicle and useful data are recorded at in the mean time.

After navigation paths are known by the vehicle after the learning process, a fuzzy-control guidance technique and a vehicle navigation method, both deigned in this study, are applied for the vehicle to navigate along the learned paths to finish desired trips.

2.2 System Configuration

In this study, we use the Amigo robot, a mini-vehicle made by ActivMedia Robotics Technologies Inc., as shown in Figure 2.1 as the test bed. There are eight ultrasonic sensors, an odometer, and a wireless 2.4G camera on this mobile vehicle.

The ultrasonic sensors are disabled in this study because they are useless for our research work. The maximum advance speed and rotation speed of the vehicle are 75 cm/sec and 300 degrees/sec, respectively, and the speed encoders encodes 39000 ticks per wheel revolution with 124 ticks per mm. The length, width, and height with the camera and the body of the Amigo vehicle are 33 cm, 28cm, and 21cm, respectively.

Figure 2.1 The Amigo vehicle used in this study.

2.2.1 Hardware Configuration

The proposed system is composed of five parts, as shown in Figure 2.2. The first part is the vehicle with an embedded control system. By a user’s commands , this system can control the two DC motors to move forward or backward or turn around, and return some status parameters of the vehicle to the user. The second part is a vision system which consists of a camera with a wireless transceiver, a wireless video signal receiver, and an imaging frame grabber. Because the signal obtained from the camera and sent to the video signal receiver is analog, a still digital image can be obtained by the imaging frame. The image grabbed in our experiments are of the resolution of 320×240 pixels for the reason of raising image processing efficiency.

The third part is a power system which consists of two kinds of batteries. One is a 9V battery to supply the power of the wireless camera; and the other is a 12V battery to supply the power of the vehicle system. The fourth part is a remote control system in a desktop computer or a notebook PC. The kernel program can be executed on this remote control system to command the vehicle through a wireless transmission system, which is the fifth part of the proposed system, consisting of a wireless access point and a wireless signal receiver. The command of the remote control system is transmitted to the wireless signal receiver on the vehicle by an access point that meets the 802.11b standard.

2.2.2 Software Configuration

In order to transmit commands to the vehicle and to retrieve status data about the vehicle, we use the ARIA which is a C⁺⁺ based open-source development environments and provides a robust interface to the vehicle. Lowest-level details or

information of the vehicle (e.g., odometer data) can be easily retrieved by means of the ARIA, and moving commands can be sent by means of the ARIA also. In other words, any developer can use the ARIA as an interface to communicate with the embedded system of the vehicle. And we use the Borland C⁺⁺ builder as the development tool in our experiments.

2.3 Proposed Learning Principle and Process

Figure 2.2 Structure of proposed system.

Vehicle

To navigate in an unknown environment, learning is necessary. Then, it is desired to develop an efficient learning strategy that can learn the knowledge of desired navigation paths. Nevertheless, in most developed learning strategies, visual environment features are the most important data that are used as the basis for localization or other purposes in navigation. Even though the knowledge of visual environments data is very useful, the processes for obtaining and analyzing these visual data are usually complicated. Therefore, we want to design a learning strategy without using visual data.

The proposed learning strategy consists of three processes: a user-control process, a data-collection process, and a data-transformation process. The user-control process is designed to control the vehicle to learn desired paths. Users can press the buttons on a graphic user interface designed in this study at will to accomplish the learning session.

As soon as the user-control process is executed by the user, the data-collection process is activated. Two types of data are collected orderly in the process, one being user-driving parameter, and the other odometer data. The former is a set of parameters collected upon the user’ actions in controlling the vehicle. And the latter is the odometer data which include the global vehicle coordinates and azimuths of the vehicle.

In addition, the path map creation process is activated after both of the user-control process and the data-collection process are finished. The collected data are then transformed into a simple graph with two types of rules, and this simple graph represents the learned path. The data of each node in the simple graph is recorded as an order list and saved into a text file for use in navigation sessions. An illustration of the learning process is shown in Figure 2.3.

2.4 Proposed Vehicle Guidance Principle and Process

The scene in each kind of indoor environment is different and usually complicated, and furniture and decorations within rooms also have difference appearances. Therefore, it is hard to differentiate between obstacles and non-obstacles in guiding the vehicle to correct and safe paths. Hence, a fuzzy-control guidance technique is proposed in this study which guides the vehicle in a natural manner. In every navigation cycle, a suitable steering angle can be obtained by this fuzzy-control

Figure 2.3 Illustration of the proposed learning process Manual learning

Start learning

Data collection

User-driving parameters Odometer data

Path map creation

Saving data for

retrieval in navigation End learning

guidance technique and the goal of obstacle avoidance can be achieved in the meanwhile. In other words, the proposed fuzzy-control guidance technique can nominally reduce the complexity of scenes in indoor environments for choosing a suitable path to navigate.

Furthermore, a vehicle navigation process with two navigation modes for different purposes is proposed. One is the single path mode for navigating along a learned path with only a starting point and a destination. In the single path mode, the learned data are retrieved from a saved file first, and all the nodes within this navigation path are set in accordance with the nodes in the learned graph exactly.

Then the navigation trip can be accomplished by traversing each node in the learned graph orderly. For this reason, two navigation strategies are proposed for the vehicle to navigate in two types of sections during the navigation process. The first strategy is used along each straight-line section between two nodes connected with a line approximately. And the second strategy is used for turning sections where the vehicle navigates around a corner.

The other mode is the area mode for navigating among multiple nodes collected from several paths. In the area mode, a desired navigation path may not be learned before. First, the learned data are retrieved from a saved file, too. After that, the desired navigation path is determined by a given starting point and a given destination which are learned in the learning process. The nodes within this navigation path are found dynamically using Dijkstra shortest path searching algorithm. After the desired navigation path is determined, a similar navigation strategy is adopted for traversing along the nodes along the navigation path. An illustration of the vehicle navigation process is shown in Figure 2.4.

Figure 2.4 An illustration of vehicle navigation process Start Navigation

Single path mode Area mode

Learned data Retrieval

Learned data Retrieval

Dynamic path planning

Vehicle guidance

End navigation

Chapter 3

Proposed Fuzzy Guidance

Techniques for Indoor Navigation

3.1 Introduction

In order to guide a small vehicle to navigate in indoor environments with complicated scenes, two fuzzy-control guidance techniques are proposed in this study for safe indoor navigation. Since the principle of fuzzy control is intuitive and experiential, we utilize fuzzy-guidance techniques for vehicle navigation. The essence of this approach is to keep equilibrium between navigation accuracy and fuzzy control.

Generally speaking, fuzzy theory can be implemented in a wide variety of fields because of its multidisciplinary nature. A system based on a fuzzy will react to triggered events autonomously in an intuitive way. Due to this characteristic of fuzzy theory, many researchers use fuzzy theory in robot-control applications. A review of the fuzzy-control concept will be described in Section 3.2. And the reason of using fuzzy-control techniques for guidance of small vehicles in this study will be described in Section 3.3.

Besides, some parameters for the proposed fuzzy model must be acquired through some image processing works. Since appearances of different kinds of indoor environments can be classified roughly into two types, namely, room and corridor, two processes of parameter acquirement for the two types of indoor environments are

proposed and will be described in Section 3.4. After necessary parameters are acquired, desired fuzzy outputs as meaningful commands for vehicle control can be obtained by a computation process based on the proposed fuzzy model. Similarly to the need of two parameter acquirement processes, two different fuzzy models are designed for vehicle guidance in the two types of indoor environments. The details of the fuzzy models will be described in Section 3.5.

3.2 Review on Fuzzy Control Concepts

The proposed fuzzy control system is derived from fuzzy inference techniques using fuzzy if-then rules and fuzzy reasoning. According to C.T. Sun [20], the basic structure of a fuzzy inference system consists of three conceptual components: a rule base, which contains the detailed fuzzy rules; a database, which defines the membership functions used in the fuzzy rules; and a reasoning mechanism, which performs the fuzzy reasoning process using the rules and certain given facts to obtain a reasonable output.

In a fuzzy inference system, when fuzzy or crisp inputs are taken into the system, the outputs are fuzzy sets in most cases. But fuzzy-set outputs are not suitable for use in our vehicle control system. Therefore, a method of defuzzification is needed to extract a crisp value which best represents a fuzzy set. Besides, in the case with crisp inputs and outputs, the fuzzy inference system just implements a nonlinear mapping from its input space to its output space. And this mapping is accomplished by a number of fuzzy if-then rules, each of which describes the local behavior of the

mapping. The proposed fuzzy control techniques belong to this case.

The procedure of a fuzzy inference system with a crisp output is shown in Figure 3.1. The first part is an input process which takes some fuzzy or crisp values as inputs into this system. And the second part is a fuzzy reasoning process which processes the inputs with fuzzy rules to yield fuzzy-set outputs. The third part is a defuzzification process which transforms the output fuzzy sets into a single crisp value. The final crisp value will be used as the basis for generating commands to guide the vehicle in the proposed fuzzy guidance techniques.

X (crisp or fuzzy)

Figure 3.1 Illustration of a fuzzy inference system.

3.3 Reasons of Using Fuzzy Guidance Techniques

The reason for using fuzzy guidance techniques in this study is three-fold, as described in the following.

1. To reduce the complexity of guidance in indoor environments.

Since the scene structures of indoor environments are usually complicated, it is generally difficult to guide a vehicle to navigate in it. One way to solve this problem is to use fuzzy guidance techniques, which presumably are more proper for use to find vehicle navigation paths. That is, it expected that complicated scene structures will not influence vehicle guidance when fuzzy guidance techniques are employed.

2. To create obstacle avoidance capability for the vehicle.

When a vehicle is navigating in indoor environments using the proposed fuzzy guidance techniques, obstacles may appear in the navigation path. An example of images grabbed by the wireless camera in our vehicle system is shown in Figure 3.1.

Obstacles are regarded in this study as furniture in indoor environments or simply as part of environments because of the use of fuzzy guidance techniques. Therefore, our method for finding safe navigation paths is simple with no necessity to differentiate between objects in navigation paths and furniture or walls in environments. In the situation shown in Figure 3.1, the vehicle is steered to the left side in our experiment to avoid possible collision by considering the paper box as part of the environments instead of as an obstacle.

3. To create an adaptive capability for the proposed vehicle system to navigate in different indoor environments.

Some vehicle systems use visual environments features (e. g., baseline, corner, manual landmark, etc.) in the guidance process. Since these kinds of visual features can be found only in common indoor environments, these vehicle navigation systems are useless when navigating in certain environments with no such visual feature. This weakness will not be found in our navigation system using the proposed fuzzy guidance techniques because any kind of object will be regarded as part of the environment. Therefore, the proposed vehicle navigation system can be used in any kind of indoor environments without any restriction on the scene structures of the navigation environments.

3.4 Parameters for Proposed Fuzzy Guidance System

According to the review on fuzzy-control concepts in Section 3.2, in order to Figure 3.2 An image of an obstacle appearing in navigation path.

acquire a crisp output value, certain crisp values must be taken into consideration in the fuzzy guidance system first. And these crisp values are related to the designed fuzzy rules which are determined in the phase of fuzzy model development. In the proposed fuzzy guidance techniques, still images captured from the wireless camera will be processed to obtain appropriate features as crisp inputs for the fuzzy guidance system in every navigation cycle. And the proposed processes of parameter acquirement for use as inputs to the proposed fuzzy models will be detailed in the following.

Because two different fuzzy guidance techniques are designed in this study, two different processes, as mentioned previously, which acquire the demanded parameters for the two fuzzy guidance techniques, are proposed. They are described in Sections 3.4.1 and 3.4.2, respectively.

3.4.1 Features for Vehicle Guidance in Rooms

In every navigation cycle, an input image captured with the wireless camera is processed to obtain two kinds of features, namely, collision-free direction, and degrees of collisions of the left and the right route sides. The values of these features are taken as inputs into the proposed fuzzy guidance system, which yields an angle as the output for use to steer the vehicle in each navigation cycle. In the following, we describe how we extract the two kinds of features by image processing techniques in the captured images.

A. Computing collision-free direction

The feature of collision-free direction means the direction into which the vehicle may be driven with no collision with the objects along the path traversed by the

vehicle. Such a feature is computed in every navigation cycle in the following way.

First, we divide an input image into 4×4 blocks and classify each image block into two classes, namely, route area and non-route one. As a preliminary step to achieve this goal, we utilize an algorithm presented in Li and Tsai [6] to locate image blocks of possible route areas. The essence of the algorithm is to use two pixel features, namely, a pixel’s grayscale value and its Sobel edge value, to identify candidate route-area pixels. A pixel in the input image with its grayscale value close to a pre-learned grayscale value of the room ground and with its Sobel edge value smaller than a pre-selected threshold is classified as a candidate route-area pixel. An example of such image pixel classification results is shown in Figs. 3.3(a) and 3.3(b). Because of color and uniformity similarities, some wall or furniture regions may be misclassified as route areas, as can be seen in the example shown in Fig. 3.3(b).

Nevertheless, we propose an algorithm in this study to remove such erroneous areas in the following, which in addition computes the above-mentioned feature of collision-free direction from the remaining correct route areas in the input image.

Algorithm 1. Computation of route areas and collision-free direction.

Step 1. Perform region growing to find as the desired route area the largest bottom region in the candidate route-area pixels extracted from the input image using [6]. (The region growing result of the above example is shown in Fig.

3.3(c).)

Step 2. Put two parallel horizontal scanning lines in a fixed lower part of the route area. If any non-route area crosses either scanning line and cuts it into several line segments, pick out the longest segment and call it a non-obstacle segment; otherwise, select the original scanning line as the non-obstacle segment. (At the end of this step, two non-obstacle segments

will be obtained.)

Step 3. Find the middle points of the two non-obstacle segments, connect them to form a line segment, and call it route segment. (The found middle points for the above example are A and B as shown in Fig. 3.3(c).)

Step 4. Compute the middle point of the route segment, and call it route center.

(The result of this step for the last example is point C in Fig. 3.3(c).)

Step 5. Connect the route center to the middle point of the bottom line of the image to form a line segment, and call it guidance line (like the line labeled L in Fig.

Step 5. Connect the route center to the middle point of the bottom line of the image to form a line segment, and call it guidance line (like the line labeled L in Fig.