CHAPTER 1 Introduction
1.4 Organization
The rest of the thesis is organized as follows. Chapter 2 introduces the overviews of the related works, including the Indoor Localization, Gait Analysis and the Inertial Sensor Measurements combined with RSSI. Chapter 3 shows the details of algorithms and main features of the methodology. Chapter 4 presents the analyze and discusses of the simulation results. Finally, the conclusions and future works are presented in Chapter 5.
4
CHAPTER 2 Related Work
Localization techniques can be classified into Indoor and outdoor environments.
GPS could be classified on the outdoor localization, because GPS performs well in outdoor open–sky environments but tall buildings prevent users from getting a line-of-sight to GPS satellites. So its accuracy degrades in urban environments.
One of the limitations of GPS is the Network, for users need to connect to the servers.
To get more precise, GPS needs more power to localize, so the battery is another issue. On the contrary, indoor localization has become more and more popular in recently years. It always uses wireless technologies like Ultra-wideband (also known as UWB), WLAN. UWB is a radio technology pioneered by Robert A.
Scholtz[4] and others which used at a very low energy level for short-range. In addition, UWB localization is triangulation like TOA (time of arrival), and TDOA (time difference of arrival). Triangulation uses the geometric properties of triangles to estimate the target location and needs line of sight between transmitter and receiver.
In this way, obstacle will cause inaccurate and UWB’s transmission distance is too short, so we need to use a lot of UWB tags. On the other hands, WLAN (include BLE) localization method is more popular for indoor localization by comparing the signal strength between users and the tags we placed. At the beginning, Bluetooth Low Energy (BLE) started as part of the Bluetooth 4.0 Core
5
Specification [5] in early 2010. BLE is a dual-mode device compatible with both Classic and low energy peripherals as Figure 1.
Figure 1: The specific and mode of Bluetooth and BLEs. Picture originated from: [5]
In addition to low energy cost, BLE can be feasible to many devices such as phones, headphone and Keyboard. BLE also provides many services to transfer data in GATT protocols. Therefore, after surveying many sensors in the markets, we adopt BLEs as the source and use it on localization.
2.1 Indoor Localization
There are many theories about how to localize, and we use RSSI from BLE to localize human trace. Honggui Li [6] proposes the method how to use the RSSI to localize by low-cost 3D positioning with Bluetooth. The proposed method also has the merits of low cost, low power consumption, high usability and high location precision. Besides, the nonlinear least square used to calculate the user’s location in 3D indoor position. The least square is a basic model, so there are many different models to improve the precise. The Weighted and Generalized Least Square algorithm and the Weighted and Extended Total Least Square are also the different solutions which combine the weights to improve from the least square.
The four Bluetooth low energy devices are uniformly deployed on the four 3D
6
positions of a “cubic” room, the four reference devices are placed on four camera stands, and fall in a 9m×9m square with different height as the Figure 2.
Figure 2: The sketch map of the experimental environment.Picture originated from:
[6]
Experimental results of Location estimation based on weighted least square algorithm are as the following. The average and standard deviation of absolute precision of location estimation for hardware experiment is 2.2739m and 1.445m.
Another issue is which place is much better to put beacon. Sudarshan S.
Chawathe [7] proposes the placement method about where to put beacon indoor.
The maximum density of beacon is limited on the budget and deployment effort.
We need to know which places are interesting locations, then using algorithm to deploy our beacon in the area. The most important problem is the processing time for receiving the Bluetooth. Bluetooth-based localization method is use the device
7
discovery which is inherently a slow process, so the discovery process prescribed by the Bluetooth standard requires a total of 10.24 seconds for completion. The response time is the problem that’s I need to overcome.
2.2 Gait Analysis
Bogdan Muset et al.[8] proposed the method how to use accelerometer and gyroscope sensors to get human moving distance as the Figure 3. By using the rotation from legs, I can roughly calculate the distance human move. Although this algorithm can only calculate particular motion pattern, it can improve the frame rate from localization. The update rate of accelerometer and gyroscope is about 33 frames, which is more than beacons. With this paper’s algorithm, the frame rate and precise of localization can be more enhancement.
Figure 3: The algorithm of human moving distance. Picture originated from: [8]
Jia Ming Deng et al.[9] also proposed the method how to use accelerometer and gyroscope from IMU sensors such as MPU6050 to implement tracking system. By
8
eliminating the gravity of accelerometer, it can work for tracking and has many advantages, such as low cost, small size, high reliability, and strong anti-jamming capability. Jia Ming Deng et al also ensure that the IMU sensors – MPU6050 can work for tracking and motion capture. In this way, I adopt the MPU6050 as the IMU sensor, which provides many libraries to use such as the pitch, roll, and yaw to detect the movement from human.
2.3 Inertial Sensor Measurements and RSSI
Gaddi Blumrosen et al.
[10]
propose the way to trace human body by RSSI and almost 50%. The RSSI based location mean and standard errors were 6.76 and 3.11 cm. In addition, the location estimations based on the Kalman filter in steady state, are lower by around 30%, with a mean and standard error of 4.13 and 1.96 cm.According to this paper, I believe it is workable to use RSSI from BLE and combine with Inertial Sensor Measurements by other filter to improve the precise in localization.
9
Figure 4: Hand movement motion tracking in 2D experiment setup. Picture originated from: [10]
10
CHAPTER 3
Localization Algorithm
We want to locate user’s position in the building. At the beginning, the IMU can calculate the distance from the movement of the human body, but I meet problems about cumulative errors from sensors. Because the limitations of hardware from IMU sensors still can’t overcome, I try other ways to get user’s position. The RSSI from BLE is another method to localize user’s position without cumulative errors.
Therefore, this chapter will introduce the methods used to localize users in section 3.1. The algorithm to aggregate Inertial Sensor Measurements and RSSI will present in section 3.2. However, some problems may occur during the performance, so the fault tolerance is important issue and will present in section 3.3.
3.1 System Environment
First, I use the acceleration from IMU system to get distance by integrating.
However, it suffers from the cumulative errors seriously. Then, I use the rotation from legs to simulate the movement from human. But, the patterns of human movement are so complex that I must find other ways to enhance the localization.
Finally, the RSSI from BLE is a workable method and it can detect human without cumulative errors. This chapter presents three steps to localize user position and provides some fault tolerance to robust the system.
11
3.1.1 Acceleration
At first, there are many sensors with accelerate can be used. When I get the signal from sensors, I found the gravity is a very serious factor in computing distance.
As we know, the relation between acceleration, velocity, and position as following:
Table 1: The formula of the position, velocity, and acceleration.
Derivative Form Integral Form, distx(index) = distx(index-1) + vx(index-1)* time + 0.5*ax(index)* time^2; (2) With these formulate, I try to use the acceleration from sensor as followings:
The acceleration signal can be modeled as:
� = �� + � + �� (3) Where ��, �, and �� are the angular velocity vector, the related angular velocity vector bias, and a white noise. To reduce the noise, I try to use the Kalman filter [11] to get more precise. However, the gravity from earth is another factor, which I want to filter out. Therefore, we can get the acceleration without gravity by rotating the axis as the Figure 5. The gravity from earth is fixed, but the sensor may rotate to get partial value from gravity. In this way, by rotating the axis can make the gravity on the virtual Z axis, named as the ′ as follows.
12
(a) (b)
(c) (d)
Figure 5: The rotation axis of the acceleration. (a) original axis (b) rotation axis(c) y-z plane (d) x-z′ plane.
In y-z plane (c), by rotating φ makes partial gravity of y focus on the new �′.
φ = tan− �� (4)
′ = √ + (5)
In x-�′plane (d), by rotating θ makes partial gravity of x focus on the new �′′, which gets all gravity.
13
θ = tan− ��′ = tan− �
√� +� (6)
To verify the distance which was calculated by the acceleration after filtering the gravity, I use the toy car to run 20 cm as the Figure 6 and the distance which calculated from acceleration using low pass filter can be precise as the Figure 7.
Figure 6 : The toy car move 20 centimeters from right to left at the Plane
Figure 7: The acceleration, velocity, distance, yaw, pitch and roll of the toy car is right, and the distance is about 20cm.
However, human may move by various ways in real life. The accumulative errors
14
from sensors are the critical defect as the Figure 8. The distance integrated from acceleration may grow up even the sensor holds still. Owing to moving suddenly to other sides such as the roll or pitch move suddenly, the gravity may be calculated to get wrong velocity and distance.
Figure 8: The experiment of moving pitch in 500ms, and the velocity distance will be wrong.
Finally, I try other ways and get the more features of MPU6050 sensors from this experience.
3.1.2 Feet Movement
The combination of gyroscope and acceleration can determine the rotation from sensors, so by calculating the rotation from legs can be one method to get user’s
15
position. To get the position from legs, we need to wear the sensors on the legs.
The sensors can detect the pitch, roll, and yaw from legs. By calculating the angles between legs can roughly calculate the step distance, as the Figure 9.
Figure 9: The distance between legs calculated by rotation.
By step and step as the following to calculate the distance from legs, we can roughly know the position from the users.
distance = L * Math.sqrt(2*(1 - Math.cos(RightUpLegX_move*Math.PI/360)));
(7)
Besides, I used the mathlab tools to simulate the rotation between legs of human
16
as Figure 10 from the scholar of Chia-Feng Lu [12]. However, the patterns of human movements are so various that it is not easy to get all the patterns of human movements. Although I can’t use this method to capture the movement from human, this technique can improve the frame rates and the precise.
Figure 10: The simulation of human movement. Picture originated from: [12]
3.1.3 Beacon Location
The RSSI from the BLE can localize the user’s position, but how can we get RSSI from BLE? The OS of the raspberry pi is based on the Linux, so I find the API of BlueZ, which provides the library of Bluetooth low energy to get information such as RSSI and the Uuid. The peripheral, acting as transmitter, can send its RSSI to let central detect its signal strength. However, the RSSI from peripheral is unstable and drift. I tried to use four different BLE dongles as Figure 11 to receive three different BLE sensors as Figure 12, and found some features of BLE. The RSSI
17
from BLE is more precise in 30 cm. The central sensor Esense D704 made from CSR plc (formerly Cambridge Silicon Radio) is better to receive signal than others, and the peripheral sensor of BOLITEK BLE is more stable to transmit RSSI as Figure 13.
IOGEAR V4.0 SeeHot V4.0
KooPin V4.0Esense D704
Figure 11: The four different BLE dongles made in Taiwan.
MBTV4 HM-10 HC – 08
Figure 12: The three different BLEs made in China.
18
19
Figure 13: The four different BLEs signals received by four BLEs dongles.
After getting the RSSI from BLE and understanding the feature of RSSI, the RSSI is more stable in 90 cm, and can be identified in 30 cm for the curve is more skew. Although the RSSI is unstable, I can get some regular in the RSSI and try following steps to calculate user’s position.
Step 1:
20
provides by the scholar of Sandeep Mistry [13].
Distance = .5∗ � / � � ��� − (8)
Which measurePower means the RSSI of one meter.
I can get real distance and RSSI to calculate the measurePower as the test data, and use this test data in my experiment to get the rough distance by measuring the RSSI. To deal with drift signals from RSSI, I use the moving average filter to reduce the noise from RSSI and improve its identification. However, the RSSI is not stable, so we try the following algorithm to enhance. At the beginning, we use the triangle localization, but it is not precise for some BLEs are not precise. Because BLE can be detected precisely within 30 cm, I sort all the BLE to get the first, second and third nearest BLEs. To use these three BLE efficiently, I give different weights on three near BLEs to improve the precision, where the weights depend on the RSSI of each BLE.
Figure 14: The position we calculated by 8 BLEs.
Step 2:
After sorting all BLEs and determine three near BLE namely BLE 、BLE 、 BLE ,where BLE is the nearest from the user and so on, then using weighting on
21
BLE and BLE to get the First_keypoint as Figure 14.
First_keypoint = BLE + BL +BLBL * Vector , , where Vector , is from BLE to BLE . (9)
The First_keypoint can be used as the reference node to combine with BLE . To get more precise, I use more weighs on the BLE to depend user’s position.
The calculation of real position from user is following:
Real position = BLE + BL + iBL_ y i * Vector , i _ y i (10) Step 3:
By this way, we can get user’s position from the RSSI. Due to the fluctuating feature of RSSI, I also utilized the rotation of user leg movement information measured by IMU sensors as Figure 15. That is, I combine the rotation of legs from MPU6050 as the credibility to reduce the errors of RSSI. The low pass filter [14] is used to filter the noise and smooth the signals. The calculation is as follows.
Positioni a = Credibility * Position I + (1- Credibility)* Position i a − , where Credibility is the change of legs from MPU6050, and range is from 0.5 m to
1m. (11)
Figure 15: The IMU sensors used to serve as the auxiliary tool.
22
Besides, I try to use the feet movement, which calculates distance from rotation of legs, to improve the frame rates. When human moves, the position can receive from one of RSSI or IMU. The RSSI is my major algorithm to calculate position, and the IMU sensors can add some values between RSSI. Because the IMU sensors may suffer from the cumulative errors, I use the RSSI signals to overcome these errors and combine these two system, INS system and RSSI system, by the low pass filter as Figure 16.
Figure 16: The IMU sensors combine with the RSSI system.
3.2 Fault tolerance
In order to deal with some problems in the middle of performance, I think some problems may occur in the show. How many BLEs should I deploy? It depends on how precise the performance needs. Besides, the frame rates of BLE is only 5~20 fps, how can I do to make sure the frame rates can meet my needs? The sensors may be broken, what should I do to avoid this happen? In this chapter, I will introduce many methods to deal above questions.
23
3.2.1 Deployment
At the first, I deployed 8 BLEs to localize the as the Figure 17.
Figure 17: The sketch map of 8 BLEs in the floor.
These BLEs can detect user’s position, but the error rate is so high that the position may drift and unstable between middle of BLEs. To improve the correct rate, I deploy 11 BLEs, which I put more three BLEs in the middle of BLEs as the Figure 18.
Figure 18: The sketch map of 11 BLEs in the floor.
Second, the 11 BLEs can improve the precise and reduce the errors and noise from RSSI, but it still suffers from drift. In this way, I deploy 21 BLEs in the floor as the Figure 19.
24
Figure 19: The sketch map of 21 BLEs in the floor.
Finally, there are 21 BLEs in the floor, so the BLEs can detect user position.
However, there are some problems about frame rate. Because of the update rates of RSSI is just 5 to 20 frames, I use some methods to solve this problem.
3.2.2 Frame Rate
In this chapter, I use some methods to ensure the frame rate can meet my needs.
First, the coordinator, raspberry pi, has four USB ports and I use two BLE dongles to receive BLE signals. If one of BLE dongles breaks, it can use the other BLE dongle to receive. In addition, I also compare the time of signals from these BLE dongles. In this way, I can compare the time of every signal and get the better signal during the limited time.
Second, the BLE may run out of power. To solve these problems, I implement an interface to change or erase the use of BLEs by Uuid as Figure 20. With these interfaces, I can control these BLEs to decide which node I need, and know which BLE is not work.
25
Figure 20: The input column of the Bluetooth’s uuid.
3.2.3 Coordinators
In the performance the coordinators, Raspberry pi, are responsible for collecting the signal of the Bluetooth Low energy, and transmit to the server. However, the coordinators, Raspberry pi, may run out of power, so I prepare two coordinators to backup. In addition, it can also detect BLE signals by four BLE dongles. With two coordinators, I can also detect and compare the user position which is more precise and adopt it. If one of the Raspberry pi is broken, the other pi can support to make the performance work successfully as the Figure 21.
Figure 21: The coordinators, raspberry pi.
26
CHAPTER 4
Experimental Evaluation
In this chapter, the results of the various localizations will be presented and the evolution test and algorithm will show in chapter 4.2. After trying to use the RSSI to trace human movement, I found that the update rate of RSSI is not enough to meet the needs. Fortunately, the IMU sensors can be used to improve the frame rate. By using these wearable sensors, it becomes possible to use the RSSI to detect the user’s position. Therefore, I will also compare the results of only using RSSI and the combination of the IMU and RSSI.
4.1 Simulation Setup
At the beginning, I used little BLE peripherals and only one BLE central to localize.
The BLE peripheral is a device which can broadcast the RSSI to let BLE central scan and detect the signals as the Figure 22.
Figure 22: The Bluetooth low energy serves as peripheral.
27
The BLE central uses the Bluetooth protocols, GATT (Generic Attribute Profile), to scan near devices for connection and each BLE peripheral has its universally unique identifier(UUID) for identification.
The coordinator is the raspberry pi 2 model B as the Figure 23. The raspberry pi is a little and light pc, which equips with a 900MHz quad-core ARM Cortex-A7 CPU,
The coordinator is the raspberry pi 2 model B as the Figure 23. The raspberry pi is a little and light pc, which equips with a 900MHz quad-core ARM Cortex-A7 CPU,