CHAPTER 2 Related Work
2.3 Inertial Sensor Measurements and RSSI
Gaddi Blumrosen et al.
[10]
propose the way to trace human body by RSSI and almost 50%. The RSSI based location mean and standard errors were 6.76 and 3.11 cm. In addition, the location estimations based on the Kalman filter in steady state, are lower by around 30%, with a mean and standard error of 4.13 and 1.96 cm.According to this paper, I believe it is workable to use RSSI from BLE and combine with Inertial Sensor Measurements by other filter to improve the precise in localization.
9
Figure 4: Hand movement motion tracking in 2D experiment setup. Picture originated from: [10]
10
CHAPTER 3
Localization Algorithm
We want to locate user’s position in the building. At the beginning, the IMU can calculate the distance from the movement of the human body, but I meet problems about cumulative errors from sensors. Because the limitations of hardware from IMU sensors still can’t overcome, I try other ways to get user’s position. The RSSI from BLE is another method to localize user’s position without cumulative errors.
Therefore, this chapter will introduce the methods used to localize users in section 3.1. The algorithm to aggregate Inertial Sensor Measurements and RSSI will present in section 3.2. However, some problems may occur during the performance, so the fault tolerance is important issue and will present in section 3.3.
3.1 System Environment
First, I use the acceleration from IMU system to get distance by integrating.
However, it suffers from the cumulative errors seriously. Then, I use the rotation from legs to simulate the movement from human. But, the patterns of human movement are so complex that I must find other ways to enhance the localization.
Finally, the RSSI from BLE is a workable method and it can detect human without cumulative errors. This chapter presents three steps to localize user position and provides some fault tolerance to robust the system.
11
3.1.1 Acceleration
At first, there are many sensors with accelerate can be used. When I get the signal from sensors, I found the gravity is a very serious factor in computing distance.
As we know, the relation between acceleration, velocity, and position as following:
Table 1: The formula of the position, velocity, and acceleration.
Derivative Form Integral Form, distx(index) = distx(index-1) + vx(index-1)* time + 0.5*ax(index)* time^2; (2) With these formulate, I try to use the acceleration from sensor as followings:
The acceleration signal can be modeled as:
� = �� + � + �� (3) Where ��, �, and �� are the angular velocity vector, the related angular velocity vector bias, and a white noise. To reduce the noise, I try to use the Kalman filter [11] to get more precise. However, the gravity from earth is another factor, which I want to filter out. Therefore, we can get the acceleration without gravity by rotating the axis as the Figure 5. The gravity from earth is fixed, but the sensor may rotate to get partial value from gravity. In this way, by rotating the axis can make the gravity on the virtual Z axis, named as the ′ as follows.
12
(a) (b)
(c) (d)
Figure 5: The rotation axis of the acceleration. (a) original axis (b) rotation axis(c) y-z plane (d) x-z′ plane.
In y-z plane (c), by rotating φ makes partial gravity of y focus on the new �′.
φ = tan− �� (4)
′ = √ + (5)
In x-�′plane (d), by rotating θ makes partial gravity of x focus on the new �′′, which gets all gravity.
13
θ = tan− ��′ = tan− �
√� +� (6)
To verify the distance which was calculated by the acceleration after filtering the gravity, I use the toy car to run 20 cm as the Figure 6 and the distance which calculated from acceleration using low pass filter can be precise as the Figure 7.
Figure 6 : The toy car move 20 centimeters from right to left at the Plane
Figure 7: The acceleration, velocity, distance, yaw, pitch and roll of the toy car is right, and the distance is about 20cm.
However, human may move by various ways in real life. The accumulative errors
14
from sensors are the critical defect as the Figure 8. The distance integrated from acceleration may grow up even the sensor holds still. Owing to moving suddenly to other sides such as the roll or pitch move suddenly, the gravity may be calculated to get wrong velocity and distance.
Figure 8: The experiment of moving pitch in 500ms, and the velocity distance will be wrong.
Finally, I try other ways and get the more features of MPU6050 sensors from this experience.
3.1.2 Feet Movement
The combination of gyroscope and acceleration can determine the rotation from sensors, so by calculating the rotation from legs can be one method to get user’s
15
position. To get the position from legs, we need to wear the sensors on the legs.
The sensors can detect the pitch, roll, and yaw from legs. By calculating the angles between legs can roughly calculate the step distance, as the Figure 9.
Figure 9: The distance between legs calculated by rotation.
By step and step as the following to calculate the distance from legs, we can roughly know the position from the users.
distance = L * Math.sqrt(2*(1 - Math.cos(RightUpLegX_move*Math.PI/360)));
(7)
Besides, I used the mathlab tools to simulate the rotation between legs of human
16
as Figure 10 from the scholar of Chia-Feng Lu [12]. However, the patterns of human movements are so various that it is not easy to get all the patterns of human movements. Although I can’t use this method to capture the movement from human, this technique can improve the frame rates and the precise.
Figure 10: The simulation of human movement. Picture originated from: [12]
3.1.3 Beacon Location
The RSSI from the BLE can localize the user’s position, but how can we get RSSI from BLE? The OS of the raspberry pi is based on the Linux, so I find the API of BlueZ, which provides the library of Bluetooth low energy to get information such as RSSI and the Uuid. The peripheral, acting as transmitter, can send its RSSI to let central detect its signal strength. However, the RSSI from peripheral is unstable and drift. I tried to use four different BLE dongles as Figure 11 to receive three different BLE sensors as Figure 12, and found some features of BLE. The RSSI
17
from BLE is more precise in 30 cm. The central sensor Esense D704 made from CSR plc (formerly Cambridge Silicon Radio) is better to receive signal than others, and the peripheral sensor of BOLITEK BLE is more stable to transmit RSSI as Figure 13.
IOGEAR V4.0 SeeHot V4.0
KooPin V4.0Esense D704
Figure 11: The four different BLE dongles made in Taiwan.
MBTV4 HM-10 HC – 08
Figure 12: The three different BLEs made in China.
18
19
Figure 13: The four different BLEs signals received by four BLEs dongles.
After getting the RSSI from BLE and understanding the feature of RSSI, the RSSI is more stable in 90 cm, and can be identified in 30 cm for the curve is more skew. Although the RSSI is unstable, I can get some regular in the RSSI and try following steps to calculate user’s position.
Step 1:
20
provides by the scholar of Sandeep Mistry [13].
Distance = .5∗ � / � � ��� − (8)
Which measurePower means the RSSI of one meter.
I can get real distance and RSSI to calculate the measurePower as the test data, and use this test data in my experiment to get the rough distance by measuring the RSSI. To deal with drift signals from RSSI, I use the moving average filter to reduce the noise from RSSI and improve its identification. However, the RSSI is not stable, so we try the following algorithm to enhance. At the beginning, we use the triangle localization, but it is not precise for some BLEs are not precise. Because BLE can be detected precisely within 30 cm, I sort all the BLE to get the first, second and third nearest BLEs. To use these three BLE efficiently, I give different weights on three near BLEs to improve the precision, where the weights depend on the RSSI of each BLE.
Figure 14: The position we calculated by 8 BLEs.
Step 2:
After sorting all BLEs and determine three near BLE namely BLE 、BLE 、 BLE ,where BLE is the nearest from the user and so on, then using weighting on
21
BLE and BLE to get the First_keypoint as Figure 14.
First_keypoint = BLE + BL +BLBL * Vector , , where Vector , is from BLE to BLE . (9)
The First_keypoint can be used as the reference node to combine with BLE . To get more precise, I use more weighs on the BLE to depend user’s position.
The calculation of real position from user is following:
Real position = BLE + BL + iBL_ y i * Vector , i _ y i (10) Step 3:
By this way, we can get user’s position from the RSSI. Due to the fluctuating feature of RSSI, I also utilized the rotation of user leg movement information measured by IMU sensors as Figure 15. That is, I combine the rotation of legs from MPU6050 as the credibility to reduce the errors of RSSI. The low pass filter [14] is used to filter the noise and smooth the signals. The calculation is as follows.
Positioni a = Credibility * Position I + (1- Credibility)* Position i a − , where Credibility is the change of legs from MPU6050, and range is from 0.5 m to
1m. (11)
Figure 15: The IMU sensors used to serve as the auxiliary tool.
22
Besides, I try to use the feet movement, which calculates distance from rotation of legs, to improve the frame rates. When human moves, the position can receive from one of RSSI or IMU. The RSSI is my major algorithm to calculate position, and the IMU sensors can add some values between RSSI. Because the IMU sensors may suffer from the cumulative errors, I use the RSSI signals to overcome these errors and combine these two system, INS system and RSSI system, by the low pass filter as Figure 16.
Figure 16: The IMU sensors combine with the RSSI system.
3.2 Fault tolerance
In order to deal with some problems in the middle of performance, I think some problems may occur in the show. How many BLEs should I deploy? It depends on how precise the performance needs. Besides, the frame rates of BLE is only 5~20 fps, how can I do to make sure the frame rates can meet my needs? The sensors may be broken, what should I do to avoid this happen? In this chapter, I will introduce many methods to deal above questions.
23
3.2.1 Deployment
At the first, I deployed 8 BLEs to localize the as the Figure 17.
Figure 17: The sketch map of 8 BLEs in the floor.
These BLEs can detect user’s position, but the error rate is so high that the position may drift and unstable between middle of BLEs. To improve the correct rate, I deploy 11 BLEs, which I put more three BLEs in the middle of BLEs as the Figure 18.
Figure 18: The sketch map of 11 BLEs in the floor.
Second, the 11 BLEs can improve the precise and reduce the errors and noise from RSSI, but it still suffers from drift. In this way, I deploy 21 BLEs in the floor as the Figure 19.
24
Figure 19: The sketch map of 21 BLEs in the floor.
Finally, there are 21 BLEs in the floor, so the BLEs can detect user position.
However, there are some problems about frame rate. Because of the update rates of RSSI is just 5 to 20 frames, I use some methods to solve this problem.
3.2.2 Frame Rate
In this chapter, I use some methods to ensure the frame rate can meet my needs.
First, the coordinator, raspberry pi, has four USB ports and I use two BLE dongles to receive BLE signals. If one of BLE dongles breaks, it can use the other BLE dongle to receive. In addition, I also compare the time of signals from these BLE dongles. In this way, I can compare the time of every signal and get the better signal during the limited time.
Second, the BLE may run out of power. To solve these problems, I implement an interface to change or erase the use of BLEs by Uuid as Figure 20. With these interfaces, I can control these BLEs to decide which node I need, and know which BLE is not work.
25
Figure 20: The input column of the Bluetooth’s uuid.
3.2.3 Coordinators
In the performance the coordinators, Raspberry pi, are responsible for collecting the signal of the Bluetooth Low energy, and transmit to the server. However, the coordinators, Raspberry pi, may run out of power, so I prepare two coordinators to backup. In addition, it can also detect BLE signals by four BLE dongles. With two coordinators, I can also detect and compare the user position which is more precise and adopt it. If one of the Raspberry pi is broken, the other pi can support to make the performance work successfully as the Figure 21.
Figure 21: The coordinators, raspberry pi.
26
CHAPTER 4
Experimental Evaluation
In this chapter, the results of the various localizations will be presented and the evolution test and algorithm will show in chapter 4.2. After trying to use the RSSI to trace human movement, I found that the update rate of RSSI is not enough to meet the needs. Fortunately, the IMU sensors can be used to improve the frame rate. By using these wearable sensors, it becomes possible to use the RSSI to detect the user’s position. Therefore, I will also compare the results of only using RSSI and the combination of the IMU and RSSI.
4.1 Simulation Setup
At the beginning, I used little BLE peripherals and only one BLE central to localize.
The BLE peripheral is a device which can broadcast the RSSI to let BLE central scan and detect the signals as the Figure 22.
Figure 22: The Bluetooth low energy serves as peripheral.
27
The BLE central uses the Bluetooth protocols, GATT (Generic Attribute Profile), to scan near devices for connection and each BLE peripheral has its universally unique identifier(UUID) for identification.
The coordinator is the raspberry pi 2 model B as the Figure 23. The raspberry pi is a little and light pc, which equips with a 900MHz quad-core ARM Cortex-A7 CPU, 1GB RAM, 4 USB ports, 40 GPIO pins, full HDMI port, Ethernet port, combined 3.5mm audio jack and composite video, camera interface (CSI), display interface (DSI), micro SD card slot, and videoCore IV 3D graphics core.
Figure 23: The coordinate hardware, raspberry pi.
When the raspberry pi coordinates all signals, pi needs to send these data to the server. To connect with server, the raspberry pi equips with Wifi dongle to transmit data to server.
The Server is a mac pro equips with 2.26 GHz (P8400) Intel Core 2 Duo Penryn with 3 MB on-chip L2 cache, 2 GB (two 1 GB) memory. Finally, the server has more resource to deal with complex operation. Therefore, some complex algorithm in the server can be operated in real time.
There is a problem about how to know the results of the localization. To see how
28
precise of the algorithm and debug results, I use the JSXGraph to create points, circles and lines. In this way, I can see where the user is as the Figure 24. Besides, I upload the results to the server. When the server receives the results, other tools such as Unity can use these data to draw many amazing graph as the Figure 25.
Figure 24: The diagram of detecting Bluetooth created by JSXGraph.
29
Figure 25: The human model created by Unity.
4.2 Simulation Results
4.2.1 Absolute position
I try to use three cases in the 650 * 200 � cubic room as the Figure 26.
Figure 26: The diagram of Bluetooth on the floor in the 650 * 200 cm.
The following is the results about deploying different BLEs to localize user position
30
as table 2.
The algorithm is in chapter 3, using weighting with each nodes, as Figure 24. As the Figure 27 shows, the more BLEs used the more precise I can calculate from user’s location. I put the BLE dongle on the floor to receive signals, the distance from BLE dongle to BLEs are 30, 50, and 100cm. Within 30 cm, I can really find the user’s location. In the 21 beacons, I ignore one beacon to make the distance from BLE dongle to BLE is 100cm. What the performance needs are not only stable but also high frame rates. Therefore, 21 beacons were used in performance.
Table 2: Deploy different BLEs and get its position
Number of
(250,150) (250,220) (250,240) (250,150) (250,220) (250,240) (250,150) (250,220) (250,240)
Calculate value
(390,218) (242,189) (263,210) (299,163) (245,171) (257,187) (298,181) (238,199) (256,204)
Error
155 cm 32 cm 32 cm 50.6 cm 49 cm 53.4 cm 57 cm 15 cm 36.4 cmGraph
31
Figure 27: The error rate of BLEs. The error value shows the difference from real position to detect position. The 21 BLEs is the more precise than others.
The architecture of the flow char is as the Figure 28 shows. There are two ways to calculate the position. One is the BLE, and another is IMU sensor.
Figure 28: The error rate of BLEs. The error value shows the difference from real position to detect position. The 21 BLEs is the more precise than others.
After receiving the raw data of the RSSI, I use the algorithm of the filter to reduce the noise and improve the precise by using the IMU in Equation 11. I actually test the
0
32
localization algorithm and walk in the cubic room from 50m to 650m and return.
After the experiment, I compare the RSSI signals and use the camera to record my movement. The results of using only one Bluetooth dongle to receive the RSSI and use the low pass filter can solve the drift of the RSSI are as the Figure 29 shows. In addition, the raspberry pi uses the two Bluetooth dongles to receive more RSSI as the Figure 30 shows. In addition, I also use the camera to capture the real movement and the mapping virtual character as the Figure 31 shows.
Figure 29: The result of using filter to get more stable signal by one signal.
0
33
Figure 30: The result of using filter to get more stable signal by two signals.
(a) (b)
(c) (d)
Figure 31: The result of real movement from left to right.(a) Left (b)Right (c)middle (d) left but with some delay time
This paper focuses on the movement of the user especially the delay time, and the drifts of the RSSI may affect the result. In order to localize the user position, I present the algorithm which take little seconds to calculate the result. The low pass filter is a good solution to filter the noise and just takes 0.08s to get the user position.
0
34
Although it may not so precise that error may be 15cm to 57cm, it can follow up the movement of the user. In addition, I also use the IMU sensors to improve its precise, so the algorithm can get user’s position in real-time as the Figure 32 and Figure 33.
Figure 32: Using only one raspberry pi and IMU to get the position.
Figure 33: Using the two raspberry pi and IMU to get the position.
0
35
In addition, I also use the camera to capture the real movement and the mapping virtual character as the Figure 34 shows.
Figure 34: The result of real movement from left to right.
The architecture of the overall experiment is as the Figure 35. First, the BLE peripheral sent signal to the Raspberry pi. Second, the Raspberry pi filters these signal and sent to the Server by wireless. Finally, the server process these signal and sent the real position by message queue.
The cost time during the pi to server and server calculate the position from users is as the Table 3.
36
Figure 35: The architecture of the overall experiment.
Table 3: The total times from Raspberry pi to server.
Step time delay
From pi to server
0.23Server process the siganl
0.08Transfer by Message queue
0.0064.2.2 Real performance in Campus
We used the Wearable Item Service runtimE (WISE) server as the platform. As shown
We used the Wearable Item Service runtimE (WISE) server as the platform. As shown