Figure 29 – Android Application Shake Recognition Mechanism
VI. Testing Performance Results
This chapter discusses testing results, and is divided into two parts:
o First part contains GPS location accuracy testing results from RTKLib;
o Second part contains architecture framework testing results o Server – App message delivery time;
o Server – App traffic lights status delivery time;
o Battery Performance results
RTKLib testing results are required in order to understand whether L1 low-‐cost receivers can be used to improve positioning accuracy and how efficient they are.
Architecture framework testing results include GCM response time and Traffic Light Delivery time. Traffic Light delivery time is one of the most important measurements that show whether the system is able to notify a blind person in a real-‐time. Battery consumption tests were conducted to evaluate android application efficiency and evaluate usage time of the application.
6.1 RTKLib testing results
L1 GPS receiver has been tested together with RTKLib to evaluate whether they are efficient to use in this kinds of applications. Although Kinematic Mode is the most relevant to these kinds of applications and is of the most interest for this project, the tests have been performed to test all the RTKLib modes.
Figure 30 – NEO Freerunner, Low-‐Cost GPS antenna
As mentioned in [22] it is possible to obtain cm-‐level accuracy using low-‐cost L1 GPS receivers together with good antennas. It is important to mention that L1 low-‐cost GPS chip has been tested together with low-‐cost GPS antenna. Both NEO Freerunner and GPS Antenna are shown on the picture below. GPS antenna supported frequency is 1575.42 MHZ, which means that this is L1 signal GPS antenna.
In RTK modes reference station information was provided by Network RTK e-‐GPS commercial service (http://www.egps.nlsc.gov.tw).
As was previously mentioned, recommended distance between a reference station and a rover should be within 10 km.
Single mode
Single mode is the simplest configuration and processing mode. It shows positioning the same way as a GNSS receiver just using ephemerides and clock data from the satellites. To make it simple, the information in single mode is not processed in any extended way, and is the same as in GPS receiver.
Figure 31 illustrates results for the Single mode with the accuracy within 10 square meters.
Figure 31 – RTKLib Single Mode Results
Location accuracy of the single equals to location accuracy of GPS receiver without any improvements and usually is in the range of 5 to 15 meters depending on the weather conditions.
All the other modes will show some improvement of GPS location accuracy.
Precise Point Positioning (PPP)
PPP positioning techniques do not require any reference base station, and location can be computed using a single GPS receiver.
There are 3 PPP modes supported by RTKLib:
• PPP Kinematic -‐ binary data from a GPS receiver is combined with real time predicted ephemerides and satellite clocks to improve the rover's position
• PPP Static – same as in Kinematic, but the receiver is static
• PPP Fixed – GPS receiver position is fixed with PPP mode
During the tests PPP positioning always make improvement over the Single’s mode results.
Picture below illustrates GPS positioning improvement from ~10 meters to ~2,3 meters (which gives about 77% of improvement).
Figure 32 – RTKLib Single Mode vs PPP static mode results
Real Time Kinematic (RTK) modes
Real-‐Time Kinematic methods or carrier-‐phase dependent methods supported in RTKLib are the following:
• Kinematic mode -‐ binary data from a moving GPS receiver is combined with raw output data from a reference base station to improve the receiver position.
• Static mode – same principal as in Kinematic, except that the rover is stationary.
• Moving-‐Baseline -‐ same principal as in Kinematic and Static, except that the distance between the receiver and the reference base station is computed regardless of the reference base station's position.
• Fixed mode – the GPS rover is fixed
Tests showed that usage of low-‐cost antenna is not efficient for Kinematic mode. RTKLib computed “FLOAT” solutions more often than “FIX” solutions. In the cases when RTKLib computes “FIX” solutions, usually accuracy is within 0,5 – 1 meter (which gives ~90-‐95%
of improvement comparing to initial 10 meters of accuracy); whereas “FLOAT” solutions provides accuracy only within 1,5 -‐ 2,5 meters (~75%-‐85% of improvement). Pictures above illustrate test results for “FIX” and “FLOAT” solutions:
Figure 33 – RTKLib RTK mode, “FIX” solution results
Figure 34 – RTKLib RTK mode, “FLOAT” solution results
6.2 Architecture framework testing results
This section contains performance results of server-‐client communication. During the following tests the cloud server had a stable wired NCTU Internet connection; and android phone had whether stable WiFi over WiMAX (commercial, provided by G1, Taiwan) connection or stable WiFi over LTE connection (test LTE network in NCTU).
The following tests will show results of Google Cloud Messaging response time and time that required by the cloud server to access traffic lights database, formulate a message and
send message to GCM server for further processing. However, it is also important to measure what time does it take for a message to reach android phone app. Unfortunately, GCM does not provide this kind of information. Another way to measure this time is to synchronize clocks between the cloud server and the android app, and then compare time difference between sending a message and receiving a message. However, Android OS does not allow precise clock synchronization on un-‐rooted phones for third-‐party applications.
In order to predict message delivery time, first GCM response time was measure and summarized with an average latency time for LTE networks, based on information from [40]. According to the authors tests in [40], average latency for TCP packets in LTE networks for downlink is 11.8 ms and 25.4 ms for uplink.
The picture below illustrates basic networking concept used during performance testing:
Figure 35 – Testing Environment (Network)
GCM response time (LTE)
It is important to measure GCM response time and predict GCM total delivery time to evaluate whether GCM service is suitable for delivering real-‐time messages.
Making HTTP POST request to GCM server and calculating time difference between sending a request and getting a response measured Google Cloud Messaging response time. In the testing request 'time_to_live' was set to 0, meaning that the message should not be stored on GCM servers if the device is not available.
When the message is processed successfully by the GCM, the HTTP response has status 200 and delivers additional information. The following is an example of successfully processed message:
{"success":1,"failure":0,"canonical_ids":0,"results":[{"message_id":"0:1371831279848529
%af3e3d02f9fd7ecd"}]}
In case a request is rejected by the GCM, the HTTP response should contain a non-‐200 status code (400, 401, or 503). The following is an example of a rejected message response from the GCM; the reason of rejection – the application is shut down and is not registered with GCM:
{"success":0,"failure":1,"canonical_ids":0,"results":[{"error":"NotRegistered"}]}
Figure 36 – GCM response time, LTE
The results of GCM response time were sum up with average downlink LTE latency time in order to get total message delivery time. Average message delivery time is 0.0841 sec.
Traffic Light Status Delivery Time (LTE)
This test measures traffic light status delivery time. To measure the delivery time, the difference time between accessing the database (to get a traffic light status) and receiving GCM successful response was calculated. Average time based on 550 iterations is 0,195 seconds and after we add LTE downlink time to predict total message delivery time, we got 0,196 sec. Traffic Lights status delivery time is a critical aspect in a navigation system for visually impaired; delivery time should be as fast as possible to guarantee real-‐time feedback. The average response time for traffic status delivery in [2] is 660 milliseconds, which is around 3 times slower than a possible delivery time in a networking based model.
Figure 37 – Traffic Light Status Delivery Time, LTE
Battery performance test
The main advantage of mobile client app running on android phone is that only one power-‐
consuming interface (Network) is used. Battery testing performance results show efficiency of using Networking together with GCM. During tests most of the times display was switched off, and GCM service was running as a background service, allowing the application to get all the messages from the cloud server.
While the application was running the following messages have been pushed:
• Location updates were sent every 3 seconds (random latitude and longitude);
• Zebra crossing notifications, that require TTS engine to perform an action after the message is received by the application, were sent every 1 minute
Additionally JSON messages from the app were sent to couchdb database every 1 second.
In these conditions a phone was able to operate for more than 11 hours. Figure 38 shows phone battery condition before the application was started. As seen from the picture, the battery is fully charged and phone has been running on battery for 17 seconds. After this screenshot was captured, the application has been started including all the communication activities mentioned above.
Figure 38 – Battery efficiency test, first stage
During the test other applications were also performing some background work, as it would be on every real working system. Figure 39 shows that the phone was running on the battery for more than 11 hours and remaining battery power was 8%:
Figure 39 – Battery efficiency test, second stage