Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Floor Plans
Built by Multi-platform Mobile Mapping Systems
Jhen-Kai Liao, Guang-Je Tsai, Kai-Wei Chiang, Hsiu-Wen Chang
The Development of an Artificial Neural Networks
Aided Image Localization Scheme
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Contents
•
Motivations
•
Methods
•
Experiment
•
Results and discussions
•
Conclusion
•
Future works
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Motivations
•
Taipei main station
2018/2/27 3
Where am I?
Where is my destination?
How can I go?
Location!
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Motivations
2018/2/27 4Accuracy
Applicability
•
Real-time
•
Convenient
•
Easy to use
•
Model tuning
•
Expert knowledge
•
Wearable device
•
Many infrastructures
•
Environmental change
•
Database construction
•
Successive images
•
Inertial
•
Radio frequency
•
Image
•
Challenges
Coordinate system?
Integration!
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Motivations
•
Sensors, universal, applications
2018/2/27 5
Accelerometer
GNSS
Photometer
Gyro
Barometer
Magnetometer
Camera
WiFi & BLE
…….
.
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Motivations
•
Pedestrian dead reckoning
2018/2/27 6
Gyro
Magnetometer
Accelerometer
• Previous calibration
• Tuning parameters in post-processing
• Other wearable sensors
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Motivations
•
Portable mobile mapping system
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Motivations
•
UAV and LiDAR
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
•
Mobile mapping (for portable MMS)
2018/2/27 9
Known
position
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
2018/2/27 10
•
Simple marker recognition
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
•
ANN aided image-based localization
•
Multilayer feedforward neural network
2018/2/27 11
Input layer:
One layer, eleven input neurons: pixel coordinates of
four marker vertexes, the area of the recognized
marker, the estimated distance from marker
distortion, and sensor orientation - 11
Hidden layer:
One layer, eight hidden neurons, sigmoid activation
function - 8
Output layer:
One layer, linear activation function, one output
neuron: distance - 1
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
•
ANN aided image-based localization
•
Simple position estimation
2018/2/27 12
Known
Known
Known
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
•
Pedestrian dead reckoning
2018/2/27 13
𝑥𝑥
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻=
δbψ, k =
δψk
Φ
𝑘𝑘−1𝐻𝐻𝐻𝐻𝐻𝐻𝑥𝑥
𝑘𝑘−1𝐻𝐻𝐻𝐻𝐻𝐻= 1 ∆t
0 1
δbψ, k−1 +
δψk−1
𝑤𝑤
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻,
𝑤𝑤
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻~N(0,
𝑄𝑄
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻)
𝑧𝑧
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻=
𝐴𝐴
𝑚𝑚−
𝐴𝐴
𝑔𝑔𝑥𝑥
𝑘𝑘𝑃𝑃𝐻𝐻𝐻𝐻= Ek Nk Lk bE, k bN,k bL,k
=
Φ
𝑘𝑘−1𝑃𝑃𝐻𝐻𝐻𝐻𝑥𝑥
𝑘𝑘−1𝑃𝑃𝐻𝐻𝐻𝐻+
𝑤𝑤
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻,
𝑤𝑤
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻~N(0,
𝑄𝑄
𝑘𝑘𝐻𝐻𝐻𝐻𝐻𝐻)
𝑧𝑧
𝑘𝑘𝑃𝑃𝐻𝐻𝐻𝐻=
[𝐸𝐸
𝑘𝑘𝑎𝑎𝑎𝑎𝑎𝑎− 𝐸𝐸
𝑘𝑘𝑝𝑝𝑎𝑎𝑝𝑝, 𝑁𝑁
𝑘𝑘𝑎𝑎𝑎𝑎𝑎𝑎− 𝑁𝑁
𝑘𝑘𝑝𝑝𝑎𝑎𝑝𝑝]
𝛷𝛷
𝑘𝑘−1𝑃𝑃𝐻𝐻𝐻𝐻=
1 0 ϑ
0 1 η
0 0 1
1 0 0
0 1 0
0 0 1
0 0 0
0 0 0
0 0 0
1 0 0
0 1 0
0 0 1
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
•
Simultaneous localization and mapping
•
For UAV MMS
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
2018/2/27 15
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Methods
•
Other image-based positioning methods for comparison
•
Simple distortion
•
Space resection
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Experiment
2018/2/27 17
•
Training and testing images: 200, 32
•
Experimental field: underground parking lot, 350 meters long, marker*12
•
Smartphone: SONY Z3
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Results and discussions
•
Accuracy of georeferenced image
2018/2/27 18 0 2 4 6 8 10 12 14 16 18 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 Check point ID E rro r (m)
Accuracy analysis of georeferenced image
Eastern error Northern error Height error
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Results and discussions
•
Accuracy of floor plan
2018/2/27 19 1 2 3 4 5 6 7 8 9 10 0 2 4 6 8 10 12 14 Check point ID 3 D po si ti o n e rr o r ( m )
Accuracy analysis of floor plan
Raw floor plan Corrected floor plan
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Results and discussions
2018/2/27 20
•
Image-based localization comparison
•
Simple marker distortion
•
Space resection
•
ANN aided image-based localization
2 3 4 5 6 7 8 9 10 0 0.5 1 1.5 2 2.5
Real distance between the camera and marker (m)
2 D P o sitio n R M S E ( m )
Accuracy analysis of image-based localizations
ANN aiding Space resection Marker distortion 0 20 40 60 80 0 5 10 0 0.5 1 1.5
Relative angle from the perpendicular (degree)
Characteristic analysis of ANN model
Real camera distance (m)
2D pos it ion e rr or ( m ) -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Results and discussions
2018/2/27 21
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Conclusions
• This study proposes an indoor navigation system based on
ANN aided image-based
localization
and
PDR
• ANN is used to provide the estimated
distance
between camera and georeferenced marker,
then combined to the orientation sensors to update PDR position
• The use of self-designed marker and simple recognition are in order to
reduce the of
image processing burden
• The results show the accurate long-term navigation with the positional error about
4
meters
and the percentage of loop closure error about
0.3%
after travelled
650 meters
• The smartphone and papers are the only required device and infrastructure
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Conclusions
• The
efficient joint operation of MMSs
is proposed to collect the
necessary information for proposed indoor navigation system:
georeferenced image
and
floor plan
• Those productions make a well demonstration of navigation
solution and has an ability of connecting the outdoor world
Positioning, Orientation and Integrated Navigation Technologies Lab
Department of Geomatics, National Cheng Kung University
Future works
• Collecting more samples and identifying other impact
factors for
model improvement
• ANN for maker recognition (ex. CNN), because the
accurate marker recognition is expected when the marker
is blurred with longer camera distance
• Heading estimation are also considered
• Of course, the cloud server or another
powerful hardware
are needed
Positioning, Orientation and Integrated Navigation Technologies Lab
