Development of Vision Aiding Pedestrian Dead Reckoning with Smartphone for Indoor Navigation

(1)

Positioning, Orientation and Integrated Navigation Technologies Lab

Department of Geomatics, National Cheng Kung University

Development of Vision Aiding Pedestrian Dead

Reckoning with Smartphone for Indoor Navigation

Authors : Shih-Huan Huang, Jhen-Ka Liao

Kai-Wei Chiang, Hsiu-Wen Chang, Thanh-Trung Duong

Presenter : Shih-Huan Huang

(2)

2 Motivation

(3)

3 Motivation

(4)

4 𝑁𝑁

_𝑘𝑘+1

= 𝑁𝑁

_𝑘𝑘

+

𝑆𝑆𝑆𝑆

_𝑘𝑘

× 𝑐𝑐𝑐𝑐𝑐𝑐

𝛼𝛼

_𝑘𝑘

𝐸𝐸

_𝑘𝑘+1

= 𝐸𝐸

_𝑘𝑘

+

𝑆𝑆𝑆𝑆

_𝑘𝑘

× 𝑐𝑐𝑠𝑠𝑠𝑠

𝛼𝛼

_𝑘𝑘

1 Step detection

2 Step length estimation

3 Heading estimation

Pedestrian Dead Reckoning

(𝐸𝐸

_𝑘𝑘

, 𝑁𝑁

_𝑘𝑘

)

(𝐸𝐸

_𝑘𝑘+1

, 𝑁𝑁

_𝑘𝑘+1

)

N

E

𝑆𝑆𝑆𝑆

_𝑘𝑘

𝑆𝑆𝑆𝑆

_𝑘𝑘

× 𝑐𝑐𝑐𝑐𝑐𝑐 𝛼𝛼

_𝑘𝑘

𝑆𝑆𝑆𝑆

_𝑘𝑘

× 𝑐𝑐𝑠𝑠𝑠𝑠 𝛼𝛼

_𝑘𝑘

𝛼𝛼

_𝑘𝑘

𝛼𝛼

_𝑘𝑘

(5)

5

• Step detection

• Signal Vector Magnitude= 𝑎𝑎𝑐𝑐𝑐𝑐𝑎𝑎

2 + 𝑎𝑎𝑐𝑐𝑐𝑐𝑎𝑎

2

(6)

6

• Step length calibration

• Average step length = distance / number of steps

• Heading estimation

• Integrated from the output of gyroscope

(7)

7

• Experimental camera phone : I-phone 5s

• Pixel size：0.0015 mm

• Image size：3264*2448

• Focal length：4.1901 mm

• FOV

• horizontal：50 degrees

• vertical：67 degrees

Image resection

(8)

8

• Single Image resection

• collinear equation

𝑎𝑎 − 𝑎𝑎

₀

= −𝑓𝑓

𝑅𝑅

11

𝑋𝑋−𝑋𝑋

0

+𝑅𝑅

12

𝑌𝑌−𝑌𝑌

0

+𝑅𝑅

13

𝑍𝑍−𝑍𝑍

0

𝑅𝑅

₃₁

𝑋𝑋−𝑋𝑋

₀

+𝑅𝑅

₃₂

𝑌𝑌−𝑌𝑌

₀

+𝑅𝑅

₃₃

𝑍𝑍−𝑍𝑍

₀

+ ∆𝑎𝑎 + 𝛿𝛿𝑎𝑎

𝑎𝑎 − 𝑎𝑎

₀

= −𝑓𝑓

𝑅𝑅

21

𝑋𝑋−𝑋𝑋

0

+𝑅𝑅

22

𝑌𝑌−𝑌𝑌

0

+𝑅𝑅

23

𝑍𝑍−𝑍𝑍

0

𝑅𝑅

₃₁

𝑋𝑋−𝑋𝑋

₀

+𝑅𝑅

₃₂

𝑌𝑌−𝑌𝑌

₀

+𝑅𝑅

₃₃

𝑍𝑍−𝑍𝑍

₀

+ ∆𝑎𝑎 + 𝛿𝛿𝑎𝑎

Image resection

Z

Camera

Y

X

(9)

9

• Observation : image coordinate (x,y)

• Known :

• Unknown : the position of camera (𝑋𝑋

_𝑜𝑜

, 𝑌𝑌

_𝑜𝑜

)

Image resection

Parameter

symbol

source

Internal orientation

_𝑎𝑎

₀

_{, 𝑎𝑎}

₀

_{, 𝑓𝑓, ∆𝑎𝑎, 𝛿𝛿𝑎𝑎, ∆𝑎𝑎, 𝛿𝛿𝑎𝑎}

Camera calibration

Attitude angle

_{𝜔𝜔, 𝜑𝜑, 𝜅𝜅}

Inertial sensors

Height

_𝑍𝑍

_𝑜𝑜

Approximation

Feature points

(10)

10 Integrated system

Analysis the necessity

Use the check points to analysis

the percentage of misclosure

Simulated Stage

Use the control points to update

PDR trajectory

Actual stage

Use the image resection results

to update PDR trajectory

Analysis the feasibility

Accuracy evaluation of image

aiding PDR

(11)

11

• Pure inertial sensor PDR

Experiment and analysis

Start point

End point

(12)

12

• Percentage of misclosure

• 𝑃𝑃(%) =

𝛥𝛥𝐸𝐸

2

+𝛥𝛥𝑁𝑁

2

∑ 𝑆𝑆𝑆𝑆

Experiment and analysis

Test Actual

steps Detected steps distance Actual (m) Detected distance (m) Distance Error (m) Misclosure (m) Percentage of Misclosure (%) 1 205 201 134.34 128.29 6.04 10.988 8.2 2 202 199 134.34 127.02 7.32 11.042 8.2 3 202 201 134.34 134.00 0.34 6.298 4.7 4 190 186 134.34 126.82 7.52 2.814 2.1 5 195 190 134.34 129.55 4.79 6.432 4.8 mean 129.14 5.20 7.515 5.6

(13)

13

• Simulated stage

• PDR trajectory is updated by known position

Experiment and analysis

(14)

14

• Simulated stage

• Accuracy evaluation

Experiment and analysis

Accuracy evaluation of pure inertial sensor PDR

Test Actual

steps Detected steps distance Actual (m) Detected distance (m) Distance error (m) Misclosure (m) Percentage of misclosure (%) 1 93 91 59.395 57.860 1.535 0.329 0.6 2 90 93 59.395 54.498 4.897 0.588 1.0 3 98 98 59.395 54.480 4.915 0.306 0.5 4 103 102 59.395 57.426 1.969 0.094 0.2 5 105 104 59.395 58.552 0.843 0.158 0.3 mean 56.563 2.838 0.295 0.5

(15)

15

• Actual stage

• Using the results of image resection to replace control points

• The selection of feature points

Experiment and analysis

(16)

16 Position Range

(m)

resection with 2

Error of image

points（m）

Error of image

resection with 4

points（m）

1

15

2.537

0.509

2

9

1.177

0.558

3

5

1.283

0.373

4

0.501

0.388

5

3

0.880

0.039

6

3

1.203

0.529

7

2 divergence

0.138

8

1 divergence

0.050

9

1 divergence

0.017 Experiment and analysis

• Actual stage

• The range of feature points should be near to the calibration range

(2m)

(17)

17

• Actual stage

• Accuracy evaluation of image resection

Experiment and analysis

Position Range (m) position Actual (m) Calculated position (m) Positioning error (m) C1 4 (24.178,0) (24.360,-0.454) 0.501 C2 3 (31.293,9.871) (32.109,10.200) 0.880 C3 3 (47.764,9.871) (46.654.9.408) 1.203 mean 0.861 Position Range (m) position Actual (m) Calculated position (m) Positioning error (m) C1 4 (24.178,0) (24.371,-0.317) 0.388 C2 3 (31.293,9.871) (31.330,9.883) 0.039 C3 3 (47.764,9.871) (47.287,9.641) 0.529 mean 0.319

Image resection with 4 feature points

Image resection with 2 feature points

(18)

18

• Actual stage

Experiment and analysis

(19)

19

• Actual stage

• Accuracy evaluation

Experiment and analysis

Test Actual

steps Detected steps distance Actual (m) Detected distance (m) Distance error (m) Misclosure (m) Percentage of Misclosure (%) 1 93 91 59.395 57.860 1.535 0.474 0.8 2 90 93 59.395 54.498 4.897 0.977 1.6 3 98 98 59.395 54.480 4.915 0.623 1.0 4 103 102 59.395 57.426 1.969 0.480 0.8 5 105 104 59.395 58.552 0.843 0.864 0.9 mean 56.563 2.838 0.684 1.0

(20)

20

• Comparison

Experiment and analysis

Misclosure (m) Percentage of misclosure (%) Pure inertial PDR 7.515 5.6 Simulated stage 0.295 0.5 Actual stage 0.684 1.0

The rising accuracy of image aiding

=

𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑖𝑖𝑖𝑖𝑝𝑝𝑝𝑝𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑃𝑃𝑃𝑃𝑅𝑅 − 𝑖𝑖𝑎𝑎𝑖𝑖𝑝𝑝𝑖𝑖𝑖𝑖 𝑠𝑠𝑖𝑖𝑖𝑖𝑠𝑠𝑝𝑝

_{𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑖𝑖𝑖𝑖𝑝𝑝𝑝𝑝𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑃𝑃𝑃𝑃𝑅𝑅}

× 100%

(21)

21

• The accuracy of image resection is affected by the

focal length and number of feature points

• Image aiding PDR drops the percentage of

misclosure from 5.6% to 1.0%

• Feature works:

• Expand the experimental field

• Achieve real- time processing on the smartphone in the future

(22)

22

• Bashir Kazemipur, Zainab Syed, Jacques Georgy, Naser El-Sheimy (2014). Vision-based Context and Height Estimation for 3D Indoor Location

• Chien-Hsun Chu, Kai-Wei Chiang, Cheng-An Lin (2013). The Performance Analysis of a Portable Mobile Mapping System with Different GNSS Processing Strategies. ION GNSS 2013 Meeting, Nashville, Tennessee, USA(EI)

• N. Bidargaddi, A. Sarela, L. Klingbeil, M. Karunanithi (2007). Detecting Walking Activity in Cardiac Rehabilitation by Using Accelerometer pp. 555–560

• Li-Hung Chen, Kuei-Ping Chang, Kai-Wei Chiang (2014). The Performance Analysis of Visual Odometry Assisted Inertial Navigation System.

• Li, Y.H. (2010): The Calibration Methodology of a Low Cost Land Vehicle Mobile Mapping System, ION GNSS 2010, Portland, OR

• Q. Ladetto, B. Merminod (2002). Digital Magnetic Compass and Gyroscope Integration for Pedestrian Navigation • Ronald Azuma, Mark Ward (1991). Space resection by collinearity Mathematics behind the optical ceiling

head-tracker.

• Ruizhi Chen, Ling Pei, Yuwei Chen(2011). A Smartphone Based PDR Solution for Indoor Navigation. ION GNSS 2011 Proceedings.

• Sheng-Cheng Yeh, Chen-Chih Chiu, Cheng-En Hung (2010). A Study of Indoor Locating and Tracking Systems Based on Wireless Local Networks with RFID Technique

• Sou-Yung Hsieh (1998). The Study of the Three Dimensional Navigation System Hardware Structure by Integrating GPS with INS.

(23)