A C R S 2 0 1 7

全文

(1)

Student:Pin-Yun Chen, Kuan-Ying Lin Adviser:Yi-Hsing Tseng

ACRS 2017

Automatic Image Matching for Space Intersection

of Spherical Panorama Images

(2)

• Introduction

• Spherical Panorama Image (SPI)

• Portable Panoramic Image Mapping System (PPIMS)

• SPI Matching Strategy

• Matching method

• Matching procedure

• Experiment Analysis

• Test field

• Image matching

• Conclusions

Outline

(3)

Introduction

(4)

SPI

• Spherical Panorama Image (SPI)

180°

360°

(X,Y,Z)?

(5)

SPI

• Spherical Panorama Image (SPI)

5

(6)

PPIMS

• Portable Panoramic Image Mapping System (PPIMS)

• Combining GNSS receiver

Camera Specs

Model Sony NEX-5N

Resolution 16 megapixels Focal length 16 mm

Pixel size 4.8 𝜇𝑚

Field of view 80° × 53°

(7)

PPIMS

• PPIMS Coordinate System

7

Z

M

Mapping frame

Y

M

X

M

X

B

Body frame

Z

B

Y

B

𝑥

𝐶

𝑧

𝐶

𝑦

𝐶

Camera frame

𝑟 𝑐 𝐵 𝑖

𝑅 𝑐 𝐵 𝑖

(8)

PPIMS

• Calibration Result of PPIMS Relative Orientation

Camera 𝒙𝑪𝑩𝒊(𝐦) 𝒚𝑪𝑩𝒊(𝐦) 𝒛𝑪𝑩𝒊(𝐦) 𝛚𝑪𝑩𝒊(°) 𝛗𝑪𝑩𝒊(°) Ҡ𝑪𝑩𝒊(°) No.1 -0.0965 0.1017 -0.0896 359.1348 109.4978 226.0600 No.2 -0.1453 0.0021 -0.0905 359.7416 109.8292 180.1320 No.3 -0.1050 -0.1021 -0.0937 0.0605 109.0724 135.5507 No.4 0.0019 -0.1517 -0.0958 357.1614 108.9608 92.6564 No.5 0.0982 -0.1041 -0.0896 359.4705 109.8352 46.0271 No.6 0.1462 -0.0023 -0.0907 359.4214 109.9843 0.1291 No.7 0.1078 0.1071 -0.0926 0.0300 110.0384 314.9899

No.8 0.0051 0.1425 -0.0886 0 110 270

Distribution of 8 cameras

Lever-arm vector & Boresight angle

(9)

PPIMS

• Formation of PPIMS SPI

9

(10)

PPIMS

• Bundle adjustment of multi-station SPIs (Lin, 2014)

• SPI center

• Ground control point

• SPI point

P (Object point) o

𝑟𝐶𝐵𝑖

𝑟𝑃𝐵

𝑝(SPI Point) 𝑐𝑖

𝑟

𝑝′𝐵

𝑥𝑆1

𝑦𝑆1 𝑧𝑆1

𝑦𝑆3 𝑥𝑆3

𝑧𝑆3

𝑥𝑆2 𝑦𝑆2 𝑧𝑆2

P

𝑺𝑷𝑰 1

𝑺𝑷𝑰 2

𝑺𝑷𝑰 3

𝑟

𝑝′𝐵

+ 𝑣

𝑟

𝑝′𝐵

= 1

𝜆′

𝑝′𝐵

∙ 𝑅

𝑀𝐵

∙ (𝑟

𝑃𝑀

− 𝑟

𝐵𝑀

) − 𝑟

𝑐𝐵𝑖

(11)

SPI Matching Strategy

(12)

Matching Method

• Matching index

SNCC (Sum of Normalized Cross Correlation) • YARD (Yet Another Reconstruction Dataprogram)

𝑁𝐶𝐶

𝑘

= σ σ(𝑓

𝑖𝑗𝑘

− 𝑓

𝑘

)(𝑔

𝑖𝑗

− 𝑔) σ σ(𝑓

𝑖𝑗𝑘

− 𝑓

𝑘

)

2

σ σ(𝑔

𝑖𝑗

− 𝑔)

2

𝑆𝑁𝐶𝐶 = 1

𝑁 ෍ 𝑁𝐶𝐶

𝑘

𝑌𝐴𝑅𝐷 = σ σ 𝑓

𝑖𝑗

− 𝑓)(𝑔

𝑖𝑗

− 𝑔

σ σ 𝑓

𝑖𝑗

− 𝑓

2

σ σ 𝑔

𝑖𝑗

− 𝑔

2

𝑓

𝑖𝑗𝑘

: The gray value of pixel i, j in match image k 𝑓

𝑘

: The mean value of match image k

𝑔

𝑖𝑗

: The gray value of pixel (i, j) in base image 𝑔: The mean value of base image

N: Number of the overlapped SPIs

𝑓

𝑖𝑗

: The gray value of pixel (i, j) in average image

▪ Based on Normalized Cross Correlation ▪ Use average image for matching

Average image ( f ) = 1

𝑁 ෍ 𝑓

𝑖𝑗k

(13)

Matching Method

• Search conjugate points along epipolar line.

13

(14)

Matching Procedure

(15)

Matching Procedure

• Back projection

• Find the possible position of interest point on other SPIs based on the collinearity condition.

15

(16)

Matching Procedure

• Matching in the object space

• Generate the virtual surface in the object space for each assumed point.

• Bilinear image resampling.

(17)

Matching Procedure

• Similarity check

• Generate similarity profile along the searching direction

• The maximum similarity locates is regarded as the object point position.

17

Base image

Match image

(18)

Experiment Analysis

(19)

Test Field

• First floor of Dept. of Geomatics in NCKU

• 5 image stations

• 6 control points

• 24 tie points

19

(20)

Test Field

• Tie points on SPIs

SPI 2

SPI 3

SPI 4

(21)

Test Field

• Tie points on SPIs

21

SPI 5

SPI 6

(22)

Test Field

• E.O. of 5 SPIs

ෝ𝝈𝐗(m) 𝝈ෝ𝐘(m) 𝝈ෝ𝐙(m) ෝ𝝈𝛚(˚) 𝝈ෝ𝛗(˚) ෝ𝝈𝛋(˚) SPI 2 ±0.006 ±0.005 ±0.004 ±0.043 ±0.030 ±0.029 SPI 3 ±0.005 ±0.005 ±0.004 ±0.042 ±0.032 ±0.028 SPI 4 ±0.004 ±0.004 ±0.003 ±0.042 ±0.032 ±0.027 SPI 5 ±0.005 ±0.004 ±0.003 ±0.042 ±0.029 ±0.025 SPI 6 ±0.005 ±0.004 ±0.003 ±0.050 ±0.036 ±0.032

Point ID ∆X (m) ∆Y (m) ∆Z (m) ∆𝐝 (m)

1 0.003 -0.007 0.004 0.008

4 0.000 -0.003 0.001 0.003

8 -0.012 0.000 0.007 0.014

27 0.001 0.000 0.000 0.001

36 0.000 0.000 -0.001 0.001

Mean. -0.002 -0.002 0.002

RMSD ±0.006 ±0.003 ±0.004

Differences of check points Distribution of image stations and check points

(23)

Image Matching

• 5 test targets

• Windows size:35×35 pixel

• Searching range: Initial position ± 0.25m

• Searching distance: 0.01m

• Different test cases

23

Case 1 Case 2 Case 3 Case 4

Match Image Type Image Space Image Space Object Space Object Space

Matching Index SNCC YARD SNCC YARD

(24)

Image Matching

Max. Similarity SPI 2 SPI 3 SPI 5 SPI 6

Case 1 0.248 Case 2 0.472 Case 3 0.232

Max. Similarity SPI 2 SPI 4 SPI 5

Case 1 0.193 Case 2 0.523 Case 3 0.563

(25)

Image Matching

25 Max. Similarity SPI 4 SPI 5

Case 1 0.242 Case 2 0.333 Case 3 0.465 Case 4 0.624

Max. Similarity SPI2 SPI4 SPI5

Case 1 0.533 Case 2 0.757 Case 3 0.706 Case 4 0.777

(26)

Image Matching

Point ID Case 1 Case 2 Case 3 Case 4

1 X O X X

4 X O O O

8 X X O O

27 O O O O

36 O O O O

O: Yes X: No

Max. Similarity SPI 2 SPI 3 SPI 5 Case 1 0.684

Case 2 0.787 Case 3 0.897

Show high similarity at central position?

(27)

Image Matching

• Maximum similarity value

27

Case 1 Case 2 Case 3 Case 4

Matching Index SNCC YARD SNCC YARD

Match Image Type Image Space Image Space Object Space Object Space Point ID

1 0.248 0.472 0.232 0.411

4 0.193 0.523 0.563 0.747

8 0.242 0.333 0.465 0.624

27 0.533 0.757 0.706 0.777

36 0.684 0.787 0.897 0.922

Average 0.380 0.574 0.573 0.696

YARD index > SNCC index

(28)

Image Matching

• Point searching results

Point ID ∆X (m) Case 1∆Y (m) ∆Z (m) ∆X (m) Case 2∆Y (m) ∆Z (m)

1 -0.007 0.002 0.006 -0.007 0.002 0.006

4 -0.026 0.043 0.007 -0.005 -0.002 -0.003

8 -0.012 0.060 -0.006 -0.012 0.060 -0.006

27 0.000 0.000 0.001 0.000 0.000 0.001

36 0.011 0.003 -0.004 0.011 0.003 -0.004

Mean -0.007 0.022 0.001 -0.003 0.013 -0.001

RMSD ±0.014 ±0.033 ±0.005 ±0.008 ±0.027 ±0.005

Point ID Case 3 Case 4

∆X (m) ∆Y (m) ∆Z (m) ∆X (m) ∆Y (m) ∆Z (m)

1 -0.007 0.002 0.006 -0.015 -0.017 0.010

4 -0.005 -0.002 -0.003 -0.005 -0.002 -0.003

8 -0.013 0.001 0.004 -0.013 0.001 0.004

27 0.000 0.000 0.001 0.000 0.000 0.001

36 0.011 0.003 -0.004 0.011 0.003 -0.004

Mean -0.003 0.001 0.001 -0.004 -0.003 0.002

Object Space > Image Space

(29)

Image Matching

• Point searching results

29 Point ID

Case 1 Case 2

∆X (m) ∆Y (m) ∆Z (m) ∆X (m) ∆Y (m) ∆Z (m)

1 -0.007 0.002 0.006 -0.007 0.002 0.006

4 -0.026 0.043 0.007 -0.005 -0.002 -0.003

8 -0.012 0.060 -0.006 -0.012 0.060 -0.006

27 0.000 0.000 0.001 0.000 0.000 0.001

36 0.011 0.003 -0.004 0.011 0.003 -0.004

Mean -0.007 0.022 0.001 -0.003 0.013 -0.001

RMSD ±0.014 ±0.033 ±0.005 ±0.008 ±0.027 ±0.005

Point ID Case 3 Case 4

∆X (m) ∆Y (m) ∆Z (m) ∆X (m) ∆Y (m) ∆Z (m)

1 -0.007 0.002 0.006 -0.015 -0.017 0.010

4 -0.005 -0.002 -0.003 -0.005 -0.002 -0.003

8 -0.013 0.001 0.004 -0.013 0.001 0.004

27 0.000 0.000 0.001 0.000 0.000 0.001

36 0.011 0.003 -0.004 0.011 0.003 -0.004

Mean -0.003 0.001 0.001 -0.004 -0.003 0.002

RMSD ±0.009 ±0.002 ±0.004 ±0.010 ±0.008 ±0.005

YARD index > SNCC index

YARD index ≅ SNCC index

(30)

Conclusions

(31)

Conclusions

• PPIMS SPIs is applied to space intersection in this study, and the precision can reach cm level with reliable quality of image station EOPs.

• The feasibility of matching strategy proposed are validated.

• Matching in the object space shows better performance for searching conjugate point.

• The similarity value using YARD index is higher than SNCC index in most cases. However, searching the object point position by YARD index is equal to SNCC index while matching in the object space.

31

(32)

Thanks for your listening!

Q & A

數據

Updating...

參考文獻

Updating...

相關主題 :