Spatio-Temporal Remote Sensing Information

1 Chia-Hao Chang, ¹ Tzu-How Chu, ² Chiou-Shann Fuh

Detecting Rice Crop Fields Distribution with Spatio-Temporal Remote Sensing Information During the First Rice Season in 2018

– A Case Study in the Erlin Township, Changhua County and Tapi Township, Yunlin County.

1

Dept. of Geography,

Dept. of Computer Science and Information Engineering,

National Taiwan University, Taipei, Taiwan

Outline

• Introduction

• Rice Crop Interpretation

• Methodology

• Data

• Farm Field Polygons and Ground Truth Data

• Sentinel-1 Synthetic Aperture Radar (SAR) Satellite

• Sentinel-1 GRD Image Data

• Results and Discussion

• SAR Image Pre-processing

• T-LBP Baseline and Threshold

• Rice Crop Interpretation

• Discussion

• Conclusion

INTRODUCTION

Rice Crop Interpretation

• With optical image data

• The Agriculture and Food Agency (AFA) uses annually two times of aerial photo to monitor rice distribution in 1

and 2

rice seasons.

• Researchers are also using different remote sensing data to develop more efficient methods to monitor rice crop fields and estimate production.

• Overall accuracy can be up to 86~98%.

• Producer accuracy of rice field can reach 88~96%.

Rice Crop Interpretation (cont.)

• The challenges for optical image

• Spatial resolution

• Acquisition date

• Cloud cover and shadow

15^th Feb. 2017

27^th Mar. 2017 6^th May. 2017 16^th Apr. 2017

Rice Crop Interpretation (cont.)

• With SAR image data

• Cheng et al. using RADARSAT-2 C-Band quad-polarized SAR images acquired from Mar. to Aug. 2009 to analyze polarimetric response of rice crop field with four-component scattering

decomposition (Y4R), including surface scattering, volume scattering, double bounce, and helix scattering (Cheng et al., 2012) .

Different rice crop stages of four-component

scattering decomposition.

(R:P_d, G:P_v, B:P_s)

Rice Crop Interpretation (cont.)

• Chu et al. using ALOS-1 PALSAR data acquired on 8

Apr. 2011, decomposed polarimetric response with Pauli basis, Y4R, and HH- VV (Chu et al., 2014) .

• Classified by Minimum Distance Classification (MDC) and verified with ground truth of 1

rice season in the Yunlin County, 2011.

• The highest rice producer accuracy is HH-VV 91.11%, but the highest overall accuracy is Pauli basis 76.62%.

Pauli Basis Y4R HH-VV

Producer of Rice 91.01% 85.39% 91.11%

Overall 76.62% 66.88% 70.32%

Rice Crop Interpretation (cont.)

• Chu et al. use temporal Sentinel-1 backscattering image (𝜎° ) to analyze the difference between rice, building, betel nut, bare soil, and banana. The 𝜎° is obviously lower than other land cover during transplant (Chu et al., 2016) .

-18.00 -16.00 -14.00 -12.00 -10.00 -8.00 -6.00 -4.00 -2.00 0.00

20160111 20160202 20160226 20160309 20160321 20160426 20160508 20160520

Backscattering in VV polarization (dB)

Acquisition Date of Sentinel-1

Rice Building Betelnut Baresoil Banana

Rice Crop Interpretation (cont.)

• Chang et al. analyzed temporal Sentinel-1 𝜎° image with Temporal Local Binary Pattern (T-LBP) in 1

rice season in Liouying Dist., Tainan County, 2016 (Chang et al., 2017) .

• Verify with ground truth field polygons, the overall accuracy is 83.55%, but producer accuracy of rice is only 57.82%.

Ground Truth

U.A.

Rice Non-Rice Total Classified

Rice 4245 3719 7964 53.30%

Non-Rice 3097 30377 33474 90.75%

Total 7342 34096 41438

P.A. 57.82% 89.09% Overall 83.55%

kappa 0.45

Rice Crop Interpretation (cont.)

• Because some fields are too thin for Sentinel-1 image data, we use 7m inner buffer to filter out those field polygons which are not wider than 14m.

• Verify with those ground truth polygons wider than 14m, overall accuracy was decreased from 83.55% to 78.87% but rice crop producer accuracy was increased from 57.82% to 60.86%.

Ground Truth

U.A.

Rice Non-Rice Total Classified

Rice 3682 1390 5072 72.59%

Non-Rice 2368 8810 11178 78.82%

Total 6050 10200 16250

P.A. 60.86% 86.37% Overall 76.87%

kappa 0.49

METHODOLOGY

Methodology

• Local Binary Pattern (LBP)

• In 1994, Ojala et al. based on the contrast between pixel and eight

neighbors, developed a local texture description method, Local

Binary Pattern (LBP) operator. In the beginning, LBP uses the sign

of eight differences between central and neighbors, recorded into

an 8-bit number (Ojala et al., 1996, Ojala et al., 2001).

Methodology (cont.)

• LBP is an efficient method used to describe local texture in 2D images. But for locally homogenous land cover type, such as water body, rice fields, grassland, and flat bare

land, the separability of LBP is not explicit.

• The agricultural land cover changes with crop’s lifecycle;

image patterns in the remote sensing data are changed

also.

Methodology (cont.)

• Temporal Local Binary Pattern (T-LBP)

• Based on LBP with 2-D texture, Chang et al.

(2017) proposed T-LBP to describe the temporal change pattern of rice pixel in time series

Sentinel-1 𝜎° image.

-18.00 -16.00 -14.00 -12.00 -10.00 -8.00 -6.00 -4.00 -2.00 0.00

20160111 20160202 20160226 20160309 20160321 20160426 20160508 20160520

Backscattering in VV polarization (dB)

Acquisition Date of Sentinel-1

Rice Building Betelnut Baresoil Banana

Land Cover Base 2⁰ 2¹ 2² 2³ 2⁴ 2⁵ T-LBP

Rice 0 1 1 1 1 1 1 63

Building 0 0 0 1 0 0 0 4

Betelnut 0 0 1 1 0 0 0 6

Baresoil 0 0 0 1 0 0 0 4

Banana 0 0 0 0 0 0 0 0

Methodology (cont.)

• Using time series of Sentinel-1 IW mode Ground Range

Detected (GRD) image data, applying LBP operator into temporal 𝜎° change of SAR images.

• Analyze rice crop’s 𝜎° change pattern in 1 ^st rice season in Erlin Township, Changhua County, and Tapi Township, Yunlin County, 2018.

Sentinel-1 GRD Data Collection

Backscatter Data Generation

Temporal Data Coregistration Define Rice

Crop T-LBP Threshold

Calculate T-

LBP Rice Fields

Interpretation

Accuracy

Verification

DATA

Data

• Polygon data are provided by AFA.

• Ground truth data were from ground survey and interpreted with UAV image in May, 2018.

Erlin Township

Tapi Township

Changhua County

Yunlin County

Rice Crop Fields

Non-Rice Crop Fields

Satellite

• The European Space Agency (ESA) launched a series of satellites called “Sentinel”, including optical and SAR

instruments from 2014.

• Sentinel-1 plan is a C-Band Synthetic Aperture Radar (SAR) with single polarization (HH or VV) and dual polarization (HH+HV or VH+VV), which have twin satellites named Sentinel-1A and

Sentinel-1B.

• The orbit height is 700km and 12-days revisit in the same track and same direction.

• To prevent satellite resources conflict, the primary modes are

Interferometric Wide swath (IW) , with VV+VH polarization over

land.

Mode Polarization Main Targets Interferometric Wide Swath (IW) VV+VH land

Wave (WV) VV open ocean

Extra-Wide Swath (EW) ---- wide area coastal, oil spill, sea-ice

Strip Map (SM) ---- Extraordinary events

Sentinel-1 GRD Image Data

• Collected 17 scenes of Sentinel-1A/B GRD image data from 3 ^rd Feb. to 22 ^nd May in 2018.

• To avoid the potential polarimetric response error from

different acquisition directions and satellite obits, all image data cover Northern Taiwan in the same orbit with

ascending direction.

Date Orbit Mode Level

Feb. 3

, 9

, 15

, 27

100 IW GRD

Mar. 5

, 11

, 17

, 23

, 29

Apr. 4

, 16

, 22

, 28

May 4

, 10

, 16

, 22

Sentinel-1 GRD Image Data (cont.)

• Separate 17 image dates into 9 datasets.

• Each dataset includes 9 image dates.

No. Date

1

Feb. 3

, 9

, 15

, 27

Mar. 5

, 11

, 17

, 23

, 29

2

Feb. 9

, 15

, 27

Mar. 5

, 11

, 17

, 23

, 29

Apr. 4

3

Feb. 15

, 27

Mar. 5

, 11

, 17

, 23

, 29

Apr. 4

, 16

No. Date

4

Feb. 27

Mar. 5

, 11

, 17

, 23

, 29

Apr. 4

, 16

, 22

5

Mar. 5

, 11

, 17

, 23

, 29

Apr. 4

, 16

, 22

, 28

6

Mar. 11

, 17

, 23

, 29

Apr. 4

, 16

, 22

, 28

May 4

No. Date

7

Mar. 11

, 17

, 23

, 29

Apr. 4

, 16

, 22

, 28

May 4

, 10

8

Mar. 17

, 23

, 29

Apr. 4

, 16

, 22

, 28

May 4

, 10

, 16

9

Mar. 23

, 29

Apr. 4

, 16

, 22

, 28

May 4

, 10

, 16

, 22

RESULTS AND

DISCUSSION

GRD Images Preprocessing

Layer Stack Clip

(a) Single swath data of GRD image

(b) Backscatter image

T-LBP Baseline and Threshold

-18.00 -16.00 -14.00 -12.00 -10.00 -8.00 -6.00 -4.00 -2.00 0.00

20160111 20160202 20160226 20160309 20160321 20160426 20160508 20160520

Backscattering in VV polarization (dB)

Acquisition Date of Sentinel-1

Rice Building Betelnut Baresoil Banana

Land Cover Base 2

2

2

2

2

2

T-LBP

Rice 0 1 1 1 1 1 1 63

Building 0 0 0 1 0 0 0 4

Betelnut 0 0 1 1 0 0 0 6

Baresoil 0 0 0 1 0 0 0 4

Banana 0 0 0 0 0 0 0 0

Ideal Temporal 𝜎° Changing Pattern

T-LBP Baseline and Threshold (cont.)

-18.00 -16.00 -14.00 -12.00 -10.00 -8.00 -6.00 -4.00 -2.00 0.00

20160111 20160202 20160226 20160309 20160321 20160426 20160508 20160520

Backscattering in VV polarization (dB)

Acquisition Date of Sentinel-1

Rice Building Betelnut Baresoil Banana

Land Cover Base 2

2

2

2

2

2

T-LBP

Rice 0 0 0 0 1 0 1 40

Building 0 1 0 0 1 1 1 57

Betelnut 0 1 0 1 1 1 0 29

Baresoil 0 1 0 0 1 0 0 9

Banana 0 0 0 0 0 0 0 0

Error Baseline Temporal 𝜎° Changing Pattern

T-LBP Baseline and Threshold (cont.)

• Calculate T-LBP with different baseline image dates.

• Based on Chang et al. (2017), T-LBP threshold was set as 200.

From 27

Feb. to 22

Apr. From 5

Mar. to 28

Apr. From 11

Mar. to 4

May

No.4 No.5 No.6

Rice Crop Interpretation

• To reduce the effect of speckle noise, we use mean value of each rice field polygon rather than each pixel value.

• 𝜎° Filter:

• Building: mean 𝜎° over -10

• Grassland: min temporal 𝜎° over -20

Ground Truth

T-LBP

Rice Non-Rice Sum. U.A.

Rice 2064 1216 3280 62.93%

Non-Rice 344 913 1257 72.63%

Sum. 2408 2129 4537

P.A. 85.71% 42.88% Overall 65.62%

kappa 0.292933 Ground Truth

T-LBP

Rice Non-Rice Sum. U.A.

Rice 1740 995 2735 63.62%

Non-Rice 668 1134 1802 62.93%

Sum. 2408 2129 4537

P.A. 72.26% 53.26% Overall 63.35%

kappa 0.257527

Ground Truth

T-LBP

Rice Non-Rice Sum. U.A.

Rice 2032 1238 3270 62.14%

Non-Rice 376 891 1267 70.32%

Sum. 2408 2129 4537

P.A. 84.39% 41.85% Overall 64.43%

kappa 0.268662

Ground Truth

T-LBP

Rice Non-Rice Sum. U.A.

Rice 1739 502 2241 77.60%

Non-Rice 669 1627 2296 70.86%

Sum. 2408 2129 4537

P.A. 72.22% 76.42% Overall 74.19%

kappa 0.484184 Ground Truth

T-LBP

Rice Non-Rice Sum. U.A.

Rice 1485 417 1902 78.08%

Non-Rice 923 1712 2635 64.97%

Sum. 2408 2129 4537

P.A. 61.67% 80.41% Overall 70.47%

kappa 0.415112

Ground Truth

T-LBP

Rice Non-Rice Sum. U.A.

Rice 1713 493 2206 77.65%

Non-Rice 695 1636 2331 70.18%

Sum. 2408 2129 4537

P.A. 71.14% 76.84% Overall 73.82%

kappa 0.477192

5

Mar. to 28

Apr.

11

Mar. to 4

May

5

Mar. to 28

Apr.

11

Mar. to 4

May

Discussion

• The highest interpret accuracy is dataset No. 4 (27 ^th Feb.

to 22 ^nd Apr.).

• Overall Accuracy: 65.62%

• Rice Producer Accuracy: 85.71%

• Non-Rice Producer Accuracy : 42.88%

• With 𝜎° filter, in producer accuracy, rice fields were

decreased but non-rice fields were obviously increased.

• Overall Accuracy: 74.19%

• Rice Producer Accuracy: 72.22%

• Non-Rice Producer Accuracy: 76.42%

CONCLUSION

Conclusion

• Although the rice fields producer accuracy can be higher than 96% with optical data, but still cannot interpret the cloud cover area.

• SAR is a positive remote sensing technique to provide

land cover information over day and night regardless of

cloud cover.

Conclusion (cont.)

• In previous study, an efficient rice field’s interpretation

method with temporal SAR data and T-LBP was proposed.

• This study demonstrates that applied appropriate filter, such as zonal aggregate or signal feature filter, can

obviously improve T-LBP interpretation accuracy.

• Reasons of interpretation error

• Some non-rice fields with similar T-LBP were misclassified.

• Rice fields are mostly in narrow rectangle in Taiwan.

References

1. C. H. Chu, K. C. Chang, R. Y. Lee, “Applying High Satellite Imagery to assist farming inventory (I)”, Agriculture and Food Agency, 2002

2. C. H. Chu, K. C. Chang, R. Y. Lee, “Applying High Satellite Imagery to assist farming inventory (II)”, Agriculture and Food Agency, 2003

3. C. H. Chu, K. C. Chang, R. Y. Lee, “Applying High Satellite Imagery to assist farming inventory (III)”, Agriculture and Food Agency, 2004

4. C. H. Chu, C. C. Liu, Y. S. Tseng, T. Y. Chou, “The Application on Investigation of Rice Field Using the High Frequency and High Resolution Satellite Images (1/3)”, Agriculture and Food Agency, 2005

5. C. H. Chu, C. C. Liu, Y. S. Tseng, C. C. Lei, “The Application on Investigation of Rice Field Using the High Frequency and High Resolution Satellite Images (2/3)”, Agriculture and Food Agency, 2006

6. C. H. Chu, C. C. Liu, Y. S. Tseng, C. C. Lei, “The Application on Investigation of Rice Field Using the High Frequency and High Resolution Satellite Images (3/3)”, Agriculture and Food Agency, 2007

7. C. H. Chu, et al., “Study on Applying High Resolution Optical Satellite Image and L-Band SAR Image to Crops Production Investigation and Reexamine Mechanism” , Agriculture and Food Agency, 2014

8. C. H. Chu, et al., “Research of Apply UAV, Remote Sensing Information and Multi-Scale Spatial Information Transform Classification Model to Improve Paddy Rice Area and Yields Investigation Technique” , Agriculture and Food Agency, 2016

9. C. H. Chang, C. S. Fuh, S. W. Wang, 2017, Rice Field Interpretation with Temporal SENTINEL-1 Synthetic Aperture Radar Image Data, CVGIP2017.

10. C. H. Chang, C. H. Chu, C. S. Fuh, 2017, Landcover Interpretation by Difference Features in Multi-Temporal Remote Sensing Images, Remote Sensing Satellite Technology Workshop.

11. F. De Zan, A. M. Guarnieri, “TOPSAR: Terrain Observation by Progressive Scans”, Geoscience and Remote Sensing, IEEE Transactions on, 44(9), pp.2352–2360, 2006

12. T. Ojala, M. Pietikäinen, D. Harwood, A comparative study of texture measures with classification based on featured distributions, Pattern Recognition, 29(1): 51-59, 1996.

13. T Ojala, K Valkealahti, E Oja, M Pietikäinen, Texture discrimination with multidimensional distributions of signed gray-level differences, Pattern Recognition, Pattern Recognition, 34, pp.727-739, 2001.

14. T. Y. Cheng, C. Y. Chu, K. S. Chen, Y. Yamaguchi, S. J. Lee, “Time Series SAR Polarimetric Analysis of Rice Crop based on Four-Component Scattering Decomposition”, ISAP2012, pp.620-623, 2012

15. ESA Sentinel Online（https://sentinel.esa.int/web/sentinel/home）

Thank You