Two datasets were processed to examine the algorithm’s performance. For the urban area dataset, we first focus on the subsite sample 11. The elevation difference threshold dhTh of filtering parameter was set as 1.5 m. The initial search radius was set at 60 m so as to be large enough to remove buildings.
1. Sensitivity of Varying Search Radius
Figure 3.23 shows the difference of filtering results by adaptive directional elevation-difference filter with varying search radius. With increasing search radius, Type I errors increased from 11.1% to 11.7%, and Type II error decreased from 11.7% to 11.3%. The filtering result with large search radius (120 m) did not tend to over-filter the ground points. As search radius increased from 60 m to 120 m, the Total error increased from 11.3% to 11.5%, a difference of only 0.2%.
Figure 3.24 shows the difference of filtering results by adaptive directional steepest-descent filter with varying search radius. The steepest-descent threshold γTh of filtering parameter was set as 12%. Similar to the result of adaptive directional elevation-difference filter, the filtering result with large search radius (120 m) did not over-filter the ground points. As search radius increased from 60 m to 120 m, the Total error increased from 11.3% to 11.4%, a difference of only 0.1%.
11.311.111.7 11.311.311.4 11.511.611.4 11.511.711.3
60 80 100 120
Search distance (m)
0 5 10 15 20
er ro r (% )
Total Type I Type II
Figure 3.23. Effect of varying initial radius on errors, filtered by directional elevation-difference algorithm.
11.311.211.5 11.211.211.2 11.411.211.5 11.411.411.6
Figure 3.24. Effect of varying initial radius on errors, filtered by adaptive steepest-descent algorithm.
2. DTM Generation Results on Urban Area
Figure 3.25 shows the shaded relief maps of six subsites within the urban area for visual inspection. The filtering result of Figure 3.25 was processed by multiple filtering procedures. The parameters used are listed in Table 3.4.
In the large objects removal stage (1), with the adaptive elevation-difference filter, the initial search radius was set at 100 m. A 1.0 m to 2.4 m elevation-difference threshold was used. The elevation-difference thresholds were determined by visual inspection. In the medium object removal stage (2), using adaptive steepest-descent filter, a 15% steepest-descent threshold was used. In the final filtering stage (3), using directional steepest-descent filter, the search radius was set at 3 m and a 3% steepest-descent threshold was used. Comparing the filtered result with reference from Figure 3.25, most non-ground objects were filtered successfully. Low buildings were left in mistakenly surrounding A [Figure 3.25(a)]
and B [Figure 3.25 (b)]. A few measurements below the bridge were mistakenly removed [C in Figure 3.25 (d)]. A few measurements surrounding a complex plaza were chopped off mistakenly [D in Figure 3.25 (e)].
The quantitative error examinations of Type I, Type II and the Total errors for the six DTMs are displayed in Figure 3.26 and Table 3.4. The computed errors over six subsites ranged from 0.1% to 10.1%, 4.7% to 9.4% and 3.5% to 9.2% for Type I, Type II and the Total errors respectively. The filtering strategy of this research is trying to minimize Type I errors. Except sample 23, Type I errors of most filtering results are smaller than Type II errors. This is because some measurements in sample 23 (complex scene) were removed mistakenly.
Unfiltered filtered reference filtered result
(a) Sample 11
(b) Sample 12
(c) Sample 21
Figure 3.25. The DTM generation results for the urban area.
A A
B B
(d) Sample 22
(e) Sample 23
(f) Sample 24
Figure 3.25. The DTM generation results for the urban area.
C C
D D
9.2
3.5
1.6
4.8
7.6 7.4 9.2
1.5
0.1
3.9
10.1
6.6 9.3
5.6
6.9 6.9
4.7
9.4
Sa m pl e 11 Sa m pl e 12 Sa m pl e 21 Sa m pl e 22 Sa m pl e 23 Sa m pl e 24
0 5 10 15
erro r (% )
Total Type I Type II
Figure 3.26. The filtering errors for sample sites.
Table 3.4 The accuracy of multiple filtering results
Sample 11 Filtered Error Parameters
Ground Non-ground Type I (%) 9.16 Initial radius (m) 100 Ground 21786 19791 1995 Type II (%) 9.28 Elevation difference (m) 2.4 Non-ground 16224 1505 14719 Total (%) 9.21 Steepest descent (%) 15 Reference
Total 38010
Sample 12 Filtered Error Parameters
Ground Non-ground Type I (%) 1.48 Initial radius (m) 100 Ground 26691 26295 396 Type II (%) 5.56 Elevation difference (m) 1.2 Non-ground 25428 1413 24015 Total (%) 3.47 Steepest descent (%) 15 Reference
Total 52119
Sample 21 Filtered Error Parameters
Ground Non-ground Type I (%) 0.08 Initial radius (m) 100 Ground 10085 10077 8 Type II (%) 6.92 Elevation difference (m) 1.0 Non-ground 2875 199 2676 Total (%) 1.60 Steepest descent (%) 15 Reference
Total 12960
Sample 22 Filtered Error Parameters
Ground Non-ground Type I (%) 3.85 Initial radius (m) 100 Ground 22504 21637 867 Type II (%) 6.90 Elevation difference (m) 1.0 Non-ground 10202 704 9498 Total (%) 4.80 Steepest descent (%) 15 Reference
Total 32706
Sample 23 Filtered Error Parameters
Ground Non-ground Type I (%) 10.13 Initial radius (m) 100 Ground 13223 11884 1339 Type II (%) 4.73 Elevation difference (m) 1.0 Non-ground 11872 562 11310 Total (%) 7.58 Steepest descent (%) 15 Reference
Total 25095
Sample 24 Filtered Error Parameters
Ground Non-ground Type I (%) 6.64 Initial radius (m) 100 Ground 5434 5073 361 Type II (%) 9.38 Elevation difference (m) 1.0 Non-ground 2058 193 1765 Total (%) 7.39 Steepest descent (%) 15 Reference
Total 7492
3. DTM Generation Results on High-relief Area
For the high-relief dataset, ground checkpoints in the area of pavement and of wet soil were used to assess the accuracy of LIDAR observations on open terrain.
The ‘difference’ or elevation error for each checkpoint is computed by subtracting the surveyed elevation of the checkpoint from the LIDAR dataset elevation interpolated at the x/y coordinate of the checkpoint. The interpolation method is the Delaunay triangulation with linear interpolation. LIDAR-derived elevation was converted into triangulated irregular networks. Differences were obtained by subtracting the reference elevation from the LIDAR elevation.
Table 3.5 shows that RMSE error of pavement area is 9.2 cm and of wet soil is 14.5 cm for H = 1100 m dataset. Table 3.6 shows that RMSE error of pavement area is 14.0 cm and of wet soil is greater than 15 cm (RMSE 20.7 cm) for H = 1800 m dataset.
This study adopted the Terrascan software for the automated filtering of LIDAR points. The software’s algorithm uses the single-return range data, that is, the last return for the multi-echo. There are four steps involved in the filtering procedure.
In the inspecting stage (1), the analyst chooses a series of representative profiles, especially cross-sections of ridges and cross-sections of dense vegetation. In the analyzing stage (2), each profile is measured for the terrain slope, vegetation density, and the largest size of buildings. The coverage was manually divided into multiple overlapping subsets according to terrain slope and vegetation so that each subset has homogenous terrain and vegetation type. After this, the parameter thresholds are determined for each subset. A small height threshold or a large window size removes some detail of the terrain or cuts some hill peaks. A large height threshold or a small window size tends to preserve non-terrain points. After the dataset has
been filtered, a preliminary DTM is derived from the ground points. In the checking stage (3), the chosen representative profiles are then viewed individually to check if the summit has been cut wrongly or the vegetation points were still retained in wooded area. It is an iterative procedure in which the analyst revises the parameter thresholds until the ground points and non-ground points are separated. Finally, in the manual editing stage (4), the shaded relief map is plotted from the DEM to check the filtering results and to manually edit the artifacts.
Two different working schemes are adopted for automatic filtering process.
The filtering for H = 1800 m used an automated procedure. In the case of H = 1100 m, the automated procedure was followed by a manual editing to refine the ground points. While processing H = 1800 m data by the automated procedures, the strategy of choosing the parameter thresholds tended to remove as many vegetation points as possible (minimizing the error of classifying non-ground points as ground measurements). On the other hand, processing the H = 1100 m data tends to incorrectly leave ground points (minimizing the error of filtering ground points mistakenly (Sithole and Vosselman, 2003). The reason is that during the manual editing process it is thought to be easier to remove unwanted points than to detect the absence of ground points and add them back in, with the working environment of this study.
Checkpoints in the area of occlusion path, landslide rock, orchard, sloped tea farm A and tea farm B were used for quantitative analysis of filtering accuracy. A 2.0 m elevation-difference threshold and a 15% steepest-descent threshold were used.
* For H = 1100 m dataset, the differences in MAE using multiple-filtering process and automatic removal technique with manual editing are displayed in Figure 3.27.
The MAEs of multiple-filtering processing are smaller than those of automatic processing with manual edits for area of occlusion path and sloped tea farm A (H
= 1100 m dataset). On the other hand, the MAEs of multiple-filtering processing are larger than those of automatic processing with manual edits for area of landslide rock, orchard and tea farm B. The total differences in MAE of automatic processing with manual edits (15.4 cm) were smaller than those of multiple-filtering processing (16.6 cm). The t-test was significant at the 0.078 level (>0.05 level).
* For H = 1800 m dataset, the MAE of multiple-filtering processing are smaller than those of automatic processing for all areas (Figure 3.28, Table 3.8, Table 3.10).
The total differences in MAE of multiple-filtering processing (22.2 cm) were smaller than those of automatic processing (25.4 cm). The t-test of total differences in MAE was significant at the 0.004 level (<0.05 level).
* Note that for H = 1100 m, in the area of sloped tea farm A (dense tea tree and sloped relief), the results of proposed algorithms obtain less mean errors (-12.6 cm vs. -26.5 cm) and less MAE (24.6 cm vs. 27.2 cm) than automatic processing with manual edits scheme (Table 3.7, Table 3.9).
* For H = 1800 m, in the area of sloped tea farm A, the results of proposed algorithms also have better performance (19.2 cm vs. 26.3 cm mean error and 26.1 cm vs. 29.0 cm MAE).
15.1 16.5
Occlusion path Landslide rock Orchard Sloped tea farm A Tea farm B 0
Figure 3.27. The comparisons of DTMs for H = 1100 m.
13.2
Occlusion path Landslide rock Sloped tea f arm A Tea f arm B 0
Table 3.5 Accuracy of DTM in bald earth areas (H = 1100 m, error in cm)
Pavement Wet soil
95% # of points 130 82
95% RMSE 8.2 13.9
95% MAE 6.4 11.8
95% Mean error -1.4 -10.2
95% Median -0.6 -11.6
95% Skew -0.310 +0.272
95% Std Dev 8.1 9.5
95% Min -20.1 -26.2
95% Max 14.1 9.9
100% # of points 137 86
100% RMSE 9.2 14.5
100% MAE 7.1 12.2
100% Mean error -1.7 -10.2
100% Skew -0.428 +0.173
100% Std Dev 9.1 10.4
100% Min -28.4 -36.3
100% Max 14.8 11.6
Table 3.6 Accuracy of DTM in bald earth areas (H = 1800 m, error in cm)
Pavement Wet soil
95% # of points 130 82
95% RMSE 11.7 20.3
95% MAE 9.9 18.4
95% Mean error -4.6 -18.4
95% Median -5.7 -17.2
95% Skew +0.119 -0.463
95% Std Dev 10.8 8.6
95% Min -36.3 -37.9
95% Max 17.3 -4.9
100% # of points 137 86
100% RMSE 14.0 20.7
100% MAE 11.2 18.5
100% Mean error -5.3 -18.5
100% Skew +0.553 -0.411
100% Std Dev 13.0 9.4
100% Min -50.4 -40.4
100% Max 21.2 1.0
Table 3.7 Accuracy of DTM for automatic processing with manual edits (H = 1100 m,
Table 3.8 Accuracy of DTM for automatic processing (H = 1800 m, error in cm)
Occlusion path Landslide rock Sloped tea farm A
Tea farm B Total
95% # of points 78 330 44 78 530 95% RMSE 18.3 25.8 37.6 40.0 26.6 95% MAE 14.3 22.6 27.2 32.7 22.5 95% Mean error -6.8 -15.7 25.8 29.3 -5.0 95% Median -4.7 -19.4 18.5 29.2 -9.4 95% Skew -0.140 0.650 1.072 0.555 0.541 95% Std Dev 17.1 20.4 27.7 27.5 26.1 95% Min -40.6 -54.6 -16.2 -43.4 -52.5 95% Max 31.3 37.7 93.5 123.5 66.3
100% # of points 82 348 46 82 558 100% RMSE 21.0 28.4 40.2 47.0 32.1 100% MAE 16.0 24.4 29.0 36.7 25.4 100% Mean error -6.3 -15.1 26.3 29.2 -3.9 100% Median -4.7 -19.4 18.5 29.2 -9.4 100% Skew 0.339 0.804 0.974 0.179 0.986 100% Std Dev 20.2 24.1 30.8 37.1 31.8 100% Min -47.5 -76.1 -29.6 -88.5 -88.5 100% Max 60.7 82.5 106.4 157.0 157.0
Table 3.9 Accuracy of DTM for multiple-filter processing (H=1100m, error in cm)
Occlusion path
Landslide rock
Orchard Sloped tea
farm A Tea farm B Total
95% # of points 78 330 119 44 78 649 95% RMSE 16.5 18.4 17.8 27.9 17.9 17.8 95% MAE 14.6 15.4 13.8 22.8 14.0 14.8 95% Mean error -14.6 14.7 8.3 -13.6 -3.8 6.2 95% Median -14.2 13.8 3.1 -10.8 -4.9 7.5 95% Skew -0.329 0.158 0.415 0.260 -0.235 -0.170 95% Std Dev 7.8 11.1 15.9 24.6 17.6 16.8 95% Min -33.6 -11.4 -19.3 -52.9 -66.9 -39.6 95% Max -1.2 42.8 39.9 45.1 34.1 40.0
100% # of points 82 348 125 46 82 683 100% RMSE 17.5 20.0 18.9 30.4 23.6 20.9 100% MAE 15.1 16.5 14.7 24.6 16.7 16.6 100% Mean error -14.8 14.5 8.4 -12.6 -3.8 5.8 100% Median -14.2 13.8 3.1 -10.8 -4.9 7.5 100% Skew -0.506 -0.468 0.359 0.680 0.067 -0.376 100% Std Dev 9.3 13.9 17.0 28.0 23.4 20.1 100% Min -43.7 -61.6 -23.8 -55.0 -74.7 -74.7 100% Max 10.1 50.6 44.3 73.4 86.7 86.7
Table 3.10 Accuracy of DTM for multiple-filter processing (H=1800m, error in cm)
Occlusion path Landslide rock Sloped tea
farm A Tea farm B Total
95% # of points 78 330 44 78 530 95% RMSE 14.8 22.5 31.6 38.3 24.4
95% MAE 11.5 18.7 24.3 33.6 19.8 95% Mean error -32.6 -11.0 19.0 33.4 -0.5 95% Median -30.3 -12.0 14.6 33.8 -3.1 95% Skew -0.160 0.221 0.349 0.155 0.370 95% Std Dev 14.9 19.6 25.5 18.9 24.4 95% Min -33.3 -49.9 -37.9 -8.2 -47.0 95% Max 27.6 43.3 76.5 79.4 61.7
100% # of points 82 348 46 82 558 100% RMSE 18.1 25.4 34.0 42.4 28.5
100% MAE 13.2 20.7 26.1 35.2 22.2 100% Mean error -5.8 -10.5 19.2 34.4 0.1 100% Median -30.3 -12.0 14.6 33.8 -3.1 100% Skew 0.480 0.488 0.348 1.573 0.714 100% Std Dev 18.2 23.2 28.4 25.0 28.5 100% Min -46.7 -73.5 -40.0 -13.7 -73.5 100% Max 74.0 70.9 88.5 161.5 161.5
CHAPTER FOUR: ERROR ASSESSMENT AND DTM VALIDATION
Evaluating LIDAR accuracy based on both land-cover types and terrain characteristics is important. The vegetation types for collecting reference points are commonly divided into basic land-cover categories such as tall weeds, brush/low trees, and forests. This chapter considers some quantitative descriptors such as vegetation angle, canopy volumes and LIDAR-derived tree height, to characterize the relationship between elevation accuracy and the types of vegetation. In this chapter, an approach was proposed to identify vegetation characteristics based on LIDAR data.
The derived vegetation information was factored into the evaluation of the impact of vegetation types on the accuracy of LIDAR-derived elevation.
This investigation compared two overlapping datasets in high-relief test site in central Taiwan. The test sites are of different type topography, with slopes ranging from 0° to 70.3°. These two datasets were obtained from different flying altitudes under leaf-on conditions. LIDAR-derived elevation was compared with in situ measurements. The procedure for evaluation of LIDAR data quality was also assessed.