Landsat:LiDAR AGB 0.0.8: Another One

Hudak-method models using LiDAR surfaces masked by both LCMAP and AOA. 2022-03-22

Evaluation Results

Last iteration: 2022-03-22 - Landsat AGB 0.0.7: Forced 0s

Change Summary

• Using the newer 1.4.0 LiDAR surfaces (no map accuracy stats yet)
• Masking using both AOA and LCMAP (classes 1, 5, 8)
• This approach generally follows our main Hudak modeling approach

	RF (ranger)	GBM (LightGBM)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	38.315	38.187	36.989	36.878
RMSE %	27.202	27.112	26.261	26.182
MBE	-2.168	-9.205	-7.341	-5.757
R2	0.783	0.792	0.802	0.802

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
  0.00673  70.70000 141.00000 140.86577 212.00000 280.45026 

Validation Results

RMSE	Min	Median	Max
Rf	34.045	34.876	35.456
Lgb	32.603	33.384	33.901
Ensemble	32.218	32.963	33.534

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.810	0.816	0.826
lgb	0.826	0.831	0.839
ensemble	0.831	0.836	0.843

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf 
0.5100569 0.4899431 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-190.091  -16.215   -0.639   17.001  185.827 

Coefficients:
                     Estimate   Std. Error t value        Pr(>|t|)
(Intercept)      -2.075665890  0.108583211 -19.116         < 2e-16
rf_pred           0.355271651  0.001971459 180.207         < 2e-16
lgb_pred          0.649312190  0.001865869 347.994         < 2e-16
rf_pred:lgb_pred  0.000036750  0.000005643   6.513 0.0000000000737
                    
(Intercept)      ***
rf_pred          ***
lgb_pred         ***
rf_pred:lgb_pred ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 32.79 on 1199996 degrees of freedom
Multiple R-squared:  0.8369,    Adjusted R-squared:  0.8369 
F-statistic: 2.052e+06 on 3 and 1199996 DF,  p-value: < 2.2e-16

Coverages

• 1 coverages:
  • testing.csv

\(n\) and \(p\)

• 50000 observations
  • 40000 training
  • 10000 testing
• 29 predictors
  • tcb, tcw, tcg, nbr, mag, ysd, chm_simard, nys_precip, nys_tmax, nys_tmin, nys_aspect, nys_dem, nys_slope, nys_twi, delta_tcb, delta_tcw, delta_tcg, delta_nbr, lcpri4, lcpri6, lcpri2, lcpri3, lcsec2, lcsec3, lcsec4, lcsec1, lcsec8, lcsec5, lcsec6

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 11

$min.node.size
[1] 2

$sample.fraction
[1] 1

$splitrule
[1] "variance"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$params
$params$num_leaves
[1] 18

$params$max_depth
[1] -1

$params$extra_trees
[1] FALSE

$params$min_data_in_leaf
[1] 10

$params$bagging_fraction
[1] 0.8

$params$bagging_freq
[1] 1

$params$feature_fraction
[1] 0.9

$params$min_data_in_bin
[1] 8

$params$lambda_l1
[1] 6

$params$lambda_l2
[1] 7

$params$learning_rate
[1] 0.1

$params$force_col_wise
[1] TRUE

$params$nrounds
[1] 6000