Landsat:LiDAR AGB 0.0.99: Final Countdown

Hudak-method models using LiDAR surfaces masked by both LCMAP and AOA. 2022-04-12

Evaluation Results

Last iteration: 2022-03-22 - Landsat AGB 0.0.8: Another One

Change Summary

• Fixed a bug in the data-creation stage which averaged overlapping LiDAR regions, rather than selecting the newest one

	RF (ranger)	GBM (LightGBM)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	37.982	36.411	35.878	36.072
RMSE %	26.984	25.868	25.489	25.627
MBE	-2.031	-1.298	-2.081	-1.656
R2	0.788	0.799	0.807	0.807

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
  0.00204  70.66485 141.33928 140.83129 211.98747 280.45026 

Validation Results

RMSE	Min	Median	Max
Rf	34.248	34.888	35.538
Lgb	33.075	33.646	34.336
Ensemble	32.583	33.204	33.880

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.808	0.816	0.825
lgb	0.820	0.828	0.836
ensemble	0.825	0.833	0.841

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf 
0.5114895 0.4885105 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-230.594  -16.533   -0.514   17.110  182.828 

Coefficients:
                     Estimate   Std. Error t value Pr(>|t|)    
(Intercept)      -1.681637820  0.110413787 -15.230   <2e-16 ***
rf_pred           0.365822385  0.002087550 175.240   <2e-16 ***
lgb_pred          0.633829523  0.001960836 323.245   <2e-16 ***
rf_pred:lgb_pred  0.000050833  0.000005697   8.922   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 33.12 on 1199996 degrees of freedom
Multiple R-squared:  0.8337,    Adjusted R-squared:  0.8337 
F-statistic: 2.005e+06 on 3 and 1199996 DF,  p-value: < 2.2e-16

Coverages

• 1 coverages:
  • testing.csv

\(n\) and \(p\)

• 50000 observations
  • 40000 training
  • 10000 testing
• 29 predictors
  • tcb, tcw, tcg, nbr, mag, ysd, chm_simard, nys_precip, nys_tmax, nys_tmin, nys_aspect, nys_dem, nys_slope, nys_twi, delta_tcb, delta_tcw, delta_tcg, delta_nbr, lcpri2, lcpri4, lcpri6, lcpri3, lcsec3, lcsec2, lcsec1, lcsec4, lcsec8, lcsec5, lcsec6

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 500

$mtry
[1] 8

$min.node.size
[1] 1

$sample.fraction
[1] 1

$splitrule
[1] "variance"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$params
$params$num_leaves
[1] 21

$params$max_depth
[1] -1

$params$extra_trees
[1] FALSE

$params$min_data_in_leaf
[1] 10

$params$bagging_fraction
[1] 0.9

$params$bagging_freq
[1] 10

$params$feature_fraction
[1] 0.7

$params$min_data_in_bin
[1] 8

$params$lambda_l1
[1] 5

$params$lambda_l2
[1] 5

$params$learning_rate
[1] 0.1

$params$force_col_wise
[1] TRUE

$params$nrounds
[1] 7000