Landsat AGB 0.0.5: High Class

Hudak-method models using LCMAP primary and secondary classifications as predictors. 2021-07-19

Evaluation Results

Last iteration: 2021-07-13 - Landsat AGB 0.0.4: Do it Better

Change Summary

• This version includes boolean variables for each LCMAP primary and secondary land cover classification.
• Sampling is done independent of land cover classification – we’re still sampling 0.05% of each surface split into 20 bins, with no attention paid to the new variables

	RF (ranger)	GBM (LightGBM)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	46.541	42.049	41.969	42.790
RMSE %	32.398	29.271	29.216	29.787
MBE	-0.001	-0.150	-0.283	-0.079
R2	0.693	0.744	0.745	0.738

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
  0.00011  71.90615 143.20532 143.63437 215.93515 306.56705 

Validation Results

RMSE	Min	Median	Max
Rf	47.012	47.715	48.489
Lgb	42.442	43.184	44.060
Ensemble	43.219	43.851	44.603

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.665	0.679	0.689
lgb	0.723	0.732	0.743
ensemble	0.717	0.727	0.736

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf 
0.5239175 0.4760825 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-243.469  -25.489   -0.016   25.922  277.713 

Coefficients:
                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)      -1.818e-01  1.816e-01  -1.001    0.317    
rf_pred           1.372e-01  2.499e-03  54.904   <2e-16 ***
lgb_pred          8.311e-01  2.538e-03 327.406   <2e-16 ***
rf_pred:lgb_pred  1.836e-04  9.875e-06  18.595   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 43.04 on 955696 degrees of freedom
Multiple R-squared:  0.7338,    Adjusted R-squared:  0.7338 
F-statistic: 8.781e+05 on 3 and 955696 DF,  p-value: < 2.2e-16

Coverages

• 18 coverages:
  • 3_county, allegany_steuben, cayuga_oswego, clinton_essex_franklin, columbia_rensselaer, erie_genesee_livingston, franklin_st_lawrence, fulton_saratoga_herkimer_franklin, great_lakes, hudson_hoosic, long_island, madison_otsego, north_east, oneida_subbasin, scoharie, southwest_b, southwest, warren_washington_essex

\(n\) and \(p\)

• 45523 observations
  • 31858 training
  • 13665 testing
• 32 predictors
  • tcb, tcw, tcg, nbr, mag, ysd, chm_simard, nys_precip, nys_tmax, nys_tmin, nys_aspect, nys_dem, nys_slope, nys_twi, delta_tcb, delta_tcw, delta_tcg, delta_nbr, lcpri2, lcpri6, lcpri3, lcpri1, lcpri5, lcpri4, lcpri8, lcsec1, lcsec4, lcsec3, lcsec2, lcsec8, lcsec6, lcsec5

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 8

$min.node.size
[1] 2

$sample.fraction
[1] 1

$splitrule
[1] "maxstat"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.1

$nrounds
[1] 1500

$num_leaves
[1] 23

$max_depth
[1] -1

$extra_trees
[1] FALSE

$min_data_in_leaf
[1] 10

$bagging_fraction
[1] 0.9

$bagging_freq
[1] 1

$feature_fraction
[1] 0.9

$min_data_in_bin
[1] 3

$lambda_l1
[1] 9

$lambda_l2
[1] 8

$force_col_wise
[1] TRUE