Landsat:Lidar AGB 0.0.6: LCMAP Stratification

Hudak-method models stratified by LCMAP and coverage area. 2021-08-19

Evaluation Results

Last iteration: 2021-07-18 - Landsat AGB 0.0.5: High Class

Change Summary

• Sampling is now stratified by LCMAP primary classification and surface area: 0.01% of pixels within each LCMAP class within each surface area is selected (so 0.01% of the entire surface is selected)
• No stratification by AGB

	RF (ranger)	GBM (LightGBM)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	32.616	32.513	32.133	32.243
RMSE %	70.490	70.268	69.446	69.683
MBE	0.317	-0.234	-0.209	0.041
R2	0.671	0.670	0.678	0.678

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00    0.00   18.31   45.67   79.07  278.74 

Validation Results

RMSE	Min	Median	Max
Rf	30.806	32.331	33.779
Lgb	30.971	32.385	33.941
Ensemble	30.526	32.008	33.458

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.639	0.675	0.701
lgb	0.644	0.672	0.695
ensemble	0.650	0.680	0.705

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf 
0.4993076 0.5006924 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-173.145  -14.771   -2.926   10.532  205.452 

Coefficients:
                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)      -1.530e+00  1.238e-01  -12.36   <2e-16 ***
rf_pred           5.617e-01  8.027e-03   69.99   <2e-16 ***
lgb_pred          4.115e-01  7.825e-03   52.59   <2e-16 ***
rf_pred:lgb_pred  6.089e-04  3.512e-05   17.34   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 31.93 on 226496 degrees of freedom
Multiple R-squared:  0.6797,    Adjusted R-squared:  0.6797 
F-statistic: 1.602e+05 on 3 and 226496 DF,  p-value: < 2.2e-16

Coverages

• 18 coverages:
  • 3_county, allegany_steuben, cayuga_oswego, clinton_essex_franklin, columbia_rensselaer, erie_genesee_livingston, franklin_st_lawrence, fulton_saratoga_herkimer_franklin, great_lakes, hudson_hoosic, long_island, madison_otsego, north_east, oneida_subbasin, scoharie, southwest_b, southwest, warren_washington_essex

\(n\) and \(p\)

• 10801 observations
  • 7551 training
  • 3250 testing
• 34 predictors
  • tcb, tcw, tcg, nbr, mag, ysd, chm_simard, nys_precip, nys_tmax, nys_tmin, nys_aspect, nys_dem, nys_slope, nys_twi, delta_tcb, delta_tcw, delta_tcg, delta_nbr, lcpri1, lcpri2, lcpri3, lcpri4, lcpri5, lcpri6, lcpri8, lcsec2, lcsec8, lcsec4, lcsec3, lcsec5, lcsec1, lcsec6, lcpri0, lcsec0

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 500

$mtry
[1] 6

$min.node.size
[1] 3

$sample.fraction
[1] 1

$splitrule
[1] "variance"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.1

$nrounds
[1] 100

$num_leaves
[1] 24

$max_depth
[1] -1

$extra_trees
[1] TRUE

$min_data_in_leaf
[1] 10

$bagging_fraction
[1] 0.9

$bagging_freq
[1] 1

$feature_fraction
[1] 0.7

$min_data_in_bin
[1] 18

$lambda_l1
[1] 2

$lambda_l2
[1] 0.1

$force_col_wise
[1] TRUE