Landsat AGB 0.0.4: Do it Better

Landsat models actually built using 5 land cover types. Using 19 LiDAR coverages. 2021-07-13

Evaluation Results

Last iteration: 2021-07-03 - Landsat AGB 0.0.3: Big Tent

Change Summary

• Iteration 0.0.3 only excluded developed lands.
• This version properly excludes developed, water, and barren LCMAP classes

	RF (ranger)	GBM (LightGBM)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	46.219	43.481	43.346	43.818
RMSE %	32.015	30.119	30.025	30.352
MBE	0.590	0.485	0.281	0.536
R2	0.701	0.732	0.733	0.730

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
  0.00188  72.45654 143.93385 144.21586 216.48722 306.56705 

Validation Results

RMSE	Min	Median	Max
Rf	46.053	46.850	47.670
Lgb	43.779	44.599	45.693
Ensemble	43.805	44.630	45.441

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.672	0.685	0.699
lgb	0.699	0.711	0.723
ensemble	0.701	0.713	0.726

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf 
0.5137954 0.4862046 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-248.799  -26.644    0.159   26.924  260.166 

Coefficients:
                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)      -1.023e+00  1.971e-01  -5.192 2.08e-07 ***
rf_pred           2.647e-01  2.963e-03  89.320  < 2e-16 ***
lgb_pred          7.023e-01  2.884e-03 243.507  < 2e-16 ***
rf_pred:lgb_pred  2.188e-04  1.023e-05  21.388  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 44.33 on 892096 degrees of freedom
Multiple R-squared:  0.7149,    Adjusted R-squared:  0.7149 
F-statistic: 7.458e+05 on 3 and 892096 DF,  p-value: < 2.2e-16

Coverages

• 18 coverages:
  • 3_county, allegany_steuben, cayuga_oswego, clinton_essex_franklin, columbia_rensselaer, erie_genesee_livingston, franklin_st_lawrence, fulton_saratoga_herkimer_franklin, great_lakes, hudson_hoosic, long_island, madison_otsego, north_east, oneida_subbasin, scoharie, southwest_b, southwest, warren_washington_essex

\(n\) and \(p\)

• 42494 observations
  • 29738 training
  • 12756 testing
• 18 predictors
  • tcb, tcw, tcg, nbr, mag, ysd, chm_simard, nys_precip, nys_tmax, nys_tmin, nys_aspect, nys_dem, nys_slope, nys_twi, delta_tcb, delta_tcw, delta_tcg, delta_nbr

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 4

$min.node.size
[1] 2

$sample.fraction
[1] 1

$splitrule
[1] "variance"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.1

$nrounds
[1] 2000

$num_leaves
[1] 13

$max_depth
[1] -1

$extra_trees
[1] FALSE

$min_data_in_leaf
[1] 10

$bagging_fraction
[1] 0.9

$bagging_freq
[1] 1

$feature_fraction
[1] 0.9

$min_data_in_bin
[1] 3

$lambda_l1
[1] 7

$lambda_l2
[1] 7

$force_col_wise
[1] TRUE