Landsat FIA AGB

Evaluation Results

Change Summary

• First edition using LANDSAT predictors trained using FIA plot data
  • Used newest FIA plot measurements from 2013 and newer (to match LiDAR years)

	RF (ranger)	GBM (LightGBM)	SVM (kernlab)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	50.440	49.085	48.579	47.824	48.244
MBE	0.169	0.295	-3.238	0.653	-0.923
R2	0.613	0.628	0.639	0.647	0.644

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00    0.00   61.56   79.10  139.79  425.00 

Validation Results

RMSE	Min	Median	Max
Rf	47.604	52.243	55.840
Lgb	46.297	50.332	53.434
Svm	46.729	51.582	55.886
Ensemble	45.509	50.254	53.854

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.557	0.599	0.646
lgb	0.585	0.618	0.679
svm	0.563	0.604	0.657
ensemble	0.590	0.628	0.682

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf       svm 
0.3390723 0.3279463 0.3329814 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred * svm_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-178.625  -26.094   -3.477   17.530  245.788 

Coefficients:
                            Estimate Std. Error t value Pr(>|t|)    
(Intercept)                1.201e+00  4.049e-01   2.966  0.00302 ** 
rf_pred                   -1.412e-01  2.824e-02  -5.003 5.67e-07 ***
lgb_pred                   2.634e-01  4.072e-02   6.468 1.00e-10 ***
svm_pred                   5.302e-01  2.300e-02  23.055  < 2e-16 ***
rf_pred:lgb_pred           4.032e-03  2.424e-04  16.635  < 2e-16 ***
rf_pred:svm_pred          -4.601e-04  2.958e-04  -1.556  0.11979    
lgb_pred:svm_pred          3.202e-03  3.888e-04   8.235  < 2e-16 ***
rf_pred:lgb_pred:svm_pred -2.878e-05  1.704e-06 -16.896  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 49.43 on 92492 degrees of freedom
Multiple R-squared:  0.6324,    Adjusted R-squared:  0.6324 
F-statistic: 2.273e+04 on 7 and 92492 DF,  p-value: < 2.2e-16

Coverages

• 0 coverages:

\(n\) and \(p\)

• 4406 observations
  • 3084 training
  • 1322 testing
• 21 predictors
  • PLOT, COUNTYCD, nbr, delta_nbr, tcb, delta_tcb, tcg, delta_tcg, tcw, delta_tcw, mag, ysd, dem, slope, aspect, twi, tmin, tmax, precip, chm, invyr

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 500

$mtry
[1] 5

$min.node.size
[1] 5

$sample.fraction
[1] 0.55

$splitrule
[1] "maxstat"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.05

$nrounds
[1] 100

$num_leaves
[1] 13

$max_depth
[1] -1

$extra_trees
[1] FALSE

$min_data_in_leaf
[1] 8

$bagging_fraction
[1] 0.4

$bagging_freq
[1] 1

$feature_fraction
[1] 0.9

$min_data_in_bin
[1] 3

$lambda_l1
[1] 5

$lambda_l2
[1] 3

$force_col_wise
[1] TRUE

SVM:

$x
agb_mgha ~ .

$kernel
[1] "laplacedot"

$type
[1] "eps-svr"

$kpar
$kpar$sigma
[1] 0.0078125


$C
[1] 32

$epsilon
[1] 0.0078125

$nu
[1] 0.2