Landsat FIA AGB - Take Two

Evaluation Results

Change Summary

• Second edition using LANDSAT predictors trained using FIA plot data
  • Used newest FIA plot measurements from 2013 and newer (to match LiDAR years)
• These models no longer cheat by having plot IDs, counties, or inventory years (whoops)

	RF (ranger)	GBM (LightGBM)	SVM (kernlab)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	50.025	49.288	48.394	47.751	48.224
MBE	0.152	0.971	1.983	0.812	1.040
R2	0.617	0.626	0.640	0.648	0.644

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00    0.00   61.56   79.10  139.79  425.00 

Validation Results

RMSE	Min	Median	Max
Rf	47.160	51.792	55.203
Lgb	46.049	50.343	54.077
Svm	46.079	50.407	54.243
Ensemble	45.133	49.831	53.484

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.564	0.605	0.654
lgb	0.578	0.619	0.678
svm	0.575	0.621	0.664
ensemble	0.596	0.634	0.685

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf       svm 
0.3374801 0.3282720 0.3342480 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred * svm_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-179.438  -25.021   -3.747   17.404  245.196 

Coefficients:
                            Estimate Std. Error t value Pr(>|t|)    
(Intercept)                5.777e-01  4.069e-01   1.420  0.15569    
rf_pred                    1.248e-01  3.827e-02   3.260  0.00112 ** 
lgb_pred                  -1.392e-01  4.548e-02  -3.060  0.00221 ** 
svm_pred                   3.928e-01  1.961e-02  20.027  < 2e-16 ***
rf_pred:lgb_pred           3.809e-03  2.118e-04  17.981  < 2e-16 ***
rf_pred:svm_pred          -1.871e-03  3.933e-04  -4.757 1.96e-06 ***
lgb_pred:svm_pred          8.120e-03  4.502e-04  18.036  < 2e-16 ***
rf_pred:lgb_pred:svm_pred -3.829e-05  1.505e-06 -25.445  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 49.04 on 92492 degrees of freedom
Multiple R-squared:  0.6382,    Adjusted R-squared:  0.6382 
F-statistic: 2.331e+04 on 7 and 92492 DF,  p-value: < 2.2e-16

Coverages

• 0 coverages:

\(n\) and \(p\)

• 4406 observations
  • 3084 training
  • 1322 testing
• 18 predictors
  • nbr, delta_nbr, tcb, delta_tcb, tcg, delta_tcg, tcw, delta_tcw, mag, ysd, dem, slope, aspect, twi, tmin, tmax, precip, chm

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 5

$min.node.size
[1] 3

$sample.fraction
[1] 0.7

$splitrule
[1] "maxstat"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.05

$nrounds
[1] 100

$num_leaves
[1] 9

$max_depth
[1] -1

$extra_trees
[1] TRUE

$min_data_in_leaf
[1] 10

$bagging_fraction
[1] 0.7

$bagging_freq
[1] 5

$feature_fraction
[1] 1

$min_data_in_bin
[1] 13

$lambda_l1
[1] 0.3

$lambda_l2
[1] 2

$force_col_wise
[1] TRUE

SVM:

$x
agb_mgha ~ .

$kernel
[1] "laplacedot"

$type
[1] "eps-svr"

$kpar
$kpar$sigma
[1] 0.03125


$C
[1] 16

$epsilon
[1] 0.25

$nu
[1] 0.2