Landsat FIA AGB 0.0.99: Final Countdown

Evaluation Results

Last version of FIA models: 2021-07-19 High Class

Change Summary

• This version will be used in the 1.0.0 ensemble.
• Uses LCMAP 1.2 (upgrade from 1.1), new disturbance metrics (re-segmented), and no plots with non-forest conditions

	RF (ranger)	GBM (LightGBM)	SVM (kernlab)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	59.360	59.994	57.352	56.990	58.112
MBE	0.732	-0.383	-4.375	-1.288	-1.359
R2	0.303	0.283	0.347	0.351	0.334

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00   89.95  134.63  136.85  179.18  425.00 

Validation Results

RMSE	Min	Median	Max
Rf	53.590	56.972	60.422
Lgb	54.247	57.070	60.789
Svm	52.304	55.653	58.862
Ensemble	52.581	55.771	58.898

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.231	0.298	0.359
lgb	0.216	0.284	0.354
svm	0.253	0.318	0.385
ensemble	0.258	0.325	0.384

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf       svm 
0.3301731 0.3325303 0.3372966 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred * svm_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-169.619  -37.546   -3.781   32.045  254.176 

Coefficients:
                              Estimate   Std. Error t value
(Intercept)               30.906801545  7.364339483   4.197
rf_pred                    0.025249766  0.131872587   0.191
lgb_pred                  -0.737581177  0.128942477  -5.720
svm_pred                   0.456834954  0.073163585   6.244
rf_pred:lgb_pred           0.005798133  0.000785825   7.378
rf_pred:svm_pred           0.000953535  0.000852493   1.119
lgb_pred:svm_pred          0.005575557  0.000938360   5.942
rf_pred:lgb_pred:svm_pred -0.000034029  0.000003859  -8.818
                                   Pr(>|t|)    
(Intercept)               0.000027124191481 ***
rf_pred                               0.848    
lgb_pred                  0.000000010709321 ***
svm_pred                  0.000000000430506 ***
rf_pred:lgb_pred          0.000000000000163 ***
rf_pred:svm_pred                      0.263    
lgb_pred:svm_pred         0.000000002840873 ***
rf_pred:lgb_pred:svm_pred           < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 55 on 42292 degrees of freedom
Multiple R-squared:  0.3303,    Adjusted R-squared:  0.3302 
F-statistic:  2980 on 7 and 42292 DF,  p-value: < 2.2e-16

Coverages

• 0 coverages:

\(n\) and \(p\)

• 2015 observations
  • 1410 training
  • 605 testing
• 26 predictors
  • nbr, delta_nbr, tcb, delta_tcb, tcg, delta_tcg, tcw, delta_tcw, mag, ysd, dem, slope, aspect, twi, tmin, tmax, precip, chm, lcpri4, lcpri2, lcpri6, lcpri3, lcsec3, lcsec1, lcsec2, lcsec4

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1500

$mtry
[1] 7

$min.node.size
[1] 3

$sample.fraction
[1] 0.7

$splitrule
[1] "variance"

$replace
[1] TRUE

$formula
agb_mgha ~ .

LGB:

$nrounds
[1] 50

$params
$params$learning_rate
[1] 0.05

$params$num_leaves
[1] 13

$params$max_depth
[1] -1

$params$extra_trees
[1] FALSE

$params$min_data_in_leaf
[1] 10

$params$bagging_fraction
[1] 0.4

$params$bagging_freq
[1] 1

$params$feature_fraction
[1] 1

$params$min_data_in_bin
[1] 18

$params$lambda_l1
[1] 3

$params$lambda_l2
[1] 0

$params$force_col_wise
[1] TRUE

SVM:

$x
agb_mgha ~ .

$kernel
[1] "laplacedot"

$type
[1] "eps-svr"

$kpar
$kpar$sigma
[1] 0.0078125


$C
[1] 16

$epsilon
[1] 0.25

$nu
[1] 0.2