Landsat FIA AGB 0.0.5: High Class

Evaluation Results

Last version of FIA models: 2021-07-06 Take Two

Comparison Hudak method: 2021-07-19 0.0.5 High Class

Change Summary

• This version includes boolean variables for each LCMAP primary and secondary land cover classification.

	RF (ranger)	GBM (LightGBM)	SVM (kernlab)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	46.998	47.494	48.081	46.714	46.540
MBE	1.058	0.154	-0.843	0.770	0.134
R2	0.666	0.658	0.650	0.669	0.673

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00    0.00   61.55   79.01  139.60  425.00 

Validation Results

RMSE	Min	Median	Max
Rf	45.144	48.646	51.930
Lgb	45.090	48.580	52.525
Svm	45.039	50.191	88.470
Ensemble	44.548	48.860	57.898

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.603	0.644	0.682
lgb	0.595	0.641	0.681
svm	0.173	0.628	0.670
ensemble	0.604	0.652	0.690

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf       svm 
0.3326046 0.3396057 0.3277897 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred * svm_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-189.558  -21.504   -4.116   16.588  277.471 

Coefficients:
                            Estimate Std. Error t value Pr(>|t|)    
(Intercept)                1.259e-01  4.117e-01   0.306  0.75969    
rf_pred                    2.317e-01  3.186e-02   7.270 3.61e-13 ***
lgb_pred                   3.614e-01  3.370e-02  10.724  < 2e-16 ***
svm_pred                   2.991e-02  1.096e-02   2.729  0.00636 ** 
rf_pred:lgb_pred           2.253e-03  1.317e-04  17.112  < 2e-16 ***
rf_pred:svm_pred           3.978e-03  3.343e-04  11.898  < 2e-16 ***
lgb_pred:svm_pred         -3.098e-04  3.402e-04  -0.911  0.36251    
rf_pred:lgb_pred:svm_pred -2.107e-05  1.246e-06 -16.911  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 48.18 on 91992 degrees of freedom
Multiple R-squared:  0.6503,    Adjusted R-squared:  0.6502 
F-statistic: 2.443e+04 on 7 and 91992 DF,  p-value: < 2.2e-16

Coverages

• 0 coverages:

\(n\) and \(p\)

• 4385 observations
  • 3069 training
  • 1316 testing
• 32 predictors
  • nbr, delta_nbr, tcb, delta_tcb, tcg, delta_tcg, tcw, delta_tcw, mag, ysd, dem, slope, aspect, twi, tmin, tmax, precip, chm, lcpri4, lcpri2, lcpri5, lcpri1, lcpri8, lcpri6, lcpri3, lcsec3, lcsec4, lcsec1, lcsec2, lcsec8, lcsec5, lcsec6

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 10

$min.node.size
[1] 2

$sample.fraction
[1] 1

$splitrule
[1] "variance"

$replace
[1] TRUE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.1

$nrounds
[1] 50

$num_leaves
[1] 14

$max_depth
[1] -1

$extra_trees
[1] TRUE

$min_data_in_leaf
[1] 10

$bagging_fraction
[1] 0.9

$bagging_freq
[1] 10

$feature_fraction
[1] 0.9

$min_data_in_bin
[1] 13

$lambda_l1
[1] 2

$lambda_l2
[1] 2

$force_col_wise
[1] TRUE

SVM:

$x
agb_mgha ~ .

$kernel
[1] "laplacedot"

$type
[1] "eps-svr"

$kpar
$kpar$sigma
[1] 0.03125


$C
[1] 16

$epsilon
[1] 0.0625

$nu
[1] 0.2