1.4.0: A Little Adjustment

Lidar-based models adding 0 AGB with low-LiDAR back in to the mix

Evaluation Results

Last iteration of these models: 2021-11-19

Change Summary

• Including plots with LiDAR below 1m and no AGB

	RF (ranger)	GBM (LightGBM)	SVM (kernlab)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	38.199	40.070	38.749	38.221	38.390
MBE	1.046	0.650	-4.556	0.221	-0.937
R2	0.775	0.759	0.774	0.775	0.776

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00   35.69  109.16  107.84  165.65  425.00 

Bootstrapping Results

Across 1000 bootstrap iterations, our ensemble model had a mean RMSE of 38.531 \(\pm\) 0.297.

RMSE Distribution

Plot Errors

Validation Results

RMSE	Min	Median	Max
Rf	31.559	38.625	46.105
Lgb	32.685	39.510	46.855
Svm	33.450	40.340	67.592
Ensemble	31.968	38.685	46.470

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.690	0.767	0.837
lgb	0.682	0.756	0.833
svm	0.302	0.744	0.820
ensemble	0.693	0.765	0.840

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf       svm 
0.3293589 0.3399255 0.3307156 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred * svm_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-132.081  -21.166    0.255   15.756  208.553 

Coefficients:
                               Estimate    Std. Error t value
(Intercept)               -12.902189382   1.493952188  -8.636
rf_pred                     0.329128827   0.101532909   3.242
lgb_pred                    0.916892912   0.111533351   8.221
svm_pred                    0.150743284   0.054204286   2.781
rf_pred:lgb_pred           -0.001388922   0.000447318  -3.105
rf_pred:svm_pred            0.003080403   0.000619760   4.970
lgb_pred:svm_pred          -0.005375208   0.000882657  -6.090
rf_pred:lgb_pred:svm_pred   0.000010535   0.000001671   6.303
                                Pr(>|t|)    
(Intercept)                      < 2e-16 ***
rf_pred                          0.00119 ** 
lgb_pred                         < 2e-16 ***
svm_pred                         0.00542 ** 
rf_pred:lgb_pred                 0.00191 ** 
rf_pred:svm_pred          0.000000675064 ***
lgb_pred:svm_pred         0.000000001155 ***
rf_pred:lgb_pred:svm_pred 0.000000000299 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 38.81 on 16692 degrees of freedom
Multiple R-squared:  0.763, Adjusted R-squared:  0.7629 
F-statistic:  7676 on 7 and 16692 DF,  p-value: < 2.2e-16

Coverages

• 18 coverages:
  • FEMA_FranklinStLawrence2016, FEMA_FultonSaratogaHerkimerFranklin2017, FEMA_GreatLakes2014, FEMA_OneidaSubbasin2016, NYSGPO_AlleganySteuben2016, NYSGPO_CayugaOswego_2018, NYSGPO_ColumbiaRensselaer2016, NYSGPO_ErieGeneseeLivingston2019, NYSGPO_MadisonOtsego_2015, NYSGPO_Southwest_spring_2017, NYSGPO_SouthwestB_fall_2017, NYSGPO_WarrenWashingtonEssex_2015, USGS_3County2014, USGS_ClintonEssexFranklin2014, USGS_LongIsland2014, USGS_NorthEast2011, USGS_NYC2014, USGS_Schoharie2014

\(n\) and \(p\)

• 811 observations
  • 559 training
  • 252 testing
• 84 predictors
  • zmean, zmean_c, max, quad_mean, quad_mean_c, cv, cv_c, z_kurt, z_skew, L2, L3, L4, L_cv, L_skew, L_kurt, h10, h20, h30, h40, h50, h60, h70, h80, h90, h95, h99, hvol, cancov, rpc1, d10, d20, d30, d40, d50, d60, d70, d80, d90, precip, tmin, tmax, twi, slope, aspect, elev, tax_code_105, tax_code_112, tax_code_120, tax_code_210, tax_code_240, tax_code_241, tax_code_260, tax_code_311, tax_code_312, tax_code_321, tax_code_322, tax_code_323, tax_code_910, tax_code_911, tax_code_912, tax_code_920, tax_code_930, tax_code_931, tax_code_932, tax_code_941, tax_code_961, tax_code_1000, tax_category_100, tax_category_200, tax_category_300, tax_category_900, lcpri4, lcpri6, lcpri3, lcpri2, lcpri5, lcsec3, lcsec2, lcsec1, lcsec4, lcsec5, lcsec8, lcpri1, lcpri8

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 34

$min.node.size
[1] 3

$sample.fraction
[1] 0.15

$splitrule
[1] "variance"

$replace
[1] TRUE

$formula
agb_mgha ~ .

LGB:

$nrounds
[1] 50

$params
$params$learning_rate
[1] 0.05

$params$num_leaves
[1] 6

$params$max_depth
[1] -1

$params$extra_trees
[1] FALSE

$params$min_data_in_leaf
[1] 10

$params$bagging_fraction
[1] 0.7

$params$bagging_freq
[1] 10

$params$feature_fraction
[1] 1

$params$min_data_in_bin
[1] 8

$params$lambda_l1
[1] 1

$params$lambda_l2
[1] 4

$params$force_col_wise
[1] TRUE

SVM:

$x
agb_mgha ~ zmean + zmean_c + max + quad_mean + quad_mean_c + 
    cv + cv_c + z_kurt + z_skew + L2 + L3 + L4 + L_cv + L_skew + 
    L_kurt + h10 + h20 + h30 + h40 + h50 + h60 + h70 + h80 + 
    h90 + h95 + h99 + hvol + cancov + rpc1 + d10 + d20 + d30 + 
    d40 + d50 + d60 + d70 + d80 + d90 + precip + tmin + tmax + 
    twi + slope + aspect + elev + tax_code_105 + tax_code_112 + 
    tax_code_120 + tax_code_210 + tax_code_240 + tax_code_241 + 
    tax_code_260 + tax_code_311 + tax_code_312 + tax_code_321 + 
    tax_code_322 + tax_code_323 + tax_code_910 + tax_code_911 + 
    tax_code_912 + tax_code_920 + tax_code_930 + tax_code_931 + 
    tax_code_932 + tax_code_941 + tax_code_961 + tax_code_1000 + 
    tax_category_100 + tax_category_200 + tax_category_300 + 
    tax_category_900

$kernel
[1] "laplacedot"

$type
[1] "eps-svr"

$kpar
$kpar$sigma
[1] 0.001953125


$C
[1] 9

$epsilon
[1] 0.00390625