Landsat AGB: Early Efforts

Evaluation Results

Change Summary

• Refactored sampling to be more efficient
  • Side effect: sampling now uses 100% of cells in bins with < 500 values

	RF (ranger)	GBM (LightGBM)	SVM (kernlab)	Ensemble (model weighted)	Ensemble (RMSE weighted)
RMSE	40.290	36.368	37.739	35.425	35.988
MBE	-0.120	0.363	0.307	-0.193	0.192
R2	0.756	0.799	0.784	0.810	0.804

AGB Distribution

summary(bind_rows(training, testing)$agb_mgha)

     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
  0.00232  71.11791 141.31125 140.99832 210.46496 294.52237 

Validation Results

RMSE	Min	Median	Max
Rf	38.862	40.851	42.229
Lgb	35.580	37.056	38.503
Svm	37.588	39.025	40.645
Ensemble	35.378	36.791	38.210

“Ensemble” here refers to the RMSE-weighted ensemble; the linear model ensemble is trained using the predictions from the validation stage (so doesn’t have any “test” data until we do the final evaluation).

R2	Min	Median	Max
rf	0.730	0.751	0.777
lgb	0.771	0.790	0.809
svm	0.749	0.768	0.785
ensemble	0.780	0.796	0.813

Metadata

Ensembles

• RMSE-weighted model weights:

      lgb        rf       svm 
0.3501030 0.3145027 0.3353944 

• Linear model weights:

Call:
lm(formula = agb_mgha ~ rf_pred * lgb_pred * svm_pred, data = pred_values)

Residuals:
     Min       1Q   Median       3Q      Max 
-181.663  -19.332    0.516   20.048  219.547 

Coefficients:
                            Estimate Std. Error t value Pr(>|t|)    
(Intercept)                5.879e+00  4.911e-01  11.971  < 2e-16 ***
rf_pred                    8.275e-02  1.239e-02   6.680  2.4e-11 ***
lgb_pred                   3.224e-01  1.354e-02  23.806  < 2e-16 ***
svm_pred                   2.688e-01  1.130e-02  23.783  < 2e-16 ***
rf_pred:lgb_pred           1.281e-03  7.957e-05  16.098  < 2e-16 ***
rf_pred:svm_pred          -2.022e-04  8.806e-05  -2.296   0.0217 *  
lgb_pred:svm_pred          1.856e-03  7.800e-05  23.796  < 2e-16 ***
rf_pred:lgb_pred:svm_pred -6.952e-06  2.557e-07 -27.184  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 36.2 on 201292 degrees of freedom
Multiple R-squared:  0.7998,    Adjusted R-squared:  0.7998 
F-statistic: 1.149e+05 on 7 and 201292 DF,  p-value: < 2.2e-16

Coverages

• 0 coverages:

\(n\) and \(p\)

• 9591 observations
  • 6713 training
  • 2878 testing
• 18 predictors
  • precip.tif, tmax.tif, tmin.tif, aspect.tif, dem.tif, slope.tif, twi.tif, lt_tcb, lt_tcw, lt_tcg, lt_nbr, delta_tcb, delta_tcw, delta_tcg, delta_nbr, ysd, mag, chm

Component Models

• Tuning used 5-fold CV
• Final hyperparameters:

Random forest:

$num.trees
[1] 1000

$mtry
[1] 7

$min.node.size
[1] 1

$sample.fraction
[1] 0.95

$splitrule
[1] "maxstat"

$replace
[1] FALSE

$formula
agb_mgha ~ .

LGB:

$learning_rate
[1] 0.1

$nrounds
[1] 500

$num_leaves
[1] 23

$max_depth
[1] -1

$extra_trees
[1] FALSE

$min_data_in_leaf
[1] 10

$bagging_fraction
[1] 0.8

$bagging_freq
[1] 1

$feature_fraction
[1] 0.8

$min_data_in_bin
[1] 23

$lambda_l1
[1] 0.5

$lambda_l2
[1] 6

$force_col_wise
[1] TRUE

SVM:

$x
agb_mgha ~ .

$kernel
[1] "laplacedot"

$type
[1] "eps-svr"

$kpar
$kpar$sigma
[1] 0.00390625


$C
[1] 256

$epsilon
[1] 0.03125

$nu
[1] 0.2