Changes
This round differs from the last iteration in the following ways:
- Predictors:
Our landsat-ensemble modeled AGB was used as a predictor in place of FIA plot-level AGB measurements.
Some landsat derived indices (TCB, TCW, TCG, NDVI, NSDS, YOD (years-since-disturbance), MAG (magnitude-of-disturbance)) were added as predictors
Some LCMAP derived predictors were added (LCPRI, LCSEC, num_unique_lcpri (number of unique lcpri classes for a given pixel))
Landsat-AGB predictors were added (1 year delta, and standard deviation of all previous AGB values at that pixel)
- Models:
- The RF models for deadwood, litter, and soil seemed to be suffering from overfitting given the drastic drop-off in performance (1:1 plots) from the train set to the test set. This is likely a result of small n. In particular note the RF train vs test results for the deadwood models.
- I included PC regression as an alternative, with components selected to include 99% of the variance in the predictor data.
- For litter and soil models I opted for a 10-fold-CV approach coupled with PC regression given the very limited amount of data. In these instances 1:1 plots and accuracy results are given for the CV results.
AGC Model
- \(n\) train: 4540
- \(n\) test: 1113
- Predictors: agb, delta_agb, sd_agb, n_unique_lcpri, mag, nbr, ndvi, nsds, tcb, tcg, tcw, yod, dem, slope, aspect, twi, tmin, tmax, precip, chm, lcpri4, lcpri2, lcpri1, lcpri8, lcpri6, lcpri5, lcpri3, lcsec3, lcsec4, lcsec2, lcsec1, lcsec8, lcsec5
Random Forest Model - Test Set 1:1
Params
$formula
agc ~ .
$num.trees
[1] 1250
$mtry
[1] 18
$min.node.size
[1] 1
$replace
[1] TRUE
$sample.fraction
[1] 0.3Test Set Accuracy
- Mean-model RMSE: 38.884
| Prediction | |
|---|---|
| RMSE | 26.848 |
| MAE | 18.101 |
| MBE | -0.523 |
| R2 | 0.523 |
BGC Model
- \(n\) train: 2790
- \(n\) test: 690
- Predictors: agb, delta_agb, sd_agb, n_unique_lcpri, mag, nbr, ndvi, nsds, tcb, tcg, tcw, yod, dem, slope, aspect, twi, tmin, tmax, precip, chm, lcpri4, lcpri2, lcpri1, lcpri6, lcpri5, lcpri3, lcsec3, lcsec4, lcsec2, lcsec1, lcsec8
Random Forest Model - Test Set 1:1
Params
$formula
bgc ~ .
$num.trees
[1] 750
$mtry
[1] 14
$min.node.size
[1] 1
$replace
[1] TRUE
$sample.fraction
[1] 0.8Test Set Accuracy
- Mean-model RMSE: 7.181
| Prediction | |
|---|---|
| RMSE | 5.240 |
| MAE | 4.064 |
| MBE | 0.083 |
| R2 | 0.467 |
Deadwood C Model
- Outliers >= [(1.5 * IQR) + 0.75th percentile] of deadwood
measurements filtered: 25
- Outlier threshold = [(1.5 * IQR) + 0.75th percentile] = 22.479
- \(n\) train: 475
- \(n\) test: 122
- Predictors: agb, delta_agb, sd_agb, n_unique_lcpri, mag, nbr, ndvi, nsds, tcb, tcg, tcw, yod, dem, slope, aspect, twi, tmin, tmax, precip, chm, lcpri2, lcpri4, lcpri6, lcsec4, lcsec2, lcsec3, lcsec1
Random Forest Model - Training Set 1:1
Random Forest Model - Test Set 1:1
Params
$formula
deadwood ~ .
$num.trees
[1] 250
$mtry
[1] 3
$min.node.size
[1] 1
$replace
[1] FALSE
$sample.fraction
[1] 0.9PC Regression - Test set 1:1
Params
Call:
lm(formula = pc_train_deadwood$deadwood ~ ., data = pc_train_deadwood[,
1:which(props > 0.99)[1]])
Residuals:
Min 1Q Median 3Q Max
-8.7386 -3.4746 -0.8526 2.5924 17.2229
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 5.95494 2.74589 2.169 0.03063 *
PC1 -0.39452 0.80037 -0.493 0.62230
PC2 0.73318 0.57908 1.266 0.20612
PC3 -0.06246 0.18406 -0.339 0.73449
PC4 -0.79785 0.13087 -6.097 2.33e-09 ***
PC5 -0.45798 0.16133 -2.839 0.00473 **
PC6 0.03258 0.16349 0.199 0.84216
PC7 0.20250 0.46855 0.432 0.66581
PC8 0.21959 0.30954 0.709 0.47844
PC9 0.10898 0.23191 0.470 0.63863
PC10 0.10103 0.21204 0.476 0.63396
PC11 0.09455 0.24413 0.387 0.69873
PC12 0.20723 0.22736 0.911 0.36253
PC13 -0.65312 0.26627 -2.453 0.01455 *
PC14 -0.42890 0.25317 -1.694 0.09093 .
PC15 -0.17342 0.28309 -0.613 0.54044
PC16 -0.51297 0.29701 -1.727 0.08483 .
PC17 0.82333 0.31087 2.648 0.00837 **
PC18 0.16248 0.37297 0.436 0.66330
PC19 0.27971 0.41584 0.673 0.50151
PC20 0.56508 0.50417 1.121 0.26296
PC21 -0.77770 0.54989 -1.414 0.15797
PC22 0.06209 0.57659 0.108 0.91430
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 4.701 on 452 degrees of freedom
Multiple R-squared: 0.2529, Adjusted R-squared: 0.2166
F-statistic: 6.956 on 22 and 452 DF, p-value: < 2.2e-16Test Set Accuracy
- Mean-model RMSE: 5.367
| lm | RF | |
|---|---|---|
| RMSE | 5.159 | 4.669 |
| MAE | 4.128 | 3.651 |
| MBE | 0.172 | 0.096 |
| R2 | 0.069 | 0.237 |
Litter C models
- \(n\) train: 120
- \(n\) test: 37
- Predictors: agb, delta_agb, sd_agb, n_unique_lcpri, mag, nbr, ndvi, nsds, tcb, tcg, tcw, yod, dem, slope, aspect, twi, tmin, tmax, precip, chm, lcpri4, lcpri6, lcsec3, lcsec2, lcsec1, lcsec4
Random Forest Model - Training data 1:1
Random Forest Model - Testing data 1:1
Params
$formula
litter ~ .
$num.trees
[1] 90
$mtry
[1] 5
$min.node.size
[1] 5
$replace
[1] FALSE
$sample.fraction
[1] 0.5Test-set Accuracy
- Mean-model RMSE: 12.039
| Prediction | |
|---|---|
| RMSE | 11.893 |
| MAE | 7.816 |
| MBE | -2.946 |
| R2 | -0.003 |
PC Regression - 10-Fold-CV 1:1
Params
Call:
lm(formula = pc_all_litter$litter ~ ., data = pc_all_litter[,
1:which(props > 0.99)[1]])
Residuals:
Min 1Q Median 3Q Max
-10.998 -2.962 -0.794 2.223 40.741
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 13.63779 4.26254 3.199 0.001719 **
PC1 0.47658 1.14361 0.417 0.677541
PC2 -1.89067 1.14459 -1.652 0.100910
PC3 -0.28811 0.30903 -0.932 0.352856
PC4 2.23189 0.39955 5.586 1.25e-07 ***
PC5 1.09095 0.40177 2.715 0.007492 **
PC6 -0.41942 0.39714 -1.056 0.292815
PC7 -1.23072 0.66184 -1.860 0.065142 .
PC8 1.06749 0.51248 2.083 0.039153 *
PC9 -0.30149 0.58114 -0.519 0.604763
PC10 -0.31558 0.58596 -0.539 0.591076
PC11 0.52893 0.66984 0.790 0.431141
PC12 -0.84674 0.57547 -1.471 0.143534
PC13 -2.23716 0.65201 -3.431 0.000800 ***
PC14 1.16774 0.67869 1.721 0.087636 .
PC15 0.07718 0.72489 0.106 0.915371
PC16 0.82620 0.78967 1.046 0.297325
PC17 -0.66106 0.89487 -0.739 0.461362
PC18 3.08090 0.98110 3.140 0.002077 **
PC19 -7.03111 1.10089 -6.387 2.58e-09 ***
PC20 4.26695 1.17876 3.620 0.000417 ***
PC21 5.69022 1.25458 4.536 1.26e-05 ***
PC22 -1.08771 1.46851 -0.741 0.460175
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 6.659 on 134 degrees of freedom
Multiple R-squared: 0.6428, Adjusted R-squared: 0.5841
F-statistic: 10.96 on 22 and 134 DF, p-value: < 2.2e-1610-Fold-CV Accuracy
- Mean-model RMSE: 10.326
| Prediction | |
|---|---|
| RMSE | 7.289 |
| MAE | 4.751 |
| MBE | -0.121 |
| R2 | 0.501 |
Soil C model
- Predictors: agb, delta_agb, sd_agb, lcpri, n_unique_lcpri, lcsec, mag, nbr, ndvi, nsds, tcb, tcg, tcw, yod, dem, slope, aspect, twi, tmin, tmax, precip, chm
PC Regression - 10-Fold CV 1:1
Params
Call:
lm(formula = pc_all_soil$soil ~ ., data = pc_all_soil[, 1:which(props >
0.99)[1]])
Residuals:
Min 1Q Median 3Q Max
-82.54 -30.97 -5.62 18.42 405.07
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 92.548 41.241 2.244 0.0279 *
PC1 1.179 7.349 0.160 0.8730
PC2 4.231 3.045 1.390 0.1689
PC3 -1.775 3.643 -0.487 0.6276
PC4 5.748 4.479 1.283 0.2035
PC5 10.965 5.352 2.049 0.0441 *
PC6 3.811 5.305 0.718 0.4748
PC7 4.921 5.931 0.830 0.4094
PC8 -7.503 6.131 -1.224 0.2250
PC9 10.756 6.871 1.565 0.1218
PC10 6.073 7.288 0.833 0.4074
PC11 10.121 7.772 1.302 0.1969
PC12 4.814 9.217 0.522 0.6030
PC13 17.006 9.548 1.781 0.0790 .
PC14 2.197 10.767 0.204 0.8389
PC15 -5.777 11.512 -0.502 0.6173
PC16 5.804 14.642 0.396 0.6930
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 62.15 on 73 degrees of freedom
Multiple R-squared: 0.2202, Adjusted R-squared: 0.04928
F-statistic: 1.288 on 16 and 73 DF, p-value: 0.228110-Fold CV Accuracy
- Mean-model RMSE: 63.741
| Prediction | |
|---|---|
| RMSE | 65.896 |
| MAE | 40.791 |
| MBE | -0.171 |
| R2 | -0.081 |