Bounded Normal Residuals - Model Based Error for Tax Parcels in WWE

Proof-of-concept model-based error estimates for AGB predictions aggregated within 2019 NYS tax parcels in the Warren, Washington, Essex LiDAR coverage. Residuals sampled from a normal distribution centered on predictions, bounded by 95% conformal bounds.

Description

Model Based Approach

Model-based errors for aggregate AGB estimates were computed following the approach described in the CEOS Aboveground Woody Biomass Product Validation Good Practices Protocol (Section 4.2.2).

To produce pixel-level residuals, “observed” values were sampled from a normal distribution, centered on the pixel-prediction with standard deviation computed from test-set residual distribution. These sampled values were bounded by the pixel 95% conformal bounds, and then differenced with the original prediction to produce a residual. 100 iterations of this process were executed to produce a pixel residual as the average residual across the 100 iterations, as well as a pixel residual variance.

An “MSE” value for each aggregation unit was computed as the sum of the following components:

1. covariance of predictions
2. average pixel residual variance
3. covariance of pixel residuals

Data

The Warren, Washington, Essex LiDAR region was leveraged for this analysis. Note that this area is relatively homogenous with respect to landcover, so we might expect these error estimates to be optimistic relative to other regions.

The LINMOD ensemble predictions were used in this analysis, but were masked to only include LCMAP tree cover and wetland classes. Pixels falling in any other class were set to NA and excluded from the analysis.

2019 tax parcels were used as aggregation units. These aggregation units make sense from an application standpoint, though perhaps more arbitrary units of aggregation would be more suitable.

Results

Summary of Error Components

Average Contributions:

1. Prediction Covariance: 5.31%
2. Average Residual Variance: 94.56%
3. Residual Covariance: 0.13%

Distribution of % Error (capped at 200%)

Parcel Errors By Parcel Size

RMSE By Parcel Size

% Error (capped at 200%) By Parcel Size

Summarized Parcel Errors for Size Groupings

Groups are exclusive, where each point along the x-axis represents the center of a 10 acre summary group. So where the x-axis says ‘55’ we are summarizing all parcels larger than 50 acres in size and smaller than or equal to 60 acres in size.

Parcel Maps

Percent errors capped at 200% for figure clarity