Ensemble Spread Error Consistency Checker

Compare case-wise ensemble-mean RMSE with unbiased member variance under finite-M exchangeability factor sqrt(1+1/M); descriptive comparison, no reliability test/independence claim and perfect zero-error ratio undefined.

Description

Ensemble spread describes variation among members, while ensemble-mean RMSE measures error against observations.

Ensemble spread describes variation among members, while ensemble-mean RMSE measures error against observations. Their expected relation depends on finite ensemble size and exchangeability assumptions.^[1]

When to use Ensemble Spread Error Consistency Checker

Use this tool to compare case-wise ensemble-mean RMSE with unbiased member variance under finite-M exchangeability factor sqrt(1+1/M); descriptive comparison, no reliability test/independence claim and perfect zero-error ratio undefined.

Prepare the observed sounding, cloud records or forecast verification data. Check pressure and height coordinates, wind conventions and the stated units.

How to use Ensemble Spread Error Consistency Checker and understand its results

Prepare the observed sounding, cloud records or forecast verification data. Check pressure and height coordinates, wind conventions and the stated units.
Enter ensembleForecastsByCase, observationsByCase using the field table and calculation rules, then run the tool.
Read ensembleMeanForecasts, ensembleMeanRmse, rmsUnbiasedEnsembleSpread, finiteMemberExpectedSpreadErrorRatio. The explanation below defines the units, signs, ordering and any flags or placeholder values.
If the tool reports an input issue, correct the values and run it again. Apply the stated assumptions when interpreting a successfully returned result.

Input field	Meaning	Value type; see rules below	Requirement
ensembleForecastsByCase	Ensemble forecasts by case	list of list of number	Required
observationsByCase	Observations by case	list of number	Required

Provide a rectangular case-by-member matrix with at least two members per case and aligned observations. Return case means, their positive RMSE, root-mean member sample variance and the finite-member-adjusted spread/error ratio. Perfect zero RMSE is rejected because the ratio is undefined.

Paired lists must be nonempty and have equal lengths.

comparison ratio = sqrt(mean unbiased member variances) × sqrt(1+1/M)/ensemble-mean RMSE

Units, indexing, boundary conditions and exceptional cases are defined in the calculation rules above.

Output field	Meaning	Value type
ensembleMeanForecasts	Ensemble mean forecasts	list of number
ensembleMeanRmse	Ensemble mean rmse	number
rmsUnbiasedEnsembleSpread	Rms unbiased ensemble spread	number
finiteMemberExpectedSpreadErrorRatio	Finite member expected spread error ratio	number

json

{
  "input": {
    "ensembleForecastsByCase": [
      [
        0,
        2
      ],
      [
        1,
        3
      ]
    ],
    "observationsByCase": [
      2,
      3
    ]
  },
  "output": {
    "ensembleMeanForecasts": [
      1,
      2
    ],
    "ensembleMeanRmse": 1,
    "rmsUnbiasedEnsembleSpread": 1.4142135623730951,
    "finiteMemberExpectedSpreadErrorRatio": 1.7320508075688772
  }
}

Limitations and assumptions

The descriptive comparison does not establish independent cases, exchangeability or probabilistic reliability.

Numeric inputs and returned numeric values must be finite; an unrepresentable returned value produces an error. Matrices are limited to 128 rows and 128 entries per row; other numeric lists are limited to 100,000 entries. Missing required fields, extra fields, wrong value types and violations of the calculation rules are input errors.

Alternative or Complementary methods

Use Rank Histogram Flatness Check Tool: Audit supplied rank histogram versus uniform bins using descriptive Pearson statistic/end-bin excess; tie randomization and independent cases are caller responsibilities, no automatic significance or calibrated ensemble claim. For a complementary task, use Anomaly Correlation Height Field: Compute uncentered area-weighted anomaly correlation relative to caller climate-normal height field; differs from Pearson centered correlation, matched area exposure required and zero anomaly norms undefined.

References

Ensemble Spread Error Consistency Checker — tool.io

Similar or alternative tools

Rank Histogram Flatness Check Tool
Audit supplied rank histogram versus uniform bins using descriptive Pearson statistic/end-bin excess; tie randomization and independent cases are caller responsibilities, no automatic significance or calibrated ensemble claim.
Anomaly Correlation Height Field
Compute uncentered area-weighted anomaly correlation relative to caller climate-normal height field; differs from Pearson centered correlation, matched area exposure required and zero anomaly norms undefined.