MET Stat Use Case: Spread–Skill Relationship

This use case demonstrates how to run VCasT to evaluate the spread–skill relationship using MET .stat files and an ensemble continuous statistics configuration.

It uses a sample configuration file (ecnt.yaml) for computing metrics like CRPS and ensemble spread.

Prerequisites

Before running the example, make sure VCasT is installed. Follow the installation steps in Quick Start Guide.

Run the Example

  1. Clone the test repository:

    git clone https://github.com/NOAA-GSL/VCasT-tests
cd VCasT-tests/examples/MET/spread-skill

  2. Run VCasT with the provided YAML file:

    vcast ecnt.yaml

    This will load and process ensemble .stat files under the ./estats directory using the configuration options defined below.

YAML Configuration Explained

Below is the content of ecnt.yaml, which configures VCasT to:

  • Use the ecnt line type for ensemble continuous verification

  • Filter by model, variable, threshold, lead time, and region

  • Aggregate the results by model, variable, and lead time

Sample ecnt.yaml configuration
 1input_stat_folder: ./estats
 2line_type: ecnt
 3date_column: "fcst_valid_beg"
 4start_date: "2022-05-01_00:00:00"
 5end_date: "2022-05-10_12:00:00"
 6
 7string_filters:
 8  model: ["RRFS_GDAS_GF.SPP.SPPT"]
 9  fcst_var: ["TMP"]
10  fcst_lead: ["000000","010000","020000","030000","040000","050000","060000","070000","080000",
11              "090000","100000","110000","120000","130000","140000","150000","160000","170000","180000",
12              "190000","200000","210000","220000","230000","240000","250000","260000","270000","280000",
13              "290000","300000","310000","320000","330000","340000","350000","360000"]
14  vx_mask: ["CONUS"]
15  fcst_lev: ["Z2"]
16
17stat_vars: ["spread_plus_oerr", "rmse", "ratio"]
18
19reformat_file: false
20output_reformat_file: "./filtered_output.data"
21
22output_file: true
23output_plot_file: "./vars.data"
24
25aggregate: true
26group_by: ["model", "fcst_var", "fcst_lead"]
27output_agg_file: "./agg.data"

Output

There are two output files:

  • vars.data - File containing all the information filtered from MET stat files with only the specific stat_vars columns.

  • agg.data — Aggregated reliability metrics.

Note

The option stat_vars: ["ratio"] means that ratio = spread_plus_oerr / rmse is also calculated.