MET Stat Use Case: Reliability Diagram

This use case demonstrates how to run VCasT to compute probabilistic metrics used for reliability diagrams using MET .stat files and a YAML configuration.

It uses a sample configuration file (pct.yaml) that filters and aggregates probabilistic forecast statistics.

Prerequisites

Before running the example, make sure VCasT is installed. Follow the installation steps in Quick Start Guide.

Run the Example

  1. Clone the test repository:

    git clone https://github.com/NOAA-GSL/VCasT-tests
cd VCasT-tests/examples/MET/reliability

  2. Run VCasT with the provided YAML file:

    vcast pct.yaml

    This will load and process the probabilistic .stat files under the ./stats directory using the configuration options defined below.

YAML Configuration Explained

Below is the content of pct.yaml, which configures VCasT to:

  • Use the pct line type for probabilistic verification

  • Filter by model, variable, threshold, lead time, and region

  • Aggregate the results by model, variable, and lead time

Sample pct.yaml configuration
 1input_stat_folder: ./stats
 2line_type: pct
 3date_column: "fcst_valid_beg"
 4start_date: "2022-05-01_01:00:00"
 5end_date: "2022-05-13_12:00:00"
 6
 7string_filters:
 8  model: ["MPAS_EXPT1_ensprob"]
 9  fcst_var: ["REFC_ENS_FREQ_ge20"]
10  fcst_lead: ["0","010000","020000","030000","040000","050000","060000","070000","080000",
11              "090000","100000","110000","120000","130000","140000","150000","160000","170000","180000",
12              "190000","200000","210000","220000","230000","240000","250000","260000","270000","280000",
13              "290000","300000","310000","320000","330000","340000","350000","360000"]
14  vx_mask: ["CONUS"]
15
16stat_vars: ["all_thresh"]
17
18reformat_file: false
19output_reformat_file: "./filtered_output.data"
20
21output_file: true
22output_plot_file: "./vars.data"
23
24aggregate: true
25group_by: ["model", "fcst_var", "fcst_lead"]
26output_agg_file: "./agg.data"

Output

There are two output files:

  • vars.data - File containing all the information filtered from MET stat files with only the specific stat_vars columns.

  • agg.data — Aggregated reliability metrics.

Note

The option stat_vars: ["all_thresh"] means that all thresholds will be used. See Table 11.8 in MET Point-Stat tool <https://metplus.readthedocs.io/projects/met/en/latest/Users_Guide/point-stat.html>