Usage

The snowdrift CLI accepts NWP model output in GRIB format, and is intended to be configurable for accepting a variety of different local area model (LAM) forecast outputs.

Input NWP parameters

The model requires the following GRIB parameters to be available in the input model in order to derive the snowdrift strength output:

  • wind speed at 10m level or u/v wind components
  • surface temperature (temperature at level 0m)
  • snowfall hourly rate or accumulated snowfall

Output NWP parameters

The software produces the following 4 new model parameters:

  • drift - snowdrift strength value
  • snowage - age in hours of top level snow
  • driftac - accumulation of significant snow drift
  • mobility - snow drift mobility factor of top level snow

Please refer to the Configuring section below for instructions on how to adjust GRIB input, GRIB output and model tuning.

--out [grib_file]

Writes out a total of 8x GRIB parameters, used for assimilation:

  • 4x drift params:
    • drift
    • snowage
    • driftac
    • mobility
  • 4x extra parameters for assimilation only:
    • temp
    • snowground
    • snowfall
    • wind

--out-drift [grib_file]

Writes only the 4x snow drift parameters, used for delivery of snowdrift results:

  • drift
  • snowage
  • driftac
  • mobility

PNG output

PNG output is very basic and mainly used for testing purposes:

$ snowdrift path/to/fc/files* --out image_folder/test --format PNG

A folder image_folder/ will be created and filled with rendered PNG images for drift, snowage, driftac and mobility.

Simple usage

The most basic usage is to print out the snowdrift with --out <grib file> assuming default --format GRIB.

$ snowdrift path/to/fc/files* --out snowdrift.grb

Note however, for operational use should always assimilate last available snowdrift --out result when calculating new snowdrift, see next section.

GRIB with assimilation

Use the --assim <prev_snowdrift.grb> argument to take in previous forecast run output from --out <snowdrift>. This ensures that the snow state is preserved between snowdrift runs. This is required for operational use of the model:

$ snowdrift path/to/fc/files* --out snowdrift.grb --assim prev_snowdrift.grb

GRIB for delivery

To also print out a forecast that includes only the 4x necessary snowdrift parameters for model delivery, use the --out-drift <delivery.grb> argument:

$ snowdrift path/to/fc/files* --out snowdrift.grb --out-drift delivery.grb --assim prev_snowdrift.grb
# deliver the delivery.grb result, e.g.
$ scp delivery.grb <destination>

Note --out-drift GRIB output is only for delivery and cannot be used for assimilation on next run. Use --out to generate complete GRIB fc set for assimilation. You can use a combination of the two to generate assimilation GRIB as well as delivery GRIB.

Assimilation

It is necessary to continously assimilate the previous snowdrift GRIB output in order to inherit the state of the snow cover from previous forecasts. This is achieved by passing the option --assim <prev snowdrift>.grb to the CLI application. WARNING: If assimilation is skipped, the snowdrift forecast will reset itself on each run, resulting in significantly reduced accuracy in the forecast run.

verify assimilation

You can verify that the assimilation step is working correctly by reading the output logs. With --assim option, a secondary dataset will be loaded and logs should indicate if the data is being incorporated in the new forecast:

2021-03-31 14:02:28,786:INFO:loading assimilation data - previous snowdrift run
2021-03-31 14:02:28,869:INFO:collecting from 'prev_drift.grb'
2021-03-31 14:02:28,872:INFO: - found 'drift'
2021-03-31 14:02:28,876:INFO: - found 'snowage'
...
-----------------------------------
Loaded Assimilation Data
-----------------------------------
  drift:
    steps:       10
    min/max:     -1.0/4.9 none
    start/stop:  2021-03-27 00:00:00/2021-03-27 09:00:00
  snowage:
    steps:       10
    min/max:     -1.0/9.0 hours
    start/stop:  2021-03-27 00:00:00/2021-03-27 09:00:00
...
-----------------------------------
2021-03-31 14:02:29,327:INFO:assimilating previous snowdrift forecast
2021-03-31 14:02:29,327:INFO: - OK
2021-03-31 14:02:29,327:INFO:snow drift algorithm
2021-03-31 14:02:29,327:INFO: - assimilation in effect, starting algorithm at step 1
2021-03-31 14:02:29,327:INFO: - snowdrift calculation step 1
2021-03-31 14:02:29,342:INFO: - snowdrift calculation step 2
...
2021-03-31 14:02:29,477:INFO: - removing time -1 assimilation step
...

Production

The following outline in production will ensure that the previous forecast results are continously assimilated on each run:

# initial run, without assimilation:
$ snowdrift latest/files* --out result.grb
$ cp result.grb prev_result.grb

# on each new forecast run, execute:
$ snowdrift latest/files* --assim prev_drift.grb --out result.grb
$ cp result.grb prev_result.grb

Alternatively, you can also write out a snowdrift forecast containing only the necessary 4x snowdrift parameters for display/delivery:

# initial run, without assimilation:
$ snowdrift latest/files* --out result.grb --out-drift delivery.grb
$ cp result.grb prev_result.grb
$ scp delivery.grb <destination>

# on each new forecast run, execute:
$ snowdrift latest/files* --assim prev_result.grb --out result.grb --out-drift delivery.grb
$ cp result.grb prev_result.grb
$ scp delivery.grb <destination>

In-Memory Compression

The snowdrift client program is currently designed to load all necessary parameters into memory in order to easily sort the data before snowdrift processing. The reason for this current architecture decision is that GRIB forecast data does not always come in sorted order.

Depending on input forecast domain size, the current implementation may consume more memory than is easily accessible on some light-weight processing nodes. However, for most typical LAM domains this processing should not exceed a few GB in RAM.

To mitigate the memory usage problem, an in-memory forecast data compression option --compress has been provided, for lossless data compression using the python bcolz module. This option will typically reduce memory usage by a factor of 3, but also reduce processing speed by a similar factor.

To apply in-memory compression, simply add the --compress argument to the CLI snowdrift application:

snowdrift latest/files* --compress --assim prev_result.grb --out result.grb --out-drift delivery.grb

TODO: Further improvements in memory usage of the software can be made by keeping a reference to data in memory on disk during processing. Depending on user requirements this can be implemented without significant further work on the software.

Configuring

The snowdrift software ships with a default configuration for input, snowdrift tuning and output, compatible with the Harmonie NWP model being run in Iceland by the Icelandic Meteorological Office.

Other model inputs may require a granular selection of input parameters, tuning of the model parameters and configuring output GRIB parameters depending on applications.

For this reason the --config <config_file>.yml option allows for detailed configuration of the snowdrift model run.

The following example config file can be edited as appropriate and passed to the snowdrift application. This example configuration file is hosted in the software development folders under config/example_config.yml.

Example config:

# defines input model parameters
InputParameters:
    # if accumulated snow in input model
    snowac:
        id: 'heightAboveGround:0:184'
        unit: 'kg m**-2'
    # alternatively if have snow fall rate,
    # snow:
    #    id: '<type of level>:<level>:<indicator of parameter>'
    snowground:
        id: 'heightAboveGround:0:65'
    temp:
        id: 'heightAboveGround:0:11'
    # if u/v wind in input model
    wind-u:
        id: 'heightAboveGround:10:33'
    wind-v:
        id: 'heightAboveGround:10:34'
    # alternatively if have total wind speed,
    # wind:
    #    id: '<type of level>:<level>:<indicator of parameter>'

# defines snowdrift model tuning (can skip this)
ModelTuning:
    snow_cover_threshold: 1.0
    snow_fall_threshold: 0.1
    wind_speed_normalization: 12.0
    wind_zero_drift: 6.0
    mobility_snowage_reduction:
        snowage: 24
        mobility: 0.6
    mobility_driftac_reduction:
        driftac1: 2.0
        mobility1: 0.6
        driftac2: 6.0
        mobility2: 0.3

# defines GRIB output (can skip this)
OutputParameters:
    type_of_level: 'heightAboveGround'
    level: 0
    # the snowdrift results
    snowage_parameter: 145
    driftac_parameter: 146
    mobility_parameter: 147
    drift_parameter: 148
    # for assimilation purposes store these pars,
    # they get loaded for time -1 assim step.
    temp_parameter: 160
    snowground_parameter: 161
    snowfall_parameter: 162
    wind_parameter: 163

InputParameters

It is necessary to provide identifiers for the following parameters when ingesting forecast GRIB data for the snowdrift model:

  • 'wind' wind speed at 10m level in m/s
  • or 'wind-u' and 'wind-v' wind components in 'm/s'
  • 'temp' surface temperature (level 0m) in °C
    • units K should be automatically converted to °C, check the data summary output when running the software
  • 'snow' snowfall hourly rate in mm/hour (kg m-2/hour)
  • or 'snowac' accumulated snowfall through the input forecast

The syntax for selecting these input parameters is:

<parname>:
    id:'<type of level>:<level>:<indicator of parameter>'

ModelTuning

Use this config section in order to tune some key parameters within the snowdrift model. The defaults that are provided have worked well with the Icelandic Met Office's Harmonie NWP for Iceland.

  • snow_cover_threshold defines in mm (kg m-2) snow water equivalent, the threshold for snow-covered ground. Areas above this threshold will be used in snowdrift calculations.
  • snow_fall_threshold defines in mm/hour (kg m-2/hour) snow water equivalent, the threshold for identifying new snowfall for resetting top layer snowage, snow mobility and driftac.
  • wind_speed_normalization is a factor for tuning the snow drift strength in relation to wind speed. See snow drift equation, $$ \text{drift} = \left(\frac{\text{wind}}{\text{wind}_{norm}}\right)^3 \times \text{mobility}$$ in model description papers below.
  • wind_zero_drift defines the minimum wind speed in m/s for drift value calculations. Below this value there is no drift.
  • mobility_snowage_reduction defines a reduction in snow mobility after certain number of hours of top layer snow age.
    • snowage threshold in hours for reduction
    • mobility new reduced mobility value
  • mobility_driftac_reduction defines a reduction in snow mobility after a certain amount of accumulated drift.
    • driftac1 first threshold, reducing to value of mobility1
    • mobility1 mobility for driftac1 threshold
    • driftac2 second threshold, reducing to value of mobility2
    • mobility2 mobility for driftac2 threshold

Theoretical basis for these tuning options is described in the following papers:

OutputParameters

The output parameters sections defines unique ids for GRIB forecast output for the 4 calculated snowdrift parameters. This usually does not require configuration unless there are conflicts with other parameter definitions being used in plotting and Weather GIS tools used by the user.

  • type_of_level should usually be heightOverGround or 105
  • level should usually be level 0 / 0m over ground
  • snowage_parameter unique number, indicator of parameter
  • driftac_parameter unique number, indicator of parameter
  • mobility_parameter unique number, indicator of parameter
  • drift_parameter unique number, indicator of parameter

Temporary parameters for storing data necessary for assimilation --assim <prev forecast>:

  • temp_parameter unique number, indicator of parameter
  • snowground_parameter unique number, indicator of parameter
  • snowfall_parameter unique number, indicator of parameter
  • wind_parameter unique number, indicator of parameter

Interpretation

The following interpretation of the snow drift strength values were originally recommended in the original development of the model and have been used at the Icelandic Meteorological Office for classifying severity of snow drift from the model:

drift value interpretation
drift < 0.09 None
0.09 =< drift =< 0.21 Low
0.21 < drift < 0.50 Moderate
0.5 =< drift High

The choice of interpretation of drift strength is another consideration for tuning the model, but as presentation of the output from the snowdrift software.

In Iceland the following color scale definition has been used for the model, but it is completely up to the user how he wishes to present the output from the model:

driftScale = (
    (-1,'white'),
    (0,'#0d0'),
    (0.09, '#0dd'),
    (0.21, '#dd0'),
    (0.5, '#d00'),
)

CLI help

Check out the current input options to the executable with --help arg

$ bin/snowdrift -h
usage: snowdrift [-h] [--config FILENAME] [--assim FILENAME] [--out FILENAME]
                 [--out-drift FILENAME] [--format FORMAT] [--show STEP] [--verify]
                 [--compress]
                 forecast_files [forecast_files ...]

positional arguments:
  forecast_files        (required) input NWP forecast in GRIB format

optional arguments:
  -h, --help            show this help message and exit
  --config FILENAME     (optional) config file for configuring the input parameters
                        and to adjust tuning of the snowdrift model
  --assim FILENAME      (recommended) assimilate previous snowdrift result into new
                        snowdrift forecast, accepts snowdrift GRIB files from previous
                        run
  --out FILENAME        save result to file(s) (can use this, if GRIB format, for
                        --assim on next run)
  --out-drift FILENAME  save file(s) with only snowdrift parameters results (can not
                        use this, if GRIB format, for --assim on next run)
  --format FORMAT       output format ['GRIB' (default) | 'PNG']
  --show STEP           show snow drift result at STEP (-1 is last step)
  --verify              only read input data and verify that necessary parameters are
                        available, or derived from input, for snowdrift calculation
  --compress            applies data compression, saving on RAM usage