Feature #50 Hurricane Matthew: Add WRF plotting to METplus run (#66)

* Initial WRF plotting script, functions, and README for the Hurricane Matthew case.

* add Dockerfile to create iwrf-metplus image and update instructions to use that container

* make script executable

* create directory containing output file if it does not already exist

* change default input and output directories to use paths assumed in Docker container

* mount directory containing WRF plotting script to container

* run WRF plotting script as part of METplus use case

* Per #50, update instructions to use I-WRF METplus Docker container that includes METplotpy, METcalcpy, METdataio, and wrf python packages. Mount the Hurricane Matthew Visualization directory so the plotting scripts can be found. The WRF plotting script is called from the METplus use case run

* remove commented code

* remove characters that should not be rendered in the docs

* cleanup usage statement

* Per request on @Trumbore on pull request review, updated WRF log output to include updated content.

---------

Co-authored-by: Jared A. Lee <jaredlee@ucar.edu>
Co-authored-by: John Halley Gotway <johnhg@ucar.edu>

docker/metplus/Dockerfile

@@ -0,0 +1,9 @@
+ARG METPLUS_TAG=6.0.0-beta5
+
+FROM dtcenter/metplus-analysis:${METPLUS_TAG}
+
+# install wrf python package from branch in PR NCAR/wrf-python#242
+# because wrf will not install from pip
+RUN python3 -m pip install --upgrade pip \
+  && python3 -m pip install git+https://github.com/DWesl/wrf-python.git@cmake-build \
+  && python3 -m pip install cartopy

docs/Users_Guide/matthewjetstream.rst

@@ -181,12 +181,13 @@ make sure that we refer to the same resource names and file paths wherever they
 Copy and paste the definitions below into your shell to define the variables before proceeding::
 
     WRF_IMAGE=ncar/iwrf:latest
-    METPLUS_IMAGE=dtcenter/metplus-dev:develop
+    METPLUS_IMAGE=ncar/iwrf:metplus-latest
     WORKING_DIR=/home/exouser
     WRF_DIR=${WORKING_DIR}/wrf/20161006_00
     METPLUS_DIR=${WORKING_DIR}/metplus
     WRF_CONFIG_DIR=${WORKING_DIR}/i-wrf/use_cases/Hurricane_Matthew/WRF
     METPLUS_CONFIG_DIR=${WORKING_DIR}/i-wrf/use_cases/Hurricane_Matthew/METplus
+    PLOT_SCRIPT_DIR=${WORKING_DIR}/i-wrf/use_cases/Hurricane_Matthew/Visualization
     OBS_DATA_VOL=data-matthew-input-obs
 
 Any time you open a new shell on your instance, you will need to perform this action
@@ -334,29 +335,34 @@ Once completed, you can view the end of an output file to confirm that it succee
 
 The output should look something like this::
 
-    Timing for main: time 2016-10-06_11:42:30 on domain   1:    0.23300 elapsed seconds
-    Timing for main: time 2016-10-06_11:45:00 on domain   1:    0.23366 elapsed seconds
-    Timing for main: time 2016-10-06_11:47:30 on domain   1:    2.77688 elapsed seconds
-    Timing for main: time 2016-10-06_11:50:00 on domain   1:    0.23415 elapsed seconds
-    Timing for main: time 2016-10-06_11:52:30 on domain   1:    0.23260 elapsed seconds
-    Timing for main: time 2016-10-06_11:55:00 on domain   1:    0.23354 elapsed seconds
-    Timing for main: time 2016-10-06_11:57:30 on domain   1:    0.23345 elapsed seconds
-    Timing for main: time 2016-10-06_12:00:00 on domain   1:    0.23407 elapsed seconds
-    Timing for Writing wrfout_d01_2016-10-06_12:00:00 for domain        1:    0.32534 elapsed seconds
-    d01 2016-10-06_12:00:00 wrf: SUCCESS COMPLETE WRF
+    Timing for main: time 2016-10-07_23:50:00 on domain 1: 0.25548 elapsed seconds
+    Timing for main: time 2016-10-07_23:52:30 on domain 1: 0.25495 elapsed seconds
+    Timing for main: time 2016-10-07_23:55:00 on domain 1: 0.25066 elapsed seconds
+    Timing for main: time 2016-10-07_23:57:30 on domain 1: 0.25231 elapsed seconds
+    Timing for main: time 2016-10-08_00:00:00 on domain 1: 0.25795 elapsed seconds
+    Timing for Writing wrfout_d01_2016-10-08_00:00:00 for domain 1: 0.68666 elapsed seconds
+    Timing for Writing wrfout_zlev_d01_2016-10-08_00:00:00 for domain 1: 0.47411 elapsed seconds
+    Timing for Writing wrfout_plev_d01_2016-10-08_00:00:00 for domain 1: 0.47619 elapsed seconds
+    Timing for Writing restart for domain 1: 1.54598 elapsed seconds
+    d01 2016-10-08_00:00:00 wrf: SUCCESS COMPLETE WRF
 
 Run METplus
 ===========
 
 After the WRF simulation has finished, you can run the METplus verification to compare the simulated results
 to the actual weather observations during the hurricane.
 The verification takes about five minutes to complete.
-We use command line options to tell the METplus container several things, including where the observed data is located,
-where the METplus configuration can be found, where the WRF output data is located, and where it should create its output files::
+We use command line options to tell the METplus container several things,
+including where the observed data is located,
+where the METplus configuration can be found,
+where the plotting scripts can be found,
+where the WRF output data is located,
+and where it should create its output files::
 
     docker run --rm -it \
       --volumes-from ${OBS_DATA_VOL} \
       -v ${METPLUS_CONFIG_DIR}:/config \
+      -v ${PLOT_SCRIPT_DIR}:/plot_scripts \
       -v ${WORKING_DIR}/wrf:/data/input/wrf \
       -v ${METPLUS_DIR}:/data/output ${METPLUS_IMAGE} \
       /metplus/METplus/ush/run_metplus.py /config/PointStat_matthew.conf

docs/Users_Guide/matthewredcloud.rst

@@ -187,12 +187,13 @@ make sure that we refer to the same resource names and file paths wherever they
 Copy and paste the definitions below into your shell to define the variables before proceeding::
 
     WRF_IMAGE=ncar/iwrf:latest
-    METPLUS_IMAGE=dtcenter/metplus-dev:develop
+    METPLUS_IMAGE=ncar/iwrf:metplus-latest
     WORKING_DIR=/home/ubuntu
     WRF_DIR=${WORKING_DIR}/wrf/20161006_00
     METPLUS_DIR=${WORKING_DIR}/metplus
     WRF_CONFIG_DIR=${WORKING_DIR}/i-wrf/use_cases/Hurricane_Matthew/WRF
     METPLUS_CONFIG_DIR=${WORKING_DIR}/i-wrf/use_cases/Hurricane_Matthew/METplus
+    PLOT_SCRIPT_DIR=${WORKING_DIR}/i-wrf/use_cases/Hurricane_Matthew/Visualization
     OBS_DATA_VOL=data-matthew-input-obs
 
 Any time you open a new shell on your instance, you will need to perform this action
@@ -340,29 +341,34 @@ Once completed, you can view the end of an output file to confirm that it succee
 
 The output should look something like this::
 
-    Timing for main: time 2016-10-06_11:42:30 on domain   1:    0.23300 elapsed seconds
-    Timing for main: time 2016-10-06_11:45:00 on domain   1:    0.23366 elapsed seconds
-    Timing for main: time 2016-10-06_11:47:30 on domain   1:    2.77688 elapsed seconds
-    Timing for main: time 2016-10-06_11:50:00 on domain   1:    0.23415 elapsed seconds
-    Timing for main: time 2016-10-06_11:52:30 on domain   1:    0.23260 elapsed seconds
-    Timing for main: time 2016-10-06_11:55:00 on domain   1:    0.23354 elapsed seconds
-    Timing for main: time 2016-10-06_11:57:30 on domain   1:    0.23345 elapsed seconds
-    Timing for main: time 2016-10-06_12:00:00 on domain   1:    0.23407 elapsed seconds
-    Timing for Writing wrfout_d01_2016-10-06_12:00:00 for domain        1:    0.32534 elapsed seconds
-    d01 2016-10-06_12:00:00 wrf: SUCCESS COMPLETE WRF
+    Timing for main: time 2016-10-07_23:50:00 on domain 1: 0.25548 elapsed seconds
+    Timing for main: time 2016-10-07_23:52:30 on domain 1: 0.25495 elapsed seconds
+    Timing for main: time 2016-10-07_23:55:00 on domain 1: 0.25066 elapsed seconds
+    Timing for main: time 2016-10-07_23:57:30 on domain 1: 0.25231 elapsed seconds
+    Timing for main: time 2016-10-08_00:00:00 on domain 1: 0.25795 elapsed seconds
+    Timing for Writing wrfout_d01_2016-10-08_00:00:00 for domain 1: 0.68666 elapsed seconds
+    Timing for Writing wrfout_zlev_d01_2016-10-08_00:00:00 for domain 1: 0.47411 elapsed seconds
+    Timing for Writing wrfout_plev_d01_2016-10-08_00:00:00 for domain 1: 0.47619 elapsed seconds
+    Timing for Writing restart for domain 1: 1.54598 elapsed seconds
+    d01 2016-10-08_00:00:00 wrf: SUCCESS COMPLETE WRF
 
 Run METplus
 ===========
 
 After the WRF simulation has finished, you can run the METplus verification to compare the simulated results
 to the actual weather observations during the hurricane.
 The verification takes about five minutes to complete.
-We use command line options to tell the METplus container several things, including where the observed data is located,
-where the METplus configuration can be found, where the WRF output data is located, and where it should create its output files::
+We use command line options to tell the METplus container several things,
+including where the observed data is located,
+where the METplus configuration can be found,
+where the plotting scripts can be found,
+where the WRF output data is located,
+and where it should create its output files::
 
     sudo docker run --rm -it \
       --volumes-from ${OBS_DATA_VOL} \
       -v ${METPLUS_CONFIG_DIR}:/config \
+      -v ${PLOT_SCRIPT_DIR}:/plot_scripts \
       -v ${WORKING_DIR}/wrf:/data/input/wrf \
       -v ${METPLUS_DIR}:/data/output ${METPLUS_IMAGE} \
       /metplus/METplus/ush/run_metplus.py /config/PointStat_matthew.conf

docs/Users_Guide/matthewwindows.rst

@@ -62,11 +62,12 @@ or changes the path entirely to use a different location on your computer::
 Now you can copy and paste the definitions below into your shell to define the other variables before proceeding::
 
     set WRF_IMAGE=ncar/iwrf:latest
-    set METPLUS_IMAGE=dtcenter/metplus-dev:develop
+    set METPLUS_IMAGE=ncar/iwrf:metplus-latest
     set WRF_DIR=%WORKING_DIR%\wrf\20161006_00
     set METPLUS_DIR=%WORKING_DIR%\metplus
     set WRF_CONFIG_DIR=%WORKING_DIR%\i-wrf-main\use_cases\Hurricane_Matthew\WRF
     set METPLUS_CONFIG_DIR=%WORKING_DIR%\i-wrf-main\use_cases\Hurricane_Matthew\METplus
+    set PLOT_SCRIPT_DIR=%WORKING_DIR%\i-wrf-main\use_cases\Hurricane_Matthew\Visualization
     set OBS_DATA_VOL=data-matthew-input-obs
 
 Any time you open a new shell on your instance, you will need to perform this action
@@ -223,29 +224,34 @@ Once completed, you can view the end of an output file to confirm that it succee
 
 The output should look something like this::
 
-    Timing for main: time 2016-10-06_11:42:30 on domain   1:    0.23300 elapsed seconds
-    Timing for main: time 2016-10-06_11:45:00 on domain   1:    0.23366 elapsed seconds
-    Timing for main: time 2016-10-06_11:47:30 on domain   1:    2.77688 elapsed seconds
-    Timing for main: time 2016-10-06_11:50:00 on domain   1:    0.23415 elapsed seconds
-    Timing for main: time 2016-10-06_11:52:30 on domain   1:    0.23260 elapsed seconds
-    Timing for main: time 2016-10-06_11:55:00 on domain   1:    0.23354 elapsed seconds
-    Timing for main: time 2016-10-06_11:57:30 on domain   1:    0.23345 elapsed seconds
-    Timing for main: time 2016-10-06_12:00:00 on domain   1:    0.23407 elapsed seconds
-    Timing for Writing wrfout_d01_2016-10-06_12:00:00 for domain        1:    0.32534 elapsed seconds
-    d01 2016-10-06_12:00:00 wrf: SUCCESS COMPLETE WRF
+    Timing for main: time 2016-10-07_23:50:00 on domain 1: 0.25548 elapsed seconds
+    Timing for main: time 2016-10-07_23:52:30 on domain 1: 0.25495 elapsed seconds
+    Timing for main: time 2016-10-07_23:55:00 on domain 1: 0.25066 elapsed seconds
+    Timing for main: time 2016-10-07_23:57:30 on domain 1: 0.25231 elapsed seconds
+    Timing for main: time 2016-10-08_00:00:00 on domain 1: 0.25795 elapsed seconds
+    Timing for Writing wrfout_d01_2016-10-08_00:00:00 for domain 1: 0.68666 elapsed seconds
+    Timing for Writing wrfout_zlev_d01_2016-10-08_00:00:00 for domain 1: 0.47411 elapsed seconds
+    Timing for Writing wrfout_plev_d01_2016-10-08_00:00:00 for domain 1: 0.47619 elapsed seconds
+    Timing for Writing restart for domain 1: 1.54598 elapsed seconds
+    d01 2016-10-08_00:00:00 wrf: SUCCESS COMPLETE WRF
 
 Run METplus
 ===========
 
 After the WRF simulation has finished, you can run the METplus verification to compare the simulated results
 to the actual weather observations during the hurricane.
 The verification takes about five minutes to complete.
-We use command line options to tell the METplus container several things, including where the observed data is located,
-where the METplus configuration can be found, where the WRF output data is located, and where it should create its output files::
+We use command line options to tell the METplus container several things,
+including where the observed data is located,
+where the METplus configuration can be found,
+where the plotting scripts can be found,
+where the WRF output data is located,
+and where it should create its output files::
 
     docker run --rm -it ^
       --volumes-from %OBS_DATA_VOL% ^
       -v %METPLUS_CONFIG_DIR%:/config ^
+      -v %PLOT_SCRIPT_DIR%:/plot_scripts ^
       -v %WORKING_DIR%\wrf:/data/input/wrf ^
       -v %METPLUS_DIR%:/data/output %METPLUS_IMAGE% ^
       /metplus/METplus/ush/run_metplus.py /config/PointStat_matthew.conf

docs/Users_Guide/running.rst

@@ -17,12 +17,24 @@ Load the apptainer module::
 
    module load apptainer
 
-Change directory to scratch and pull the containers from DockerHub.
+Change directory to scratch and pull the images from DockerHub.
 This will create a `.sif` file in the current directory::
 
-   apptainer pull ${SCRATCH}/metplus-dev_develop.sif docker://dtcenter/metplus-dev:develop
+   apptainer pull ${SCRATCH}/iwrf-metplus.sif docker://ncar/iwrf:metplus-latest
    apptainer pull ${SCRATCH}/data-matthew-input-obs.sif oras://registry-1.docker.io/ncar/iwrf:data-matthew-input-obs
 
+.. note::
+
+   If an error is displayed when attempting to pull the METplus image,
+   creating a DockerHub account and authenticating through apptainer may be
+   necessary.
+
+   ::
+
+      apptainer remote login --username {USERNAME} docker://docker.io
+
+   where **{USERNAME}** is your DockerHub username.
+
 Create a directory to store the output::
 
    mkdir ${SCRATCH}/metplus_out
@@ -48,21 +60,25 @@ using the --bind argument)
 * Config directory containing METplus use case configuration file
    * Local: ${SCRATCH}/i-wrf/use_cases/Hurricane_Matthew/METplus
    * Container: /config
+* Plot script directory containing WRF plotting scripts
+   * Local: ${SCRATCH}/i-wrf/use_cases/Hurricane_Matthew/Visualization
+   * Container: /plot_scripts
 * Output directory to write output
    * Local: ${SCRATCH}/metplus_out
    * Container: /data/output
 
 ::
 
    LOCAL_METPLUS_CONFIG_DIR=${SCRATCH}/i-wrf/use_cases/Hurricane_Matthew/METplus
+   LOCAL_PLOT_SCRIPT_DIR=${SCRATCH}/i-wrf/use_cases/Hurricane_Matthew/Visualization
    LOCAL_FCST_INPUT_DIR=/glade/derecho/scratch/jaredlee/nsf_i-wrf/matthew
    LOCAL_OUTPUT_DIR=${SCRATCH}/metplus_out
 
-   export APPTAINER_BIND="${SCRATCH}/data-matthew-input-obs.sif:/data/input/obs:image-src=/,${LOCAL_METPLUS_CONFIG_DIR}:/config,${LOCAL_FCST_INPUT_DIR}:/data/input/wrf,${LOCAL_OUTPUT_DIR}:/data/output"
+   export APPTAINER_BIND="${SCRATCH}/data-matthew-input-obs.sif:/data/input/obs:image-src=/,${LOCAL_METPLUS_CONFIG_DIR}:/config,${LOCAL_FCST_INPUT_DIR}:/data/input/wrf,${LOCAL_OUTPUT_DIR}:/data/output,${LOCAL_PLOT_SCRIPT_DIR}:/plot_scripts"
 
 Execute the run_metplus.py command inside the container to run the use case::
 
-   apptainer exec ${SCRATCH}/metplus-dev_develop.sif /metplus/METplus/ush/run_metplus.py /config/PointStat_matthew.conf
+   apptainer exec ${SCRATCH}/iwrf-metplus.sif /metplus/METplus/ush/run_metplus.py /config/PointStat_matthew.conf
 
 Check that the output data was created locally::
 

use_cases/Hurricane_Matthew/METplus/PointStat_matthew.conf

@@ -8,7 +8,7 @@
 # https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-list
 ###
 
-PROCESS_LIST = MADIS2NC(metar), MADIS2NC(raob), PointStat(surface), PointStat(upper_air)
+PROCESS_LIST = MADIS2NC(metar), MADIS2NC(raob), PointStat(surface), PointStat(upper_air), UserScript(wrf_plot)
 
 
 ###
@@ -163,3 +163,8 @@ OBTYPE =
 POINT_STAT_OUTPUT_PREFIX = {instance}
 
 POINT_STAT_MASK_GRID = FULL
+
+[wrf_plot]
+
+USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE
+USER_SCRIPT_COMMAND=/plot_scripts/plot_wrf.py

use_cases/Hurricane_Matthew/Visualization/README.md

@@ -0,0 +1,38 @@
+# plot_wrf
+Python scripts and functions to make nice plots of WRF model output using Cartopy.
+
+plot_wrf.py can be called with a set of arguments (use option -h to get the current usage statement):
+
+```
+> python plot_wrf.py -h
+usage: plot_wrf.py [-h] [-w WRF_DIR_PARENT] [-o OUT_DIR_PARENT] [-f CYCLE_DT_FIRST] [-l CYCLE_DT_LAST]
+                   [-i CYCLE_STRIDE_H] [-b BEG_LEAD_TIME] [-e END_LEAD_TIME] [-s STR_LEAD_TIME] [-d DOMAIN]
+
+options:
+  -h, --help            show this help message and exit
+  -w WRF_DIR_PARENT, --wrf_dir_parent WRF_DIR_PARENT
+                        string specifying the directory path to the parent WRF output directories, above any
+                        experiment or cycle datetime subdirectories (default: /data/input/wrf)
+  -o OUT_DIR_PARENT, --out_dir_parent OUT_DIR_PARENT
+                        string specifying the directory path to the parent plot directories (default: /data/output/wrf)
+  -f CYCLE_DT_FIRST, --cycle_dt_first CYCLE_DT_FIRST
+                        beginning date/time of first WRF simulation [YYYYMMDD_HH] (default: 20161006_00)
+  -l CYCLE_DT_LAST, --cycle_dt_last CYCLE_DT_LAST
+                        beginning date/time of last WRF simulation [YYYYMMDD_HH]
+  -i CYCLE_STRIDE_H, --cycle_stride_h CYCLE_STRIDE_H
+                        stride in hours between cycles (default: 24)
+  -b BEG_LEAD_TIME, --beg_lead_time BEG_LEAD_TIME
+                        beginning lead time for plotting WRF simulations [HH:MM] (default: 00:00)
+  -e END_LEAD_TIME, --end_lead_time END_LEAD_TIME
+                        ending lead time for plotting WRF simulations [HH:MM] (default: 48:00)
+  -s STR_LEAD_TIME, --str_lead_time STR_LEAD_TIME
+                        stride to create plots every N minutes (default: 180)
+  -d DOMAIN, --domain DOMAIN
+                        WRF domain number to be plotted (default: 1)
+```
+
+The plot_wrf.parse_args function creates a dictionary of options that is then passed to the main routine. Doing this via a dictionary object should make it simpler to add even more customization/options in the future, requiring changes in fewer places than passing numerous positional arguments around.
+
+The plot_wrf script opens specified wrfout files in sequence, reads in user-specified variables (currently set with options like plot_TERRAIN = True and plot_T2 = False in the main function), creates a dictionary of plotting options that is then passed to map_funcs.map_plot to create and save each plot to a file. Inside the main function there are also user-settable boolean flags to turn on/off plotting surface wind barb overlays, labeled stations/cities, etc.
+
+Both these requested variables for plotting and other plot customization options could eventually be changed to be passed in on the command line to not require users to modify the script itself before running it, but that is left for future development.