-
Notifications
You must be signed in to change notification settings - Fork 15
Running FV3‐JEDI Hofx Validation
Hofx validation is done to compare FV3-JEDI observation operators against the expected GSI result. We are not able to compare MPAS-JEDI against a GSI baseline, thus all validation is done in the context of FV3-JEDI only.
- NOAA RDHPCS Hera
- NOAA RDHPCS Orion
- FV3 (validation only done with FV3-JEDI vs GSI)
git clone --recurse-submodules https://github.com/NOAA-EMC/GSI.git
cd GSI/ush
./build.sh
Follow the instruction at: https://github.com/NOAA-EMC/RDASApp/wiki/Clone-and-Build-RDASapp
RDASApp/rrfs-test/scripts
bash setup_experiment.sh # Make sure to edit USER INPUT block first
It is good practice to start from a single ob, if possible. It makes debugging much easier, but takes a little bit more effort to get set up. For a single obs test, we will create our own single ob file based on a real ob from prepbufr rather than using the built in single ob option in JEDI. The purpose is to create a single bufr file that is then run through bufr2ioda converter to have a matching single ob IODA file. Then run both through GSI and JEDI respectively and compare the result.
Let's find a real ob that we want to duplicate. The following example uses MSONET data as an example:
module use RDASApp/modulefiles
module load RDAS/${platform}.intel
split_by_subset prepbufr # splits a BUFR file into subsets
readbp MSONET # this will print reports one at a time (press enter to iterate)
Looping through the MSONET data with readbp, you might find an ob that we could duplicate like this:
#--------------------------------------------------------------------------------
#MESSAGE: MSONET 1 102 2022052619
#STATION: LWEF1 278.61 27.44
#TIME: 2022052619 -0.23
#ELV: 43.00
#TYPE: 540. ******** 195.
#DATA:
# LVL CAT POB SPH TOB ZOB UOB VOB
# 1 0 1008.4(3) 13838.0(3) 33.9(2) 43.0(3) *******(*) *******(*)
#--------------------------------------------------------------------------------
Once you find an ob to duplicate you can modify GSI's Single_Observation code to create the ob type that you need for your work. You don't have to exactly match the ob information found in readbp. For example, you could change the ob type from 195 to 188 (which is also mesonet). You could also change the station name incase GSI black lists this particular station.
(The rest of these instructions are under construction)
While single ob tests can be useful for initial testing, it is still important to run a full obs test to complete validation. The first step to create the GSI-diag based IODA observation file is to first run the GSI. If working with a specific ob platform such as MSONET data, you can split your prepbufr data by subset first. In addition to split_by_subset there are some other useful tools to be aware of:
module use RDASApp/modulefiles
module load RDAS/${platform}.intel
gettab ${bufrfile} # this will print the internal BUFR table
readbp ${bufrfile} # this will print reports one at a time
readmp ${bufrfile} # similar to readbp but prints more info
split_by_subset ${bufrfile} # splits a BUFR file into subsets
This will create BUFR files like MSONET, ADPSFC, ADPUPA, etc. that were extracted from the original prepbufr file containing only obs from those specific platforms/subsets. Here you might want to link/copy your prepbufr that was split by subset as prepbufr to the GSI run directory. For example:
cd ${YOUR_EXPERIMENT_DIR}/gsi_2022052619
split_by_subset prepbufr
mv prepbufr prepbufr.save
ln -sf MSONET prepbufr
Now run the GSI experiment
cd ${YOUR_EXPERIMENT_DIR}/gsi_2022052619
sbatch run_gsi.sh
After GSI has finished running, you will need to use the diag files to create the GSI-based IODA files
bash run_gsi_ncdiag.sh ${diag} # Edit this script. Currently assumes sfc conv obs.
where ${diag} is the name of your diagnostic file, for example: diag_conv_t_ges.2022052619. This will create a GSI-IODA file in obsout/. The file named is like sfc_tsen_obs_2022052619.nc4 for surface temperature obs, for example.
***Note: If this script fails, it could be that your obstype is not in the script located here $RDASApp/build/lib/python3.10/pyiodaconv/gsi_ncdiag.py this is the "installed" gsi_ncdiag.py script. If it is not, please add it to RDASApp/sorc/iodaconv/src/gsi_ncdiag/gsi_ncdiag.py and open a PR to https://github.com/JCSDA-internal/ioda-converters.
For demonstration purposes use the provided testinput/msonet_airTemperature.yaml.
cd ${YOUR_EXPERIMENT_DIR}/rrfs-data_fv3jedi_2022052619
cp ${YOUR_EXPERIMENT_DIR}/gsi_2022052619/obsout/sfc_tsen_obs_2022052619.nc4 .
ln -sf testinput/msonet_airTemperature.yaml .
sbatch run_fv3jedi.sh msonet_airTemperature.yaml
For quick sanity checks, there is a convienient python script to plot hofx ratios.
cd ${YOUR_EXPERIMENT_DIR}/rrfs-data_fv3jedi_2022052619
module use ${YOUR_PATH_TO_RDASAPP}/modulefiles
module load EVA/${platform}
python fv3jedi_gsi_validation.py -d ${diag} -v ${variable} -o ${obtype}
where ${diag} is the JEDI diag file like MSONET_hofxs_${variable}_2022052619.nc4, ${variable} is a variable name such as airTemperature, specificHumidity, etc. and ${obtype} is the BUFR ob types such as 188 for mesonet state variables or 288 for mesonet wind variables.
A helper script (run_eva.sh) is provided to get users started running EVA and meant for demonstration purposes. The script that is included assumes mesonet data, but could be easily modified to run other observation types.
cd ${YOUR_EXPERIMENT_DIR}/eva
ln -sf ../rrfs-data_fv3jedi_2022052619/msonet_airTemperature.yaml .
ln -sf ../rrfs-data_fv3jedi_2022052619/MSONET_hofxs_airTemperature_2022052619.nc4 .
bash run_eva.sh eva_mesonet_MSONET_hofxs_airTemperature_2022052619.yaml
Once finished, EVA will create three subdirectories containting a variety of validation figures: observation_scatter_plots, histograms, and map_plots.
***Note: If eva fails, please check if you have another python/conda environment setup or you are not in the da group.