-
Notifications
You must be signed in to change notification settings - Fork 15
Running FV3‐JEDI Hofx Validation
Hofx validation is done to compare FV3-JEDI observation operators against the expected GSI result. We are not able to compare MPAS-JEDI against a GSI baseline, thus all validation is done in the context of FV3-JEDI only.
- NOAA RDHPCS Hera
- NOAA RDHPCS Orion (coming soon)
- FV3 (validation only done with FV3-JEDI vs GSI)
git clone --recurse-submodules https://github.com/NOAA-EMC/GSI.git
cd GSI/ush
./build.sh
Follow the instruction at: https://github.com/NOAA-EMC/RDASApp/wiki/Clone-and-Build-RDASapp
RDASApp/rrfs-test/scripts
bash setup_experiment.sh # Make sure to edit USER INPUT block first
It is good practice to start from a single ob, if possible. It makes debugging much easier, but takes a little bit more effort to get set up. For a single obs test, we will create our own single ob file based on a real ob from prepbufr rather than using the built in single ob option in JEDI. The purpose is to create a single bufr file that is then run through bufr2ioda converter to have a matching single ob IODA file. Then run both through GSI and JEDI respectively and compare the result.
Let's find a real ob that we want to duplicate. The following example uses MSONET data as an example:
module use RDASApp/modulefiles
module load RDAS/${platform}.intel
split_by_subset prepbufr # splits a BUFR file into subsets
readbp MSONET # this will print reports one at a time (press enter to iterate)
Looping through the MSONET data with readbp, you might find an ob that we could duplicate like this:
#--------------------------------------------------------------------------------
#MESSAGE: MSONET 1 102 2022052619
#STATION: LWEF1 278.61 27.44
#TIME: 2022052619 -0.23
#ELV: 43.00
#TYPE: 540. ******** 195.
#DATA:
# LVL CAT POB SPH TOB ZOB UOB VOB
# 1 0 1008.4(3) 13838.0(3) 33.9(2) 43.0(3) *******(*) *******(*)
#--------------------------------------------------------------------------------
Once you find an ob to duplicate you can modify GSI's Single_Observation code to create the ob type that you need for your work. You don't have to exactly match the ob information found in readbp. For example, you could change the ob type from 195 to 188 (which is also mesonet). You could also change the station name incase GSI black lists this particular station.
(The rest of these instructions are under construction)
While single ob tests can be useful for initial testing, it is still important to run a full obs test to complete validation. It is recommended to run single ob tests first.
Configure your GSI experiment as desired.
The first thing you might want to do is split your prebufr data by subset.
module use RDASApp/modulefiles
module load RDAS/${platform}.intel
gettab ${bufrfile} # this will print the internal BUFR table
readbp ${bufrfile} # this will print reports one at a time
readmp ${bufrfile} # similar to readbp but prints more info
split_by_subset ${bufrfile} # splits a BUFR file into subsets
This will create BUFR files like MSONET, ADPSFC, ADPUPA, etc. that were extracted from the original prepbufr file containing only obs from those specific platforms/subsets. Here you might want to link/copy your prepbufr that was split by subset as prepbufr to the GSI run directory.
For example:
cd ${YOUR_EXPERIMENT_DIR}/gsi_2022052619
split_by_subset prepbufr
mv prepbufr prepbufr.save
ln -sf MSONET prepbufr
Now run the GSI experiment
cd ${YOUR_EXPERIMENT_DIR}/gsi_2022052619
sbatch run_gsi_${platform}.sh
After GSI has finished running, you will need to use the diag files to create the GSI-based IODA files
bash run_gsi_ncdiag_${platform}.sh ${diag} # Edit this script. Currently assumes sfc conv obs.
where ${diag} is the name of your diagnostic file, for example: diag_conv_t_ges.2022052619. This will create a GSI-IODA file in obsout/. The file named is like sfc_tsen_obs_2022052619.nc4 for surface temperature obs, for example.
Note: If this script fails, it could be that your obstype is not in the script located here $RDASApp/build/lib/python3.10/pyiodaconv/gsi_ncdiag.py this is the "installed" gsi_ncdiag.py script. If it is not, please add it to RDASApp/sorc/iodaconv/src/gsi_ncdiag/gsi_ncdiag.py and open a PR to https://github.com/JCSDA-internal/ioda-converters.
cd ${YOUR_EXPERIMENT_DIR}/rundir-rrfs_fv3jedi_hyb_2022052619
cp ${YOUR_EXPERIMENT_DIR}/gsi_2022052619/obsout/${GSI-IODA} .
sbatch run_fv3jedi_${platform}.sh $yaml # make sure your yaml points to the GSI-IODA file on previous line
For quick sanity checks, there is a convienient python script to plot hofx ratios.
cd ${YOUR_EXPERIMENT_DIR}/rrfs-data_fv3jedi_2022052619
module use ${YOUR_PATH_TO_RDASAPP}/modulefiles
module load EVA/${platform}
python fv3jedi_gsi_validation.py -d ${diag} -v ${variable} -o ${obtype}
where ${diag} is the JEDI diag file like MSONET_hofxs_${variable}_2022052619.nc4, ${variable} is a variable name such as airTemperature, specificHumidity, etc. and ${obtype} is the BUFR ob types such as 188 for mesonet state variables or 288 for mesonet wind variables.
A helper script (run_eva_hera.sh) is provided to get users started running EVA and meant for demonstration purposes. The script that is included assumes mesonet data, but could be easily modified to run other observation types.
cd ${YOUR_EXPERIMENT_DIR}/eva
ln -sf ../rrfs-data_fv3jedi_2022052619/msonet_airTemperature.yaml .
ln -sf ../rrfs-data_fv3jedi_2022052619/MSONET_hofxs_airTemperature_2022052619.nc4 .
bash run_eva_hera.sh eva_mesonet_MSONET_hofxs_airTemperature_2022052619.yaml
Once finished, EVA will create three subdirectories containting a variety of validation figures: observation_scatter_plots, histograms, and map_plots.