Skip to content

Running FV3‐JEDI or MPAS‐JEDI ctest and setting up an experiment

Jump to bottom
delippi edited this page Apr 24, 2024 · 17 revisions

Running FV3-JEDI or MPAS-JEDI ctest and setting up an experiment

Currently supported platforms

  • NOAA RDHPCS Hera
  • NOAA RDHPCS Orion (coming soon)

Currently supported dycores

  • FV3
  • MPAS

Clone and build RDASApp

Follow the instruction at: https://github.com/NOAA-EMC/RDASApp/wiki/Clone-and-Build-RDASapp

Run ctest

export SLURM_ACCOUNT=$account # for tcsh use: setenv SLURM_ACCOUNT $account
cd RDASApp/build
ctest -VV -R rrfs_fv3jedi_hyb_2022052619
ctest -VV -R rrfs_mpasjedi_2022052619_Ens3Dvar

Where $account is the slurm resource account (e.g., fv3-cam, da-cpu, etc.).

Getting test case data

When you build using the -r option, it will copy the staged RRFS JEDI data to the following paths. More experiments will be added in the future.

RDASApp/bundle/rrfs-test/rundir-rrfs_fv3jedi_hyb_2022052619 # when DYCORE=FV3
RDASApp/bundle/rrfs-test/rundir-rrfs_mpasjedi_2022052619_Ens3Dvar # when DYCORE=MPAS

There is a convienient script to set up your experiment from the ctest data located in the following path. Just make sure to edit the USER INPUT block of the script to set up the experiment to point to your installation of RDASApp as well as to the location of your desired experiment directory, the desired dycore for which you are setting up an experiment for, the platform you are running on, and finally your slurm account information.

cd RDASApp/rrfs-test/scripts
bash ./setup_experiment.sh # edit USER INPUT block before running.

Running your own experiments with the ctest case

In your newly created experiment directory, you will have a run script run_${dycore}jedi_${platform}.sh which was copied and edited in your directory by the setup_experiment.sh script. Just provide this script with a yaml configuration and that is all you need to run a JEDI analysis.

sbatch run_${dycore}jedi_${platform}.sh $inputYaml
tail -f jedi.log

Where $platform is one of the supported options orion or hera, dycore is one of fv3 or mpas, and $inputYaml is a yaml like the one provided in the test case testinput/rrfs_fv3jedi_hyb_2022052619.yaml or testinput/msonet_singleob_airTemperature.yaml.

Visualizing Fv3-JEDI results (currently only for FV3-JEDI)

A simple python script was created for demonstration purposes to visualize the single ob output (results from sbatch run_fv3jedi_hera.sh testinput/msonet_singleob_airTemperature.yaml). You can use this as a guide for visualizing your own results.

cd rundir-rrfs_fv3jedi_hyb_2022052619
module use RDASApp/modulefiles
module load EVA/$platform
python fv3jedi-increment.py

Please note that sometimes you may have to run module load EVA/$platform twice. It should show (eva) to denote that the EVA conda environment has been activated.

Clone this wiki locally