Skip to content

Commit

Permalink
add a Evaluator class for evaluation
Browse files Browse the repository at this point in the history
  • Loading branch information
OuyangWenyu committed Mar 27, 2024
1 parent 206ba2b commit f3c1d38
Show file tree
Hide file tree
Showing 10 changed files with 273 additions and 366 deletions.
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ $ python prepare_data.py --origin_data_dir <your_data_directory_for_hydromodel>
To run calibration with CAMLES dataset, you can use the following code:

```Shell
$ python calibrate_xaj.py --exp camels --warmup_length 365 --model {\"name\":\"xaj_mz\",\"source_type\":\"sources\",\"source_book\":\"HF\"} --algorithm {\"name\":\"SCE_UA\",\"random_seed\":1234,\"rep\":5000,\"ngs\":20,\"kstop\":3,\"peps\":0.1,\"pcento\":0.1}
$ python calibrate_xaj.py --exp example --warmup_length 365 --model {\"name\":\"xaj_mz\",\"source_type\":\"sources\",\"source_book\":\"HF\"} --algorithm {\"name\":\"SCE_UA\",\"random_seed\":1234,\"rep\":5000,\"ngs\":20,\"kstop\":3,\"peps\":0.1,\"pcento\":0.1}
```

To use your own data, run the following code:
Expand All @@ -105,12 +105,18 @@ $ python calibrate_xaj.py --exp example --warmup_length 365 --model {\"name\":\"
# python calibrate_xaj.py --exp <name of directory of the prepared data> --warmup_length <hydromodel need some warm-up period> --model <model function parameters> --algorithm <calibration algorithm parameters>
```

Then you can evaluate the calibrated model with the following code:

```Shell
$ python evaluate_xaj.py --exp example
```

### See the results

Run the following code:
Run the following code to see the results of the evaluation:

```Shell
$ python datapostprocess4calibrate.py --exp example
$ python post_process.py --exp example
```

You will get two metrics files in the "example" directory: "basins_test_metrics_mean_all_cases.csv" and "basins_test_metrics_median_all_cases.csv". The first one is the mean metrics of the testing period -- one row means the mean metrics of all basins in a case, and the second one is the median metrics.
Expand Down
2 changes: 1 addition & 1 deletion hydromodel/datasets/data_preprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,7 @@ def get_ts_from_diffsource(data_type, data_dir, periods, basin_ids):
return ts_data


def get_pe_q_from_ts(ts_xr_dataset):
def _get_pe_q_from_ts(ts_xr_dataset):
"""Transform the time series data to the format that can be used in the calibration process
Parameters
Expand Down
3 changes: 1 addition & 2 deletions hydromodel/trainers/calibrate_ga.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,9 @@
from hydroutils import hydro_file, hydro_stat


from datasets.data_postprocess import plot_sim_and_obs
from hydromodel.datasets.data_postprocess import plot_sim_and_obs, plot_train_iteration
from hydromodel.models.model_config import MODEL_PARAM_DICT
from hydromodel.models.model_dict import MODEL_DICT, rmse43darr
from datasets.data_postprocess import plot_train_iteration


def evaluate(individual, x_input, y_true, warmup_length, model):
Expand Down
Loading

0 comments on commit f3c1d38

Please sign in to comment.