Feature/nl format

[drnimbusrain]

@zmoon @angehung5 Since we have over 40 NL options and growing, need to organize better. I also added some python/README changes for running global data and an example slurm script to help. Please let me know ASAP what you think so we can update develop going forward.

[angehung5]

I changed the required cpu time and mem in the slurm script and readme based on my slurm settings on Hopper. Time might be a little off since I set ntask=12, but mem should be accurate (could be less depending on the case but it would be safer to put 60G).

[drnimbusrain]

@angehung5 Please see my comments on your suggested memory updates here.

[drnimbusrain]

@angehung5 A single node/task with your suggestion of 60GB memory may not work, please test this example.

I don't think we really need to increase ntasks because its not parallel code.

From my experience using a single node/task requires like about 256 GB memory.

[drnimbusrain]

@angehung5 A single node/task with your suggestion of 60GB memory may not work, please test this example.

I don't think we really need to increase ntasks because its not parallel code.

From my experience using a single node/task requires like about 256 GB memory.

[angehung5]

Tested on Hopper. For three time steps with node=1 and ntask=1:

Cores: 1
CPU Utilized: 01:12:51
CPU Efficiency: 95.37% of 01:16:23 core-walltime
Job Wall-clock time: 01:16:23
Memory Utilized: 9.57 GB

I put 12G just in case.

[drnimbusrain]

1:16

@angehung5 OK, thank you. Is this for running both the python global process script and canopy-app for those times?

[angehung5]

1:16

@angehung5 OK, thank you. Is this for running both the python global process script and canopy-app for those times?

global process script only.

[drnimbusrain]

1:16

@angehung5 OK, thank you. Is this for running both the python global process script and canopy-app for those times?

global process script only.

@angehung5 Right, then the memory suggestion is not correct, and would need to be largely increased for canopy-app too. We need to test run both at same time/job.

[drnimbusrain]

Closing this PR due to updated new PR from @angehung5 built on latest commit.