Error when using git-fleximod for noresm2_5_alpha08

[TomasTorsvik]

Describe the bug
Please provide a clear and concise description of what the bug is.

• NorESM version: noresm2_5_alpha08
• HPC platform: betzy; login-2
• Compiler (if applicable):
• Compset (if applicable):
• Resolution (if applicable):
• Error message (if applicable):

$ ./bin/git-fleximod update
          parallelio up to date.
                 mct up to date.
          ccs_config up to date.
                cime up to date.
Recursively checking out submodules of cime
               share up to date.
                blom up to date.
Recursively checking out submodules of blom
                 cam up to date.
Recursively checking out submodules of cam
Sparse checkout clubb fxsparse ../.clubb_sparse_checkout
Successfully checked out                clubb at clubb_4ncar_20240605_73d60f6_gpufixes_posinf
Sparse checkout cosp2 fxsparse ../.cosp_sparse_checkout
Successfully checked out                cosp2 at v2.1.4cesm
Recursively checking out submodules of hemco
Recursively checking out submodules of HEMCO
Skipping optional component             ali_arms
Skipping optional component              cloud_j
Skipping optional component             geoschem
Skipping optional component                 hetp
Skipping optional component                 mpas
               cdeps up to date.
Recursively checking out submodules of cdeps
               cice6 up to date.
Recursively checking out submodules of cice6
Recursively checking out submodules of src
Cmd('git') failed due to: exit code(128)
  cmdline: git fetch origin refs/tags/cismwrap_2_2_002_noresm_v1:refs/tags/cismwrap_2_2_002_noresm_v1
  stderr: 'fatal: couldn't find remote ref refs/tags/cismwrap_2_2_002_noresm_v1'

To Reproduce
Steps to reproduce the behavior:

1. clone NorESM
2. git checkout noresm2_5_alpha08
3. ./bin/git-fleximod update

Expected behavior
After running git-fleximod, I have cism component with git status : HEAD detached at 41843ef

[gold2718]

@TomasTorsvik, was this on a fresh clone? If not, what was the tree status before you ran the update command?

I cannot reproduce this. Here are my steps:

git clone https://github.com/NorESMhub/NorESM alpha08
cd alpha08
git checkout noresm2_5_alpha08
./bin/git-fleximod update

I performed this sequence successfully on login-3 of Betzy and also on login-2 and my local workstation.
git version was 2.45.1

I think this may have been a glitch or a temporary disk issue on login-2 as I cannot reproduce this. Closing.
Please re-open if you still have problems?

[TomasTorsvik]

@gold2718 , @mvertens - CISM is pointing to the ESCOMP repository in the noresm_develop branch. This should be changed to point to the NorESMhub repository. Or am I missing something?

[submodule "cism"]
path = components/cism
url = https://github.com/ESCOMP/CISM-wrapper
fxtag = cismwrap_2_2_002_noresm_v1
fxrequired = ToplevelRequired
fxDONOTUSEurl = https://github.com/ESCOMP/CISM-wrapper

[mvertens]

@TomasTorsvik - so this is one of the real problems with git-fleximod. In my case it didn't find the tag and it just went on.
Your suggested fix is correct but we also need to update the submodule correctly for cism.

If I do the following:

$git submodule status
3640d984aad755c2b536abe2ddbcb428cca130ae ccs_config (ccs_config_noresm0.0.38)
f4a63723ead19bf5391abf21e5d49bd3741f7e39 cime (cime6.1.28_noresm_v0)
816981ab062ae466851bd4d239ecf3d912f94011 components/blom (dev1.6.1.5-27-g816981a)
85f35cca4f65927f1a398ccbb0dc2a462b6be4fd components/cam (noresm2_5_016_cam6_4_041)
f6bc97483a1bfb7352c6c5610a13ed898a86990b components/cdeps (cdeps1.0.53)
b0bf4fef4aa944c57c0972b66411882e20d3868d components/cice (cesm_cice6_5_0_20240702_noresm_v2)
41843ef8fed91fcf60e2ea217c4f6f2ee5133c5d components/cism (cismwrap_2_2_005) <=======
ca5cddbcfe543b89e8f9e6b5fb1941b8c8297925 components/clm (ctsm5.3.11-noresm_v1)
1d456df89ce4a1aefe7cf9f5564c53ebe3dab41f components/cmeps (cmeps1.0.20_noresm_v0)
e2ffe00004cc416cfc8bcfae2a949474075c1d1f components/mosart (mosart1.1.02)
4e0ac3d700a151e46a9219ed8773928881e3ba2b components/ww3 (ww3_interface_noresm0.0.15)
82b0071e69d14330b75d23b0bc68543ebea9aadc libraries/mct (MCT_2.11.0)
f52ade075619b32fa141993b5665b0fe099befc2 libraries/parallelio (pio2_5_9-172-gf52ade07)
b26285a01ba00c2c7f31182708827096b1bf93a8 share (share1.1.2_noresm_v0)
-e13f2c61fe6a5136ae1fa034abb5e6973ed40077 tools/statistical_ensemble_test/pyCECT

If I apply your suggested fix - which is to change the .gitmodules at the top level to be

[submodule "cism"]
path = components/cism
url = https://github.com/NorESMhub/CISM-wrapper <====
fxtag = cismwrap_2_2_002_noresm_v1
fxrequired = ToplevelRequired
fxDONOTUSEurl = https://github.com/NorESMhub/CISM-wrapper <====

the the following results:

$ git submodule status
 3640d984aad755c2b536abe2ddbcb428cca130ae ccs_config (ccs_config_noresm0.0.38)
 f4a63723ead19bf5391abf21e5d49bd3741f7e39 cime (cime6.1.28_noresm_v0)
 816981ab062ae466851bd4d239ecf3d912f94011 components/blom (dev1.6.1.5-27-g816981a)
 85f35cca4f65927f1a398ccbb0dc2a462b6be4fd components/cam (noresm2_5_016_cam6_4_041)
 f6bc97483a1bfb7352c6c5610a13ed898a86990b components/cdeps (cdeps1.0.53)
 b0bf4fef4aa944c57c0972b66411882e20d3868d components/cice (cesm_cice6_5_0_20240702_noresm_v2)
+34c8df43e4ab87bf9e8629c48393b73592734753 components/cism (cismwrap_2_2_002_noresm_v1) <=====
 ca5cddbcfe543b89e8f9e6b5fb1941b8c8297925 components/clm (ctsm5.3.11-noresm_v1)
 1d456df89ce4a1aefe7cf9f5564c53ebe3dab41f components/cmeps (cmeps1.0.20_noresm_v0)
 e2ffe00004cc416cfc8bcfae2a949474075c1d1f components/mosart (mosart1.1.02)
 4e0ac3d700a151e46a9219ed8773928881e3ba2b components/ww3 (ww3_interface_noresm0.0.15)
 82b0071e69d14330b75d23b0bc68543ebea9aadc libraries/mct (MCT_2.11.0)
 f52ade075619b32fa141993b5665b0fe099befc2 libraries/parallelio (pio2_5_9-172-gf52ade07)
 b26285a01ba00c2c7f31182708827096b1bf93a8 share (share1.1.2_noresm_v0)
-e13f2c61fe6a5136ae1fa034abb5e6973ed40077 tools/statistical_ensemble_test/pyCECT

You will have checked out the right branch BUT it is not consistent with the internal git submodule entry inside the .git directory tree. So that needs to be fixed as well.

@gold2718, @mvdebolskiy and I have talked today about how we want to proceed to with addressing NorESM needs. For now I think we need to work with git-fleximod and understand how to fix things and at the same time work to implement a better solution for our needs.

What we plan to do is to create a noresm2_5_alpha08b tag that has the correct fixes for cism and an updated ChangeLog.
@adagj - since you are not using active cism in your compset I think it's fine to use the sandbox you create with noresm2_5_alpha08 for the upcoming simulation.

[TomasTorsvik]

@mvertens - thanks for the update. It's not a major problem for me, since I'm not running with CISM. It breaks my test script, but I can work around it manually.

[oyvindseland]

I am getting the same error message after

git clone https://github.com/NorESMhub/NorESM.git
cd NorESM
git checkout noresm2_5_alpha08
./bin/git-fleximod update

Trying to switch node

[TomasTorsvik]

@oyvindseland - Switching node will probably not change anything. The entry in .gitmodule for cism is wrong in the repository, because there is no tag cismwrap_2_2_002_noresm_v1 in the ESCOMP repository. This can be fixed after checking out noresm2_5_alpha08 by editing the url under the cism entry:

[submodule "cism"]
path = components/cism
url = https://github.com/NorESMhub/CISM-wrapper
fxtag = cismwrap_2_2_002_noresm_v1
fxrequired = ToplevelRequired
fxDONOTUSEurl = https://github.com/NorESMhub/CISM-wrapper

[oyvindseland]

OK. I guess it will be updated in the next tag.

I am setting up a 20 year simulation and will add the information in the simulation overview

[mvertens]

There is a new PR for noresm2_5_alpha08b that will fix this. But I think for this simulation not having a correct cism checkout should not be a problem.

[TomasTorsvik]

Fixed with tag noresm2_5_alpha08.