Internal: Add logic to define and parse METplus inter-component version dependencies

[JohnHalleyGotway]

Describe the New Feature

This topic was discussed during the METplus All Hands meeting on April 24, 2024 and documented in this section of the METplus Engineering meeting notes. Each METplus coordinated release consists of a collection of METplus component releases. The integration testing performed in METplus (for all METplus components) and METviewer (for METplus-Analysis components) needs to know which component versions correspond with each other.

Currently, METplus uses numerical offsets to compute corresponding component version numbers. METplus version X.Y corresponds to:

• MET version X+6.Y
• METviewer version X.Y
• METplotpy version X-3.Y
• METcalcpy version X-3.Y
• METdataio version X-3.Y

While this logic does work, it does NOT work for METexpress which releases on a different cycle. And these relationships may break in the future depending on future funding levels. Note also, that this logic is not currently well-implemented for handling METviewer dependencies.

The METplus team decided that it would prefer to move from a numerical offset solution to a table lookup. Recommend adding a Python file to the METplus repo to define a table for all prior METplus coordinated releases along with utility functions to parse that table. Also consider how these version dependencies relate to the manage-externals and avoid duplication of logic as much as possible.

• Define a table with 7 columns (one each for METplus, MET, METviewer, METplotpy, METcalcpy, METdataio, and METexpress) and 1 row for each prior coordinated release.
• Define a utility function that...
  • Takes as input a current component and version number.
  • Takes as input a requested METplus component name.
  • Takes as input a requested version "type", such as release version number (e.g. vX.Y.Z), branch name (e.g. main_vX.Y or develop), or DockerHub tag (e.g. MET-11.1-latest).
  • Provides as output the corresponding version name.
• Many questions remain:
  • How to get the latest bugfix number for the vX.Y release? Perhaps use a vX.Y-latest GitHub tag? And define a GitHub action to auto-update the vX.Y-latest tags?
  • When should develop be returned? And what about beta and rc releases?
  • What about DockerHub image names?
• Once this functionality exists, update METplus to make use of it.
• Remember to update the METplus Release Guide to include updating this coordinated release table.
• Define a METviewer issues to retrieve this table lookup from the METplus develop branch and make good use of it. -- Make use of METplus version lookup script METviewer#554 was created to address this

The goal throughout is avoiding hard-coding METplus component version dependencies and instead define that logic in a single spot.

Acceptance Testing

List input data types and sources.
Describe tests required for new functionality.

Time Estimate

3 days?

Sub-Issues

Consider breaking the new feature down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2782544 (NOAA Base)

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED CYCLE ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
• Will need a METviewer issue at a minimum.

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Add any new Python packages to the METplus Components Python Requirements table.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[georgemccabe]

The script and lookup table were added with PR #2733. I am leaving this issue open because I still need to add instructions to update the version numbers to the Release Guide.