Internal: Improve NetCDF file diff to match MET diff functionality

[georgemccabe]

This came up in the comments of issue #2704.

Describe the Enhancement

Improve the logic in the METplus diff code to catch the differences flagged by the MET diff tests but not in METplus. @JohnHalleyGotway will comment to provide some sample files to use for testing.

Time Estimate

Estimate the amount of work required here.
Issues should represent approximately 1 to 3 days of work.

Sub-Issues

Consider breaking the enhancement down into sub-issues.

• Add a checkbox for each sub-issue here.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

@natalieb-noaa is funded through the DTC to work on METplus tasks.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as a METplus-Wrappers-X.Y.Z version, Consider for Next Release, or Backlog of Development Ideas
• For a METplus-Wrappers-X.Y.Z version, select the METplus-Wrappers-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Add any new Python packages to the METplus Components Python Requirements table.
• For any new datasets, an entry to the METplus Verification Datasets Guide.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: METplus-Wrappers-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

Please note that differences to NetCDF MTD output file were flagged in this MET GHA Testing Workflow run. The diff.zip artifact can be found here:
diff.zip

In that zip file compare:
mtd/mtd_BASIC_20100517_010000V_obj_TRUTH.nc vs mtd/mtd_BASIC_20100517_010000V_obj_OUTPUT.nc

Where the order of the dimensions and details of the global attributes, as seen in output from ncdump -h.
While the order of the dimensions and variables really do not matter, we really should be checking for differences in the global and variable attributes. In MET the comp_nc.R script calls the compareNc utility which calls ncdump -h followed by diff to check for NetCDF header diffs. I'd recommend using a Python-only solution rather than adding a dependency on ncdump, if possible and convenient.

[JohnHalleyGotway]

Reassigned this issue @natalieb-noaa so that she can work on it in conjunction with dtcenter/MET#2718, which updates MET to use the differencing logic provided by METplus. Seems like it'd be good to do these at the same time.