Reduction process: effective-instrument geometry.

[ekapadi]

Description of work

At the end of the reduction process, the instrument associated with each output workspace is modified. A new effective instrument is substituted for the instrument attached to each workspace. This instrument has the same number of pixels as there are group-ids, and the location of each pixel is set to the mean location of the unmasked original pixels participating in that pixel group. By implication, this substitution results in there being one pixel per spectrum in each output workspace. (An exception to this substitution is the combined pixel-mask workspace, which is not modified.)

Explanation of work

This commit includes the following changes:

• A new EffectiveInstrumentRecipe implemented as a subrecipe called by ReductionRecipe for each grouping;

• Modification of ReductionIngredients to include the unmasked PixelGroups;

• Modification of SousChef.prepReductionIngredients to include the unmasked PixelGroups;

• Modification of existing unit tests, and implementation of new unit tests to verify the new subrecipe's execution.

A known issue: EditInstrumentGeometry deletes, but does not regenerate the XML associated with the newly-edited instrument. Unfortunately, this tends to generate a lot of misleading log messages when the associated workspaces are saved, and then reloaded. This isn't so much an issue with EditInstrumentGeometry specifically, but rather with the Mantid instrument framework.

To test

Existing unit tests have been modified, and new unit tests have been added to cover the new subrecipe. A new CIS-test script has been added at tests/cis_tests/effective_instrument_script.py.

Dev testing

In addition to running the unit tests, and the CIS-test script, the GUI-panels should be run for the full set of workflows. There should be no observable behavior changes. (With the exception of the log messages associated with the new output changes, and a warning message associated with the "known issue" indicated above.) Note that you'll also need the newly-saved output from the reduction workflow to test this PR.

The instrument associated with any output workspace can be examined directly. It should now have the same number of pixels as there are spectra in the workspace. Please see (and run through) the new CIS-test script at: tests/cis_tests/effective_instrument_script.py.

CIS testing

The new changes function automatically as part of the reduction process.
The instruments attached to the reduction-output workspaces should be examined, and verified to be as expected.
For comparison purposes, or in the case a work-around is required, the "application.yml" flag reduction.output.useEffectiveInstrument can be used to turn off the effective-instrument substitution.

Link to EWM item

EWM#6549

Verification

• [ x] the author has read the EWM story and acceptance critera
• the reviewer has read the EWM story and acceptance criteria
• the reviewer certifies the acceptance criteria below reflect the criteria in EWM

Acceptance Criteria

1. Where possible, all new changes are covered by unit tests.

2. Reduction output workspaces will have a new effective instrument that has the same number of pixels as there are spectra in the workspace.  (And by assumption, these workspaces will have a number of spectra that correspond to the number of group-ids in the associated grouping.)

3. The name of the effective instrument will be "SNAP_<grouping name>".

4. Locations of each effective pixel will correspond to those of the unmasked PGP values for that pixel's group-id.  Note that tests/cis_tests/effective_instrument_script.py can be used to obtain and verify these values.  Mantid-workbench instrument view might also be a useful way to verify this quickly.

This list is for ease of reference, and does not replace reading the EWM story as part of the review. Verify this list matches the EWM story before reviewing.

• acceptance criterion 1
• acceptance criterion 2
• acceptance criterion 3
• acceptance criterion 4

[ekapadi]

I'm adding a cis-test script to help with verifying acceptance criterion #4.

[ekapadi]

I ran into a snag with the acceptance criteria, so please wait to review this PR. There are some required Mantid changes in order to get the CIS-test to work. :(

[codecov]

Codecov Report

Attention: Patch coverage is 96.34146% with 3 lines in your changes missing coverage. Please review.

Project coverage is 96.51%. Comparing base (bb8cf23) to head (fc4cfe9).
Report is 2 commits behind head on next.

Files with missing lines	Patch %	Lines
...napred/backend/recipe/EffectiveInstrumentRecipe.py	92.50%	3 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             next     #461       +/-   ##
===========================================
+ Coverage   59.51%   96.51%   +37.00%     
===========================================
  Files          64       66        +2     
  Lines        4836     4962      +126     
===========================================
+ Hits         2878     4789     +1911     
+ Misses       1958      173     -1785     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ekapadi]

Everything is now ready for review! :)

[walshmm]

Calibration and Normalization run without issue.
Reduction produces workspaces with the expected instrument name
but I'm not seeing any instrument in the instrument view on completion of reduction

[walshmm]

they reload with the new instrument but still none in the instrument viewer

[walshmm]

(the cis test script did pass with my data subbed in)