Tone Equalizer curve fixes

[jenshannoschwalm]

Tone equalizer fixes

• Fix a check in get_luminance_from_buffer()
• pseudo_solve() problems are tracked while committing the parameters.
  This means we always have valid reports even if parameters didn't change and
  we re-open the image in darkroom.
• some refactoring
• some correct use of gboolean instead of int
• Use gboolean for 'checks' when calling pseudo_solve()

Fixes #17866

[TurboGit]

I still can reproduce the issue with this PR:

[TurboGit]

Still something wrong...

[jenshannoschwalm]

Added additional reports while in interactive mode, squashed and force-pushed.

@TurboGit i checked again and i can't reproduce a non-report any longer. Please note

• you might check while using -d params and modify the curve-smoothing slider. The warning only appears if the is a non-solved situation so your third curve would be mathematically ok - although looking quite strange - , your forth curve should be reported.
• runtime checks should report only if tone equalizer module is active.

Let me know if you still have the issue.
Release note might be: Fixed missing control logs about bad advanced tone equalizer curve

[TurboGit]

Still reproduce, when the curve go crazy (4th picture above) I get:

    38.2297 Cholesky decomposition returned NaNs
    38.2297 Cholesky LU triangular descent returned NaNs
    38.2297 Cholesky LU triangular ascent returned NaNs

On the 3rd example above, I see nothing reported and my picture gets really wrong so the "crazy" curve is applied.

[jenshannoschwalm]

On the 3rd example above, I see nothing reported and my picture gets really wrong so the "crazy" curve is applied.

Right. But how to solve that? I mean, that's how the solver works it's doesn't care about crazy curves right now, just about non solvable, basically it just checks for "avoid div by zero". (See choleski.h _choleski_decompose_safe, _triangular_descent_safe and _triangular_ascent_safe.

We could probably use some eps instead of comparing to zero?
Do that now and here or in another PR possibly even for 5.2?

[jenshannoschwalm]

@TurboGit i added a version with tests against close-to-zero instead of being zero. Looks really good. As the solver is not used elsewhere it might even be possible to use it for 5.0

[TurboGit]

Hum... I still have the issue on my side with this latest version. I'll say let's this for 5.2, I don't feel like changing this so close from the release.

[jenshannoschwalm]

Not sure if i misunderstood. Do you have a situation where a) there are logs about bad data but b) no control_log message? If so, could you share the xmp?

[TurboGit]

With this situation:

I have nothing in the debug log on console and no control log.

XMP:
6Q1B7618.CR2.xmp.txt

[jenshannoschwalm]

Edited comment: I see and agree.

So for now only the reporting in color equalizer has been fixed and we sill require a better solution for the solver.