Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convolve incorrectly handles large gaps in Q values #140

Open
bmaranville opened this issue Oct 4, 2021 · 2 comments
Open

Convolve incorrectly handles large gaps in Q values #140

bmaranville opened this issue Oct 4, 2021 · 2 comments

Comments

@bmaranville
Copy link
Member

In a dataset with a gap in Q (from masking, for instance) the current convolve_gaussian algorithm finds the left edge of the convolution window by moving forward through the x-values until x >= (xo - limit), then steps backwards while (x > xo - limit). This has the effect of always selecting one point beyond the end of (xo-limit), even if that point is very far away from the desired window. I think we'd be better served by dropping the second loop, and just keeping points where x >= (xo - limit).

@bmaranville
Copy link
Member Author

Why is the limit of Gaussian convolution set by this?

LOG_RESLIMIT = -6.90775527898213703123
limit = sqrt(-2.*sigma*sigma * LOG_RESLIMIT)

The result of this is that

limit == 3.716922 * sigma

while I thought in other communications it had been agreed to settle on 3-sigma as the cutoff?

@pkienzle
Copy link
Member

pkienzle commented Oct 7, 2021

Confused reasoning on my part.

As mentioned in the comment above the code, this is 0.001 of the peak value of the normal distribution. At the time I was trying to do a good approximation of the full Gaussian convolution and chose that as the cutoff. The code I was modifying used a pdf of 0.001 as the cutoff.

If we believe that resolution on reflectometers have truncated tails, then 3σ is a better choice. Feel free to change it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants