You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hey all,
seems that the sanity check is using absolute error (difference in results) in checking definition of the vectorspace. When using a large-valued vector, this error is sometimes not achievable due to machine precision.
In my example, the inner products are of the order of 1e8 with a 1e-9 absolute error. This causes the check to fail with the default tolerance of 1e-12.
To me, it makes sense to turn this into a check on relative error (ratio of results) rather than absolute (difference of results) as to avoid machine precision issues.
Hey all,
seems that the sanity check is using absolute error (difference in results) in checking definition of the vectorspace. When using a large-valued vector, this error is sometimes not achievable due to machine precision.
In my example, the inner products are of the order of 1e8 with a 1e-9 absolute error. This causes the check to fail with the default tolerance of 1e-12.
To me, it makes sense to turn this into a check on relative error (ratio of results) rather than absolute (difference of results) as to avoid machine precision issues.
modred/modred/vectorspace.py
Line 156 in bbe35c8
modred/modred/vectorspace.py
Lines 202 to 207 in bbe35c8
The text was updated successfully, but these errors were encountered: