You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With Rational<BigInt> values, incorrect infinities or NaNs or zeroes are returned for doubleValue when the numerators or denominator or both are too large themselves to fit inside a Double, even when the values themselves fit.
It would be great to document this behavior or even to correct for it.
Example test program:
import Foundation
import NumberKit
var x: Rational<BigInt> = 1
while true {
print("\(x)")
print("numerator.doubleValue = \(x.numerator.doubleValue)")
print("denominator.doubleValue = \(x.denominator.doubleValue)")
print("doubleValue = \(x.doubleValue)")
if x.doubleValue.isNaN {
break
}
x *= 101
x /= 100
print()
}
The text was updated successfully, but these errors were encountered:
detreville
changed the title
Incorrect infinities and NanN and zeroes with doubleValue
Incorrect infinities and NaNs and zeroes with doubleValue
Jun 17, 2023
detreville
changed the title
Incorrect infinities and NaNs and zeroes with doubleValue
doubleValue returns incorrect infinities and NaNs and zeroes
Jun 17, 2023
With
Rational<BigInt>
values, incorrect infinities or NaNs or zeroes are returned fordoubleValue
when the numerators or denominator or both are too large themselves to fit inside aDouble
, even when the values themselves fit.It would be great to document this behavior or even to correct for it.
Example test program:
Example test output:
...
The text was updated successfully, but these errors were encountered: