Question about floating point
On 08/28/2018 07:11 AM, Frank Millman wrote:
> Hi all
> I know about this gotcha -
>>>> x = 1.1 + 2.2
> According to the docs, the reason is that "numbers like 1.1 and 2.2 do
> not have exact representations in binary floating point."
> So when I do this -
>>>> y = 3.3
> what exactly is happening? What is 'y' at this point?
> Or if I do this -
>>>> z = (1.1 + 2.2) * 10 / 10
> What makes it different from the first example?
> Frank Millman
Python uses what the C folks call doubles, and what IEEE calls a
binary64. If you look down at the precision chart, you'll see that for
a value V, you have precision limits on the order of (V * 1e-16). If
you strip off the 3.3 that you wanted from the result that you got, your
error there is 3e-16, and all that's rounded for display, so all you can
really talk about is order of magnitude.
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.