osdir.com


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Question about floating point


On Tue, 28 Aug 2018 at 15:50, Frank Millman <frank at chagford.com> wrote:
>
> "Frank Millman"  wrote in message news:pm3l2m$kv4$1 at blaine.gmane.org...
> >
> > I know about this gotcha -
> >
> > >>> x = 1.1 + 2.2
> > >>> x
> > 3.3000000000000003
> >
> [...]
> >
> > >>> y = 3.3
> > >>> y
> > 3.3
> >
> [...]
> >
> > >>> z = (1.1 + 2.2) * 10 / 10
> > >>> z
> > 3.3
>
> Thanks to Chris and Stephen for the replies.
>
> They were interesting, but actually did not answer the question that I
> forgot to ask!
>
> The reason for my query is this. I am assisting someone with an application
> involving monetary values. I have been trying to explain why they should use
> the decimal module. They have had a counter-argument from someone else who
> says they should just use the rounding technique in my third example above.
>
> My gut-feel says that they are wrong, but I don't have the knowledge to
> prove it. I think a lot depends on what they are going to do with this value
> down the line - maybe it is 'safe enough' for their purposes, so I don't
> want to overstate my case. Are there edge cases where that rounding method
> could fail?

That rounding method is not suitable. As mentioned by others you
should at least use round() if working with floats. Here's an example
that shows that it doesn't work in the way you want:

>>> 0.11 - 0.1
0.009999999999999995
>>> (0.11 - 0.1) * 10 / 10
0.009999999999999995
>>> round(0.11 - 0.1, 2)
0.01

I wouldn't use floats for this though.

--
Oscar