Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

With python, I get 2 even when using Decimals.

    Python 2.7.3 (default, Oct 26 2016, 21:01:49)
    [GCC 4.6.3] on linux2
    Type "help", "copyright", "credits" or "license" for more 
    information.
    >>> from decimal import *
    >>> getcontext().prec
    28
    >>> a=Decimal(9999999999999999.0)
    >>> b=Decimal(9999999999999998.0)
    >>> a-b
    Decimal('2')
That is unexpected.


The issue is that 9999999999999999.0 == 10000000000000000.0. You need to pass a string: Decimal('9999999999999999.0')


This is because you create a python float (64-bit) before passing this float to the Decimal class.

  >>> a = 9999999999999999.0
  >>> a
  1e+16


  >>> Decimal('9999999999999999.0')-Decimal('9999999999999998.0')
  Decimal('1.0')




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: