NO

Author Topic: Precision problem with doubles  (Read 4782 times)

cane

  • Guest
Precision problem with doubles
« on: October 17, 2005, 11:37:35 AM »
Hi Pelle, I was trying to develop a little program to dump (bin & hex) FPU values (IEEE double precision floating point only). I've compiled it with PellesC 4 and I've noticed a certain lack of precision compared to other compilers (I tried DMC, BCC, VC, OpenWatcom and GCC).

Compile & run the program I've attached and enter 23.85 when prompted.

You'll notice that PellesC will give you these dumps:

IEEE bin: 0 10000000011 0111110110011001100110011001100110011001100110011001
RAW hex:  40 37 D9 99 99 99 99 99

Which are wrong in the last 2 binary digits; while other compilers will give you the correct dumps:

IEEE bin: 0 10000000011 0111110110011001100110011001100110011001100110011010
RAW hex:  40 37 D9 99 99 99 99 9A

Hope this helps.

Thanks, bye.

Offline Pelle

  • Administrator
  • Member
  • *****
  • Posts: 2266
    • http://www.smorgasbordet.com
Precision problem with doubles
« Reply #1 on: October 17, 2005, 04:52:30 PM »
It's a known problem with low priority. I might look at it in the future.

Pelle
/Pelle

cane

  • Guest
Precision problem with doubles
« Reply #2 on: October 17, 2005, 10:57:27 PM »
I don't want to sound pedant, but a bug like this seems rather critical, and its correction should be prioritized.
And not only because it prevents me from using PellesC for my present project, but also beacuse:
  • the representation is wrong per se;
  • error propagation, happening during math operations, produces and amplifies totally unreliable results.

Online AlexN

  • Global Moderator
  • Member
  • *****
  • Posts: 394
    • Alex's Link Sammlung
Precision problem with doubles
« Reply #3 on: October 18, 2005, 07:57:15 AM »
Quote from: "Pelle"
It's a known problem with low priority. I might look at it in the future.


Until Pelle will patch this problem, you can try to link with msvcrt.lib. In this case your example works correct.
best regards
 Alex ;)