- Welcome to Pelles C forum.
Pelles C forum
News:
Download Pelles C here: http://www.pellesc.se
Recent posts
#71
Tips & tricks / Re: UTF-8 multilanguage consol...
Last post by Michele - September 18, 2026, 09:26:00 AMTimovjl you successfully changed the codepage to UTF-8, in fact many characters are correctly displayed.
I agree that now the problem is the console font that doesn't contains the full set.
A known font known to render all charset is 'Consolas', try changing the face font with:
If still doesn't it confirms that the fonts in your OS version are limited.
You said that on a Win10/11 system it works?
I agree that now the problem is the console font that doesn't contains the full set.
A known font known to render all charset is 'Consolas', try changing the face font with:
Code Select
BOOL SetConsoleFontToConsolas(void)
{
HANDLE hConsole = GetStdHandle(STD_OUTPUT_HANDLE);
if (hConsole == INVALID_HANDLE_VALUE) return FALSE;
CONSOLE_FONT_INFOEX cfi;
cfi.cbSize = sizeof(CONSOLE_FONT_INFOEX);
// Get the current font info first to preserve defaults
if (!GetCurrentConsoleFontEx(hConsole, FALSE, &cfi))
{
return FALSE;
}
// Set font properties
cfi.dwFontSize.X = 0; // Width (0 lets Windows calculate based on height)
cfi.dwFontSize.Y = 16; // Height (Font size in points/pixels)
cfi.FontFamily = FF_DONTCARE;
cfi.FontWeight = FW_NORMAL;
// Specify a TrueType font that supports UTF-8 Unicode characters
// Common choices: L"Cascadia Code", L"Consolas", L"Lucida Console"
wcscpy_s(cfi.FaceName, LF_FACESIZE, L"Consolas");
// Apply the new font
return SetCurrentConsoleFontEx(hConsole, FALSE, &cfi);
}If still doesn't it confirms that the fonts in your OS version are limited.
You said that on a Win10/11 system it works?
#72
Tips & tricks / Re: UTF-8 multilanguage consol...
Last post by TimoVJL - September 18, 2026, 09:03:07 AMWindows 7 x64 EN console.
Less fonts than Windows 1x versions.
Less fonts than Windows 1x versions.
#73
Tips & tricks / Re: UTF-8 multilanguage consol...
Last post by Michele - September 17, 2026, 10:06:40 PMHello Vortex, I imagine that the problem is the OS version.
I will not add mine, just report what Gemini says:
I will not add mine, just report what Gemini says:
QuoteThe functions SetConsoleOutputCP and SetConsoleCP along with the CP_UTF8 code page constant (code page 65001) have been supported since Windows 2000.
However, the minimal OS version depends on what you are trying to achieve:
Basic Function Support (API availability): Windows 2000 or later. Calling SetConsoleOutputCP(65001) is supported on all Windows NT-based OS versions starting from 2000, as well as Windows XP, Vista, 7, 8, and 10.
Functional UTF-8 Console Rendering: Windows 10 Version 1903 (Build 18362) or later.
Why the distinction matters:
While the functions exist on older OS versions, UTF-8 support inside cmd.exe on legacy releases (Windows 7/8/early Win 10) was notoriously buggy:
Legacy Bugs: On older versions like Windows 7, setting the console to code page 65001 frequently broke C runtime output routines (printf), caused character truncation, or crashed batch scripts.
Font Rendering: Legacy consoles couldn't render multi-byte UTF-8 sequences properly unless using specific TrueType fonts.
Full ConPTY Overhaul: Microsoft fully overhauled the Windows Console infrastructure in Windows 10 version 1903 (Build 18362)—the exact same release that introduced manifest-based UTF-8 ACP support—making UTF-8 output reliable and stable in the terminal.
#74
Tips & tricks / Re: UTF-8 multilanguage consol...
Last post by Vortex - September 17, 2026, 10:01:22 PMHi Michele,
Thanks for your code. Testing your executable on my Windows 7 Sp1 64-bit, I get a lot of garbled characters, probably a limitation of Windows 7.
Thanks for your code. Testing your executable on my Windows 7 Sp1 64-bit, I get a lot of garbled characters, probably a limitation of Windows 7.
#75
Tips & tricks / UTF-8 with windows mixed ANSI/...
Last post by Michele - September 17, 2026, 09:52:41 PMThis other example shows how to use the manifest to change the process codepage for a window program.
When we change the process codepage to UTF-8 the ANSI version of many (warning not all!) windows messages automatically translates from process codepage to internal UTF-16 (unicode) format.
Winapi explicitly allows the mixed use of ANSI and WIDE messages providing behind the scenes the translation from between the process codepage and unicode. This means that you can create an ANSI window/control and send to it wide messages or the reverse, and also means that you are free to create a unicode or ansi application as you like and use which encoding you prefer.
But if your codepage doesn't cover the whole range of alphabets, i.e. if you use a typical latin western 1252 encoding, supplying an utf8 string will produce garbage output.
The trick is to set the process codepage to UTF-8 using the specific manifest command, but this feature has benn introduced starting from Windows 10 Version 1903 (Build 18362), and will be silently ignored in previous versions. The manifest code is:
The idea was born creating a program based on sqlite3 that is UTF-8 based.
If your system does'nt supports the process codepage change you can use the standar conversion functions:
When we change the process codepage to UTF-8 the ANSI version of many (warning not all!) windows messages automatically translates from process codepage to internal UTF-16 (unicode) format.
Winapi explicitly allows the mixed use of ANSI and WIDE messages providing behind the scenes the translation from between the process codepage and unicode. This means that you can create an ANSI window/control and send to it wide messages or the reverse, and also means that you are free to create a unicode or ansi application as you like and use which encoding you prefer.
But if your codepage doesn't cover the whole range of alphabets, i.e. if you use a typical latin western 1252 encoding, supplying an utf8 string will produce garbage output.
The trick is to set the process codepage to UTF-8 using the specific manifest command, but this feature has benn introduced starting from Windows 10 Version 1903 (Build 18362), and will be silently ignored in previous versions. The manifest code is:
Code Select
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings>
<activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
</windowsSettings>
</application>
The attached sample demonstrate how it works. You can test it in both versions ANSI and WIDECHAR, just change the '#if 1' to '#if 0' to compile in ANSI mode.The idea was born creating a program based on sqlite3 that is UTF-8 based.
If your system does'nt supports the process codepage change you can use the standar conversion functions:
Code Select
#include <windows.h>
#include <stdlib.h>
#include <stdio.h>
// 1. UTF-8 (multibyte char*) -> UTF-16 (wchar_t*)
wchar_t* Utf8ToUtf16(const char* utf8_str) {
if (!utf8_str) return NULL;
// Get required buffer size in wchar_t elements (including null-terminator)
int size_needed = MultiByteToWideChar(CP_UTF8, 0, utf8_str, -1, NULL, 0);
if (size_needed == 0) return NULL;
wchar_t* wstr = (wchar_t*)malloc(size_needed * sizeof(wchar_t));
if (!wstr) return NULL;
// Perform conversion
MultiByteToWideChar(CP_UTF8, 0, utf8_str, -1, wstr, size_needed);
return wstr; // Caller must free()
}
// 2. UTF-16 (wchar_t*) -> UTF-8 (multibyte char*)
char* Utf16ToUtf8(const wchar_t* wstr) {
if (!wstr) return NULL;
// Get required buffer size in bytes (including null-terminator)
int size_needed = WideCharToMultiByte(CP_UTF8, 0, wstr, -1, NULL, 0, NULL, NULL);
if (size_needed == 0) return NULL;
char* utf8_str = (char*)malloc(size_needed);
if (!utf8_str) return NULL;
// Perform conversion
WideCharToMultiByte(CP_UTF8, 0, wstr, -1, utf8_str, size_needed, NULL, NULL);
return utf8_str; // Caller must free()
}
#76
Tips & tricks / UTF-8 multilanguage console pr...
Last post by Michele - September 17, 2026, 08:38:58 PMSomewhere on this forum there was a discussion about how to print multilanguage strings on console output.
The small attached sample shows the code that use the console codepage set functions using the UTF-8 encoding that is, at least for me, a very powerful option because while allows the multilanguage text let you use the simple and efficient ansi text functions (strcpy, ctrcat, strcmp, ...).
The small attached sample shows the code that use the console codepage set functions using the UTF-8 encoding that is, at least for me, a very powerful option because while allows the multilanguage text let you use the simple and efficient ansi text functions (strcpy, ctrcat, strcmp, ...).
#77
Add-ins / Re: Add-Ins
Last post by TimoVJL - September 17, 2026, 04:45:37 PMWhen someone want to see compiled C-code in assembler, easist way is to use Add-In, that show podump.exe output from object file in poide Output window in easy way.
The poide have over 50 Add-Ins with source code, so lot of code for everyone to study.
https://forum.pellesc.de/index.php?msg=29747
The poide have over 50 Add-Ins with source code, so lot of code for everyone to study.
https://forum.pellesc.de/index.php?msg=29747
#78
Add-ins / Re: Add-Ins
Last post by John Z - September 14, 2026, 09:55:47 PMDefinitely agree with you Timo ! The ability to do Add-In's is exceptional. User feature requests are where I started to develop Add-In's from requests for simple button on the toolbar to Linecounter+ to Copy Mode and Hint. You are the king of the Add-In's in my book.
Many might not know that a few revision back Pelle significantly enhanced the Add-In's capability for the Debug mode. Seems like a lot of potential there.
John Z
Many might not know that a few revision back Pelle significantly enhanced the Add-In's capability for the Debug mode. Seems like a lot of potential there.
John Z
#79
Bug reports / warning #2131 fires on defined...
Last post by KEL26 - September 12, 2026, 08:18:54 PMSubject: warning #2131 fires on defined shifts, and on expressions with no shift
Pelles C 14.50, Windows x64 and x86. At every optimisation level, and never
without one, the compiler reports undefined behaviour that is not present.
Minimal case, no headers needed:
Compiled with:
gives, one for each function:
Two things about that.
FIRST, the shift in pack_a is not undefined. in[1] & 0x0f is subject to the
usual arithmetic conversions, so the left operand of << has type int, with a
value of at most 15. Shifting it by 8 is fully defined.
SECOND, pack_b contains no shift operator at all. The multiplication by 256
is being rewritten as a shift, and the message then describes the rewritten
form as though it were what was written.
Remove the /O switch and both warnings disappear, on both targets, which is
what places this in an optimisation stage rather than the front end.
WHAT IS AND IS NOT AFFECTED
Sixteen one-line cases were compiled in twenty-four configurations: four
optimisation settings, two targets, three warning levels. The same six lines
warn in every configuration that has a /O switch, and none warns without one.
the same right-hand side, assigned to unsigned char warns
... to signed char warns
... to unsigned short, short, int, unsigned silent
the multiplication form, to unsigned char warns
the multiplication form, to unsigned short silent
shift by 7, to unsigned char silent
shift by 9, to unsigned char warns, "by 9 bits"
So the destination width decides it, and the shift count is being compared
against 8 rather than against the width of int.
The message also mis-spells the type for a signed char destination:
THE GENERATED CODE IS CORRECT
This is a diagnostic matter only. The same computation was run over all
65,536 input pairs under four Pelles C builds -- no /O, -Ot, -Os, -Ox -- and
under a second C compiler. All five produced identical checksums, 8355840 for
the 8-bit destination and 134184960 for the 16-bit one. Nothing is
miscompiled.
WHY IT IS WORTH FIXING
That expression is the ordinary way to unpack two 12-bit samples from three
bytes, and it appears verbatim in a widely used image library, inside a block
guarded by a run-time test for 12-bit precision. A build kept deliberately
clean gains a warning that says "undefined" about code that has none.
The condition being detected does look like a real one: the shifted bits are
lost in the narrowing conversion. The compiler already has #2215, "possible
loss of data or unexpected result", and emits it elsewhere in the same build.
Reporting these sites as #2215 would say something true.
A self-contained reproducer, the sixteen-case file, the value comparison and
a script that regenerates every figure above are attached.
Pelles C 14.50, Windows x64 and x86. At every optimisation level, and never
without one, the compiler reports undefined behaviour that is not present.
Minimal case, no headers needed:
Code Select
typedef unsigned char u8;
void pack_a(const u8 *in, u8 *out)
{
out[0] = (u8)(((in[1] & 0x0f) << 8) | in[2]);
}
void pack_b(const u8 *in, u8 *out)
{
out[0] = (u8)((((int)in[1]) & 0x0f) * 256 | in[2]);
}
Compiled with:
Code Select
pocc -Tx64-coff -std:C17 -Ze -Zx -Go -Ot -W1 -c d006_minimal.c
gives, one for each function:
Code Select
warning #2131: Shifting an 'unsigned char' by 8 bits is undefined.
Two things about that.
FIRST, the shift in pack_a is not undefined. in[1] & 0x0f is subject to the
usual arithmetic conversions, so the left operand of << has type int, with a
value of at most 15. Shifting it by 8 is fully defined.
SECOND, pack_b contains no shift operator at all. The multiplication by 256
is being rewritten as a shift, and the message then describes the rewritten
form as though it were what was written.
Remove the /O switch and both warnings disappear, on both targets, which is
what places this in an optimisation stage rather than the front end.
WHAT IS AND IS NOT AFFECTED
Sixteen one-line cases were compiled in twenty-four configurations: four
optimisation settings, two targets, three warning levels. The same six lines
warn in every configuration that has a /O switch, and none warns without one.
the same right-hand side, assigned to unsigned char warns
... to signed char warns
... to unsigned short, short, int, unsigned silent
the multiplication form, to unsigned char warns
the multiplication form, to unsigned short silent
shift by 7, to unsigned char silent
shift by 9, to unsigned char warns, "by 9 bits"
So the destination width decides it, and the shift count is being compared
against 8 rather than against the width of int.
The message also mis-spells the type for a signed char destination:
Code Select
warning #2131: Shifting an 'char' by 8 bits is undefined.
THE GENERATED CODE IS CORRECT
This is a diagnostic matter only. The same computation was run over all
65,536 input pairs under four Pelles C builds -- no /O, -Ot, -Os, -Ox -- and
under a second C compiler. All five produced identical checksums, 8355840 for
the 8-bit destination and 134184960 for the 16-bit one. Nothing is
miscompiled.
WHY IT IS WORTH FIXING
That expression is the ordinary way to unpack two 12-bit samples from three
bytes, and it appears verbatim in a widely used image library, inside a block
guarded by a run-time test for 12-bit precision. A build kept deliberately
clean gains a warning that says "undefined" about code that has none.
The condition being detected does look like a real one: the shifted bits are
lost in the narrowing conversion. The compiler already has #2215, "possible
loss of data or unexpected result", and emits it elsewhere in the same build.
Reporting these sites as #2215 would say something true.
A self-contained reproducer, the sixteen-case file, the value comparison and
a script that regenerates every figure above are attached.
#80
Bug reports / The digits beyond the fifteent...
Last post by KEL26 - September 12, 2026, 05:17:23 PMSummary
On Pelles C 14.50 (x64), the conversion of a double to decimal text produces
digits that are correct to about fifteen significant figures. Digits
requested beyond that are produced, but they are wrong, not merely
imprecise.
The practical consequence is that the round-trip guarantee implied by
DECIMAL_DIG does not hold:
-------------------------------- ------------------------------------------
DECIMAL_DIG declared by 17
<float.h>
DBL_DECIMAL_DIG declared by 17
<float.h>
finite normal doubles printed 48,191 of 100,000 -- 48.19%
with %.17g that do not read back
as the same value
-------------------------------- ------------------------------------------
The same test program, built with another toolchain's runtime on the same
machine and run on the same values, reports zero failures.
The input side is not affected: strtod reads a correctly rounded
seventeen-digit decimal back to the exact bit pattern. Hexadecimal output,
%a, is not affected either. The fault appears to lie solely in the
binary-to-decimal conversion used by the printf family.
A single self-contained test program is attached. It needs no data files, no
arguments and no other libraries, and it runs in under a second.
Environment
------------------------------------- -------------------------------------
Compiler Pelles ISO C Compiler, Version
14.50.0, x64
Switches -Tx64-coff -std:C17 -Ze -Zx
Link crt64.lib kernel32.lib, console
subsystem
Optimisation irrelevant: identical output
unoptimised and at -Ot, -Os and -Ox
FLT_EVAL_METHOD 0
------------------------------------- -------------------------------------
The optimiser was eliminated first, by rebuilding at every optimisation
level and comparing the output byte for byte. All four builds agree with one
another and all four disagree with the correctly rounded digits, so this is
a runtime library matter and not a code generation one.
How it was found
The work in progress was unrelated: making a C++ image processing library
usable from Pelles C by compiling its C++ once into an object file and
presenting a pure C interface, so that a Pelles C program can call it
without a DLL. In that arrangement the C++ code is compiled by another
compiler, but every C function it calls -- malloc, memcpy, fopen, printf,
cos, sin -- is resolved at the final link by Pelles C's own crt64.lib. That
is the point of the design: one C runtime, one heap, one set of file
handles.
To check that the C interface altered nothing, thirteen image processing
computations were written twice: once in C, calling through the interface,
and once in C++, calling the library directly. The two sets of output images
were compared byte for byte. Twelve of the thirteen were identical. One
differed in a single pixel, by one grey level.
Chasing that one pixel is what turned this up. The chain of elimination was:
1. The optimiser was ruled out, as described above.
2. Both computations were instrumented to compare their floating point in
memory, stage by stage, before anything was cast to a byte. All nine
stages were bit-identical, so the interface was not altering anything.
3. The small amount of arithmetic in the caller was compiled by both
compilers from the same source and compared: 51,062 floats,
bit-identical.
4. What remained was the mathematical library. Because the design routes
every C call to crt64.lib, the transform in question was using Pelles
C's cos and sin where the C++ comparison used the other toolchain's.
Measured over 4,096 angles: 2,061 of 8,192 values differ, by at most one
unit in the last place. That is not a fault in anything -- neither
library claims correctly rounded transcendentals -- and it fully
explains one pixel in 65,536 landing on the boundary between grey 31 and
grey 32.
The present report arises from step 4. While printing the cos values for
comparison, the two runtimes printed different decimal text for identical
bit patterns. That is not a question about mathematics; it is a question
about printf, and it turned out to be a real one.
The minimal test
decimal_dig_test.c, attached, does two things.
Part one takes three doubles as exact IEEE-754 bit patterns, so that no
decimal parsing happens before the test, and compares printf's output
against the correctly rounded digits. Those expected strings were obtained
independently, by arbitrary-precision arithmetic, and are written into the
program as literals.
Part two generates 100,000 finite normal doubles from a four-line xorshift
generator -- so the sample is identical on every machine and every run --
prints each with %.*g at DECIMAL_DIG, reads it back with strtod, and
compares the bits.
strtod is used as the reader because it was verified separately, against
exact bit patterns, to be correct on this implementation. If it were not,
part one would fail in the same direction and the two parts would disagree.
Results
Part one, on Pelles C 14.50
bits 3feffff9606a6fbc
%.17g expected 0.99999684172825409 got 0.999996841728254 <-- DIFFERS
%.20f expected 0.99999684172825409334 got 0.99999684172825400310 <-- DIFFERS
bits 3fefe9cdad01883a
%.17g expected 0.99729045667869021 got 0.99729045667869013 <-- DIFFERS
%.20f expected 0.99729045667869020697 got 0.99729045667869013413 <-- DIFFERS
bits 400921fb54442d18
%.17g expected 3.1415926535897931 got 3.1415926535897932 <-- DIFFERS
%.20f expected 3.14159265358979311600 got 3.14159265358979318989 <-- DIFFERS
Two features are worth separating.
The digits beyond the fifteenth are wrong. In the first value the correct
digits are ...254093 34, and what is produced is ...254003 10. This is not a
rounding difference at the last digit; the sixteenth digit onwards is a
different number.
Fewer digits are produced than were requested. %.17g on the first value
returns 0.999996841728254, which is fifteen significant figures, not
seventeen. In the 100,000-value sample this happens in 9.84% of cases.
Part two, on Pelles C 14.50
values tested 100000
failed to read back as the same double 48191 (48.19%)
printed with fewer digits than requested 9836 (9.84%)
The same program, another runtime, same machine, same values
values tested 100000
failed to read back as the same double 0 (0.00%)
How it varies with the precision requested
Over 4,000 finite normal doubles spanning the whole exponent range, counting
strings that differ from the correctly rounded conversion:
conversion Pelles C 14.50 control
------------ ---------------- ---------
%.15g 3.33% 0.00%
%.16g 29.93% 0.00%
%.17g 85.47% 0.00%
%.18g 97.75% 0.00%
And the round-trip failures over the same sample:
conversion Pelles C 14.50 control
------------ ---------------- ---------
%.16g 61.88% 44.77%
%.17g 48.10% 0.00%
%.18g 47.45% 0.00%
The %.16g row is expected to fail on both, since sixteen digits cannot name
every double; it is included as a control on the method. The %.17g row is
the one that matters, and the two columns should both be zero.
What is not affected
These were tested and are correct, which may help to localise the fault.
strtod is correct. Five exact seventeen-digit decimals were parsed and each
produced the expected bit pattern.
%a is correct. The hexadecimal form was checked across every binary exponent
from 2^-1070 to 2^1023, 2,094 values, and separately on DBL_MAX, DBL_MIN,
the largest subnormal and the smallest subnormal. All were exact. %a is
therefore a sound workaround for anyone who needs to write a double as text
without losing it.
The optimiser is not involved. Unoptimised, -Ot, -Os and -Ox all produce
identical output.
Why it matters
<float.h> on this implementation declares DECIMAL_DIG = 17 and
DBL_DECIMAL_DIG = 17. The C standard describes DECIMAL_DIG as the number of
decimal digits such that any floating value may be converted to decimal and
back with no change of value (C11, 5.2.4.2.2, paragraph 12). Seventeen is
the correct value for IEEE-754 double. The declaration is therefore right;
what is missing is the conversion that would make it true.
The consequence is quiet, which is what makes it serious. Any program that
writes double data as text -- a configuration file, a comma-separated
export, a log, a checkpoint to be resumed from, a scientific data file, a
coordinate in a header -- and that writes more than fifteen significant
figures in the belief that it is preserving the value, is altering the value
at the moment of writing, and nothing in the program will say so. The loss
is at the level of one or two units in the last place, which is exactly the
level at which it survives casual inspection and then accumulates.
A false lead, recorded so that nobody repeats it
An earlier version of the measuring program appeared to show %a producing a
grossly wrong result for DBL_MAX: 0x8.181818181818p-255 in place of
0x1.fffffffffffffp+1023. That was a fault in the measuring program, not in
Pelles C. The same program had earlier printed DBL_MAX with %.20f into a
128-byte buffer, and that conversion needs more than 330 characters. The
buffer overflow corrupted what followed.
It is recorded here because it was nearly reported as a defect, and because
the only reason it was not is that the result was checked a second time by a
different route.
How to reproduce
pocc -Tx64-coff -std:C17 -Ze -Zx decimal_dig_test.c
polink -subsystem:console -machine:x64 decimal_dig_test.obj ^
crt64.lib kernel32.lib -out:decimal_dig_test.exe
decimal_dig_test
The program returns 0 if both parts pass and 1 otherwise. For a comparison,
the identical source builds unchanged with any other C compiler.
Suggested area to look at
The pattern -- correct to about fifteen significant figures, then wrong
rather than absent, together with %g sometimes yielding fewer significant
digits than requested -- is consistent with the conversion being carried out
through an intermediate of limited precision, with the remaining digits
generated from a value that has already lost them, rather than by an exact
arbitrary-precision method such as Steele and White's Dragon4 or Grisu with
an exact fallback.
strtod on the same implementation is exact, so the necessary
arbitrary-precision machinery may already be present on the input side.
File attached 'decimal_dig_test.c':
/* ===========================================================================
* decimal_dig_test.c
* ---------------------------------------------------------------------------
* A self-contained test of one thing only: whether printf can write a double
* with DECIMAL_DIG significant digits such that strtod reads back the same
* value.
*
* <float.h> declares
*
* DECIMAL_DIG the number of decimal digits needed to hold any
* floating value of the widest supported type so that
* converting it to decimal and back changes nothing
* DBL_DECIMAL_DIG the same, for double
*
* The C standard describes DECIMAL_DIG (C11 5.2.4.2.2, paragraph 12) as the
* number of digits for which that round trip is guaranteed. This program
* simply takes the implementation at its word and checks.
*
* IT NEEDS NOTHING. No data files, no libraries beyond the standard ones,
* no command line arguments. It runs in under a second.
*
* HOW IT AVOIDS TESTING ITSELF
*
* Part one sets its values from EXACT BIT PATTERNS, so no decimal
* parsing takes place before the test. Each expected string was
* obtained independently, from arbitrary-precision arithmetic, and is
* written into this file as a literal.
*
* Part two uses strtod to read the printed text back. That is safe
* here: strtod was checked separately against exact bit patterns and is
* correct on this implementation. If it were not, part one would fail
* in the same way and the two parts would disagree.
*
* BUILD, Pelles C:
* pocc -Tx64-coff -std:C17 -Ze -Zx decimal_dig_test.c
* polink -subsystem:console -machine:x64 decimal_dig_test.obj \
* crt64.lib kernel32.lib -out:decimal_dig_test.exe
*
* BUILD, any other C compiler, for comparison:
* cc -std=c11 -O1 decimal_dig_test.c -o decimal_dig_test
* ======================================================================== */
On Pelles C 14.50 (x64), the conversion of a double to decimal text produces
digits that are correct to about fifteen significant figures. Digits
requested beyond that are produced, but they are wrong, not merely
imprecise.
The practical consequence is that the round-trip guarantee implied by
DECIMAL_DIG does not hold:
-------------------------------- ------------------------------------------
DECIMAL_DIG declared by 17
<float.h>
DBL_DECIMAL_DIG declared by 17
<float.h>
finite normal doubles printed 48,191 of 100,000 -- 48.19%
with %.17g that do not read back
as the same value
-------------------------------- ------------------------------------------
The same test program, built with another toolchain's runtime on the same
machine and run on the same values, reports zero failures.
The input side is not affected: strtod reads a correctly rounded
seventeen-digit decimal back to the exact bit pattern. Hexadecimal output,
%a, is not affected either. The fault appears to lie solely in the
binary-to-decimal conversion used by the printf family.
A single self-contained test program is attached. It needs no data files, no
arguments and no other libraries, and it runs in under a second.
Environment
------------------------------------- -------------------------------------
Compiler Pelles ISO C Compiler, Version
14.50.0, x64
Switches -Tx64-coff -std:C17 -Ze -Zx
Link crt64.lib kernel32.lib, console
subsystem
Optimisation irrelevant: identical output
unoptimised and at -Ot, -Os and -Ox
FLT_EVAL_METHOD 0
------------------------------------- -------------------------------------
The optimiser was eliminated first, by rebuilding at every optimisation
level and comparing the output byte for byte. All four builds agree with one
another and all four disagree with the correctly rounded digits, so this is
a runtime library matter and not a code generation one.
How it was found
The work in progress was unrelated: making a C++ image processing library
usable from Pelles C by compiling its C++ once into an object file and
presenting a pure C interface, so that a Pelles C program can call it
without a DLL. In that arrangement the C++ code is compiled by another
compiler, but every C function it calls -- malloc, memcpy, fopen, printf,
cos, sin -- is resolved at the final link by Pelles C's own crt64.lib. That
is the point of the design: one C runtime, one heap, one set of file
handles.
To check that the C interface altered nothing, thirteen image processing
computations were written twice: once in C, calling through the interface,
and once in C++, calling the library directly. The two sets of output images
were compared byte for byte. Twelve of the thirteen were identical. One
differed in a single pixel, by one grey level.
Chasing that one pixel is what turned this up. The chain of elimination was:
1. The optimiser was ruled out, as described above.
2. Both computations were instrumented to compare their floating point in
memory, stage by stage, before anything was cast to a byte. All nine
stages were bit-identical, so the interface was not altering anything.
3. The small amount of arithmetic in the caller was compiled by both
compilers from the same source and compared: 51,062 floats,
bit-identical.
4. What remained was the mathematical library. Because the design routes
every C call to crt64.lib, the transform in question was using Pelles
C's cos and sin where the C++ comparison used the other toolchain's.
Measured over 4,096 angles: 2,061 of 8,192 values differ, by at most one
unit in the last place. That is not a fault in anything -- neither
library claims correctly rounded transcendentals -- and it fully
explains one pixel in 65,536 landing on the boundary between grey 31 and
grey 32.
The present report arises from step 4. While printing the cos values for
comparison, the two runtimes printed different decimal text for identical
bit patterns. That is not a question about mathematics; it is a question
about printf, and it turned out to be a real one.
The minimal test
decimal_dig_test.c, attached, does two things.
Part one takes three doubles as exact IEEE-754 bit patterns, so that no
decimal parsing happens before the test, and compares printf's output
against the correctly rounded digits. Those expected strings were obtained
independently, by arbitrary-precision arithmetic, and are written into the
program as literals.
Part two generates 100,000 finite normal doubles from a four-line xorshift
generator -- so the sample is identical on every machine and every run --
prints each with %.*g at DECIMAL_DIG, reads it back with strtod, and
compares the bits.
strtod is used as the reader because it was verified separately, against
exact bit patterns, to be correct on this implementation. If it were not,
part one would fail in the same direction and the two parts would disagree.
Results
Part one, on Pelles C 14.50
bits 3feffff9606a6fbc
%.17g expected 0.99999684172825409 got 0.999996841728254 <-- DIFFERS
%.20f expected 0.99999684172825409334 got 0.99999684172825400310 <-- DIFFERS
bits 3fefe9cdad01883a
%.17g expected 0.99729045667869021 got 0.99729045667869013 <-- DIFFERS
%.20f expected 0.99729045667869020697 got 0.99729045667869013413 <-- DIFFERS
bits 400921fb54442d18
%.17g expected 3.1415926535897931 got 3.1415926535897932 <-- DIFFERS
%.20f expected 3.14159265358979311600 got 3.14159265358979318989 <-- DIFFERS
Two features are worth separating.
The digits beyond the fifteenth are wrong. In the first value the correct
digits are ...254093 34, and what is produced is ...254003 10. This is not a
rounding difference at the last digit; the sixteenth digit onwards is a
different number.
Fewer digits are produced than were requested. %.17g on the first value
returns 0.999996841728254, which is fifteen significant figures, not
seventeen. In the 100,000-value sample this happens in 9.84% of cases.
Part two, on Pelles C 14.50
values tested 100000
failed to read back as the same double 48191 (48.19%)
printed with fewer digits than requested 9836 (9.84%)
The same program, another runtime, same machine, same values
values tested 100000
failed to read back as the same double 0 (0.00%)
How it varies with the precision requested
Over 4,000 finite normal doubles spanning the whole exponent range, counting
strings that differ from the correctly rounded conversion:
conversion Pelles C 14.50 control
------------ ---------------- ---------
%.15g 3.33% 0.00%
%.16g 29.93% 0.00%
%.17g 85.47% 0.00%
%.18g 97.75% 0.00%
And the round-trip failures over the same sample:
conversion Pelles C 14.50 control
------------ ---------------- ---------
%.16g 61.88% 44.77%
%.17g 48.10% 0.00%
%.18g 47.45% 0.00%
The %.16g row is expected to fail on both, since sixteen digits cannot name
every double; it is included as a control on the method. The %.17g row is
the one that matters, and the two columns should both be zero.
What is not affected
These were tested and are correct, which may help to localise the fault.
strtod is correct. Five exact seventeen-digit decimals were parsed and each
produced the expected bit pattern.
%a is correct. The hexadecimal form was checked across every binary exponent
from 2^-1070 to 2^1023, 2,094 values, and separately on DBL_MAX, DBL_MIN,
the largest subnormal and the smallest subnormal. All were exact. %a is
therefore a sound workaround for anyone who needs to write a double as text
without losing it.
The optimiser is not involved. Unoptimised, -Ot, -Os and -Ox all produce
identical output.
Why it matters
<float.h> on this implementation declares DECIMAL_DIG = 17 and
DBL_DECIMAL_DIG = 17. The C standard describes DECIMAL_DIG as the number of
decimal digits such that any floating value may be converted to decimal and
back with no change of value (C11, 5.2.4.2.2, paragraph 12). Seventeen is
the correct value for IEEE-754 double. The declaration is therefore right;
what is missing is the conversion that would make it true.
The consequence is quiet, which is what makes it serious. Any program that
writes double data as text -- a configuration file, a comma-separated
export, a log, a checkpoint to be resumed from, a scientific data file, a
coordinate in a header -- and that writes more than fifteen significant
figures in the belief that it is preserving the value, is altering the value
at the moment of writing, and nothing in the program will say so. The loss
is at the level of one or two units in the last place, which is exactly the
level at which it survives casual inspection and then accumulates.
A false lead, recorded so that nobody repeats it
An earlier version of the measuring program appeared to show %a producing a
grossly wrong result for DBL_MAX: 0x8.181818181818p-255 in place of
0x1.fffffffffffffp+1023. That was a fault in the measuring program, not in
Pelles C. The same program had earlier printed DBL_MAX with %.20f into a
128-byte buffer, and that conversion needs more than 330 characters. The
buffer overflow corrupted what followed.
It is recorded here because it was nearly reported as a defect, and because
the only reason it was not is that the result was checked a second time by a
different route.
How to reproduce
pocc -Tx64-coff -std:C17 -Ze -Zx decimal_dig_test.c
polink -subsystem:console -machine:x64 decimal_dig_test.obj ^
crt64.lib kernel32.lib -out:decimal_dig_test.exe
decimal_dig_test
The program returns 0 if both parts pass and 1 otherwise. For a comparison,
the identical source builds unchanged with any other C compiler.
Suggested area to look at
The pattern -- correct to about fifteen significant figures, then wrong
rather than absent, together with %g sometimes yielding fewer significant
digits than requested -- is consistent with the conversion being carried out
through an intermediate of limited precision, with the remaining digits
generated from a value that has already lost them, rather than by an exact
arbitrary-precision method such as Steele and White's Dragon4 or Grisu with
an exact fallback.
strtod on the same implementation is exact, so the necessary
arbitrary-precision machinery may already be present on the input side.
File attached 'decimal_dig_test.c':
/* ===========================================================================
* decimal_dig_test.c
* ---------------------------------------------------------------------------
* A self-contained test of one thing only: whether printf can write a double
* with DECIMAL_DIG significant digits such that strtod reads back the same
* value.
*
* <float.h> declares
*
* DECIMAL_DIG the number of decimal digits needed to hold any
* floating value of the widest supported type so that
* converting it to decimal and back changes nothing
* DBL_DECIMAL_DIG the same, for double
*
* The C standard describes DECIMAL_DIG (C11 5.2.4.2.2, paragraph 12) as the
* number of digits for which that round trip is guaranteed. This program
* simply takes the implementation at its word and checks.
*
* IT NEEDS NOTHING. No data files, no libraries beyond the standard ones,
* no command line arguments. It runs in under a second.
*
* HOW IT AVOIDS TESTING ITSELF
*
* Part one sets its values from EXACT BIT PATTERNS, so no decimal
* parsing takes place before the test. Each expected string was
* obtained independently, from arbitrary-precision arithmetic, and is
* written into this file as a literal.
*
* Part two uses strtod to read the printed text back. That is safe
* here: strtod was checked separately against exact bit patterns and is
* correct on this implementation. If it were not, part one would fail
* in the same way and the two parts would disagree.
*
* BUILD, Pelles C:
* pocc -Tx64-coff -std:C17 -Ze -Zx decimal_dig_test.c
* polink -subsystem:console -machine:x64 decimal_dig_test.obj \
* crt64.lib kernel32.lib -out:decimal_dig_test.exe
*
* BUILD, any other C compiler, for comparison:
* cc -std=c11 -O1 decimal_dig_test.c -o decimal_dig_test
* ======================================================================== */