Recent posts

#1

Bug reports / Wrong code at every /O level: ...

Last post by KEL26 - Today at 11:47:36 AM

Wrong code at every /O level: two masked zero-tests joined by && lose their masks


SUMMARY

When two masked zero-tests on the SAME variable are joined by &&, and either
mask is a run-time value rather than a compile-time constant, the generated
code discards both masks and tests the variable against itself.

This function

Code Select Expand

unsigned int decide(unsigned int x, unsigned int m)
{
    return !((x & m) != 0) && ((x & 0x100u) == 0);
}

is compiled as though it read  return x == 0;

No diagnostic is issued, at any warning level.


AFFECTED

  Compiler          Pelles ISO C Compiler, Version 14.50.0
  Targets           -Tx64-coff and -Tx86-coff, both affected
  Levels            /Os, /Ot, /Ox, /O1, /O2 -- every one
  Not affected      compiling with no /O switch
  Independent of    /Ze, /Zx, /Go, and the /std: setting
                    (verified across nine switch combinations,
                     and with /std:C99, /std:C11, /std:C17)


WHAT THE STANDARD REQUIRES

ISO/IEC 9899:2018 6.5.10p4 (&), 6.5.3.3p5 (!) and 6.5.13p3 (&&) give, for
x = 0x00000279 and m = 2:

Code Select Expand

x & m             = 0x279 & 0x002 = 0
(x & m) != 0      = 0
!((x & m) != 0)   = 1
x & 0x100u        = 0x279 & 0x100 = 0
(x & 0x100u) == 0 = 1
1 && 1            = 1

decide(0x279, 2) shall return 1.  It returns 0.

There is no undefined behaviour in the function.  Every operand is
unsigned int, so no signed overflow and no implementation-defined signed
conversion arises.  There is no shift, no pointer arithmetic, no aliasing and
no uninitialised read, and neither operand of && has a side effect.


GENERATED CODE

Compiled -Tx64-coff -std:C17 with no /O switch -- correct:

Code Select Expand

decide:
  mov   dword ptr [rsp+8],ecx
  mov   dword ptr [rsp+10],edx
  mov   eax,dword ptr [rsp+10]
  test  dword ptr [rsp+8],eax        ; x & m
  jne   L_false
  test  dword ptr [rsp+8],100        ; x & 0x100
  jne   L_false
  mov   eax,1
  ret
L_false:
  mov   eax,0
  ret

The same source, -Tx64-coff -Ot -std:C17 -- wrong:

Code Select Expand

decide:
  xor   eax,eax
  test  ecx,ecx                      ; x is tested against ITSELF
  sete  al
  ret

edx, which holds the parameter m, is never read.  The constant 0x100 has
disappeared as well.

The 32-bit target, -Tx86-coff /Ot, shows the same fault:

Code Select Expand

_decide:
  xor   eax,eax
  cmp   dword ptr [esp+4],0
  sete  al
  ret


WHAT APPEARS TO BE HAPPENING

Fusing two masked zero-tests on one variable,

    (x & m) == 0 && (x & n) == 0   ==>   (x & (m | n)) == 0

is a valid and worthwhile transformation, and it is performed CORRECTLY when
both masks are compile-time constants.  With m = 0x002 and n = 0x100 written
literally, the emitted code is exactly right:

Code Select Expand

both_const:
  xor   eax,eax
  test  ecx,102                      ; 0x002 | 0x100 = 0x102   CORRECT
  sete  al
  ret

The fault appears when either mask is not a constant.  Rather than computing
m | n at run time, or declining the transformation, the emitted test takes the
value register as its second operand.

A run-time mask can be built perfectly well when the fusion is written out by
hand, so the capability is clearly there:

Code Select Expand

unsigned int fused_by_hand(unsigned int x, unsigned int m)
{
    return ((x & m) | (x & 0x100u)) == 0;
}

fused_by_hand:
  xor   eax,eax
  or    edx,100                      ; m | 0x100
  test  ecx,edx
  sete  al
  ret                                ; CORRECT


WHICH FORMS ARE AFFECTED

All compiled -Tx64-coff -Ot -std:C17.

Code Select Expand

  form                                              emitted test    verdict
  ----------------------------------------------------------------------
  both masks compile-time constants                 test ecx,102    correct
  first mask a parameter, second constant           test ecx,ecx    WRONG
  first mask constant, second a parameter           test ecx,ecx    WRONG
  both masks parameters                             test ecx,ecx    WRONG
  three conjuncts, first mask a parameter           test ecx,ecx    WRONG
  signed int operands, otherwise identical          test ecx,ecx    WRONG
  shared value left in first &, right in second     test ecx,ecx    WRONG
  shared value is the right operand of both         two tests       correct
  shared operand is a variable mask, values differ  two tests       correct
  shared operand is a constant mask, values differ  two tests       correct
  both masks are enumeration constants              test ecx,14000  correct
  single masked test, variable mask, no &&          test ecx,edx    correct
  the two tests on different variables              two tests       correct
  fusion written by hand with |                     or, then test   correct
  || with != 0 (the De Morgan dual)                 two tests       correct
  ----------------------------------------------------------------------

Stated exactly: the fault arises when the LEFT operand of the & in an earlier
conjunct reappears, in either position, in a later conjunct's masked
zero-test, and at least one of the remaining operands is not a compile-time
constant.

Two further points, both measured rather than assumed:

  - An enumeration constant behaves exactly as a literal does.  Masks
    NO_UGLY = 0x10000 and CONSERVE_MEMORY = 0x4000, declared in an enum, fuse
    correctly to  test ecx,14000  -- byte for byte the code the literals give.

  - If the shared operand is the MASK rather than the value, as in
    (h & 3) == 0 && (w & 3) == 0 where two different values are tested against
    one constant, no fusion is attempted and the code is correct.

Signedness does not matter.  Hoisting the sub-expressions into ordinary
non-volatile locals does NOT avoid it.


CONTROL

The identical source compiled by MinGW-w64 GCC 13 at -O2 performs the same
fusion and gets it right:

Code Select Expand

decide:
  xor    %eax,%eax
  or     $0x1,%dh          # sets bit 8 of edx: m | 0x100
  test   %ecx,%edx
  sete   %al
  ret


WHERE THIS WAS FOUND IN REAL CODE

Building libtiff 4.7.2 at /Ot.  The function TIFFWriteEncodedStrip, in the
source file tif_write.c, contains

Code Select Expand

if (!isFillOrder(tif, td->td_fillorder) &&
    (tif->tif_flags & TIFF_NOBITREV) == 0)
    TIFFReverseBits((uint8_t *)data, cc);

with

Code Select Expand

#define isFillOrder(tif, o)  (((tif)->tif_flags & (o)) != 0)
#define TIFF_NOBITREV        0x00100U

At run time tif_flags = 0x00100279 and td_fillorder = 2, so the condition is
true and the bit reversal must happen.  The optimised build evaluated it as
false, skipped the reversal, and wrote every CCITT Group 3 fax image
bit-reversed while still tagging it FillOrder = 2.  The files opened
without complaint; the pixels were wrong.

Twelve sites in that library match this trigger.  No diagnostic was issued
for any of them.


TO REPRODUCE IN THIRTY SECONDS

The attached pellesc_1450_optimiser_bug.c is self-checking.  Compile it twice
and run both:

Code Select Expand

pocc   -Tx64-coff      -std:C17 -c pellesc_1450_optimiser_bug.c
polink -subsystem:console -machine:x64 pellesc_1450_optimiser_bug.obj ^
       crt64.lib kernel32.lib -out:noopt.exe
noopt.exe        ->  "All 7 cases correct."      exit status 0

pocc   -Tx64-coff -Ot  -std:C17 -c pellesc_1450_optimiser_bug.c
polink -subsystem:console -machine:x64 pellesc_1450_optimiser_bug.obj ^
       crt64.lib kernel32.lib -out:opt.exe
opt.exe          ->  "4 of 7 cases WRONG."       exit status 1


ATTACHED FILES

Code Select Expand

  src/pellesc_1450_optimiser_bug.c   self-checking reproducer, run it twice
  src/bug.c                          the single function, for disassembly
  src/variants.c                     ten of the forms in the table above
  src/characterise.c                 the same test across operand values
  src/hypothesis.c                   demonstrates the transformation performed
  src/shared_mask.c                  shared mask versus shared value
  src/position.c                     which operand position matters
  src/enum_masks.c                   enumeration constants as masks

  disassembly/bug_no_optimisation_x64.asm   correct
  disassembly/bug_Ot_x64.asm                WRONG
  disassembly/bug_Ot_x86.asm                WRONG
  disassembly/variants_Ot_x64.asm           all variants
  disassembly/shared_mask_Ot_x64.asm
  disassembly/position_Ot_x64.asm
  disassembly/enum_masks_Ot_x64.asm
  disassembly/bug_gcc_O2_control.asm        GCC 13 -O2, correct

All listings were produced with  podump /DISASM  on the object files built
from exactly the attached sources.

#2

Assembly discussions / Re: Find characters inside a s...

Last post by Vortex - Yesterday at 09:37:55 PM

Hi Timo,

Sorry for the late reply. I modified my posts above to present a better algo. Here are the comments :

Code Select Expand

OPTION PROLOGUE:NONE
OPTION EPILOGUE:NONE

FindChar PROC src:DWORD,char:DWORD

eax to point the string to search for a specific character :

    mov     eax,DWORD PTR [esp+4]

ecx holding the character to be found :

    mov     ecx,DWORD PTR [esp+8]

Decrement eax so modifying eax should not interfer with the only jmp jz

    dec     eax
@@:
    inc     eax

Get a byte from eax pointing the string

    mov     dl,BYTE PTR [eax]

Check if it's NULL terminator

    test    dl,dl

If dl is NULL set dh to 1

    setz    dh

If ( dl XOR cl ) == 0 the we found the char we were looking for.

dl XOR cl is zero if dl == cl :

    xor     dl,cl
    setz    ch

If none of the conditions above are met then return back to the top of the loop :   

    or      ch,dh
    jz      @b
    retn    8

FindChar ENDP

OPTION PROLOGUE:PrologueDef
OPTION EPILOGUE:EpilogueDef

#3

Add-ins / Add-Ins

Last post by TimoVJL - September 08, 2026, 11:39:08 AM

The poide.exe have an useful Add-In interface, so why not using it.

For example testing small code and see an assembly in Output window might be useful.
Also creating 32/64 bit projects easily.

Users could tell, what kind of tools they need to help their programming situations.

Hardly many IDE have same features.

#4

Bug reports / Re: optimiser emits ud2 (inval...

Last post by Michele - September 03, 2026, 08:38:21 AM

=====================================

Please, to all helpful users, read all of my text in each section,  before 'helping' me out - I explained the problem as clear as I could, with code and words - please explain using sentences rather than just pure code - the penny will then, hopefully, drop.

Kelly PellesC isn't open source, the only person that could take a corrective action is Pelle.
What the other users are doing is to supply other test cases for Pelle to hopefully help debugging.
Those aren't answers to help directly with your case.
We have to wait for Pelle now.

And I still couldn't understand the sense to insert such opcode, if the compiler meet a problem should fail under compilation phase, not let a time bomb instruction code in the executable, unless it was there to help compiler debug. And for some reason Pelle forget to remove it when the compiler was released.

#5

Bug reports / Re: optimiser emits ud2 (inval...

Last post by KEL26 - September 03, 2026, 01:39:12 AM

=====================================

Please, to all helpful users, read all of my text in each section,  before 'helping' me out - I explained the problem as clear as I could, with code and words - please explain using sentences rather than just pure code - the penny will then, hopefully, drop.
I am a professional programmer and I have been debugging C-language with (almost) 'ease' for years.  How to detect the bug, this code, has been shown in my zipped folder, that I attached for Dear Pelle and for all who wish to see, I appreciate explanations with any code you post to help me and others.

=====================================

Robert, thank you. Your printf line gives the clearest picture of the fault so far. Under /O2 it prints all two hundred stored values cycling 0 to 43, and it does so in both of your optimised runs. That is the bug in full: 300 modulo 256 is 44.

The change to the checksum line does not alter that, and your own output shows it. The printed array is identical, and wrong, in both /O2 runs. Only the checksum differs, and the reason is that

Code Select Expand

sum += (*data)++;

never reads the element at index i. It reads the first element two hundred times, adding the old value and incrementing it, so from zero it sums 0+1+...+199 = 19900 whatever the other 199 bytes contain. I checked this by filling the array with 0xFF in every byte but the first, which was set to zero: that loop still prints 19900.

So the fix changed the measurement rather than the data. The array your program prints is the real result. It is wrong under /O2 and right without it, which is exactly what Timo's element check found and what has now gone to Pelle. Your run makes it three machines.

#6

Bug reports / Re: optimiser emits ud2 (inval...

Last post by Robert - September 02, 2026, 10:54:39 PM

Code Select Expand

int __cdecl printf(const char * restrict format, ...);
int __cdecl main(void)
{
    int i;
    unsigned long sum = 0;
    unsigned char *pdata;    // pointer to data
    unsigned char data[200];    // safer stack place at end ?

    pdata = data;
    for (i = 0; i < 200; i++)
        *pdata++ = (unsigned char)(i % 300);

    pdata = data;
    for (i = 0; i < 200; i++)
        sum += *pdata++;
        //sum += (*data)++;

    printf("checksum %lu (correct 19900)\n", sum);
    return 0;
}
also with

Code Select Expand

sum += (*data)++;output

Code Select Expand

checksum 19900 (correct 19900)
EDIT: a simple check

Code Select Expand

int __cdecl printf(const char * restrict format, ...);
int __cdecl main(void)
{
    unsigned char *pdata; // pointer to data
    int i;
    unsigned char data[200];

    pdata = data;
    for (i = 0; i < 200; i++)
    {
        *pdata = (unsigned char)(i % 300);
        if (*pdata != i)
        {
            printf("\nerror: %u != %u\n", i, *pdata);
            break;
        }
        printf("%u ", *pdata);
        pdata++;
    }
    return 0;
}



Code Select Expand

int __cdecl printf(const char * restrict format, ...);
int __cdecl main(void)
{
    int i;
    unsigned long sum = 0;
    unsigned char *pdata;    // pointer to data
    unsigned char data[200];    // safer stack place at end ?

    pdata = data;
    for (i = 0; i < 200; i++)
        *pdata++ = (unsigned char)(i % 300);

    pdata = data;
    for (i = 0; i < 200; i++)
        {
        printf("%i ", data[i]);
        // sum += *pdata++;
         sum += (*data)++;
        }

    printf("checksum %lu (correct 19900)\n", sum);
    return 0;
}

/*
/O2 optimization
using "sum += *pdata++;"
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 checksum 4060 (correct 19900)
*/

/*
/O2 optimization
using "sum += (*data)++;"
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 checksum 19900 (correct 19900)
*/
 
/*
No /O2 optimization
using "sum += *pdata++;"
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 checksum 19900 (correct 19900)
*/

/*
No /O2 optimization
using "sum += (*data)++;"
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 checksum 19900 (correct 19900)
*/

#7

Assembly discussions / NetUserGetInfo example

Last post by Vortex - September 02, 2026, 09:38:12 PM

Based on the NetUserGetInfo API function, command-line application displaying the lockout status of a local account :

Code Select Expand

include LockStatus.inc

.data

f1      dw 'The account %s is NOT locked.',0
f2      dw 'The account %s is locked.',0
f3      db 'Usage : LockStatus <Account>'
        db 13,10,0

sTable  dd OFFSET f1,OFFSET f2
ErrMsg  db 'NetUserGetInfo failed.',0

.data?

ui4     dd ? ; USER_INFO_4
args    dd ?

.code

start:

    call    main
    invoke  ExitProcess,eax
   
main PROC uses esi

    invoke  GetCommandLineW
    invoke  CommandLineToArgvW,eax,ADDR args

    cmp     args,1
    jne     @f

    invoke  printf,ADDR f3
    ret
@@:
    mov     esi,eax
    invoke  NetUserGetInfo,0,\
            DWORD PTR [eax+4],4,ADDR ui4

    test    eax,eax
    jz      @f

    invoke  printf,ADDR ErrMsg
    xor     eax,eax
    ret
@@:
    mov     eax,USER_INFO_4.usri4_flags[ui4]
    and     eax,UF_LOCKOUT
    shr     eax,2
    lea     edx,[sTable+eax]
    shr     eax,2
    push    eax
    invoke  wprintf,DWORD PTR [edx],\
            DWORD PTR [esi+4]

    invoke  NetApiBufferFree,ui4
    pop     eax
    ret

main ENDP

END start


#8

Bug reports / Re: optimiser emits ud2 (inval...

Last post by TimoVJL - September 02, 2026, 08:29:49 PM

Kelly,
You found a bug and now we have a small example code for it, so now just wait Pelle's opinion.
Using pointer was just an another test for it.

EDIT: for minimal executable for debugging, but needs msvcrt.libs :

Code Select Expand

#pragma comment(lib, "msvcrt.lib")
int __cdecl printf(const char * restrict format, ...);
int __stdcall ExitProcess(int);
void __cdecl mainCRTStartup(void)
{
    unsigned char *pdata; // pointer to data
    int i;
    unsigned char data[200];

    pdata = data;
    for (i = 0; i < 200; i++)
    {
        *pdata = (unsigned char)(i % 300);
        if (*pdata != i)
        {
            printf("\nerror: %u != %u\n", i, *pdata);
            break;
        }
        printf("%u ", *pdata);
        pdata++;
    }

    ExitProcess(0);
}
EDIT: useful Add-Ins ?

https://forum.pellesc.de/index.php?topic=5248.0

https://forum.pellesc.de/index.php?topic=4467.0

https://forum.pellesc.de/index.php?msg=26516

#9

Bug reports / Re: optimiser emits ud2 (inval...

Last post by KEL26 - September 02, 2026, 06:49:20 PM

Please reply with the information I request. Thank you, TimoVJL.

Your second program is a better test than my checksum,
because it can fail on the exact element that goes wrong, and I have adopted it.

Compiled exactly as you posted it, with no changes, on Pelles C 14.50.0 for x64:

Code Select Expand

pocc -Tx64-coff -std:C17 -Ze -Zx -W1 -Ot check.c
polink -subsystem:console -machine:x64 check.obj crt64.lib kernel32.lib -out:check.exe
check.exe

it prints

Code Select Expand

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43
error: 44 != 0

The same with -Os and with -Ox. With no -O switch at all it prints all
two hundred values and no error line.

The object code for the store that your check reads back, at -Ot:

Code Select Expand

mov     dl,2C            ; 0x2C is 44, and 300 mod 256 is 44
mov     al,dil
xor     ah,ah
div     dl               ; the 8-bit divide
mov     al,ah            ; the remainder
mov     byte ptr [rsi],al
cmp     byte ptr [rsi],dil

44 mod 44 is 0, which is the 0 in the error line.

Your first program, the pointer checksum,
behaves the same way here: 4060 at -Ot, -Os and -Ox,
and 19900 with no optimisation.
So the pointer form does not avoid it on this machine,
and neither does the position of the array in the frame.

That leaves exactly one difference between us: the optimisation switch.
Would you run your own check with -Ot given explicitly on the command line, as above,
and say what it prints? If it prints the error line, we have the same result.
If it prints all two hundred values for you at -Ot,
then something differs between our installations that I would very much like to identify,
and the first line of

Code Select Expand

pocc /?

would help.

For anyone hitting this today, the form I would still suggest is

Code Select Expand

int t = i % 300;
*pdata = (unsigned char)t;  /* see my note below, just in case any one wants to write to me! */

which keeps the remainder in an int object, so the narrowing to unsigned char
is never part of the same expression and the rewrite has nothing to attach to.

Code Select Expand

N.B.
Why *pdata is right in TimoVJL's loop

TimoVJL's check walks the array with the pointer itself:
it stores through *pdata, compares, and then does pdata++ at the bottom of the loop.
Inside that loop the current element is *pdata, full stop.
My suggestion keeps his loop shape and changes only the two lines that matter — compute in int,
then narrow — so he can drop it straight into his own program.
I verified it: at -Ot it stores all two hundred elements correctly and the check never fires.


Using pdata[i] inside TimoVJL's loop would compile, but combined with his pdata++ it would
advance twice per iteration and write to the wrong places.
Using *pdata inside an indexed loop would write every value to element zero.
Each form is right in its own loop; *pdata[i] is right in neither.



#10

Assembly discussions / Re: Find characters inside a s...

Last post by Vortex - September 02, 2026, 01:01:28 PM

PathFindExtension :

https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/nf-shlwapi-pathfindextensiona

PathCchFindExtension :

https://learn.microsoft.com/en-us/windows/win32/api/pathcch/nf-pathcch-pathcchfindextension