Pelles C forum

Pelles C => Bug reports => Topic started by: akko on July 22, 2012, 11:59:41 PM

Title: memmove sporadic misbehavior
Post by: akko on July 22, 2012, 11:59:41 PM
The enclosed code is created by PC rel. 7 for memmove.
Irritatingly I get sporadic wrong results with overlapping moves, i.e. the move-direction seems wrong.

I am now guessing in the wild that it might depend on the actual varying address location, i.e. higher addresses may be treated as negative. Compiled with gcc or msvc I have no problems.

My guess is that the b.m. jle at [00401112] should be an unsigned jbe instead. Any other ideas???

Greetings
Akko
   
Code: [Select]
  [004010E0] 56                     push              esi
  [004010E1] 57                     push              edi
  [004010E2] 8B7C240C               mov               edi,dword ptr [esp+C]
  [004010E6] 8B742410               mov               esi,dword ptr [esp+10]
  [004010EA] 8B4C2414               mov               ecx,dword ptr [esp+14]
  [004010EE] 89C8                   mov               eax,ecx
  [004010F0] 01F0                   add               eax,esi
  [004010F2] 39F7                   cmp               edi,esi
  [004010F4] 7604                   jbe               004010FA
  [004010F6] 39C7                   cmp               edi,eax
  [004010F8] 720A                   jb                00401104
  [004010FA] 5F                     pop               edi
  [004010FB] 5E                     pop               esi
  [004010FC] E9FF010000             jmp               00401300
  [00401101] 8D7F00                 lea               edi,[edi+0]
  [00401104] FD                     std               
  [00401105] 01CE                   add               esi,ecx
  [00401107] 01CF                   add               edi,ecx
  [00401109] 89C8                   mov               eax,ecx
  [0040110B] 89F9                   mov               ecx,edi
  [0040110D] 83E103                 and               ecx,3
  [00401110] 29C8                   sub               eax,ecx
  [00401112] 7E1E                   jle               00401132 ; <--- ? not jbe?
  [00401114] 83EE01                 sub               esi,1
  [00401117] 83EF01                 sub               edi,1
  [0040111A] F3A4                   rep movsb         
  [0040111C] 83EE03                 sub               esi,3
  [0040111F] 83EF03                 sub               edi,3
  [00401122] 89C1                   mov               ecx,eax
  [00401124] 83E003                 and               eax,3
  [00401127] C1E902                 shr               ecx,2
  [0040112A] F3A5                   rep movsd         
  [0040112C] 83C603                 add               esi,3
  [0040112F] 83C703                 add               edi,3
  [00401132] 01C1                   add               ecx,eax
  [00401134] F3A4                   rep movsb         
  [00401136] FC                     cld               
  [00401137] 8B44240C               mov               eax,dword ptr [esp+C]
  [0040113B] 5F                     pop               edi
  [0040113C] 5E                     pop               esi
  [0040113D] C3                     ret               
Title: Re: memmove sporadic misbehavior
Post by: Bitbeisser on July 23, 2012, 06:09:03 AM
Other ideas? Well, none you probably like to hear.

memmove is a very basic function of every C compiler, so it is highly unlikely (though not 100% impossible)  that the function itself is in any way producing by itself a "sporadic behaviour"...

In +30 years of programming in C (and 36 years now total), such "sporadic behaviour" is almost always a bug in the user program.

But without a reproducible example of source code, I doubt that this is in any way to determine where the source for your apparent problems are. The assembler output of a library function is here  unfortunately of no help whatsoever...

Ralf
Title: Re: memmove sporadic misbehavior
Post by: akko on July 23, 2012, 09:16:23 AM
Thanks, of course you are right, that's why it had cost me a lot of debugging time....
Could you please check this on your machine with PC7:

Code: [Select]
// PC7 memmove test

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

int main(void)
{ char *str;

str = malloc(6);
strcpy(str,"abc");
printf("moving %s :",str);

memmove(str+1,str,2);
printf("\naab ? %s",str);

memmove(str,str+1,2);
printf("\nabb ? %s\n",str);

return 0;
}
Title: Re: memmove sporadic misbehavior
Post by: Bitbeisser on July 23, 2012, 08:39:14 PM
Well, certainly an odd one, but rather than a problem with memmove, I suspect a subtle issue with memory allocation and alignment. Below is a slightly modified version of your test program, with shows that the second version produces the result you're expecting. As I did this little test just before heading out for work, I don't quite know yet WHY, would have to look into this when I get back later today, unless someone else beats me to it...  ;)

Code: [Select]
// PC7 memmove test

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

int main(void)
{ char *str;
char st2 [10];
 
str = malloc(10);
strcpy(str,"abcde");
printf("moving %s :\n",str);

memmove (str+1, str, 2);
printf("aab ? %s\n",str);

memmove (str, str+1, 2);
printf("abb ? %s\n",str);
//=====================
strcpy(st2,"abcde");
printf("moving %s :\n",st2);

memmove (st2+1, st2, 2);
printf("aab ? %s\n",st2);

memmove (st2, st2+1, 2);
printf("abb ? %s\n",st2);


return 0;
}
Ralf
Title: Re: memmove sporadic misbehavior
Post by: akko on July 24, 2012, 12:43:28 PM
Yeah, it's an odd bug. It works when the string is in the stack, but it seems to be a really severe bug when the string is allocated from the heap or system.
I don't believe that this is caused by address misalignment (and even then the result should be correct, although the execution would be nanoseconds slower).
My "gut feeling" directs me back to Pelles code (see disassembler listing in my first post). I guess for optimization he introduced

 [0040112A] F3A5                   rep movsd         

which moves 32-bit cells blockwise, although memmove is just a character transfer between memories.

Perhaps this should be posted under bug reports.
Title: Re: memmove sporadic misbehavior
Post by: CommonTater on July 24, 2012, 03:56:06 PM
If you suspect an optimizer bug... first try to compile your code with all optimizations off...

Project -> Project Options -> Compiler -> Optimizations  == None.

It does look like a bug but you should try to track it down as far as you can before reporting it.
Title: Re: memmove sporadic misbehavior
Post by: akko on July 24, 2012, 04:24:28 PM
Certainly I did that before, and to be sure on 2 different machines, one XP and the other Win7.
Output is on both system:

moving abc :
aab ? abbc
abb ? bbbc
Press any key to continue...

So it not just moves wrong, it also prolongates the string from 3 to 4 characters.
One cannot complain, you get an extra char for free.  :-))
Title: Re: memmove sporadic misbehavior
Post by: Bitbeisser on July 24, 2012, 05:59:28 PM
If you suspect an optimizer bug... first try to compile your code with all optimizations off...

Project -> Project Options -> Compiler -> Optimizations  == None.

It does look like a bug but you should try to track it down as far as you can before reporting it.
I forgot to mention that I already tried that, to no avail.
Did not have the time to look into this any further, as I didn't not get off work until 3am last night and just now (9am) try with the help of some strong coffee to get up and out of the house again...  :(

Ralf
Title: Re: memmove sporadic misbehavior
Post by: Bitbeisser on July 24, 2012, 06:05:04 PM
Certainly I did that before, and to be sure on 2 different machines, one XP and the other Win7.
Output is on both system:

moving abc :
aab ? abbc
abb ? bbbc
Press any key to continue...

So it not just moves wrong, it also prolongates the string from 3 to 4 characters.
If you check out my example, where I lengthen the strings/space, you'll see that it doesn't happen there. But it seems to me as if there is some word size issue, which causes the malloc'd string to be address with a strange offset. But as mentioned before, it's a bit of a bad timing for me to look into this more closely. I suspect something subtle, it isn't the default optimization though, I tried that before I had to go yesterday...
Quote
One cannot complain, you get an extra char for free.  :-))
Well, that's what I would call a positive attitude...  ;)

Ralf
Title: Re: memmove sporadic misbehavior
Post by: TimoVJL on July 24, 2012, 08:14:06 PM
Test project for testing memmove difference between 7 and 6.5.
Just extracted asm code from both versions.
Title: Re: memmove sporadic misbehavior
Post by: CommonTater on July 24, 2012, 09:15:49 PM
I forgot to mention that I already tried that, to no avail.
Did not have the time to look into this any further, as I didn't not get off work until 3am last night and just now (9am) try with the help of some strong coffee to get up and out of the house again...  :(

Hi Ralf...
I suspected both you an akko might have tried it with no optimizations but I thought I'd suggest it anyway so it wasn't overlooked.

Yep... I'm a "second cuppa java" type in the mornings too... These old eyes don't open as quickly as they used to :D
 
 

 
Title: Re: memmove sporadic misbehavior
Post by: akko on July 24, 2012, 09:25:39 PM
Thanks timovjl

I tested and can confirm that the 6.5 asm code works here,
the "subtly different" 7.0 code is buggy.

 
Title: Re: memmove sporadic misbehavior
Post by: Bitbeisser on July 25, 2012, 08:07:36 PM
I forgot to mention that I already tried that, to no avail.
Did not have the time to look into this any further, as I didn't not get off work until 3am last night and just now (9am) try with the help of some strong coffee to get up and out of the house again...  :(

Hi Ralf...
I suspected both you an akko might have tried it with no optimizations but I thought I'd suggest it anyway so it wasn't overlooked.
No, that's an absolute negative on my part. I explicitly went into the project settings and disabled the optimization, to no avail, that's why I mentioned it in my post...

I just happened to been coming back from work way after 2am the last couple of days, so I have not been able to follow up on this so far.

Ralf
Title: Re: memmove sporadic misbehavior
Post by: Bitbeisser on July 26, 2012, 07:18:41 AM
Yeah, it's an odd bug. It works when the string is in the stack, but it seems to be a really severe bug when the string is allocated from the heap or system.
I don't believe that this is caused by address misalignment (and even then the result should be correct, although the execution would be nanoseconds slower).
My "gut feeling" directs me back to Pelles code (see disassembler listing in my first post). I guess for optimization he introduced

 [0040112A] F3A5                   rep movsd         

which moves 32-bit cells blockwise, although memmove is just a character transfer between memories.
Well, that doens't seem to be correct either, as the "offset" by which the move seems to work wrong is a word size (16bit) not a double word (32 bit)...

Ralf
Title: Re: memmove sporadic misbehavior
Post by: TimoVJL on July 26, 2012, 01:29:34 PM
Akko, test this:
Quote
Any other ideas???
Code: [Select]
mov ecx, edi
sub esi, 1    ; <--+
sub edi, 1    ; <-+|
and ecx, 3    ;   ||
sub eax, ecx  ;   ||
jle L00401702 ;   ||
;sub esi, 1   ;>--|+
;sub edi, 1   ;>--+
rep movsb
Title: Re: memmove sporadic misbehavior
Post by: dienstag on October 01, 2012, 10:35:55 AM
Yesterday I stumbled over this comment in an older project of mine:

Code: [Select]
G200:   // edi is aligned now. Move 4 bytes at a time
        mov     edx, ecx
        shr     ecx, 2

            // Do not use rep movsd in negative direction, errors on AMD's

G210: sub edi, 4
mov eax, [edi + esi]
mov [edi], eax
dec ecx
jnz G210

The rep movsd is not the source of the problem here, since only two bytes were moved. However, there were other processor bugs reported for rep movs* for AMD as well as Intel processors (search for 'bug "rep movs"'). Occurence of these bugs usuallay depends on a number of side conditions, so they are hard to reproduce.

I could not reproduce the problem on a Windows XP, Athlon, Pelles 4.50.113.

The original code I used in the example above is from Agner Fog, www.agner.org/optimize (GPL).
Title: Re: memmove sporadic misbehavior
Post by: CommonTater on October 01, 2012, 01:40:54 PM
If you are using Pelles C 4.50 you need to update... the current version is 7.00 available HERE (http://www.pellesc.de/index.php?page=download&lang=en&version=7.00)