NO

Author Topic: _msize returning wrong values  (Read 13424 times)

karnak

  • Guest
_msize returning wrong values
« on: April 10, 2014, 12:39:46 AM »
hi,

I recently discovered the _msize() function which returns the size of dynamically allocated memory, and decided to write a small test program to check it out. However instead of giving me the correct sizes of 10 and 20 it actually gives 16 and 24.

I decided to see what mingw 32-bit would give so I ran it in Codeblocks which did indeed return the correct values of 10 and 20.
(had to include <malloc.h> and change the format specifier from %zu to %u)

Anyone any ideas?

Code: [Select]

#include <stdio.h>
#include <stdlib.h>
//#include <malloc.h>

#define SIZE_ONE 10
#define SIZE_TWO 20

int main(void)
{
char* array_ptr;
char* temp;
int i;
size_t array_size;

array_ptr = malloc(sizeof(char) * SIZE_ONE);
if(array_ptr == NULL)
{
fprintf(stderr,"Malloc error - Out Of Memory.");
exit(EXIT_FAILURE);
}

array_size = _msize(array_ptr);

printf("malloc'd array size = %zu\n\n",array_size);

for(i = 0;i < SIZE_ONE;i++)
{
array_ptr[i] = '0' + i;
printf(" %c",array_ptr[i]);
}

printf("\n\n");

temp = array_ptr;

array_ptr = realloc(array_ptr,sizeof(char) * SIZE_TWO);
if(array_ptr == NULL)
{
fprintf(stderr,"Realloc error - Out Of Memory.");
free(temp);
exit(EXIT_FAILURE);
}

array_size = _msize(array_ptr);

printf("malloc'd array size = %zu\n\n",array_size);

for(i = SIZE_ONE;i < SIZE_TWO;i++)
{
array_ptr[i] = 'A' + i - SIZE_ONE;
printf(" %c",array_ptr[i]);
}

printf("\n\n");

for(i = 0;i < SIZE_TWO;i++)
{
printf(" %c",array_ptr[i]);
}

printf("\n\n");
   
    return 0;
}


CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #1 on: April 10, 2014, 02:07:39 AM »
hi,

I recently discovered the _msize() function which returns the size of dynamically allocated memory, and decided to write a small test program to check it out. However instead of giving me the correct sizes of 10 and 20 it actually gives 16 and 24.

When _msize works correctly is does not reaturn what you said in malloc(), it returns the actual size of the memory block allocated by the system.  Depending on windows versions this could be in blocks of 8, 16 or 32 characters.  Thus if you ask for 1 you will get 8, 10 will get you 16, 20 will get you the next multiple of 8 which is 24.
 
It's harmless unless you are trying to figure out something like the length of a struct... for that you would use sizeof() or for the length of a string you would use strlen().  For an array size (per your example) use  Sizeof(element) * NumberOfElements.
 
 
 
 

 

karnak

  • Guest
Re: _msize returning wrong values
« Reply #2 on: April 10, 2014, 05:10:40 AM »
Gotcha - its the blocks allocated rather than the bytes requested.

Still - wonder why there is a difference between Pelles C and Codeblocks - on the same 64 bit win7 computer?

I would have thought the allocated memory fetch would be handled by the OS.

Edit: I just checked and for 1 byte requested allocation _msize returns 16 in Pelles C and 1 in Codeblocks. So it seems in Pelles C its returning the blocks allocated and in Codeblocks its returning the bytes requested.

btw I know good programming practise would avoid it, but I wondered if that means that in any malloc'd allocation that didn't allocate a full block those 1 - 15 bytes or whatever are actually available for use? I wouldn't do it, but I can't think of a reason why they wouldn't be usable?
« Last Edit: April 10, 2014, 05:56:26 AM by karnak »

Offline frankie

  • Global Moderator
  • Member
  • *****
  • Posts: 2096
Re: _msize returning wrong values
« Reply #3 on: April 10, 2014, 10:10:18 AM »
As CommonTater explained the memory allocater reserves so many baytes to guarantee that the next allocation request will get memory starting on a boundary suiting for any kind of data.
Normally this is 8 byte for 32 bits, 16 for 64 bits systems.
To use the spare memory is a good programming practice to roud allocation request to the minimal allocable block,
Anyway _msize  as per MS definition, that PellesC follows in this case, returns the size of blocks allocated not the bytes requested (see here).
I would add that if you use the function to check for memory leaks or usage to get the real space allocated is more indicative.
While you can use OS functions to allocate memory, C standard memory allocator (malloc, calloc, etc) allocates a big chunk from the OS than creates substructures to handle memory requests allocated from that chunk. For small or frequent allocation/reallocations this could be much more efficient than enter the OS each time.
« Last Edit: April 10, 2014, 10:15:13 AM by frankie »
It is better to be hated for what you are than to be loved for what you are not. - Andre Gide

Offline jj2007

  • Member
  • *
  • Posts: 536
Re: _msize returning wrong values
« Reply #4 on: April 10, 2014, 12:06:45 PM »
...guarantee that the next allocation request will get memory starting on a boundary suiting for any kind of data.
Normally this is 8 byte for 32 bits, 16 for 64 bits systems.

MSDN, malloc:
Quote
The storage space pointed to by the return value is guaranteed to be suitably aligned for storage of any type of object.

That's a bold statement, and I'd like to test it with _mm_load_ps aka movaps but get compile errors.
PellesC\Include\xmmintrin.h uses _mm_load_ps but it seems to be undefined.

Offline frankie

  • Global Moderator
  • Member
  • *****
  • Posts: 2096
Re: _msize returning wrong values
« Reply #5 on: April 10, 2014, 12:59:51 PM »
That's a bold statement, and I'd like to test it with _mm_load_ps aka movaps but get compile errors.
PellesC\Include\xmmintrin.h uses _mm_load_ps but it seems to be undefined.

you have to include <xmmintrin.h>.
Probably my statement above in uncorrect, PellesC must align always on 16bytes boundary for 32 and 64 bits if xmm set have to be supported.
In case of misalignement will be trigged an exception (GP fault from CPU).

I just made a quick check, the function _mm_load_ps is defined only for 64bits.
So malloc for 32 maybe uses 8byte alignement.
Anyway you should be able to use these functions on 32 bits using this trick:
Code: [Select]
#define tmp __POCC_TARGET__
#undef __POCC_TARGET__
#define __POCC_TARGET__ 3
#include <xmmintrin.h>
#undef __POCC_TARGET__
#define __POCC_TARGET__ tmp

Then you must use _mm_malloc and _mm_free to allocate memory for your dynamic variables (for static or local variables prepend __declspec(align(16)) ).

I have took a look at the headers, but not tested it. Maybe doesn't work if Pelle removed the functionalities from compiler and the _mm_malloc from 32 bits libraries.
« Last Edit: April 10, 2014, 01:16:50 PM by frankie »
It is better to be hated for what you are than to be loved for what you are not. - Andre Gide

Offline jj2007

  • Member
  • *
  • Posts: 536
Re: _msize returning wrong values
« Reply #6 on: April 10, 2014, 01:52:48 PM »
Frankie,
This is less complicated:

  long long q=0x1234567812345678;
  __asm movaps xmm0, q;


However, the problem is elsewhere:

int main(int argc, char* argv[]) {
  long long *q=malloc(32);
  __asm movaps xmm0, q;
}


Works fine in 50% of all cases (while for the other 50%, malloc returns a pointer to an 8- but not 16-byte aligned memory area, in spite of Microsoft's bold promises...).

CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #7 on: April 10, 2014, 03:43:05 PM »
That's a bold statement, and I'd like to test it with _mm_load_ps aka movaps but get compile errors.
PellesC\Include\xmmintrin.h uses _mm_load_ps but it seems to be undefined.

Ok, moving on.....
 

CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #8 on: April 10, 2014, 03:50:56 PM »
Gotcha - its the blocks allocated rather than the bytes requested.

Still - wonder why there is a difference between Pelles C and Codeblocks - on the same 64 bit win7 computer?

Look in the Pelles C help file. There is an appendix that explains Pelle's memory allocator.  Although I can't find the link right now there is also a writeup that explains the MinGW allocator.  You should probably study up on both.
 
Most modern compilers are using Heaps, which behave as Frankie and I have explained.  By aligning the data on certain boundaries the OS can be far more efficient at accessing memory and, yes, the performance differences are noticeable on repetative memory intense tasks.
 
Heaps are explained here...
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366711(v=vs.85).aspx
 
What matters most is that it _msize returns actual space allocated, not space used.
 

CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #9 on: April 10, 2014, 03:53:30 PM »
btw I know good programming practise would avoid it, but I wondered if that means that in any malloc'd allocation that didn't allocate a full block those 1 - 15 bytes or whatever are actually available for use? I wouldn't do it, but I can't think of a reason why they wouldn't be usable?

You could probably use them... in fact some system exploits do exactly that.  15 bytes is enough space to set up a jmp instruction to malicious code. 
 
But you are correct, it would be horrificly bad practice to try.
 

CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #10 on: April 10, 2014, 03:54:46 PM »
Frankie,
This is less complicated:

  long long q=0x1234567812345678;
  __asm movaps xmm0, q;


However, the problem is elsewhere:

int main(int argc, char* argv[]) {
  long long *q=malloc(32);
  __asm movaps xmm0, q;
}


Works fine in 50% of all cases (while for the other 50%, malloc returns a pointer to an 8- but not 16-byte aligned memory area, in spite of Microsoft's bold promises...).

Damn it JJ ... this is a forum for C programming.  Nobody gives a flying rats ass about your irrelevent asm garbage.  It just confuses people and if the moderators had even the first lick of sense they would declare it "off topic". 
 
As you've been told before ... Learn C or get lost!

 
« Last Edit: April 10, 2014, 04:29:16 PM by CommonTater »

Offline jj2007

  • Member
  • *
  • Posts: 536
Re: _msize returning wrong values
« Reply #11 on: April 10, 2014, 05:15:39 PM »
Damn it JJ ... this is a forum for C programming.  Nobody gives a flying rats ass about your irrelevent asm garbage.  It just confuses people and if the moderators had even the first lick of sense they would declare it "off topic". 
 
As you've been told before ... Learn C or get lost!

Tater,

Get a shrink before it's too late. You may be a reasonable coder but your social skills are below zero.

On topic: The point is that malloc uses HeapAlloc under the hood, which returns 8-byte aligned memory. Has nothing to do with assembly or C but it causes bugs that are difficult to chase.

czerny

  • Guest
Re: _msize returning wrong values
« Reply #12 on: April 10, 2014, 05:29:44 PM »
Damn it JJ ... this is a forum for C programming.  Nobody gives a flying rats ass about your irrelevent asm garbage.  It just confuses people and if the moderators had even the first lick of sense they would declare it "off topic". 
 
As you've been told before ... Learn C or get lost!
This forum is not a pure C forum. I personally appreciate JJs contributions and it is always an enrichment for me, to learn something new, asm for example.

CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #13 on: April 10, 2014, 06:37:30 PM »
Damn it JJ ... this is a forum for C programming.  Nobody gives a flying rats ass about your irrelevent asm garbage.  It just confuses people and if the moderators had even the first lick of sense they would declare it "off topic". 
 
As you've been told before ... Learn C or get lost!
This forum is not a pure C forum. I personally appreciate JJs contributions and it is always an enrichment for me, to learn something new, asm for example.

Then you should join the MASM forum where JJ is 100% on topic... but just as ridiculous.
 

CommonTater

  • Guest
Re: _msize returning wrong values
« Reply #14 on: April 10, 2014, 06:39:31 PM »
On topic: The point is that malloc uses HeapAlloc under the hood, which returns 8-byte aligned memory. Has nothing to do with assembly or C but it causes bugs that are difficult to chase.

On topic ... the user did not ask for a mass discussion of various memory allocation schemes. He just wanted to know why _msize gave him an unexpected result.  He was asking about a C language function... Get it? Not ASM... C.  You diverging off into your usual bullshit about asm only confuses and does not give meaningful answers.
 
You are not helping anyone and you should simply stop posting.

 
 
« Last Edit: April 10, 2014, 06:43:46 PM by CommonTater »