EZ memory management avoids lost malloc branches

[Rainbow Sally]

As in all things there are tradeoffs.  This utility won't be useful in all cases, but it does address a very real problem.

Let's say you are using something like scandir to read files and folders. Not only is memory malloc'ed for the names of the files and folders, but the array that points to those names is also malloc'ed.

name pointers array [name1] [name2] [name3]  --> grows as required.

The above can be realloc'ed or whatever to accommodate growth, but what happens when the names themselves are malloc'ed?  No prob... until you want to free the stuff and do another run.  No idea how many names to free.  No idea whether or not they have already been freed.  Lots of problems.  And if you free the pointers array first, forget it.  You can NEVER free the names after that.

This branching malloc stuff is what this utility addresses.  It keeps all the memory under one official malloc so it doesn't matter what branches off of what.

This version is meant to hold temp data such as the output of a scandir function, so it simply recycles when the buffer is exhausted but you can easily modify it to grow forever if you wish, and it will still maintain all the mallocs under one main pointer.

And it's only a few lines of code...

Code: [Select]


// 64k recycling buffer for stuff
#define TMP_ALLOC_BUFFSIZE (0x10000)

static int tmp_alloc_max = TMP_ALLOC_BUFFSIZE; // default
static char* tmp_alloc_mem = 0;
static int tmp_alloc_ptr = 0;

void tmp_alloc_init(void){ tmp_alloc_mem = malloc(tmp_alloc_max);}

void* tmp_alloc(int size)
{
if(tmp_alloc_mem == 0) tmp_alloc_init();
if(&tmp_alloc_mem[tmp_alloc_ptr+size+4] > &tmp_alloc_mem[tmp_alloc_max])
tmp_alloc_ptr=0;
char *ret = &tmp_alloc_mem[tmp_alloc_ptr];
// erase the block of memory here if you need clean
// blocks of memory, but it will cost you some time.
// Also, align your pointer if you plan to use this for
// data other than strings.
tmp_alloc_ptr += size + 4; // unaligned
return ret;
}

void* tmp_calloc(int blksize, int nblks)
{
return tmp_alloc(blksize*nblks);
}

void tmp_alloc_free(void)
{
free(tmp_alloc_mem);
tmp_alloc_mem = 0; // signal united
tmp_alloc_ptr = 0; // ready to restart
}



Usually you won't ever need to execute the <name>_free function, but you can do this to reset the buffer, or to resize it (by changing the value tmp_alloc_max).

If you have problems with your screen going white when you are debugging, try this instead of your usual way of allocating memory.  It's easy enough to switch back after the app is debugged.

.

[Gerome]

Hi,

Seems fine, but...

Imagine i want to use your personal alloc & calloc functions to allocate char* of a size > 64 kb, for example i need to use 4 char* of 4 MB each.
char *a, *b, *c, *d;
if i branch'em to your alloc function by telling it i wanted (1024*1024)*4 bytes, i only only have 64 Kb available for my pointer...

Or is there something i've missed ?
Thanks for your help.

[Rainbow Sally]

Hi Gerome.

Hi,

Seems fine, but...

Imagine i want to use your personal alloc & calloc functions to allocate char* of a size > 64 kb, for example i need to use 4 char* of 4 MB each.
char *a, *b, *c, *d;
if i branch'em to your alloc function by telling it i wanted (1024*1024)*4 bytes, i only only have 64 Kb available for my pointer...

Or is there something i've missed ?
Thanks for your help.

You are quite right.  The solution would be to resize the buffer, which can be done with a realloc without losing any memory.  Just change the size of the "max" variable and realloc to that.  Or allocate as much memory as you can get away with straight off the bat.

But actually, since you DO know the number of pointers in the example you cite, you can easily get away with a normal malloc-free arrangement.  

It's when your array of pointers are likely to be variable sizes or when you don't even know how many dimensions your arrays will be dim-ed to that you can get into sticky situations with malloc branches.

In the example I was working on, belive it or not, there was a

 

Code: [Select]

  dirent ***varName

variable required.

The final variable was a folder or file name in some arbitrary directory.  

How many?  Which to free first?  

How many chances do we get to get it right?  (All are pointers and all can be freed in any order.  I.e., there's no type checking to help us here. )

...And how much time will the OS take to do it's memory management?  

We can do ours pretty quickly though accomodating large allocations would require a tad bit of extra code.

Stick with malloc for what you are doing now, but consider a recycling buffer for those temp thingy's we all have need for so often, and for arrays of arrays of pointers to pointers to pointers it's even more convenient.

.