BITMAPFILEHEADER packing

[drgott]

i ran into a funny crash situation when trying to save a bitmap to a file.  there are a few different ways to implement this, and i painstakingly tried them all, but they all crashed at the same spot, that is, where BITMAPFILEHEADER struct is populated.

there appears to be a known issue regarding this: http://bremedios.blogspot.com/2008/07/crashes-accessing-valid-memory-due-to.html, but to read the relevant msdn pages and looking at code samples, there is no hint of any problem, and the operation is presented as almost trivial, if a little tedious (building file headers by hand, due to lack of support on the wince platform for certain bitmap operations.)

once you get beyond searching for "crash" and begin homing in on the "packing" aspect, you come across some explanations as to why things were happening, but nothing relating to what to do about them.  (the suggested remedy in the link above doesn't seem to do anything; the application still crashed where the bfSize (DWORD) member of the BITMAPFILEHEADER struct is populated, even though a different struct type was defined and declared without the #pragma.  the application built cleanly but still crashed at the same spot.)

so, my question is: does this situation ring a bell with anyone, especially as it relates to pelles c?  the pelles c wingdi.h file contains the #pack pragmas that are mentioned in articles i've seen. 

now, it has been mentioned to me here that code meant to run on the armh4i processor may not behave correctly on an armh4 chip (my device), but that was in the context of a 3rd party .dll built for the armh4i.  in this case, i'm building with the normal pelles tools for armh4, with which i've not had a problem.  but this is the first time i've had to deal with manipulating a bitmap at this level.  since there seems to be a known issue, and before continuing to bang my head, i wanted to run things by the forum.

thanks, as usual, for any assistance
-go

[CommonTater]

It would help if you could post the problematic segment of your code... in particular the parts where you are instantiating the struct and assigning it's values...
 
I'm just speculating here but it sounds like the problem could be data alignment...  The #pragma pack() is probably required to ensure a known data alignment in the headers.  Consider that structs often end up being different sizes on 16, 32 and 64 bit operating systems because of the way data is aligned in memory. 
 
Consider:

Code: [Select]

typedef struct t_TEST
  {  SHORT x;
     CHAR  y; }
  TEST, *PTEST;
This struct with a SHORT and a CHAR would likely end up being 3 bytes on a 16 bit OS, but 4 byte data aligment for a 32 bit os might increase it to 5 bytes, and 8 byte alignment on a 64 bit os could increase it to at least 9. Just to make it worse, there's no guarantee that this alignment will hold true across Windows, Linux, OSx etc... so in order to ensure that bitmaps exchange correctly across all platforms some control over alignment is necessary... hence the #pragma controlling it.
 
From wingdi.h...

Code: [Select]

#include <pshpack2.h>          // does #pragma pack(push,2)
typedef struct tagBITMAPFILEHEADER {
    WORD bfType;
    DWORD bfSize;
    WORD bfReserved1;
    WORD bfReserved2;
    DWORD bfOffBits;
} BITMAPFILEHEADER, *LPBITMAPFILEHEADER, *PBITMAPFILEHEADER;
#include <poppack.h>       // does #pragma pack(pop)

So long as the struct is correctly instantiated in memory, I see no reason your app should crash when assigning a value to bfSize, except that a data misalignment is occurring.... But hey I've been wrong before.
 
 

Finally, reading the linked article, I have to remind you that alignment and data size are not the same thing.  The article mistakenly assumes that because it contains 32 bit values the struct has to align on 4 byte boundaries, and it does not.  A 32 bit value will simply occupy 2 alignment points... like this...

0                1                2               3
|----------------|----------------|---------------|----------------|
|------WORD------|--CHAR--|padding|-------------DWORD--------------|
 

Unless there's something horridly wrong with the author's compiler this should be no problem.

 

[drgott]

thanks.  speculation or not, you have recapped the situation perfectly.  and it does indeed seem to be an alignment issue.  my question was if the situation was familiar to anyone using pelles c?  has no one tried to save a bitmap to a file to pocketpc? 

in response to your request, the code snippet - which is one of a number available and that i've tried - is simple:
    BITMAPFILEHEADER hdr;       // the bitmap file-header 

    // start to populate it
    hdr.bfType = 0x4d42;
    hdr.bfReserved1 = 0;
    hdr.bfReserved2 = 0;

    // crash here
    hdr.bfSize = (DWORD) (sizeof(BITMAPFILEHEADER) + pbih->biSize + pbih->biClrUsed * sizeof(RGBQUAD) + pbih->biSizeImage);

obviously, the last assignment references some other variables assigned elsewhere, but it's immaterial; you can assign pretty much any value you want to hdr.bfSize for the purposes of testing.  the application will fail at that point at the point of assignment regardless of which code snippet i use.  in other words,

    hdr.bfSize = 10;
    hdr.bfSize = (DWORD) 10;
    int foo = 10;
    hdr.bfSize = foo;
    hdr.bfSize = (DWORD) foo;
    and numerous other efforts to assign the member all fail at the point of assignment.  (or really soon after)


it doesn't matter the order in which you populate the struct.  it doesn't matter if you change the scope of the header struct by moving it out of the function and making it a global.

the problem reminds me of that float or double issue that was addressed in version 6, i think.  in that case
as well, merely trying to access a float which had been passed caused the failure.

the articles mentioning the crash and alignment problems, didn't say which compiler(s) they were using, but  the examples that i looked at all came nicely wrapped in vs projects.  and, presumably, the original msdn example used vs.  in other words, the implication is that all of the examples will run fine if you build with vs. 

can this be a situation similar to the float/double problem pre-v6?  if it is, i realize it's not likely to be remedied at this stage for pocketpc, and i'll get on with my life.  but, if it isn't, and someone has been saving bitmaps under pelles c, i'd like to know that there is hope.

-go

[CommonTater]

thanks.  speculation or not, you have recapped the situation perfectly.  and it does indeed seem to be an alignment issue.  my question was if the situation was familiar to anyone using pelles c?  has no one tried to save a bitmap to a file to pocketpc? 

I'm sure people have, but I'm not one of them  

in response to your request, the code snippet - which is one of a number available and that i've tried - is simple:
    BITMAPFILEHEADER hdr;       // the bitmap file-header 

This may or may not make a difference, but as a matter of good practice you should use:

Code: [Select]

BITMAPFILEHEADER hdr = {0};
so that the struct is free of garbage when you start.  Odd as it may sound, Windows internal functions don't always read structs the way we fill them out and on occasion I've found that clearning the struct with " = {0} " solves the problem... Case in point the GetOpenFilename() function.  I've had trouble getting the dialog to show unless the struct is reset to 0 first.  This may not solve your boggle but it's good practice none the less.

    hdr.bfSize = (DWORD) (sizeof(BITMAPFILEHEADER) + pbih->biSize + pbih->biClrUsed * sizeof(RGBQUAD) + pbih->biSizeImage);

Yikes... Nothing like a killer code line... The only hitch here is that casting to (DWORD) might cause problems if any of the other variables happened to contain a negative number.

it doesn't matter the order in which you populate the struct.  it doesn't matter if you change the scope of the header struct by moving it out of the function and making it a global.

Have you tried creating it off the stack (i.e. in the heap) with calloc()? 

Code: [Select]

PBITMAPFILEHEADER hdr = calloc(sizeof(BITMAPFILEHEADER),1);

It may be an issue of stack alignment not liking the struct's alignment.

the articles mentioning the crash and alignment problems, didn't say which compiler(s) they were using, but  the examples that i looked at all came nicely wrapped in vs projects.  and, presumably, the original msdn example used vs.  in other words, the implication is that all of the examples will run fine if you build with vs. 

And they probably will... However the C in VC++ is very non-standard and often you can do things with it you can't do on other compilers, without modifications to your source code.

can this be a situation similar to the float/double problem pre-v6?  if it is, i realize it's not likely to be remedied at this stage for pocketpc, and i'll get on with my life.  but, if it isn't, and someone has been saving bitmaps under pelles c, i'd like to know that there is hope.

I don't see where writing to the struct's DWORD is any different than writing to any other DWORD.  It doesn't make sense that writing to that one value is causing your problem.  If it was a compiler problem with writing DWORD values I'm very sure we'd see it elsewhere as well.  So this is a bit of a boggle...

I'm hoping one of the others can step up here and give you real guidance, as I'm running out of suggestions   ....

[TimoVJL]

This example runs without error in MS Device emulator

Code: [Select]

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

#pragma optimize( none)

int __cdecl WinMainCRTStartup(void)
{
BITMAPFILEHEADER hdr; // the bitmap file-header
// start to populate it
hdr.bfType = 0x4d42;
hdr.bfReserved1 = 0;
hdr.bfReserved2 = 0;
// crash here ?
hdr.bfSize = 10;
hdr.bfSize = (DWORD)10;
int foo = 10;
hdr.bfSize = foo;
hdr.bfSize = (DWORD)foo;
MessageBoxW(0, L"Done", L"Test", 0);
ExitThread(0);
}PS: with PellesC 7 debugging don't work any more, it brokes connection to Device Emulator.
EDIT: crash when optimizations are off, "Exception: Datatype Misalignment"

[drgott]

indeed, that example does work.  and if i bury your code (slightly) modified inside the wizard's template, it still works just fine.  that's partly good news; the bad news being something unfortunate is happening elsewhere and is shy about telling me where.  in any case, thanks.

-go

[drgott]

however, if you take your example and add a little - as if you were actually saving the bitmap - you should crash

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

int __cdecl WinMainCRTStartup(void)
{
   HANDLE hFile = CreateFile(L"test.bmp", GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
   if (hFile != INVALID_HANDLE_VALUE)
   {
      DWORD dwByteCount;

      // begin your example
       BITMAPFILEHEADER hdr;    // the bitmap file-header
       // start to populate it
       hdr.bfType = 0x4d42;
       hdr.bfReserved1 = 0;
       hdr.bfReserved2 = 0;
       // crash here ?
       hdr.bfSize = 10;
       hdr.bfSize = (DWORD)10;
       int foo = 10;
       hdr.bfSize = foo;
       hdr.bfSize = (DWORD)foo;
      // end your example

      // add new stuff  - start to write the bitmap
                             // begin with the BITMAPFILEHEADER

      WriteFile(hFile, &hdr, sizeof(BITMAPFILEHEADER), &dwByteCount, NULL);
      // crash should have occurred

      // saving the bitmap involves a couple more writes that would go here      
                             // please let me know if you see the "Done" message.  i don't
                           
      CloseHandle(hFile);
      MessageBoxW(0, L"Done", L"Test", 0);
   }
   else
      MessageBox(NULL, L"Could not create file",L"",MB_OK);

       ExitThread(0);
}

[TimoVJL]

Test this:

Code: [Select]

#define WIN32_LEAN_AND_MEAN
#include <windows.h>

typedef struct tagBMPFILEHEADER2 {
WORD filler;
BITMAPFILEHEADER bmfh;
} BMPFILEHEADER2;

int __cdecl WinMainCRTStartup(void)
{
HANDLE hFile = CreateFile(L"test.bmp", GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
if (hFile != INVALID_HANDLE_VALUE)
{
DWORD dwByteCount;

// begin your example
//BITMAPFILEHEADER hdr; // the bitmap file-header
BMPFILEHEADER2 hdr2;
// start to populate it
hdr2.bmfh.bfType = 0x4d42;
hdr2.bmfh.bfSize = 10;
hdr2.bmfh.bfReserved1 = 0;
hdr2.bmfh.bfReserved2 = 0;
hdr2.bmfh.bfOffBits = 0;

// add new stuff  - start to write the bitmap
// begin with the BITMAPFILEHEADER

WriteFile(hFile, &hdr2.bmfh, sizeof(BITMAPFILEHEADER), &dwByteCount, NULL);

// saving the bitmap involves a couple more writes that would go here     
// please let me know if you see the "Done" message.  i don't

CloseHandle(hFile);
MessageBoxW(0, L"Done", L"Test", 0);
}
else
MessageBox(NULL, L"Could not create file", L"", MB_OK);

ExitThread(0);
}

[drgott]

yep.  if you have the time, tell me who's being "tricked" by this and what's going to bite me by using it?  and is this related to something specific, eg, the processor, the os, a .dll ...?  i understand packing and aligning (although i haven't had to deal with it since porting something from 8bit to 16bits 25 years ago.)  i'm always curious as to why something that is supposed to work doesn't when the rules are followed.  not looking to blame, and nothing wrong with a nice workaround.  just looking for closure.  in any case, thanks.

-go   

[TimoVJL]

From http://dev.emcelettronica.com/introduction-to-arm-assembly-language#comment-2553

Data alignment

The ARM CPU can only access DWORDs in memory that are aligned on addresses that are divisible by 4. Likewise, it can only access 16-bit values on addresses divisible by 2. An unaligned access will result in a Datatype Misalignment exception.

[drgott]

ok. nice work.  probably best to let it go at this point in spite of the questions that jump to mind immediately.  thanks.
-go

[CommonTater]

Another possibility for the exception might come from this line...

Code: [Select]

hdr.bfSize = (DWORD) (sizeof(BITMAPFILEHEADER) + pbih->biSize +
                        pbih->biClrUsed * sizeof(RGBQUAD) + pbih->biSizeImage);

1) Check to be sure pibh is properly instantiated... if it's not, the exception may look like a failure at hdr.bfsize while actually being caused by accessing the other struct.

2) It is also possible the alignment of the other struct is problematic in the same way Timo pointed out for this one.

(BTW... Good work Timo... I probably wouldn't have come up with that one at all!)

[dieting]

I bury your code (slightly) modified inside the wizard's template, it still works just fine.  that's partly good news; the bad news being something unfortunate is happening elsewhere and is shy about telling me where.  in any case, thanks.??

[Bitbeisser]

I bury your code (slightly) modified inside the wizard's template, it still works just fine.  that's partly good news; the bad news being something unfortunate is happening elsewhere and is shy about telling me where.  in any case, thanks.??

ICU2!