Console Mode from WinMain() entry point

[DMac]

I wanted to debug an x86 console application on Windows 8.1 but could not due to a bug[^].  Until it's fixed here's a work around.

Code: [Select]

BOOL ConsoleMode(void)
{
    int hCrt;
    FILE *hf;

    AllocConsole();
    hCrt = _open_osfhandle((long)GetStdHandle(STD_OUTPUT_HANDLE), _O_TEXT);
    hf = _fdopen(hCrt, "w");
    *stdout = *hf;
    if (ERROR_SUCCESS != setvbuf(stdout, NULL, _IONBF, 0))
        return FALSE;
    hCrt = _open_osfhandle((long)GetStdHandle(STD_INPUT_HANDLE), _O_TEXT);
    hf = _fdopen(hCrt, "r");
    *stdin = *hf;
    if (ERROR_SUCCESS != setvbuf(stdin, NULL, _IONBF, 0))
        return FALSE;
    hCrt = _open_osfhandle((long)GetStdHandle(STD_ERROR_HANDLE), _O_TEXT);
    hf = _fdopen(hCrt, "w");
    *stderr = *hf;
    if (ERROR_SUCCESS != setvbuf(stderr, NULL, _IONBF, 0))
        return FALSE;

    return TRUE;
}

//Change int main() to int WINAPI WinMain() and call SetConsoleMode()
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nShowCmd)
{
    if (!ConsoleMode())
        return 0;

    //Original entry point code follows
}


[jcfuller]

My favorite debugging tool:
http://www.codeproject.com/Articles/693260/zTrace-debugging-utility

James

[henrin]

Thank you  DMac. I do'nt have a win8.x computer.
I am just thinking about upgrading from win7 to win10, and that could in fact be a downgrading !

About the bug, I understand that the AllocConsole did not work, or more precisely, dit not initialize correctly so that further standard I/O crashed.
My question: was it a problem with _cprintf (or _cwprintf) or in other I/O functions?

Thank you for this deep investigation.

[DMac]

Hello Henrin,

No, As I recall, it was possible to build and run an application as a release.  However if you built a debug version you would get the access violation once you tried to execute the application.

The good news is that Pelle fixed this bug and so the work around should not be necessary.  See here.