UTF-8 multilanguage console printout (and input)

Started by Michele, Yesterday at 08:38:58 PM

Previous topic - Next topic

Michele

Somewhere on this forum there was a discussion about how to print multilanguage strings on console output.
The small attached sample shows the code that use the console codepage set functions using the UTF-8 encoding that is, at least for me, a very powerful option because while allows the multilanguage text let you use the simple and efficient ansi text functions (strcpy, ctrcat, strcmp, ...).

Vortex

Hi Michele,

Thanks for your code. Testing your executable on my Windows 7 Sp1 64-bit, I get a lot of garbled characters, probably a limitation of Windows 7.
Code it... That's all...

Michele

Hello Vortex, I imagine that the problem is the OS version.
I will not add mine, just report what Gemini says:
QuoteThe functions SetConsoleOutputCP and SetConsoleCP along with the CP_UTF8 code page constant (code page 65001) have been supported since Windows 2000.

However, the minimal OS version depends on what you are trying to achieve:

Basic Function Support (API availability): Windows 2000 or later. Calling SetConsoleOutputCP(65001) is supported on all Windows NT-based OS versions starting from 2000, as well as Windows XP, Vista, 7, 8, and 10.

Functional UTF-8 Console Rendering: Windows 10 Version 1903 (Build 18362) or later.

Why the distinction matters:
While the functions exist on older OS versions, UTF-8 support inside cmd.exe on legacy releases (Windows 7/8/early Win 10) was notoriously buggy:

Legacy Bugs: On older versions like Windows 7, setting the console to code page 65001 frequently broke C runtime output routines (printf), caused character truncation, or crashed batch scripts.

Font Rendering: Legacy consoles couldn't render multi-byte UTF-8 sequences properly unless using specific TrueType fonts.

Full ConPTY Overhaul: Microsoft fully overhauled the Windows Console infrastructure in Windows 10 version 1903 (Build 18362)—the exact same release that introduced manifest-based UTF-8 ACP support—making UTF-8 output reliable and stable in the terminal.

TimoVJL

Windows 7 x64 EN console.

Less fonts than Windows 1x versions.

May the source be with you

Michele

#4
Timovjl you successfully changed the codepage to UTF-8, in fact many characters are correctly displayed.
I agree that now the problem is the console font that doesn't contains the full set.
A known font known to render all charset is 'Consolas', try changing the face font with:
BOOL SetConsoleFontToConsolas(void)
{
    HANDLE hConsole = GetStdHandle(STD_OUTPUT_HANDLE);
    if (hConsole == INVALID_HANDLE_VALUE) return FALSE;

    CONSOLE_FONT_INFOEX cfi;
    cfi.cbSize = sizeof(CONSOLE_FONT_INFOEX);

    // Get the current font info first to preserve defaults
    if (!GetCurrentConsoleFontEx(hConsole, FALSE, &cfi))
    {
        return FALSE;
    }

    // Set font properties
    cfi.dwFontSize.X = 0;   // Width (0 lets Windows calculate based on height)
    cfi.dwFontSize.Y = 16;  // Height (Font size in points/pixels)
    cfi.FontFamily = FF_DONTCARE;
    cfi.FontWeight = FW_NORMAL;

    // Specify a TrueType font that supports UTF-8 Unicode characters
    // Common choices: L"Cascadia Code", L"Consolas", L"Lucida Console"
    wcscpy_s(cfi.FaceName, LF_FACESIZE, L"Consolas");

    // Apply the new font
    return SetCurrentConsoleFontEx(hConsole, FALSE, &cfi);
}

If still doesn't it confirms that the fonts in your OS version are limited.
You said that on a Win10/11 system it works?

TimoVJL

#5
Windows 7 have Consolas font, but version 5.? and Windows 1x have version 7 font family.
Less interest to console, albeit file / folder names can have many kind of chars.
May the source be with you