Issue in editor with Unicode (v12RC1) - *SOLVED*

[Fool-DupleX]

I'm trying to contribute a French translation for Pelles C. Globally, this works fine, but I've encountered an issue while editing the HTML start page. I followed the instructions on the wiki for creating a translation package.

So the problem is easy (it seems !) : I double-clicked on the IDR_HTM_WELCOME resource to edit it, change the HTML code to display a French page, no problem, saved the project and compiled. Worked like a charm. However, when I reopened the project after relaunching the IDE, I went on to make a small fix in the HTML and when I opened it, it displayed funny, with spaces in between characters. Actually, these are not spaces, but NULL chars. I tried to change the Encoding property, which was set to ANSI, to UTF-16LE. That did not change anything, apparently. HOWEVER, when I reloaded again, the HTML displayed with 3 NULL chars in between ASCII chars. I suspect the IDE saves the HTML code in Unicode somehow but loads it as ANSI, and again, and again.

I did not see the problem reported yet apparently, so here I am. Thanks !

Edit: forgot to mention that I also changed the language property of IDR_HTM_WELCOME to French before starting editing the HTML code.

[John Z]

I tried to duplicate what you said but it does not appear to happen on my installation (version 11 right now).

I used rsrc0009.ppj  opened welcome.htm it showed UTF-16LE.
I built the dll.
I then edited, using Pelles C editor, the welcome.htm adding Chinese, saved the file, rebuilt the dll.
Closed the project, reopened the project and looked at welcome.htm the Chinese
was still there and the file was still UTF-16LE....

I'll try on Pelles V12 later today....

John Z

[Fool-DupleX]

Thanks for investigating this. Here are some screenshots. Modus operandi : I just unzipped the rsrc0009 in a folder, opened IDR_HTM_WELCOME. It displays as in the first screenshot. I modified "en-us" in "fr-fr" and saved the project, closed the tab and reopened IDR_HTM_WELCOME, it now displays as in the second screenshot. With a copy/paste, I was able to determine that what looks like spaces is actually NULL chars.

Edit: you opened the welcome.htm file, I opened the resource IDR_HTM_WELCOME in eng.rc.
Edit 2: and the welcome.htm file is now in Unicode, when it was in plain ASCII before. - No, it was already Unicode in the zip, my bad.

[Fool-DupleX]

I found a difference : after saving the resource, the welcome.htm file is updated and lacks the 0xfffe at the beginning. Why is there a 0xfffe in the first place ? This is afaik an undefined Unicode character.

[John Z]

Hi DupleX,

Yes, definite difference on how it is opened.  Try it the way I did, it may/should at least let you
make progress in completing your translation.  I opened it as you did and my Chinese is still there but
I did not try to make another edit and save .....

The Pelles Guru's will need to look further at opening it using the resource ID rather than clicking
the file under the resource in the project list window.

Just FYI - in Pelles you can also open the entire resource file as text, just right click on the .rc and select open as text.
Often it is easier and faster to make resource edits this way. 

John Z

[John Z]

I found a difference : after saving the resource, the welcome.htm file is updated and lacks the 0xfffe at the beginning. Why is there a 0xfffe in the first place ? This is afaik an undefined Unicode character.

That is a Byte Order Mark and is supposed "serve as an implicit marker or signature to identify the file as a Unicode file."

John Z

[Fool-DupleX]

Okay, so there's hope :-) I will do as you suggest, thanks !

[HellOfMice]

Bon courage compatriote et bienvenue.

[Fool-DupleX]

Merci HellOfMice ! 

Since I'm new here, what is the process to report a bug ? Is this thread enough, or should I contact Pelle ?

[HellOfMice]

There is a thread on the forum for that.
You can setup the editor for creating/openninng UTF-8 files.
I don't like Pelle editor but the other tools are very very good.

[John Z]

Merci HellOfMice ! 

Since I'm new here, what is the process to report a bug ? Is this thread enough, or should I contact Pelle ?

This is the right place: Bug Report section.  Pelle reviews it, and others help verify, disprove, or provide workarounds based on the report.  Each 'BUG' report is a separate subject/topic entry.

John Z

[Pelle]

You want the BOM also when saving the text resource?? Why didn't you say so...??
(OK, I will fix it...)

[Fool-DupleX]

Actually, I do not know if the BOM is the issue or not. But there's a consistency problem between loading and saving a text resource. IDR_HTM_WELCOME in my case. Save it once, it will not open properly the next time (see my screenshots). null chars are seen as ascii at some point somewhere.

[Pelle]

A) I can see that a check for a BOM is always performed when loading a text resource (just like when loading a text file).
B) I can see that the code for writing a BOM is always excluded when saving a text resource (unlike when saving a text file).

I can't say exactly why B) because this code hasn't changed since 2010, and I don't have history going that far back...

[Fool-DupleX]

Let me do some more tests to try and pinpoint the problem. Will come back asap.