Pelles C forum

Pelles C => Bug reports => Topic started by: Fool-DupleX on April 26, 2023, 11:48:09 AM

Title: Issue in editor with Unicode (v12RC1) - *SOLVED*
Post by: Fool-DupleX on April 26, 2023, 11:48:09 AM
I'm trying to contribute a French translation for Pelles C. Globally, this works fine, but I've encountered an issue while editing the HTML start page. I followed the instructions on the wiki for creating a translation package.

So the problem is easy (it seems !) : I double-clicked on the IDR_HTM_WELCOME resource to edit it, change the HTML code to display a French page, no problem, saved the project and compiled. Worked like a charm. However, when I reopened the project after relaunching the IDE, I went on to make a small fix in the HTML and when I opened it, it displayed funny, with spaces in between characters. Actually, these are not spaces, but NULL chars. I tried to change the Encoding property, which was set to ANSI, to UTF-16LE. That did not change anything, apparently. HOWEVER, when I reloaded again, the HTML displayed with 3 NULL chars in between ASCII chars. I suspect the IDE saves the HTML code in Unicode somehow but loads it as ANSI, and again, and again.

I did not see the problem reported yet apparently, so here I am. Thanks !

Edit: forgot to mention that I also changed the language property of IDR_HTM_WELCOME to French before starting editing the HTML code.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: John Z on April 26, 2023, 02:08:50 PM
I tried to duplicate what you said but it does not appear to happen on my installation (version 11 right now).

I used rsrc0009.ppj  opened welcome.htm it showed UTF-16LE.
I built the dll.
I then edited, using Pelles C editor, the welcome.htm adding Chinese, saved the file, rebuilt the dll.
Closed the project, reopened the project and looked at welcome.htm the Chinese
was still there and the file was still UTF-16LE....

I'll try on Pelles V12 later today....

John Z
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on April 26, 2023, 02:33:02 PM
Thanks for investigating this. Here are some screenshots. Modus operandi : I just unzipped the rsrc0009 in a folder, opened IDR_HTM_WELCOME. It displays as in the first screenshot. I modified "en-us" in "fr-fr" and saved the project, closed the tab and reopened IDR_HTM_WELCOME, it now displays as in the second screenshot. With a copy/paste, I was able to determine that what looks like spaces is actually NULL chars.

Edit: you opened the welcome.htm file, I opened the resource IDR_HTM_WELCOME in eng.rc.
Edit 2: and the welcome.htm file is now in Unicode, when it was in plain ASCII before. - No, it was already Unicode in the zip, my bad.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on April 26, 2023, 02:51:37 PM
I found a difference : after saving the resource, the welcome.htm file is updated and lacks the 0xfffe at the beginning. Why is there a 0xfffe in the first place ? This is afaik an undefined Unicode character.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: John Z on April 26, 2023, 02:57:07 PM
Hi DupleX,

Yes, definite difference on how it is opened.  Try it the way I did, it may/should at least let you
make progress in completing your translation.  I opened it as you did and my Chinese is still there but
I did not try to make another edit and save .....

The Pelles Guru's will need to look further at opening it using the resource ID rather than clicking
the file under the resource in the project list window.

Just FYI - in Pelles you can also open the entire resource file as text, just right click on the .rc and select open as text.
Often it is easier and faster to make resource edits this way. 

John Z
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: John Z on April 26, 2023, 03:02:05 PM
I found a difference : after saving the resource, the welcome.htm file is updated and lacks the 0xfffe at the beginning. Why is there a 0xfffe in the first place ? This is afaik an undefined Unicode character.

That is a Byte Order Mark and is supposed "serve as an implicit marker or signature to identify the file as a Unicode file."

John Z
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on April 26, 2023, 03:34:47 PM
Okay, so there's hope :-) I will do as you suggest, thanks !
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: HellOfMice on April 27, 2023, 04:33:20 AM
Bon courage compatriote et bienvenue.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on April 27, 2023, 08:21:13 AM
Merci HellOfMice !  :)

Since I'm new here, what is the process to report a bug ? Is this thread enough, or should I contact Pelle ?
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: HellOfMice on April 27, 2023, 09:21:30 AM
There is a thread on the forum for that.
You can setup the editor for creating/openninng UTF-8 files.
I don't like Pelle editor but the other tools are very very good.



Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: John Z on April 29, 2023, 01:41:44 PM
Merci HellOfMice !  :)

Since I'm new here, what is the process to report a bug ? Is this thread enough, or should I contact Pelle ?

This is the right place: Bug Report section.  Pelle reviews it, and others help verify, disprove, or provide workarounds based on the report.  Each 'BUG' report is a separate subject/topic entry.

John Z
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Pelle on April 29, 2023, 04:24:47 PM
You want the BOM also when saving the text resource?? Why didn't you say so...??
(OK, I will fix it...)
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on May 01, 2023, 01:39:02 PM
Actually, I do not know if the BOM is the issue or not. But there's a consistency problem between loading and saving a text resource. IDR_HTM_WELCOME in my case. Save it once, it will not open properly the next time (see my screenshots). null chars are seen as ascii at some point somewhere.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Pelle on May 01, 2023, 04:57:41 PM
A) I can see that a check for a BOM is always performed when loading a text resource (just like when loading a text file).
B) I can see that the code for writing a BOM is always excluded when saving a text resource (unlike when saving a text file).

I can't say exactly why B) because this code hasn't changed since 2010, and I don't have history going that far back...
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on May 02, 2023, 09:39:44 AM
Let me do some more tests to try and pinpoint the problem. Will come back asap.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on May 02, 2023, 10:18:01 AM
Okay, it seems the issue is more subtle. Scenario :

1. Open freshly unzipped rsrc0009 project, then open the HTML\IDR_HTM_WELCOME in eng.rc by double-clicking on it
2. Modify html, save it, close that window. Note: the BOM is removed from the .htm file.
3. Reopen the resource IDR_HTM_WELCOME by double-clicking on it, it displays funny. You may do it several times, same behaviour every time.
4. Close the window.
5. Edit manually the .htm file with another editor, add the bom, save. Reopen IDR_HTM_WELCOME. *it stills displays funny*
6. Close and re-run Pelle's C, redo step 1, now it displays correctly. Update : it also works if simply closing the project or the eng.rc window and reopening it, no need to close the IDE.

So for sure, if there's a bom at the beginning of the file, saving the resource should leave that bom in place, for consistency. But it should not be added if there was no BOM in the first place. That's one thing. But also it looks like there is like a flag somewhere that's not updated properly when reopening a resource that has been modified in parallel with another editor (I think it's just not reloaded at all actually).

That's my best guess. Note that the bug is not high priority, there are workarounds. But it is disturbing when you do not expect it. Your body starts to shiver, for no mere mortal can resist the evil of Pelle's C ! HAHAHAHA... ;D

Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Pelle on May 02, 2023, 11:04:41 AM
Well, have you tried with RC2... ?!
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Fool-DupleX on May 02, 2023, 01:04:12 PM
Oh, I did not expect such a quick remedy !  :D

Yes, it works, the original bug I mentioned is fixed ! Thank you so much !

There's still one slight issue: when the resource points to an external file, like it is the case with IDR_HTM_WELCOME, if the external file is modified by another tool, the IDE only sees the old version. You must close the eng.rc file and reopen it for the IDR_HTM_WELCOME to be updated in the IDE (just closing and reopening IDR_HTM_WELCOME itself does not update its contents).

When this happens with a source code file, Pelles C shows a pop-up asking if you want to reload it.

But that's minor.
Title: Re: Issue in editor with Unicode (v12RC1) ?
Post by: Pelle on May 02, 2023, 06:01:24 PM
Oh, I did not expect such a quick remedy !  :D
Just lucky timing...

There's still one slight issue: when the resource points to an external file, like it is the case with IDR_HTM_WELCOME, if the external file is modified by another tool, the IDE only sees the old version. You must close the eng.rc file and reopen it for the IDR_HTM_WELCOME to be updated in the IDE (just closing and reopening IDR_HTM_WELCOME itself does not update its contents).
This would require a longer chain of references, for a case that doesn't happen too often. Should work in a perfect world, but in this world it's not worth the trouble...