NO

Author Topic: Issue in editor with Unicode (v12RC1) - *SOLVED*  (Read 1177 times)

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Issue in editor with Unicode (v12RC1) - *SOLVED*
« on: April 26, 2023, 11:48:09 AM »
I'm trying to contribute a French translation for Pelles C. Globally, this works fine, but I've encountered an issue while editing the HTML start page. I followed the instructions on the wiki for creating a translation package.

So the problem is easy (it seems !) : I double-clicked on the IDR_HTM_WELCOME resource to edit it, change the HTML code to display a French page, no problem, saved the project and compiled. Worked like a charm. However, when I reopened the project after relaunching the IDE, I went on to make a small fix in the HTML and when I opened it, it displayed funny, with spaces in between characters. Actually, these are not spaces, but NULL chars. I tried to change the Encoding property, which was set to ANSI, to UTF-16LE. That did not change anything, apparently. HOWEVER, when I reloaded again, the HTML displayed with 3 NULL chars in between ASCII chars. I suspect the IDE saves the HTML code in Unicode somehow but loads it as ANSI, and again, and again.

I did not see the problem reported yet apparently, so here I am. Thanks !

Edit: forgot to mention that I also changed the language property of IDR_HTM_WELCOME to French before starting editing the HTML code.
« Last Edit: May 03, 2023, 02:21:02 PM by Fool-DupleX »

Offline John Z

  • Member
  • *
  • Posts: 790
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #1 on: April 26, 2023, 02:08:50 PM »
I tried to duplicate what you said but it does not appear to happen on my installation (version 11 right now).

I used rsrc0009.ppj  opened welcome.htm it showed UTF-16LE.
I built the dll.
I then edited, using Pelles C editor, the welcome.htm adding Chinese, saved the file, rebuilt the dll.
Closed the project, reopened the project and looked at welcome.htm the Chinese
was still there and the file was still UTF-16LE....

I'll try on Pelles V12 later today....

John Z

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #2 on: April 26, 2023, 02:33:02 PM »
Thanks for investigating this. Here are some screenshots. Modus operandi : I just unzipped the rsrc0009 in a folder, opened IDR_HTM_WELCOME. It displays as in the first screenshot. I modified "en-us" in "fr-fr" and saved the project, closed the tab and reopened IDR_HTM_WELCOME, it now displays as in the second screenshot. With a copy/paste, I was able to determine that what looks like spaces is actually NULL chars.

Edit: you opened the welcome.htm file, I opened the resource IDR_HTM_WELCOME in eng.rc.
Edit 2: and the welcome.htm file is now in Unicode, when it was in plain ASCII before. - No, it was already Unicode in the zip, my bad.
« Last Edit: April 26, 2023, 02:56:36 PM by Fool-DupleX »

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #3 on: April 26, 2023, 02:51:37 PM »
I found a difference : after saving the resource, the welcome.htm file is updated and lacks the 0xfffe at the beginning. Why is there a 0xfffe in the first place ? This is afaik an undefined Unicode character.
« Last Edit: April 26, 2023, 02:54:01 PM by Fool-DupleX »

Offline John Z

  • Member
  • *
  • Posts: 790
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #4 on: April 26, 2023, 02:57:07 PM »
Hi DupleX,

Yes, definite difference on how it is opened.  Try it the way I did, it may/should at least let you
make progress in completing your translation.  I opened it as you did and my Chinese is still there but
I did not try to make another edit and save .....

The Pelles Guru's will need to look further at opening it using the resource ID rather than clicking
the file under the resource in the project list window.

Just FYI - in Pelles you can also open the entire resource file as text, just right click on the .rc and select open as text.
Often it is easier and faster to make resource edits this way. 

John Z

Offline John Z

  • Member
  • *
  • Posts: 790
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #5 on: April 26, 2023, 03:02:05 PM »
I found a difference : after saving the resource, the welcome.htm file is updated and lacks the 0xfffe at the beginning. Why is there a 0xfffe in the first place ? This is afaik an undefined Unicode character.

That is a Byte Order Mark and is supposed "serve as an implicit marker or signature to identify the file as a Unicode file."

John Z

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #6 on: April 26, 2023, 03:34:47 PM »
Okay, so there's hope :-) I will do as you suggest, thanks !

Offline HellOfMice

  • Member
  • *
  • Posts: 62
  • Never be pleased, always improve
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #7 on: April 27, 2023, 04:33:20 AM »
Bon courage compatriote et bienvenue.
--------------------------------
Kenavo

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #8 on: April 27, 2023, 08:21:13 AM »
Merci HellOfMice !  :)

Since I'm new here, what is the process to report a bug ? Is this thread enough, or should I contact Pelle ?

Offline HellOfMice

  • Member
  • *
  • Posts: 62
  • Never be pleased, always improve
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #9 on: April 27, 2023, 09:21:30 AM »
There is a thread on the forum for that.
You can setup the editor for creating/openninng UTF-8 files.
I don't like Pelle editor but the other tools are very very good.



--------------------------------
Kenavo

Offline John Z

  • Member
  • *
  • Posts: 790
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #10 on: April 29, 2023, 01:41:44 PM »
Merci HellOfMice !  :)

Since I'm new here, what is the process to report a bug ? Is this thread enough, or should I contact Pelle ?

This is the right place: Bug Report section.  Pelle reviews it, and others help verify, disprove, or provide workarounds based on the report.  Each 'BUG' report is a separate subject/topic entry.

John Z

Offline Pelle

  • Administrator
  • Member
  • *****
  • Posts: 2266
    • http://www.smorgasbordet.com
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #11 on: April 29, 2023, 04:24:47 PM »
You want the BOM also when saving the text resource?? Why didn't you say so...??
(OK, I will fix it...)
/Pelle

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #12 on: May 01, 2023, 01:39:02 PM »
Actually, I do not know if the BOM is the issue or not. But there's a consistency problem between loading and saving a text resource. IDR_HTM_WELCOME in my case. Save it once, it will not open properly the next time (see my screenshots). null chars are seen as ascii at some point somewhere.

Offline Pelle

  • Administrator
  • Member
  • *****
  • Posts: 2266
    • http://www.smorgasbordet.com
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #13 on: May 01, 2023, 04:57:41 PM »
A) I can see that a check for a BOM is always performed when loading a text resource (just like when loading a text file).
B) I can see that the code for writing a BOM is always excluded when saving a text resource (unlike when saving a text file).

I can't say exactly why B) because this code hasn't changed since 2010, and I don't have history going that far back...
/Pelle

Offline Fool-DupleX

  • Member
  • *
  • Posts: 17
Re: Issue in editor with Unicode (v12RC1) ?
« Reply #14 on: May 02, 2023, 09:39:44 AM »
Let me do some more tests to try and pinpoint the problem. Will come back asap.