Mojibake in TC player settings

Bug reports will be moved here when the described bug has been fixed

Moderators: white, Hacker, petermad, Stefan2

User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Mojibake in TC player settings

Post by *Usher »

What type is used for LNG string 5835? When I use LNG and MNU files converted to UTF-8, the string is displayed as mojibake. Other strings are properly converted, only this one is displayed as ANSI, see the screenshot:
https://1drv.ms/u/s!Aqm4ZaWdL2tfglDdF_V9Is8RgM2K?e=3bO5JJ
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

1. This control seems to use a delphi control which isn't capable of unicode (32-bit only). I will in replace it.

2. Please do not use UTF-8 for language files unless you need to use characters outside the chosen codepage, because it makes them unusable on Windows 9x/ME.

To use UTF-8, you need to:
1. set the second line of the second line of the lng file to
codepage=65001
2. Not use a byte order mark BOM for any of the language files
3. Usae UTF-8 in all of them
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

2ghisler(Author)
I converted LNG and MNU to UTF-8 for use in Windows XP+. It's portable solution for use on other computers which may use other codepages.
ad 1. Yes, I added codepage=65001 for UTF-8.
ad 2. EDITED: LNG and INC files may have BOM, MNU files have to be without BOM, I haven't tested INC files yet.
ad 3. Unfortunately, TC doesn't work with LNG and MNU files in UTF-16 (codepage=1200), which should be universal solution for TC Unicode builds (as wincmd.ini and other ini files).

Side note: Lister in Windows XP doesn't work with UTF-16 (codepage 1200) added as a custom encoding. It gives error message:
This code page is not installed (Control Panel)!
Andrzej P. Wozniak
Polish subforum moderator
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

ghisler(Author) wrote: 2021-05-23, 08:33 UTC 1. This control seems to use a delphi control which isn't capable of unicode (32-bit only). I will in replace it.
There are more such old controls. See for example that screenshot: tc-about-unreg-rus-utf8.png

Note also that the splash screen should be resized because of longer version number: tc-splash-rus.png
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

There are more such old controls. See for example that screenshot
This isn't one, it's a regular MessageBox function, not using any Delphi controls.
Note also that the splash screen should be resized because of longer version number
This is only a problem during the beta phase and will not be changed, sorry.
I converted LNG and MNU to UTF-8 for use in Windows XP+. It's portable solution for use on other computers which may use other codepages.
Please don't use Unicode UTF-8 for that. Instead, use your local codepage (e.g. 1250 for central Europe) and set
codepage=1250
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

ghisler(Author) wrote: 2021-05-23, 17:05 UTC
There are more such old controls. See for example that screenshot
This isn't one, it's a regular MessageBox function, not using any Delphi controls.
How can you explain that some strings in Russian are displayed properly and some other (in red rectangles) aren't? I suspect it's a problem with concatenation…
ghisler(Author) wrote: 2021-05-23, 17:05 UTC
Note also that the splash screen should be resized because of longer version number
This is only a problem during the beta phase and will not be changed, sorry.
There are two places with problems. First see the first red rectangle - the last letter of the Russian word "версия" isn't fully displayed. The vertical line of that letter is overlapped by the field with version number. The same problem exists in TC 9.51 in Russian.
ghisler(Author) wrote: 2021-05-23, 17:05 UTC
I converted LNG and MNU to UTF-8 for use in Windows XP+. It's portable solution for use on other computers which may use other codepages.
Please don't use Unicode UTF-8 for that. Instead, use your local codepage (e.g. 1250 for central Europe) and set
codepage=1250
So how can I add some items in Russian?
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

How can you explain that some strings in Russian are displayed properly and some other (in red rectangles) aren't? I suspect it's a problem with concatenation…
No, the start of the about box uses string 673, which is scrambled so ordinary users don't modify it. The scrambled text has to use the same codepage as the rest of the lng file. So if you used CP1251 (cyrillic) for the encoded text and UTF-8 for the rest, you will get scrambled text.
So how can I add some items in Russian?
You can't (and shouldn't) add Russian to a non-Russian translation. For Russian translation, use codepage 1251.
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

ghisler(Author) wrote: 2021-05-24, 08:16 UTC
How can you explain that some strings in Russian are displayed properly and some other (in red rectangles) aren't? I suspect it's a problem with concatenation…
No, the start of the about box uses string 673, which is scrambled so ordinary users don't modify it.
So I don't care about any scrambled string.
ghisler(Author) wrote: 2021-05-24, 08:16 UTC
So how can I add some items in Russian?
You can't (and shouldn't) add Russian to a non-Russian translation. For Russian translation, use codepage 1251.
Oh, really?
I can use UTF-8 with mixed languages and it works for me. See this screenshot: tc-change-lng.png
Don't tell me it's a miracle.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

Why would you want to use mixed languages?
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

1. Editing only Unicode files is easier. I don't want to switch codepages every time here and there or use hex editor to edit plain text file.
2. I prefer to use native terms for tools in a specific language. It's a better reminder than (RUS) or (UKR) added.
3. Finally, you have mentioned about moving TC to full Unicode support. So that's just my 3 grosze, eurocents, pennies… worth.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

You can use a text editor where you can change the encoding, for example "Compare by content" in Total Commander - that's what I do when copying strings from Unicode to a language file.
Author of Total Commander
https://www.ghisler.com
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

The problem with string 5835 should be fixed, please try it (Lister configuration, 3rd tab, for example with Romanian).
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

2ghisler(Author)
It looks good with my Polish lng file in UTF-8. However, Russian lng file isn't updated yet. Such things should be tested for text containing characters, which are clearly different from US-ASCII, Windows-1252 and system settings.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Mojibake in TC player settings

Post by *ghisler(Author) »

As I wrote, you could try with Romanian, because it uses Unicode encoding now.
Author of Total Commander
https://www.ghisler.com
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Mojibake in TC player settings

Post by *Usher »

String 5835 looks good in Romanian UTF-8. However, some scrambled strings aren't converted to UTF-8 as you can see on "About Total Commander" screenshot: https://1drv.ms/u/s!Aqm4ZaWdL2tfglSspXWIhO_W-osu?e=vSl9wL
Andrzej P. Wozniak
Polish subforum moderator
Post Reply