Lister and UTF-8 option
Moderators: white, Hacker, petermad, Stefan2
Lister and UTF-8 option
Hi,
TC does not change automaticly the lister vue to UTF-8 if the TXT file does not contain the UTF-8 signature (at the beginning of the file).
Today, to change the view to UTF-8 for a file in lister, we have to do Options / UTF-8 or use the 7 shortcut.
Can we change to 8 this shortcut to be more mnemonic ? Or add a second shortcut : 8 for UTF-8 view ?
Another wish but not so easy to implement : Can we have a new option in Lister to use UTF-8 has default for text file ?
Best regards.
TC does not change automaticly the lister vue to UTF-8 if the TXT file does not contain the UTF-8 signature (at the beginning of the file).
Today, to change the view to UTF-8 for a file in lister, we have to do Options / UTF-8 or use the 7 shortcut.
Can we change to 8 this shortcut to be more mnemonic ? Or add a second shortcut : 8 for UTF-8 view ?
Another wish but not so easy to implement : Can we have a new option in Lister to use UTF-8 has default for text file ?
Best regards.
#22273 Personal licence
- ghisler(Author)
- Site Admin
- Posts: 48138
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
How should TC know that a file is in UTF8 format? If it contains only English text, it looks the same as ANSI text...
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Hi Christian,ghisler(Author) wrote:How should TC know that a file is in UTF8 format? If it contains only English text, it looks the same as ANSI text...
If the file have not UTF-8 signature, TC can't know that it should read it in UTF-8.
But I would like to have a special option to open the text files (files that TC opens in "text only" format by default) with UTF-8 format by default.
I am french and I use UTF-8 format by default for all my text files (but without using UTF-8 signature) with PSPad free text editor ( http://www.pspad.com/ ).
It will give no changes for English text but it is not the case for French text...
What about adding also 8 for shortcut to UTF-8 format in Lister ?
Regards
#22273 Personal licence
For English text, it looks the same, so it also doesn't matter whether the lister is switched to ANSI or UTF-8ghisler(Author) wrote:How should TC know that a file is in UTF8 format? If it contains only English text, it looks the same as ANSI text...
For non-English text, however, it should be possible to "guess" the format even when there's no signature in the file (for example, verify that all bytes >= 0x80 fall into valid UTF-8 sequences... maybe it would be good enough?).
- ghisler(Author)
- Site Admin
- Posts: 48138
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Currently lister doesn't scan the entire file when loading it, so such a check would take a long time with big files. On the other side, scanning only a small part of the file could lead to incorrect results.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Right, scanning of the whole file is not a good idea if the file is really big - but I think that a smaller block (32kB?) can give quite a reliable result (if the number of 0x80+ characters exceeds certain limit, of course; the format of UTF-8 sequences is quite special).ghisler(Author) wrote:Currently lister doesn't scan the entire file when loading it, so such a check would take a long time with big files. On the other side, scanning only a small part of the file could lead to incorrect results.
Maybe this "text format auto-detection" could be an optional feature (enable/disabled in Lister options). Detecting Unicode files (without BOF signature) should be possible in a similar way.
Well, one could argue that not scanning only a small part never leads to the correct result. I think people would rather the lister at least tried to make an educated guess, based on a small part of the file, than that it did nothing.ghisler(Author) wrote:Currently lister doesn't scan the entire file when loading it, so such a check would take a long time with big files. On the other side, scanning only a small part of the file could lead to incorrect results.
I can send you a program I wrote to determine the encoding of files at work (based on an algorithm found in the Unix utility "file"). It's written in Ruby, but it should be easy enough to follow even if you're not familiar with the language.
Re: Lister and UTF-8 option
Is there any news on this?
I don't need any autodetection. I'd just like lister to always start in mode "7" instead of mode "1".
I don't need any autodetection. I'd just like lister to always start in mode "7" instead of mode "1".
Re: Lister and UTF-8 option
Install the CudaLister plugin.
You can set UTF-8 as default for opening files and it also has many advantages compared to pure Lister.
The options are reached by the context menu in any open file.
https://totalcmd.net/plugring/CudaLister.html
Windows 11 Home x64 Version 23H2 (OS Build 22631.3593)
TC 11.03 x64 / x86
Everything 1.5.0.1376a (x64), Everything Toolbar 1.3.3, Listary Pro 6.3.0.78
QAP 11.6.3.3 x64
TC 11.03 x64 / x86
Everything 1.5.0.1376a (x64), Everything Toolbar 1.3.3, Listary Pro 6.3.0.78
QAP 11.6.3.3 x64
Re: Lister and UTF-8 option
tommy0910,
HTH
Roman
Configuration - Options - Edit/View - External Viewer - Default:I'd just like lister to always start in mode "7" instead of mode "1".
Code: Select all
%COMMANDER_EXE% /S=L:T7
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
Re: Lister and UTF-8 option
Thx Great!
Re: Lister and UTF-8 option
Where should be "Default" in External Viewer settings? I can not find...Hacker wrote: ↑2020-06-24, 09:07 UTC tommy0910,Configuration - Options - Edit/View - External Viewer - Default:I'd just like lister to always start in mode "7" instead of mode "1".HTHCode: Select all
%COMMANDER_EXE% /S=L:T7
Roman
Image: https://diogenesfest.com/temp/TC-External-Viewer-settings.png
Re: Lister and UTF-8 option
amesh wrote: ↑2021-09-01, 09:55 UTCWhere should be "Default" in External Viewer settings? I can not find...Hacker wrote: ↑2020-06-24, 09:07 UTC tommy0910,Configuration - Options - Edit/View - External Viewer - Default:I'd just like lister to always start in mode "7" instead of mode "1".HTHCode: Select all
%COMMANDER_EXE% /S=L:T7
Roman
Image: https://diogenesfest.com/temp/TC-External-Viewer-settings.png
The text box (edit control) behind of "Default:"