Fixed - Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Bug reports will be moved here when the described bug has been fixed

Moderators: white, Hacker, petermad, Stefan2

Slavic
Senior Member
Senior Member
Posts: 290
Joined: 2006-02-26, 15:41 UTC
Location: Montenegro

Fixed - Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Slavic »

When viewing the text files, Lister allows to select current text (by mouse or select all in menu) and copy it to Clipboard. This works well with UTF-8 texts in UTF-8 mode, but in Explorer Preview mode the command "Copy as text" copies the UTF-8 character as plain text (ANSI). As a result, the copied text is unreadable.

How to reproduce
1. Create a test file with non-Western encoding. I use the text in Russian which means "Cyrillic text UTF-8."

Code: Select all

Кириллический текст UTF-8.

Copy it to Notepad and save in UTF-8 encoding, as file testcyr8.txt for example.
2. Open the empty Notepad. Open the file in Lister, by default the mode should be UTF-8 (7) - Lister correctly detects it.
3. Use the keyboard commands Ctrl+A, Ctrl+C or menu Edit: Select all, Copy as text. Paste in Notepad as a reference.
4. Change Lister mode to Explorer Preview (8). Select the text or part of it with mouse, use Ctrl+C. Paste in Notepad, press Enter (for new line). The copy is good.
5. Use the Lister menu Select all, Copy as text. Paste in Notepad. The copy is wrong.

Possible reason
When in the Explorer Preview mode, the commands Ctrl+A, Ctrl+C are passed directly to external handler, which correctly works with Unicode. But the menu Copy works with the internal Lister buffer, and in mode 8 Lister doesn't recognise buffer as UTF-8 (like recognises it in mode 7). The command from Copy menu uses the "raw" buffer as ANSI text instead of Unicode.

Possible solutions
General: in mode 8 don't process these menu commands internally, instead try to pass them to view handler as keyboard combinations Ctrl+A, Ctrl+C.
or
Alternative: in mode 8 detect UTF-8 text in internal buffer and copy it to Clipboard as Unicode.
Last edited by Slavic on 2021-04-29, 17:37 UTC, edited 1 time in total.
Desktop: Windows 11 Pro 23H2, TC 11.03(RC). Mobile: Pixel 5a, Android 14, TC 3.42b5
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *ghisler(Author) »

I don't know how you managed to view text in Explorer preview mode, but it doesn't work here.

When I try to switch a text file to mode 8, it automatically switches to an appropriate text mode. So if the file is UTF-8, then Lister will switch to mode 7.

Furthermore, "Copy as text" does not work in a true Explorer preview mode. For example, when viewing a csv (comma separated values) file in mode 8, choosing "Copy as text" does nothing. Ctrl+A, Ctrl+C does work, though, and is handle directly by the preview handler. For csv, it works with Unicode here, e.g. with Chinese.
Author of Total Commander
https://www.ghisler.com
Slavic
Senior Member
Senior Member
Posts: 290
Joined: 2006-02-26, 15:41 UTC
Location: Montenegro

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Slavic »

It does work here without any special settings. Please note that it's not a default Explorer Preview mode, no checkbox at "Define view method by file type". Lister shows the files by default settings using autodetect, then mode for text can be changed freely, at least in TC 10 beta 7. Our test file is opened in mode 7 as UTF-8, then we can select mode 8 using the key 8 or menu Options. Interestingly, in mode 8 there is a narrow grey vertical line left of the text area; maybe it's shown only in Windows 8, I haven't tested.

In mode 8 for text (unlike images or multimedia) the sub-items in the menu Edit are not disabled. Using the "Select all", then "Copy as text" (via menu, not by key combinations) we get the text to clipboard and can paste it in other program like Notepad. I described this above.
Desktop: Windows 11 Pro 23H2, TC 11.03(RC). Mobile: Pixel 5a, Android 14, TC 3.42b5
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Usher »

2Slavic
I suspect that you have installed some programmer's editor with shell extension.
Andrzej P. Wozniak
Polish subforum moderator
Slavic
Senior Member
Senior Member
Posts: 290
Joined: 2006-02-26, 15:41 UTC
Location: Montenegro

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Slavic »

Usher, please don't suspect me, I don't have any external shell extension or resident program which might affect copying to clipboard. The only background process, besides the standard in Windows 8.1, is Classic Start menu. Also some OS services here are switched from automatic start to manual, but their absence should not be detrimental.

Could you tell more exactly, which of my steps aren't reproduced as I described? This might be helpful, maybe my TC internal configuration is different, maybe some "old heritage" or non-default OS settings may affect. I can test on another PC with different TC installation or settings.
Desktop: Windows 11 Pro 23H2, TC 11.03(RC). Mobile: Pixel 5a, Android 14, TC 3.42b5
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *ghisler(Author) »

Does the Lister title bar show
Lister <Explorer> - File name
or just
Lister - File name
?

If it shows the first of the two, then Lister is indeed using an Explorer preview handler. If it shows the second, then it switched back to a text view mode.
In this case, can you check the "Show" menu which mode is checked, please?
Author of Total Commander
https://www.ghisler.com
User avatar
petermad
Power Member
Power Member
Posts: 14739
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *petermad »

2ghisler(Author)
can you check the "Show" menu
I guess you mean the "Options" menu item? There is no "Show" item in Listers menu.
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50b4 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6450
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Horst.Epp »

Slavic wrote: 2021-04-27, 15:23 UTC It does work here without any special settings. Please note that it's not a default Explorer Preview mode, no checkbox at "Define view method by file type". Lister shows the files by default settings using autodetect, then mode for text can be changed freely, at least in TC 10 beta 7. Our test file is opened in mode 7 as UTF-8, then we can select mode 8 using the key 8 or menu Options. Interestingly, in mode 8 there is a narrow grey vertical line left of the text area; maybe it's shown only in Windows 8, I haven't tested.

In mode 8 for text (unlike images or multimedia) the sub-items in the menu Edit are not disabled. Using the "Select all", then "Copy as text" (via menu, not by key combinations) we get the text to clipboard and can paste it in other program like Notepad. I described this above.
That does not happen here
if you are in a real Explorer preview (shown in the title and shown enabled in the Options).
In this case the Lister entries under Edit are enabled but simply do nothing.
The Windows text preview module has no such functions so thats expected behaviour.
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Usher »

Slavic wrote: 2021-04-27, 18:43 UTCCould you tell more exactly, which of my steps aren't reproduced as I described?
I'm sorry, I can't help you any more. I don't use Windows 8.x, so I can only suspect some things. Oh, there's one more thing to test - very long file paths/names.

I just know that some programs are more aggressive than you can expect. For example, you can find many places which report problems with well known ACDSee viewer.
Andrzej P. Wozniak
Polish subforum moderator
Slavic
Senior Member
Senior Member
Posts: 290
Joined: 2006-02-26, 15:41 UTC
Location: Montenegro

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Slavic »

Lister, menu Options:
Image: https://prnt.sc/1279vom

Lister, menu Edit:
Image: https://prnt.sc/1279xp5

Notepad:
Image: https://prnt.sc/1279yj5
Desktop: Windows 11 Pro 23H2, TC 11.03(RC). Mobile: Pixel 5a, Android 14, TC 3.42b5
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Usher »

2Slavic
Right click the same file directly in Windows Explorer and show the screenshot of your context menu, please. Or use Explorer Preview for some source code (html, c, whatever).
Andrzej P. Wozniak
Polish subforum moderator
Slavic
Senior Member
Senior Member
Posts: 290
Joined: 2006-02-26, 15:41 UTC
Location: Montenegro

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Slavic »

Explorer, context menu: Image: https://prnt.sc/127c7ws
Desktop: Windows 11 Pro 23H2, TC 11.03(RC). Mobile: Pixel 5a, Android 14, TC 3.42b5
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6450
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Horst.Epp »

Slavic wrote: 2021-04-28, 13:59 UTC Explorer, context menu: Image: https://prnt.sc/127c7ws
That is not the context menu of a text Explorer Preview, its a file context menu.
You have to right click inside of an Explorer preview to see any preview handler functions.
There are preview handlers which have context menu functions like the .eml preview handler
but the Microsoft text preview handler has no context menu at all, so you can't copy text from it.
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
Slavic
Senior Member
Senior Member
Posts: 290
Joined: 2006-02-26, 15:41 UTC
Location: Montenegro

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Slavic »

Horst.Epp wrote: 2021-04-28, 15:56 UTC the Microsoft text preview handler has no context menu at all, so you can't copy text from it.
Sorry, but this is wrong and clearly means that you haven't even tried to reproduce the bug as I've described. Otherwise you would know that even if the Explorer Preview handler doesn't have the built-in context menu for text, the key combinations Ctrl+A and Ctrl+C do work perfectly. And the mouse text selection works as well. And UTF-8 text is copied correctly this way. Can you try to reproduce this now, at last?

As for screenshot above, I took it exactly as Usher asked me: directly in Windows Explorer. What is a reason to do this, please ask him, but not tell that I did it wrong way. Because otherwise it looks like you ask me to take a screenshot of non-existent context menu in Lister mode 8.

Sorry, sometimes I may be close to losing a patience, It's hard to explain the same again and again...
Desktop: Windows 11 Pro 23H2, TC 11.03(RC). Mobile: Pixel 5a, Android 14, TC 3.42b5
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: Lister: in Explorer Preview mode the Copy menu copies UTF-8 text as ANSI

Post by *Usher »

Horst.Epp wrote: 2021-04-28, 15:56 UTC
Slavic wrote: 2021-04-28, 13:59 UTC Explorer, context menu: Image: https://prnt.sc/127c7ws
That is not the context menu of a text Explorer Preview, its a file context menu.
That's exactly what I have asked. Now I'd like to see content of 3 submenus…

@Horst.Epp
Horst, do you have "Share with" submenu?
Horst.Epp wrote: 2021-04-28, 15:56 UTCThere are preview handlers which have context menu functions like the .eml preview handler but the Microsoft text preview handler has no context menu at all, so you can't copy text from it.
That's your paraphrase of my second suggestion - repeat tests for more specific text files.
Andrzej P. Wozniak
Polish subforum moderator
Post Reply