FTP : UNICODE.TXT is binary… !

[Clo]

Hello !

- Set the FTP transfer mode as “Automatic”

- Download some *.txt files UNICODE

- You get a message telling :

“WARNING! You try to transfer a binary file in text mode!
The target file will probably be unusable! Abort transfer? [Yes] [No]”

• AFAIK, TXT-Unicode is not «Binary»… Hence, I guess there is a BUG somewhere …
TC 6.54a - Win XP-Pro SP1

 Kind regards,
Claude
Clo

[Gral]

Read this:
http://www.ghisler.ch/board/viewtopic.php?p=73851#73851
UNICODE contains many 0x00 chars.

[ado]

- Set the FTP transfer mode as “Automatic”
- Download some *.txt files UNICODE

When you are choosing automatic, you also define extension for text mode and I am pretty sure, .txt is by default defined as text file

AFAIK, TXT-Unicode is not «Binary»… Hence, I guess there is a BUG somewhere …

I am affraid TXT-Unicode is «Binary» if it is encoded as 16 bit unicode (not UTF8), because if you have there some normal english or french text, each another byte is 0x00. Just take a look through lister -> 3
So...for me it looks like that your BUG is at http://perso.wanadoo.fr/charries/relais/bug02.gif ...am I right ?

ado

[Clo]

2Gral

Hello !

- Thanks ! I have indeed the entry WarnWrongMode=0 in the Tutorial at the right location

- However, this doesn't the trick, since supposing a real error should occur = no warning at all…
- Most certainly, whether I can trust the “Automatic” feature, there is not a big risk…
- In such a case, maybe TC might care the extension too, which is stated alright here, and that this extension takes precedence ?

- It could be interesting to know which behaviours the standalone FTP-clients have - I no longer use any for… Bhuuuu … !

 Kind regards,
Claude
Clo

[van Dusen]

For information only:

Only files, which contains pure ASCII (character codes x00..x7F = U+0000..U+007F) are binary identical to the Unicode representation of the same file, if UTF-8 transformation format is used. UTF-8 uses 1, 2, 3 or 4 bytes for encoding of one character, UTF-16 uses 2 or 4 (2*2) bytes and UTF-32 uses always 4 bytes.

Characters with codes above U+007F are encoded with at least 2 bytes (even in UTF-8), even though the characters U+0000..U+00FF are the same as in ISO-8859-1 (apart from coderange U+0080..U009F, ISO-8859-1 is identical to Windows-1252).

x00-bytes are rare in UTF-8 text files, it is used for character U+0000 only.

[ghisler(Author)]

Indeed Unicode files must be downloaded in binary mode, not text mode, otherwise the ftp server will damage the files!

[Clo]

2ghisler(Author)

Good morning,

• Then, the issue is : WHY TC doesn't switch to “Binary” when set in Automatic mode ?

- In such a case, the message might be :
“WARNING! TC tries to transfer a binary file in text mode! …”
- Since TC is able to display such a message, it might be able too to switch to the right mode, ignoring the *.TXT extension, might it not ?
• BTW : I downloaded some small Unicode files as text, and they are not damaged, all are perfectly readable in a good editor like i.e. Notepad2 … ?

 M.f.G.
Claude
Clo

[Flint]

Clo
TC cannot with 100% guarantee determine if some file is binary or not, and if it's binary - will it be damaged on upload/download or not: it depends of the server. You've had luck that your particular files from your particular server were downloaded correctly despite of having "incorrect" bytes in it. It's even possible that uploading the same files back to the same server (in text mode, of course) will damage them. For example, I've uploaded on my web-site some Unicode file and downloaded it in text mode: it became damaged (Unix-like new-line characters tried to be replaced with Windows-like ones - but in such a manner as if they were non-Unicode ones).

So, TC can only determine (with some probability) that the file is not ANSI-text and warn about it. But it does not tell that the file will definitely be damaged after such an operation, it just warns that it's possible.

Then, the issue is : WHY TC doesn't switch to “Binary” when set in Automatic mode ?

The reason is that it doesn't know if the file will really be damaged. Maybe, it was your purpose - to transfer your binary file in text mode.

- In such a case, the message might be :
“WARNING! TC tries to transfer a binary file in text mode! …”

Eh... I cannot see the reason of such a change. Now the message is:
"WARNING! You try to transfer a binary file in text mode! ..."
And it's really not TC, it is you who tries to transfer the file. TC just performs the operation you told him to perform.

- Since TC is able to display such a message, it might be able too to switch to the right mode, ignoring the *.TXT extension, might it not ?

Not ignoring - but in addition. E.g. display not a standard Yes/No message, but smth like "What do you want to do? Continue text/Stop transfer/Transfer in Binary mode/Switch to Binary mode permanently" But I can't imagine a good interface for it yet... Such long text on the buttons would not be a good idea, I think. But such an idea itself looks to be useful.

[Clo]

2Flint

Hello K. !

And it's really not TC, it is you who tries to transfer the file.

• I disagree totally ! If I choose "Automatic", I no longer decide, it's well TC which decides by itself !

- About the FTP interface :
- Please, just give a glance HERE, it's quite possible to use buttons…
• Note : to get the original capture, the TC title bar is indeed on the "inactive" state, so "red" here, that's not very nice… It's the FTP stuff which has the focus…

 Kind regards,
Claude
Clo

[Flint]

Clo

• I disagree totally ! If I choose "Automatic", I no longer decides, it's well TC which decides itself !

Ah, I see. When I performed experiments, I used forced Text mode - and forgot that in Auto mode TC gives this warning for every Unicode TXT-file. Maybe, you are right then...

- Please, just give a glance HERE, it's quite possible to use buttons…

This is not exactly what I meant. My idea was: now TC shows this message, and I suppose it should show something like that.

[Clo]

2Flint

Good evening!

…and I suppose it should show something like that. …

- Yes, this is nice, I support !
- I just wished to show you an idea about the general design to improve the using of FTP… and change the transfer mode without that damned drop-down list… IMHO, buttons are always easier to use than lists [OT, but your opinion should be highly appreciated ! ]
- So finally, almost the same using that your buttons.

 Kind regards,
Claude
Clo

[Flint]

Clo

IMHO, buttons are always easier to use than lists [OT, but your opinion should be highly appreciated ! ]

In general I completely agree with you. Selecting some action by pressing a button is always easier and faster. But unfortunately, buttons are not always possible to use. Especially in TC which can be translated into various languages. I don't know about French, but in Russian the translation is in approx. 90% cases longer than the English original phrase. Current listbox can be translated into Russian almost without problems. Adding 3 buttons of the same length as this list-box, of course, is impossible, so they have to be much shorter, and hence they will contain only small short words that are 1) less informative than the current lines and 2) much harder to translate into other languages.

[Clo]

2Flint

- Hello!

- In French, it's longer for 99% of the strings
• Fortunately, there are workarounds with TC, since one can choose the fonts, and I have a narrow special one which saves the situation many times ! And with Delphi, one can set an info-text for each button in addition as well… (I guess ?)

• In Russian, it seems for me that the characters are larger (no different graphisms for Upper¦Lowercase, just the same characters as two sizes…) maybe the words themselves are not the longest… ? P P ?

{ Aside : • In fact, I showed you that example -which is not in TC, but in his “fellow-soft” XnView - to know your opinion about the shown request, because there was a pretty hot row… and I will hammer the nail again, I'm very obstinate when I think I'm right ! So, feasible or not ? That is the question ! }

—> Back to topic : Finally, I guess it might be possible to improve both the GUI for FTP, and the behaviour of TC while setting « Automatic » transfer-mode with Unicode text-files.

Kind regards,
Claude
Clo

[Flint]

Clo

• In Russian, it seems for me that the characters are larger (no different graphisms for Upper¦Lowercase, just the same characters as two sizes…) maybe the words themselves are not the longest… ? P P ?

I think, both effects take place. I translate texts using EmEditor - it uses monospace font by default, and I see there that Russian translation is almost always longer than the English original phrase.

{ Aside : • In fact, I showed you that example -which is not in TC, but in his “fellow-soft” XnView - to know your opinion about the shown request, because there was a pretty hot row… and I will hammer the nail again, I'm very obstinate when I think I'm right ! So, feasible or not ? That is the question ! }

I wouldn't set such a question only. The way should be not only feasible, but also convenient in use in all senses.

Finally, I guess it might be possible to improve both the GUI for FTP, and the behaviour of TC while setting « Automatic » transfer-mode with Unicode text-files.

Quite agree.

[ghisler(Author)]

WHY TC doesn't switch to “Binary” when set in Automatic mode ?

When you switch to Automatic mode, TC asks you which file names/extensions should be considered as text, and which as binary. If you specify *.txt as text, then Total Commander will always use text mode if the file has extension .txt. So to avoid this, you need to give the file an extension which is NOT on the autmoatic text list.