TCBL new version (1.04 )

[kesdoputr]

In addition, I have tried

Code: Select all

param=-i %UL playlist "%T"

It seems good on the listfile, but unfortunately the second line(first file) can't be read by player.
The second line will increase a invisible space that let filesize incrase 3bytes.(compare with file manually edit with same content)

It might be the BOM maker because %UL has BOM

[nsp]

Code: Select all

param=-i %UL playlist "%T"

TCBL do not support Unicode.. Only ansi list have to be passed as parameter. By default codepage is the same as installed windows but you can change it if it does not match the list encoding.
internally TCBL only works with single byte character.

[kesdoputr]

Oh~~That's sad, if TC has variable that UTF-8 filelist without BOM then it's work~~

thanks for your reply

[kesdoputr]

Just a idea, tcbl will read the list(%L) and perform cmd on everyline.
I think if it is possible to add a option control tcbl to skip first serveral bytes?

ex.If i use %UL, i have tried tcbl can read the utf-8 contents correctly but will cause error because %UL contains BOM maker,
so tcbl will read and cause there are 3bytes EF BB BF before the first line.

If tcbl can skip the first 3 bytes then the result will all ok. I can use tcbl to generate UTF-8 format correctly from %UL.

Just a suggestion, thanks for your reading.

[nsp]

Unfortunately this is not as simple as this.
Even if the BOM is identified and skipped, in UTF8 most char are using single byte as for ANSI but accentuated ans some special char are encoded in 2 or more bytes which is not supported inside TCBL.

The design target creation of bat file using cmd as interpreter. Some command do not support full Unicode without some tweak...
A new version targeting powershell or python or .... would be safer but need a more robust redesign including SuperWDX replacement.

Currently i do not have the need nor the time to do it.

[kesdoputr]

Ok i understand, thanks for your reply

[Usher]

65001 stands for Unicode UTF-8, but INI files should be only in ANSI codepages (125x etc.) or Unicode UTF-16LE (1200).

TC supports UTF-8 in some other text files, but that support is inconsistent. For example LNG files may be in UTF-8 with BOM, but MNU files in UTF-8 must be without BOM.

[kesdoputr]

I found BOMRemover on github
https://github.com/TheZoc/BOMRemover

●Remove the BOM maker of %UL variable to a temp file

[em_UTF8NoBOM]
cmd=%ComSpec% /c "%COMMANDER_PATH%\PLUGINS\ETC\BOMRemover.exe"
param=%UL %%temp%%\utf8nb.txt

●Make filelist by tcbl from the temp file

[em_Filelist]
cmd=%ComSpec% /c ""%COMMANDER_PATH%\PLUGINS\ETC\tcbl.exe" -i""
param=%%temp%%\utf8nb.txt filelist "%T"

●Make a combine cmd by TCFS2

[em_MakeFilelist]
cmd=%COMMANDER_PATH%\PLUGINS\ETC\TCFS2.exe
param=/u- /ef "tem(`em_UTF8NoBOM`) delay(150) tem(`em_Filelist`)"

Finally bind a alias to em_MakeFilelist, now the result list is UTF-8 without BOM maker.

It must not the best solution, but temporary solve my problem now.
Maybe in the future TC will has it's own UTF-8 list file variable without the BOM, then everything will be simple:)

[nsp]

65001 stands for Unicode UTF-8, but INI files should be only in ANSI codepages (125x etc.) or Unicode UTF-16LE (1200).

TC supports UTF-8 in some other text files, but that support is inconsistent. For example LNG files may be in UTF-8 with BOM, but MNU files in UTF-8 must be without BOM.

@Usher,
Here we do not target TotalCommander support for UTF* or unicode but support for the TCBL external tool which do not support unicode. It support all codepage with single byte character encoding.
So %UL and %WL even if used with codepage 65001 or 10000..

With UTF8 : Bom will not be interpretted and if multi-byte char are sent this can break character buffer.
With UTF16 : this will be the same in worse.

It must not the best solution, but temporary solve my problem now.
Maybe in the future TC will has it's own UTF-8 list file variable without the BOM, then everything will be simple:)

@kesdoputr,
This cannot be the solution as if multi-byte char are part of the list, you will break character buffers ($n, $p $f... )--> The resulting list will be truncated !
Standard way in windows for UTF8 is to have BOM so i'm pretty sure TC will not generate filelist without BOM even if it is easier.. !

For your simple case, you can use a fairly simple vb-script to read all lines and add the word "file " in front of each line.

[nsp]

@kesdoputr
Addition to previous post..
You can use a bat like the following:

Code: Select all

@REM UFT8Appender.bat
@echo off
chcp 65001
REM  %~dp0\gecho -ne \xEF\xBB\xBF> "%3"
for /F "usebackq tokens=*" %%A in ("%2") do echo %1 '%%A'>>"%3"

call from TC

Code: Select all

[em_UTF8Playlist]
command=<pathTo>\UFT8Appender.bat
params=file %UL %T\playlist.txt

The generated file will not have BOM, if you want one you should use gecho from gnuwin32 coreutils + liiconv2.dd +libintl3.dll

[kesdoputr]

It must not the best solution, but temporary solve my problem now.
Maybe in the future TC will has it's own UTF-8 list file variable without the BOM, then everything will be simple:)

@kesdoputr,
This cannot be the solution as if multi-byte char are part of the list, you will break character buffers ($n, $p $f... )--> The resulting list will be truncated !
Standard way in windows for UTF8 is to have BOM so i'm pretty sure TC will not generate filelist without BOM even if it is easier.. !

For your simple case, you can use a fairly simple vb-script to read all lines and add the word "file " in front of each line.

It's strange but in my case tcbl seems can read the multi-byte correctly?
I have 4 files with korean filename
https://i.imgur.com/mu8srgG.png

And i writh below in tcbl.ini

Code: Select all

[test]
ini=o $1\test.txt
ini=x
ini=q
ini=k
cmd=$n
cmd=$p
cmd=$f

Then write a tc cmd(the utf8nb.txt is generated from %UL by BOMRemover)

Code: Select all

[em_test]
cmd=%ComSpec% /c ""%COMMANDER_PATH%\PLUGINS\ETC\tcbl.exe" -i""
param=%%temp%%\utf8nb.txt test "%T"

The result test.txt content is below, the korean char can be read correctly.
https://i.imgur.com/20rBkui.png
Next i open a new txt file and use copy/paste to manually edit with same content and save to text2.txt

Then i compare two text file by beyond compare, and they are exactly same
https://i.imgur.com/Q7Qt40Z.png

===========================================
BTW,In my test, the rusult is the same when input is %L(ANSI) between directly use cmdline and use ini
https://i.imgur.com/9Fo6e5R.png

generate l1 and l2

[em_lll1]
cmd=%ComSpec% /c ""%COMMANDER_PATH%\PLUGINS\ETC\tcbl.exe" -q""
param=%L echo file '$f'$A """"%Ttest_l1.txt""""

[em_lll2]
cmd=%ComSpec% /c ""%COMMANDER_PATH%\PLUGINS\ETC\tcbl.exe" -i""
param=%L lll2 "%T"

[lll2]
ini=o $1\test_l2.txt
ini=x
ini=q
ini=k

But the rusult is different when input is %UL(UTF-8) between directly use cmdline and use ini
https://i.imgur.com/U4LNVMp.png

generate u1 and u2

[em_uuu1]
cmd=%ComSpec% /c ""%COMMANDER_PATH%\PLUGINS\ETC\tcbl.exe" -q""
param=%UL echo file '$f'$A """"%Ttest_u1.txt""""

[em_uuu2]
cmd=%ComSpec% /c ""%COMMANDER_PATH%\PLUGINS\ETC\tcbl.exe" -i""
param=%UL uuu2 "%T"

[uuu2]
ini=o $1\test_u2.txt
ini=x
ini=q
ini=k
cmd=file '$f'

I don't know why but in my case tcbl seems can read correctly when use ini.

Anyway, thanks for your reading, sorry thake your time on my question and thanks for your reply.

[kesdoputr]

@kesdoputr
Addition to previous post..
You can use a bat like the following:

Code: Select all

@REM UFT8Appender.bat
@echo off
chcp 65001
REM  %~dp0\gecho -ne \xEF\xBB\xBF> "%3"
for /F "usebackq tokens=*" %%A in ("%2") do echo %1 '%%A'>>"%3"

call from TC

Code: Select all

[em_UTF8Playlist]
command=<pathTo>\UFT8Appender.bat
params=file %UL %T\playlist.txt

The generated file will not have BOM, if you want one you should use gecho from gnuwin32 coreutils + liiconv2.dd +libintl3.dll

Thanks for your reply, i will try this method later.
Is the three files gecho.exe libiconv2.dll libintl3.dll should be put together with the UFT8Appender.bat?

[kesdoputr]

Hello, i have tried the UFT8Appender.bat and generate a file named playlist1.txt

Next i open a new text file and manually edit with same content and save to playlist2.txt

Then i found two file size is different by 3bytes.

In this example, playlist1 and playlist2 all are UTF-8 without BOM, but playlist1 is 3 bytes more then playlist2.
https://i.imgur.com/s7IiClo.png
In the first line between ' and G is EF BB BF, which is the BOM maker.

I think it is because %UL has BOM, when UFT8Appender.bat read the first line and output, it still output the BOM bytes
even if the output file is actually no BOM.

https://i.imgur.com/3kyNV59.png
In the above picture, the cursor is just between ' and G and seems no other char between them, but there exist a invisible space.
Then i need to press right twice to move the cursor to the right side of G.

[opicron]

For the live of me I cant get the following parameter to work:

Code: Select all

magick c:\tse\in.jpg -gravity center -background white -extent "%[fx:w*1.1]x%[fx:h*1.1]" c:\TSE\out.jpg

It either removes the % when I use %% or \% and the [f are removed somehow.

I confirm this command works in cmdprompts.


This is as far as I can get:

Code: Select all

%L c:\TC\tools\imagemagick\magick $f -gravity center -background white -extent \"[fx:w*1.1]x[fx:h*1.1]\" $x.png

Inserting %% in front of the [fx parts does NOT work.

[opicron]

I finally found it:

Code: Select all

%L c:\TC\tools\imagemagick\magick $f -gravity center -background white -extent \"%%%%[fx:w*1.1]x%%%%[fx:h*1.1]\" $x.png

Sigh..