Fully functional substitute for the internal zip - with support for TC's packing precise customization

[petermad]

Version 8.51a produces zip-archives which (in my case) do not have incompatibility issues with some other software

I Checked the WCMZIP64.DLL zip packer application extension - since TC 8.51a it has been changed for version 9.00, 9.12, 9.20, 9.22 and 9.50
2Dalai

2odeon
Well, it's always a good idea to test with a current software version before complaining about an issue

Seems to me that odeon did use th current TC version 10.00. He later informed us that if he goes back to version 8.51a the problem at hand was not an issue.

---
It could be interesting if odeon would test the later versions I mention here above, to see when the problem first occurs.

[Dalai]

Seems to me that odeon did use th current TC version 10.00. He later informed us that if he goes back to version 8.51a the problem at hand was not an issue.

It's possible to understand the posts both ways, it hasn't been stated which version (or architecture) was/is used before.

It could be interesting if odeon would test the later versions I mention here above, to see when the problem first occurs.

Yes, indeed. Note that the changes were not that often in the 32-bit ZIP library wcmzip32.dll (no changes between 9.12 and 10.0). It could also be helpful to get some sample archives to perhaps analyze them what exactly is different between them.

Regards
Dalai

[petermad]

Note that the changes were not that often in the 32-bit ZIP library wcmzip32.dll (no changes between 9.12 and 10.0)

I can confirm that for 32bit TC.


I have tried packing some files (English charaters in filenames only and compression 6) with the aformentioned TC versions (8.51a, 9.00, 9.12, 9.20, 9.22 and 9.50) - the zipped files were excactly the same.

So there must be some other setting that is different in odeon's TC 8.51a and 10.00

[odeon]

2Dalai

Which version did you use previously?

The issue was first discovered in the latest version of TC, 10.0, which I currently use.

Does it work in one of the latest stable TC versions, like 8.52a, 9.0a, 9.51 or 10.0?

I've tried all the stable releases from 8.51a to 10, and found that 8.51a was the last to produce zips that could be opened by GoldenDict.

The only difference I found between different versions' zips was their properties in Winrar tab, for 8.51a-made zip it was simply 'ZIP', from version 8.52 and above up to the current 10th version - 'ZIP64'.

I believe that is the cause of the one known incompatibility issue. File sizes of zips64 made by various versions of TC (as well as their architectures x32/x64) were identical, while the 8.51a-made 'simple zip' was a bit smaller (I used the x64 8.51a version).

I used both GoldenDict versions x32 and x64 with same results.

Can this somehow be configured in TC? I.e., to force making simple zip archives instead of the zip64 ones?

[Dalai]

Are these archives larger than 2 or 4 GiB in size? Does TC ask the following (or similar) question?

Code: Select all

---------------------------
Total Commander
---------------------------
Warning: The total size of all selected files (unpacked) is > 2 GB.
If the resulting archive is larger than 2 GB, it will be unreadable by some ZIP unpackers.
Continue anyway?

Wondering why this message appears when packing files larger than 4 GB, yet it claims to be about 2 GB sizes. Or, in other words, this message doesn't appear when packing files between 2 and 4 GB. Smells like a bug to me, either in code or the message wording.

Anyway, I can only create ZIP64 archives when the files are larger than 4 GiB, regardless of TC version or bitness. In other words, I can't confirm your observation, but maybe it depends on some other factors. It would be good if you could provide some screenshots of the archive's properties, which may allow us to get an idea of what the issue is. Or some sample archives if possible.

Regards
Dalai

[odeon]

2Dalai

Working zip-archive Windows properties (right mouse button - 'Properties'):
https://ibb.co/2jxPHwM
Not working zip:
https://ibb.co/XCCbb84

the experimental file (folder in this case) is less than 0.5 GB in total, no warnings ever come out

Also as I used fresh installed (previous) versions of TC, no configuration changes whatsoever had been made, just default settings.

2petermad

It could be interesting if odeon would test the later versions I mention here above, to see when the problem first occurs.

It started with 8.52, when zipping began to produce ZIP64 archives by default.
The tested folder's name is only Latin letters, no spaces. This is also true for the files included inside this folder.

Oh btw the system codepage is Chinese (China), I'll change it to English next time it's possible to restart and check again to provide conditions equal to your systems.

Here, viewtopic.php?p=129238#p129238 Christian writes that

TC automatically uses 64-bit file offsets if the ZIP file becomes larger than 4 GB

I think that for some reason after the 8.51a version ZIP64 became forced automatically even with smaller files. It would be great to configure this setting manually, if it's possible and reasonable

[Dalai]

Huh, it seems I found the cause for this:

Code: Select all

10.08.15 Release Total Commander 8.52 beta 1 (32/64)
[...]
17.07.15 Fixed: ZIP packer: Add zip64 central directory record if number of files >= 2^16 (65536), otherwise Explorer wouldn't show all the files (32/64)

Since you have more than 86,000 files in your archive, TC adds a ZIP64 header. So it's just the number of files in the archive, not their size or characters in the filename.

Just pack less than 2^16 files next time and it will work even in newer TC versions .

Regards
Dalai

[odeon]

2Dalai
Great discovery, congrats on that! Now we know all the causes and circumstances ))
Anyway I can't pack less files, because all of them are needed. So I'll just use the 8.51a version, that's it.
But again, for the flexibility reason, I think that would be great to add this control to the configuration. Like, you know, use 'Before and including 8.51a' or 'After 8.51a' zipping mode. To suit both Explorer fans and those who use software products which are sensitive to this specifics

[Dalai]

In my opinion this is a bug in that other software. Is there any chance you can report this as a bug to the author of the other software? I can't remember reports about "incompatible" ZIP archives over the last couple of years (which doesn't mean much since I'm getting older, too ).

Regards
Dalai

[petermad]

The only difference I found between different versions' zips was their properties in Winrar tab, for 8.51a-made zip it was simply 'ZIP', from version 8.52 and above up to the current 10th version - 'ZIP64'.

That's not what I found, but I also only tested packing a few files not > 65536 files.

So I'll just use the 8.51a version, that's it.

There is still my solution with Total7zip - or does that not work either with > 65536 files?

[odeon]

2Dalai

In my opinion this is a bug in that other software

Yes, GoldenDict (other software) just doesn't support zip64 archives. More a shortcoming or a drawback than a bug actually. This feature was never announced to be supported. And there's no much chance that it will be implemented, the development is very limited now, but who knows.


2petermad

That's not what I found, but I also only tested packing a few files not > 65536 files

If the amount of files is less than 65536 items, an ordinary zip file is made, which is (roughly) similar in all TC versions

There is still my solution with Total7zip - or does that not work either with > 65536 files?

Your solution works, as I wrote earlier, and works exactly with my 'experimental' folder, which contains more than 65536 files
(I used your method with the amendment of changing archive format from 7z to zip in 7z plugin options after clicking configure button in the TC's packing menu, otherwise compression process stalled at 0%, perhaps I had to wait some more time).
So that's a working method, and exactly answers the thread's title (though the discussion had finally led to previously unexpected findings and conclusions).

Thank you for all your answers and support!

[odeon]

2petermad
Tried your method again more thoroughly - it works without any additional steps I thought were needed.
Very convenient, just the same as using the internal zip, but the result is working in my case (i.e. compatible with older software which doesn't support Zip64 format).

[petermad]

2odeon

it works without any additional steps

That's it - for some reason, if you try to make changes via the configuration - then it does NOT work.

You can see more about direct packing/unpacking other formats in the readme.rtf file that comes with the plugin:

from the readme.rtf file:

You can use Total7zip with other formats and you can create bzip2, gzip, tar, wim, xz and zip archives as well.
This is like my own PackerPlugins section looks -
[PackerPlugins]
7z=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
wim=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
xz=731,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
bzip2=731,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
7zip_gzip=731,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
7zip_tar=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
7zip_zip=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
gzip=731,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
tar=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
zip=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
rar=196,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
bz2=196,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
arj=196,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
cab=708,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
flv=708,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
gz=716,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
lzh=735,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx
jar=212,%COMMANDER_PATH%\Plugins\wcx\Total7zip\Total7zip.wcx

[odeon]

2petermad
A versatile goodie! Thanks, previously used it and didn't even know how advanced this plugin is...

Do you think it's a good idea, to post a feature proposal, i.e. to add configuration option (not in the main interface maybe, but in the wincmd.ini?) to choose between:

1) automatic Zip64 when archived files are too big or too numerous;
and
2) old-fashioned 8.51a version and before method (for compatibility with older non-Zip64 supportive software).

Both methods are ready, no additional development is needed (as far as I know), just to add the needed switch to the code (this job also requires its aim, surely, pls see below):

Your method through 7zip plugin has good usability and the result (I guess) is both Explorer and older non-Zip64 software compatible (i.e., better than the old-fashioned pre-8.52 version way). But it requires a user to take some investigation into this, and also why TC-made zips are not 'BAD', or at least 'WORSE' than 7z and Winrar's (default user wouldn't be aware of all these specific things we've found out here). Seeing the switch between the new-fashioned Zip64 method and the old-school one, users will get the difference and make the desired result suitable for their needs.

Now, most such users just try&fail then remember that TC's zips are not 100% usable and use other software (or 8.51a) for this purpose ever since. I had personally heard it from many, before faced this myself

[Dalai]

2odeon
It's not so simple, although you're probably right about the implementation being easy.

Either a switch is added and affects the creation of all ZIP archives which would make the creation of archives larger than 4 GiB impossible because ZIP64 is required for those. Or such a switch would only apply to the thing I quoted from TC's history.txt, but that would require very precise and careful wording as TC wouldn't create ZIP64 archives anymore even if there are more than 2^16 files (along with the drawbacks mentioned in history.txt) but it would still do so if the files are larger than 4 GiB.

Also keep in mind that it might be a very limited case not many users would benefit from. And, there's the usual stuff like users forgetting about settings they set months or years ago.

All that being said, I don't say you shouldn't make a suggestion. But in the end it's Ghisler's decision if and how it's implemented.

As I said: not so simple because everything has consequences. I hope I didn't sound too pessimistic .

Regards
Dalai