Multicore internal ZIP
Moderators: Hacker, petermad, Stefan2, white
Multicore internal ZIP
Well, I would like to suggest, if it is possible, to add multicore compression for internal zip archiver, for making really quick backups... is that possible?
- ghisler(Author)
- Site Admin
- Posts: 50873
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
This would be possible when packing many small files, because each file is compressed separately (no solid archives), but not when packing 1 large file. The ZIP compression doesn't look like it could be parallelized well, because the compression results of later bytes depend on the results of previous bytes.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
- ghisler(Author)
- Site Admin
- Posts: 50873
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Sorry, that's not something which could be done over night, it's very complex to get it right. In the worst case, it would be even slower, because the disk operations to read all the various files take a considerable amount (and reading from multiple files at the same time is a very bad idea).
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
I wouldn't be asking if it hasn't been done in 7zip's zip compression - it takes multiple cores and compresses files; when dealing with one big file, it does employ just one core; but when dealing with folder with a lot of smaller files, it is considerably faster. Hard drives are not so slow as ten years ago, and they can handle reading four files at the same time.