Hacker wrote:alexanderwdark,
Very cool, thanks a lot! Hopefully finally a useful packer (that stores file dates and other attributes, etc.) that uses the PAQ algorithms.
Roman
What use for PAQ8 do you find?
PAQ8P has very strong compression, but very bad speed, PAQ9A - more good speed, but not so good compression. LPAQ8 - fast version of PAQ8 (implemented in text compression mode).
PAQ8P - for one-time compression and storing for years
I know the PAQ family, though use different terms to describe their speed (LPAQ / PAQ9: very-extremely slow; PAQ8 - unusable).
"One time compression and storing for years"?
I.e. partition backup? Your computer spends a month compressing 20 GB. Then the partition fails. You need to wait another month to restore it. Not cool.
ADDED: BTW, do you have ECC memory? Because without it PAQ8 is unlikely to (de)compress a big file correctly.
Last edited by m^2 on 2008-12-29, 16:28 UTC, edited 1 time in total.
Implementing format - to big job and not in plugin concept. Plugin works with its own open-text-header PPMP format, with universal structure (also used in DarkCryptTC, etc.), just as file - to file packer (or from TC to tar.ppmp - this is great TC's feature)
So, It's easy to add some algorithm, just for fun. But implementing format, (even very nice and great) with full multi-file structure - to much time and debugging for one coder.
For PEA - possible to make some implementation, but compatibility - must be another project.
m^2 wrote:I know the PAQ family, though use different terms to describe their speed (LPAQ / PAQ9: very-extremely slow; PAQ8 - unusable).
"One time compression and storing for years"?
I.e. partition backup? Your computer spends a month compressing 20 GB. Then the partition fails. You need to wait another month to restore it. Not cool.
ADDED: BTW, do you have ECC memory? Because without it PAQ8 is unlikely to (de)compress a big file correctly.
Do you have ECC memory - of course, not. Looking at some tests (maximumcompression, etc.), I see, testers had decompressed ENWiki and other file-setes by PAQ8 and CRC was ok. Plugin has not target to be some of state-of-art packers or commertial-quality product - this plugins stays with it's source, and someone can make stable releases. Implementing PAQ, etc. - just experimental, "just a hack"
P.S. Of course, LZMA, BZIP2 - state of art in packing. PPMD - king of text compression. For quick and high-level compression rar3 -m5 with it's ppmd text processor - better way.
According to Corsair, you can expect 1 memory error per 107 hours in 1800 MB that PAQ8 uses.
ENWIK9 takes 1 SIGB.
My computer (Pentium D@2.66ghz) gets 6-8 KB/s depending on a file when using PAQ8p1 exp04.
I didn't run it on ENWIK, but with 8 KB/s it's gonna take 33.9 hours to (de)compress and gives me ~68% of not encountering any error. Actually not every error is going to change compression output, so it's somewhat less problematic...and faster computers help too.
But the 20 GB example would indeed take a month. If Corsair is correct, it has a small chance of working well. But I'm not going to check.
ADDED:
rar -m1 is "rather fast" for me.
ADDED2:
PAQ8 is cool, fine that you added it. But I just can't see ANY practical use for it.
andres992 wrote:Is it possible to make a plugin for the *.pea format (PeaZip)?
Another very interesting challenge would be a plugin to produce the smallest standard (deflate) zip possible, using KZip...
Implementing format - to big job and not in plugin concept. Plugin works with its own open-text-header PPMP format, with universal structure (also used in DarkCryptTC, etc.), just as file - to file packer (or from TC to tar.ppmp - this is great TC's feature)
For PEA - possible to make some implementation, but compatibility - must be another project.
I was thinking of possible other/new plugins (not this one) anyway. The PEA format is a new one but offers quite nice compression and is yet unsupported by TC/plugins. Getting the smallest possible zip (e.g. in the case of smaller files like JavaSE files for mobile applications where there are strict size limits) could have big practical importance sometimes.
But I am not skilled enough to write such plugins myself, so my post was quite a theoretical offtopic. Sorry for that.
m^2 wrote:
PAQ8 is cool, fine that you added it. But I just can't see ANY practical use for it.
Yes, It's very cool algo, I think, PAQ can be optimized by it's authors, may be by some preprocessors and pre-sorters, and, sometime, whe can get algorithm with compression much better than rar for all typical datasets with rar's speed.
2008.12.29 Updated bzip2 engine to latest version, code now works faster, implemented also original bzip engine ver.0.21 by Julian Seward (from OS BSD ported version, compression better than in wide-used bzip2)
andres992 wrote:
I was thinking of possible other/new plugins (not this one) anyway. The PEA format is a new one but offers quite nice compression and is yet unsupported by TC/plugins. Getting the smallest possible zip (e.g. in the case of smaller files like JavaSE files for mobile applications where there are strict size limits) could have big practical importance sometimes.
But I am not skilled enough to write such plugins myself, so my post was quite a theoretical offtopic. Sorry for that.
Thanks for info about PEA, this is very good opensource project. And your idea is nice.