[WCX] ZPAQ

Discuss and announce Total Commander plugins, addons and other useful tools here, both their usage and their development.

Moderators: white, Hacker, petermad, Stefan2

Post Reply
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Re: [WCX] ZPAQ

Post by *milo1012 »

chewi wrote: 2020-06-04, 06:33 UTC Is it possible to update plugin to be based on zpaq 7.15 ?
(sorry for the late reply)
Well, there wouldn't be any real benefit from it compared to the 7.07 core. Even the changes worth porting are negligible and don't add features or were just meant for the CLI program. Sadly Matt overhauled some major portions of the code that I encapsulated in my own wrapper, so it's quite some work for little benefit. I might update libzpaq to 7.15 eventually, but we'll see.
I didn't have much time to work on the plug-in recently, but I hope this will change soon.
TC plugins: PCREsearch and RegXtract
User avatar
nsp
Power Member
Power Member
Posts: 1802
Joined: 2005-12-04, 08:39 UTC
Location: Lyon (FRANCE)
Contact:

Re: [WCX] ZPAQ

Post by *nsp »

Could you give a look at zpaqfranz a forked version by Franco Corbelli.
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Re: [WCX] ZPAQ

Post by *milo1012 »

2nsp
Yes, I'm aware of that fork. In fact I'm working on some features that might come in handy. But I can't say when I'm done with a new version.
TC plugins: PCREsearch and RegXtract
User avatar
nsp
Power Member
Power Member
Posts: 1802
Joined: 2005-12-04, 08:39 UTC
Location: Lyon (FRANCE)
Contact:

Re: [WCX] ZPAQ

Post by *nsp »

Thanks, In the meantime i use command line to do incremental save / versioning.
User avatar
hlloyge
Member
Member
Posts: 131
Joined: 2006-11-02, 23:14 UTC

Re: [WCX] ZPAQ

Post by *hlloyge »

Hello!
Plugin is not available for download any more, as wincmd.ru domain is non-existent :)
Срок регистрации для домена закончился.
I'd say it's not renewed and I suspect it won't be for some time.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48005
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: [WCX] ZPAQ

Post by *ghisler(Author) »

The plugin is on totalcmd.net, the sister site of wincmd.ru:
https://totalcmd.net/plugring/ZPAQ.html

The maintainer is outside of Russia now, and probably has troubles renewing the Russian domain.
Author of Total Commander
https://www.ghisler.com
User avatar
nsp
Power Member
Power Member
Posts: 1802
Joined: 2005-12-04, 08:39 UTC
Location: Lyon (FRANCE)
Contact:

Re: [WCX] ZPAQ

Post by *nsp »

Just in Case, give a look on this Post.
User avatar
fcorbelli
Junior Member
Junior Member
Posts: 26
Joined: 2023-12-18, 17:07 UTC
Location: Italy
Contact:

Re: [WCX] ZPAQ

Post by *fcorbelli »

Hello everybody, can I help somewhat?
zpaqfranz's author. Yes, my name is Franco Corbelli => the zpaqfranz :)
User avatar
Hacker
Moderator
Moderator
Posts: 13040
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Re: [WCX] ZPAQ

Post by *Hacker »

fcorbelli,
OMG, the legend, welcome! 8)

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
User avatar
fcorbelli
Junior Member
Junior Member
Posts: 26
Joined: 2023-12-18, 17:07 UTC
Location: Italy
Contact:

Re: [WCX] ZPAQ

Post by *fcorbelli »

Hacker wrote: 2023-12-18, 19:08 UTC fcorbelli,
OMG, the legend, welcome! 8)

Roman
Tnx

This plugin is really complex
Quite a lot of work is needed to understand this "thing",

Currently working on experimental ADS-based filelist
After first tests extraction is too slow (for huge filelist, 1GB+), LZ4 to the rescue :)

Maybe in the WE I'll take a look for total commander
zpaqfranz's author. Yes, my name is Franco Corbelli => the zpaqfranz :)
User avatar
Brumm
Junior Member
Junior Member
Posts: 98
Joined: 2014-11-18, 11:00 UTC

Re: [WCX] ZPAQ

Post by *Brumm »

I’ll test it. :)
Last edited by Brumm on 2023-12-22, 13:44 UTC, edited 1 time in total.
Dateimanagerfetischist.
User avatar
fcorbelli
Junior Member
Junior Member
Posts: 26
Joined: 2023-12-18, 17:07 UTC
Location: Italy
Contact:

Re: [WCX] ZPAQ

Post by *fcorbelli »

I think that the plugin should be embedded inside the upstream release with a ifdef TCPLUGIN

Just to be automatically updated at every release

A good starting point is the plugin author, or I can "fork"
Seems rather complex, but I like the challenges
zpaqfranz's author. Yes, my name is Franco Corbelli => the zpaqfranz :)
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Re: [WCX] ZPAQ

Post by *milo1012 »

2fcorbelli
It's interesting to see the offer, but I don't get what's the exact point or intention here:
Do you want to make the plugin compatible with zpaqfranz? Then I would appreciate that you do some fork, and call it wcx*franz or whatever, but don't just continue it by adding features. It's not dead from my side (see below).

I appreciate your fork of ZPAQ, but I would prefer to keep the last official zpaq separate of yours, since the main focus of the plug-in was the compatibility with the official program, which is quite high IMO. I didn't get any bug reports, plus I use it for myself quite often and haven't found any major problems so far.

For the code side just some quick reminder: Matt left us some "challenges" in his code. I also contacted him some years ago about these and some other "obscurities", but he didn't seem to have much interest of fixing or clarify all of them.
For me the main points code-wise were/are:
  • multiple C/C++ instructions per line - which is most annoying if you want to get into the code
  • the far from perfect "jidac" wrapper with some "quirks"
  • the non-standard UTF-8 encoding (see my code comments)
  • missing block and/or hash fragment size statistics in the main code, which is annoying if you want to continue an archive with different fragment size
So what I basically did was to take the 7.07 code, wrap the multi-instructions in separate lines, remove the non-Windows stuff, building a wrapper around the jidac stuff. The I added the GUI stuff for the config dialog. I didn't mess with the main features at all, to keep things as compatible as possible.

As for a new version:
Look, I already made some changes to the plug-in, but not enough to satisfy a new version:
  • the new "mixed version" listing, which was discussed in the past - but it's not completed yet
  • some GUI improvements
  • speedup of the SHA calculation (which was discussed quite intense in the encoding forums), but again: not completed yet
  • maybe merging the newer 7.15 code into it - but like I already said: there are no real improvements besides the zpaq core itself
After that is done, I wanted to look at some other features of your fork any maybe add some.
I also had the idea of having the (whole) file crc/sha stored as a "hidden" field: since TC's file sync is a a mess with ZPAQ, because of slowness due to non-synchronized extractions. ...but I didn't get far yet and I will probably not add it, since it probably breaks compatibility.

So to emphasize this again: I prefer to keep things comparable and compatible to the official version. Plus: the plug-in's main focus is TC compatibility. You have to keep that in mind if you mess around with the wcx functions or break compatibility by messing with the main functions. I already had my share of bugs and quirks when it comes to TC's behavior.

I can't make promises, but I will work on some release soon. Besides the sha speedup, I'm open for other suggestion of features I should add (which don't break compatibility).

In the meantime, feel free to make some fork, if you really think it's worth it.
TC plugins: PCREsearch and RegXtract
User avatar
fcorbelli
Junior Member
Junior Member
Posts: 26
Joined: 2023-12-18, 17:07 UTC
Location: Italy
Contact:

Re: [WCX] ZPAQ

Post by *fcorbelli »

milo1012 wrote: 2023-12-19, 14:25 UTC 2fcorbelli
It's interesting to see the offer, but I don't get what's the exact point or intention here:
Do you want to make the plugin compatible with zpaqfranz?
Only if more than 2 users ask :D

(...)
[*] speedup of the SHA calculation (which was discussed quite intense in the encoding forums), but again: not completed yet
Changing 1 lines in mahoney's deduplicator speed up about 10%
You can implement, just like me, HW acceleration (for newer CPUs)
I also had the idea of having the (whole) file crc/sha stored as a "hidden" field
Already in zpaqfranz.
You can store every kind of hash, and CRC too to detect SHA-1 collisions. Even entire files (!)
So to emphasize this again: I prefer to keep things comparable and compatible to the official version.
OK, not a big deal
The "official version" is 6 years old, slow and full of problems
In the meantime, feel free to make some fork, if you really think it's worth it.
In fact no, because I already written a "text-gui" (for Windows) and a "real GUI" (for Windows), now working on other implementations (like LZ4-ADS-based-filelist)
zpaqfranz's author. Yes, my name is Franco Corbelli => the zpaqfranz :)
User avatar
fcorbelli
Junior Member
Junior Member
Posts: 26
Joined: 2023-12-18, 17:07 UTC
Location: Italy
Contact:

Re: [WCX] ZPAQ

Post by *fcorbelli »

Here's, BTW, the deduplicator optimization

Code: Select all

		while (true) {
					c=in.get();
					if (c!=EOF) {
						if (c==o1[c1])
							h=(h+c+1)*314159265u, ++hits;
						else
							h=(h+c+1)*271828182u;
						o1[c1]=c;
						c1=c;
						sha1.put(c); ///// THIS IS INEFFICIENT
						fragbuf[sz++]=c;
					}
					if (c==EOF
							|| sz>=MAX_FRAGMENT
							|| (fragment<=22 && h<(1u<<(22-fragment)) && sz>=MIN_FRAGMENT))
						break;
				}
				assert(sz<=MAX_FRAGMENT);
				total_done+=sz;
		sha1.write(&fragbuf[0],sz); /// CHANGE TO THIS ONE
		
zpaqfranz's author. Yes, my name is Franco Corbelli => the zpaqfranz :)
Post Reply