[WCX] ZPAQ

Discuss and announce Total Commander plugins, addons and other useful tools here, both their usage and their development.

Moderators: sheep, Hacker, Stefan2, white

Post Reply
User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-08-21, 12:56 UTC

Hacker wrote:of course, for the <timestamp> part the existing custom date string can be used, just surround it by < > so that there are no possible collisions
There can be no collisions anyway, since ZPAQ and the format specification always makes sure that each version differs by at least one second, so a single timestamp string is enough to make names not colliding anywhere.
I don't like putting forbidden characters anywhere, that's why I wrote "WYSIWYG" (what you see is what you get). Users may be confused that the name isn't extracted like it was seen in the archive.
And BTW, if I'd use that very string from the "Show archive version names as detailed timestamp" option, users are free to put anything before and after the main timestamp anyway. So you may put similar looking characters from the Unicode plane in this string by yourself w/o worrying about them being deleted or replaced, like I suggested here:
http://www.ghisler.ch/board/viewtopic.php?p=303101#303101
Anyway, I will need to give this some thought.
Hacker wrote:Well, as I wrote, two options:
a) latest name ("dirECTory")
b) name with <timestamp>
I understood that, but like I said: I need to treat dir names internally as case-insensitive anway. So, no matter if I try to separate dirs or not, they'd end up in the same location.
The b) option is not doable anyway, because I can't list an individual file on more than one location and making it possibly extract twice (the same reason why the "Show the default archive view in extra dir" option "moves" the newest files to that extra dir).

Hacker wrote:I would just like to add, if I extract Filename.docx from version 52, please also use dirname from version 52 and dir timestamp from version 52.
I need to check if this would be possible, but again: violation of WYSIWYG.
And when extracting a file when the output dir exists but differs in case, Windows API functions (which ZPAQ uses) would of course keep the old dir name, just like when overwriting case-differing files/dirs in TC.
TC plugins: PCREsearch and RegXtract

User avatar
Hacker
Moderator
Moderator
Posts: 11395
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker » 2016-08-21, 13:15 UTC

milo1012,
There can be no collisions anyway, since ZPAQ and the format specification always makes sure that each version differs by at least one second, so a single timestamp string is enough to make names not colliding anywhere.
Well, for there to be no collisions you would have to show the timestamp for all files, not just the ones having multiple versions.
So you may put similar looking characters from the Unicode plane in this string by yourself w/o worrying about them being deleted or replaced
Yes well I only suggested surrounding the timestamp with < > to avoid collisions with filenames that by chance would already have a timestamp in their original name. Now I understand you want to show timestamps for all files, even those that only have one version. I don't know if it's better or worse than my idea but I currently don't have any objections.
violation of WYSIWYG
Well, we are adults, if we set something in options we probably understand the consequences. ;) WYSIWYG is a good guiding principle but since the whole concept of the option I am asking for goes against it I would not dwell on it.
Windows API functions (which ZPAQ uses) would of course keep the old dir name, just like when overwriting case-differing files/dirs in TC.
That is no problem.

Thanks
Roman
Last edited by Hacker on 2016-08-21, 14:16 UTC, edited 1 time in total.
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.

User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-08-21, 13:29 UTC

Hacker wrote:Now I understand you want to show timestamps for all files, even those that only have one version.
Yes, that's how I understood your idea.
If I wanted to do this only for files with multiple versions I'd need another map in memory, another round of checking before even reporting the first file to TC, another point of slowdown when listing huge archives (> 50k files)...
I will need to give both concepts some thoughts and see how to code plays along with it.
TC plugins: PCREsearch and RegXtract

User avatar
ZoSTeR
Power Member
Power Member
Posts: 920
Joined: 2004-07-29, 11:00 UTC

Post by *ZoSTeR » 2016-09-18, 20:48 UTC

It seems that the plugin v1.4a can't unpack Zpaq v7.15 files.

I can enter the archive but the unpacking fails.

User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-09-18, 21:10 UTC

2ZoSTeR
I just tested several random archives created with 7.15: unpacking works as expected with the plug-in.

What options did you use to create the archive?
Note that only journaling archives work, streaming archives don't work due to some code changes in 7.07 (not sure if I can add support for them again - but they are not using the benefits of the archiver format anyway).

Have you tried switching the archive view ("Show all archive versions")?
If this doesn't help: Could you create some (small) sample archive that I can test?
TC plugins: PCREsearch and RegXtract

User avatar
Hacker
Moderator
Moderator
Posts: 11395
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker » 2016-09-18, 21:46 UTC

ZoSTeR,
Works fine here. I suggest to only use one thread (set in the plugin options).

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.

User avatar
ZoSTeR
Power Member
Power Member
Posts: 920
Joined: 2004-07-29, 11:00 UTC

Post by *ZoSTeR » 2016-09-19, 05:15 UTC

I used "7paq64.exe a ... -m4"

Ok unpack all with "Show all archive versions" works.

User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-09-19, 11:22 UTC

ZoSTeR wrote:Ok unpack all with "Show all archive versions" works.
So this means that it didn't work w/o it? Strange.
Like I said, if you can provide some sample archive and tell me how exactly you tried to unpack (entering the archive or simply unpacking the whole (outer) archive), I could investigate further.
TC plugins: PCREsearch and RegXtract

User avatar
ZoSTeR
Power Member
Power Member
Posts: 920
Joined: 2004-07-29, 11:00 UTC

Post by *ZoSTeR » 2016-09-19, 15:51 UTC

I've checked once more:

With "Show all archive version" I can unpack from inside the archive as well as from the whole archive (Alt+F9).

Without it I can only unpack from inside the archive. With Alt+F9 I get "Disk read error" / "Fehler beim Lesen".

The first level of the archive only consist of one folder if that matters.

User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-09-19, 17:56 UTC

I tried several archives with different path structures, and all of them work with Alt+F9 and "Show all archive versions" disabled.

Does changing the target path (target dir) help? And is it reproducible on a different machine with the same archive?
TC plugins: PCREsearch and RegXtract

User avatar
ZoSTeR
Power Member
Power Member
Posts: 920
Joined: 2004-07-29, 11:00 UTC

Post by *ZoSTeR » 2016-09-19, 19:43 UTC

I was able reproduce the problem with a very simple setup.

Created with zpaq64.exe v7.15 compressing "\TestDir\TestFile.txt"

Code: Select all

zpaq64.exe a TestArc.zpaq .\TestDir\*.* -m4
Unpacking with plugin v1.4a x86, TC 8.52a x86, Win10 1607 x64

TestArc.b64:

Code: Select all

MIME-Version: 1.0
Content-Type: application/octet-stream; name="TestArc.zpaq"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="TestArc.zpaq"

N2tTdKAxg9OMsiiw03pQUQIBBwAAAAAAAAAAAWpEQzIwMTYwOTE5MTkzNDAxYzAwMDAwMDAwMDEA
OCBqREMBAAAAAAAJAKQBAAAAAAAAAAAAAP0Sdf/7NYS2/dDOLteujg/xw8PgYv83a1N0oDGD04yy
KLDTelBRAgEOAAkQABQAABJoh/9YcjgAAWpEQzIwMTYwOTE5MTkzNDAxZDAwMDAwMDAwMDEAMTUg
akRDAQAAAAABPQEuAe//Lw0EDBQcNwE3AjcDNwQ4y4JQRwiDWAcB3wAvM0cBNwJCrwPvAC8eAs8D
NwNC1wJQDwOvB4E3A0LXA1BDjwVYRwE3AT8KQtcCUBoaRwM3AQcB3wEvPUPvAi84Qq8B3wEvFULX
AVAPAkKvAYGBNwJC1wFQGho/GkLXAVAHAs8CSEKvA4E3AkLXAlAaGhpHAjcBP70HAd8CLzkHA+sn
NEI3BkM3Bw8DRwHJWAKqg1gPBEGLUB8CQ+8ALwgaRWARCTk/80E3BAcGDwPRUAcHiVgENwEHAd8D
LytD7wEvJkKvAd8BLxRC1wFQDwKvAYGBNwJC1wFQGho/CULXAVAaRwQ3AT/PBwHfBC8iQ+8HLx0P
BEJgOQlBNwRC1whQQ48IWAcCAjcC3wAvAwQ3ATgANDw8vAGAQEoAAAAAAAAAAP2+GItWIQRK4pZS
lbIh5Y8dumCedf83a1N0oDGD04yyKLDTelBRAgEHAAAAAAAAAAABakRDMjAxNjA5MTkxOTM0MDFo
MDAwMDAwMDAwMQAyOCBqREMBAAAAAAAdAKQBAAC2DRIbQ4o4DDQ9XsPCA3VkuC/+8wMAAAAAAAAA
/QzqBFE+RR3Jc9tp36dYV4O9WcSR/zdrU3SgMYPTjLIosNN6UFECAQ4ACRAAFAAAEmiH/1hyOAAB
akRDMjAxNjA5MTkxOTM0MDFpMDAwMDAwMDAwMQA0OCBqREMBAAAAAAFhAS4B7/8vDQQMFBw3ATcC
NwM3BDjLglBHCINYBwHfAC8zRwE3AkKvA+8ALx4CzwM3A0LXAlAPA68HgTcDQtcDUEOPBVhHATcB
PwpC1wJQGhpHAzcBBwHfAS89Q+8CLzhCrwHfAS8VQtcBUA8CQq8BgYE3AkLXAVAaGj8aQtcBUAcC
zwJIQq8DgTcCQtcCUBoaGkcCNwE/vQcB3wIvOQcD6yc0QjcGQzcHDwNHAclYAqqDWA8EQYtQHwJD
7wAvCBpFYBEJOT/zQTcEBwYPA9FQBweJWAQ3AQcB3wMvK0PvAS8mQq8B3wEvFELXAVAPAq8BgYE3
AkLXAVAaGj8JQtcBUBpHBDcBP88HAd8ELyJD7wcvHQ8EQmA5CUE3BELXCFBDjwhYBwICNwLfAC8D
BDcBOABUozNqo7CSAABweaEqm6MjSpNrAneM0tjKXOjw6AAKAAAA7kAAAAACAAAAAgAAAAAAAAAA
/dy9SRCSKtLcYJBq24Ap6tV7UZZJ/w==

User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-09-19, 20:27 UTC

Thanks!
I can reproduce it too.

The problem seems to emerge from zpaq storing the complete relative path "as is" internally, including the dot/period :

Code: Select all

- 2016-09-19 19:32:04            3 A     ./TestDir/TestFile.txt
(honestly: it's just stupid to store it)
And a simple "." seems not be a valid path component for TC, so it's a TC error message you see.

Anyway, it should be fixable from my side, though I don't know when I'll have the time for it.
TC plugins: PCREsearch and RegXtract

User avatar
Hacker
Moderator
Moderator
Posts: 11395
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker » 2016-10-28, 18:21 UTC

milo1012,
Any chance you had time to think about that timestamp-showing thing, please?

TIA
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.

User avatar
milo1012
Power Member
Power Member
Posts: 1109
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 » 2016-10-28, 23:09 UTC

I did some checks against my data structures and found that both concepts are feasible, as I can rename the actual file name quite easily before reporting it to TC, but I can't really say how much effort it takes to implement that until I start working on it, especially for how to make it play along the other listing options. I'm currently working on the proposed custom "-until" listing and some other things for a new version. After that is released I will see to the timestamp option.

BTW, it would help if you could briefly illustrate your proposal with a(nother) short example, i.e. how a sample archive structure would look like in TC without such option (showing all version dirs, I assume) and how it would look with it being in use (single and multiple file versions), just to make sure that we don't have misunderstandings.
TC plugins: PCREsearch and RegXtract

User avatar
Hacker
Moderator
Moderator
Posts: 11395
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker » 2016-10-29, 20:51 UTC

milo1012,
both concepts are feasible
Which two concepts do you mean?
if you could briefly illustrate your proposal with a(nother) short example, i.e. how a sample archive structure would look like in TC without such option (showing all version dirs, I assume) and how it would look with it being in use (single and multiple file versions)
Well, without such an option, I really don't have any specific preference, can be as it is now?

As for how the view should look like with such an option, I guess the easiest way would be like this:

c:\Backup.zpaq\ImportantDir\ImportantDocument <2016-08-10 19:33:02>.docx
c:\Backup.zpaq\ImportantDir\ImportantDocument <2016-07-16 15:21:31>.docx

- an alternative (or an option) would be to not append the timestamp to files that only have one version.
- another alternative (or an option) would be not to append the timestamp the the newest version.

But these are optional, I honestly don't know if they would make the overview better or worse.


I'd love to have three options for filenames upon extraction:

1. Always create filenames with timestamps appended
- no problems with extracting several versions of one file
- problems if the filenames are really long and we reach the 255 char limit

2. Never create filenames with timestamps appended
- when extracting several versions of one file the overwrite dialog would be presented
- no problems with filename length
- you get the original filenames and don't need to use the MRT to remove the timestamp

3. Only append timestamps to filenames in case multiple versions of the same file are extracted
- IMHO the best of the two above mentioned options

Concerning files that were deleted and are not present in newest versions of the backup I would not treat them any differently - consider the newest version as the current one.

Any questions, comments, ideas, just ask. :)

Thanks
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.

Post Reply