Total Commander

Hi to all.

Please could you add xxHash (xxh3, the fastest hash algorithm: xxh3-avx2 is faster than system's RAM too) to TC's next version?

https://github.com/Cyan4973/xxHash/releases/tag/v0.8.0
https://github.com/Cyan4973/xxHash
https://github.com/Cyan4973/xxHash/wiki/Performance-comparison
https://github.com/Cyan4973/xxHash/wiki/Collision-ratio-comparison


Many thanks.

For what purpose?

2vdijken
Well, you should start with the reasonable assumption that Axis has a purpose/need/want for this particular hashing method, otherwise why would they ask for it?

Still, i wouldn't expect Christian to do this. There are quite a number of "exotic" hashing algorithms ("exotic" as in lacking widespread usage outside of narrow scenarios unrelated to PC mass storage such as communications/transmissions, low-perf embedded systems, etc.; and no, the mere existence of some package does not equate usage) that barely anybody uses. So what would be the upside of Christian spending the time and effort in implementing and maintaining an "exotic" hashing method in TC that quite likely barely any user will use in the foreseeable future?

I guess something "exotic" like this would be a case/suggestion for a packer plug-in (which then doesn't actually pack files, but rather generates a text file with the hash values of the files being "packed"), dependent on volunteer effort. If it ever happens in the future that xxHash will be on the verge of widespread popularity, it might then be the time to ponder whether it makes sense to include it into TC itself.

To see how efficient it could be, you could try to build
a wdx to show hash in a column
a wcx to create a hash file (compress)
a wlx lister to verify hashed files

vdijken wrote: ↑2021-03-03, 10:05 UTC For what purpose?

Speed: It's the fastest non-cryptographic hash algorithm, working at RAM speed limit.
I really need speed for hashing very large data (GB's): to me It's more useful than many TC's Sha algorithm variants.

It's less and less "exotic ": go to the bottom of the web page https://cyan4973.github.io/xxHash/ (starting from "xxHash is used by").

Cheers

+1 for this request.
As plugin will also be fine.

Axis,
You could try to suggest adding xxHash to LotsOfHashes.

Roman

Hacker wrote: ↑2021-03-03, 19:46 UTC Axis,
You could try to suggest adding xxHash to LotsOfHashes.

I think that's a dead project (2013) and imho tc's native integration, inside "Create Checksum File(s) (CRC32, MD5, SHA1)..." menu, is way better and really more useful than many SHA's.

Alternatively the cryptographic hash BLAKE3 would also be fine.


Please , Thanks

Cheers

My vote for xxHash. I tested xxHash today on my old laptop it's insanely fast.

Old topic but still good to revive it a little.
xxhash and specially xxhash3 64bit are really fast and could be internally used by TC to hash but also to replace md5 when looking for duplicates by content.

I have considered it, but there seem to be multiple incompatible versions of xxhash...

ghisler(Author) wrote: ↑2022-03-18, 11:32 UTC I have considered it, but there seem to be multiple incompatible versions of xxhash...

you have multiple xxhash versions 32/64/128 and xxhash3 and some implementation are not purely following standards. But if you just consider xxhas3 64 to replace md5 calculation in duplicate finding (by content) and hash calculation after copy you have a benefit anyway. I use the command line version to hash big files and it is much faster than any md5sum i found.

The creation is not the problem - I could just create one version and be done. But then 1000 people complain that they wanted to check an xxhash created by some other tool and it fails because it's one of those many other variations.

My suggestion is to use a modern efficient hash (xxhash3 or MetroHash) function to validate file operation or duplicate finder by content only...
But you are right as many implementation can use different seed to do the xxhash many tools with different result is not a "universal" hash for file exchange...