Total Commander

Total Commander (7.50) ignores all sparse properties of files. When copying, it inflates them to full size, with some files this introduces tens of gigabytes of bloat. When reporting disk space, it reports the incorrect inflated size. Explorer reports files size correctly but also inflates when copying. Win32 console copy command also inflates files.

Actually the whole Windows world seems to largely ignore the NTFS file system and only Linuxes support it correctly. As a Linux-dummy I have a hard time moving to a new HD since I cannot copy sparse files from the old one without inflating them.

Linux cp command handles sparse files, it seems from the documentation.

I have not found a sparse-aware Windows file utility by googling.

It would be very nice if Total Commander were able to handle sparse files correctly.

Edited Win32 console copy command part... I think I once got it to copy correctly within one volume but I currently cannot reproduce this.

Here's some actual information on sparse files (Microsoft only provides fluff) and example source code:

http://www.flexhex.com/docs/articles/sparse-files.phtml

Far Manager seems to have some NTFS features support, incl. sparse files... checking into it...

What happens if you archive those files: zip, tar.gz, tar.bz etc. And then unpack them when they've been transferred to the new Hard Drive?

Balderstrom wrote:What happens if you archive those files: zip, tar.gz, tar.bz etc. And then unpack them when they've been transferred to the new Hard Drive?

Archives definitely don't support sparse files. So packers will pack entire file (with zero byte areas for every missing block), and after unpacking file will be the same as in case of regular copy. Special handling is required to copy/pack sparse files (need to enum existing blocks via DeviceIoControl and read only theese blocks).

A solid Rar would pack those 0's as almost no space.

Balderstrom wrote:A solid Rar would pack those 0's as almost no space.

Most of archives would pack those 0's as almost no space (even NTFS compression would - I have 1-GB file that consists of same bytes and it takes only 64 MB of physical space).
But sparse structure would lost. Sparse file size can exceed volume size while unpacked files not.

BTW it doesn't matter is archive solid or not - solid archives are useful only when you pack multiple files with similar contents (it compresses them as solid data block).

I'm not very familiar witrh Far Manager, couldn't get it to copy sparse.

The cmdline NTFS alt. stream copy tool sparse.zip/CopyStream/cs.exe at the bottom of this page did the trick though (lightning fast compared to bloatcopy) - the original version of it, in Streamtools, does not, but this is the modified sparse-aware variant. It says "source code", but there is a binary file included.

http://www.flexhex.com/docs/articles/sparse-files.phtml

It's quite pity, TC doesn't still support sparse file during e.g. copy operation

Indeed.

TC has embraced the "MS approach". Which is exactly the thing that scared people from Explorer to Windows/Total commander in the first place : )

No, I switched from my own copy functions to CopyFileEx because there were just too many problems with badly written device drivers for custom hardware.

ghisler(Author) wrote:No, I switched from my own copy functions to CopyFileEx because there were just too many problems with badly written device drivers for custom hardware.

Sorry friend, but I haven't the foggiest idea what "No, I switched from my own copy functions to CopyFileEx" actually means, especially in the context of this thread

I'm a non-programmer.

In older versions, I used a loop of ReadFile/WriteFile, so supporting sparse files would have been possible. But now I'm using CopyFileEx to copy files, and it does not have an option to copy sparse files.

ghisler(Author) wrote:In older versions, I used a loop of ReadFile/WriteFile, so supporting sparse files would have been possible. But now I'm using CopyFileEx to copy files, and it does not have an option to copy sparse files.

How about adding option "Use old copy method (ReadFile/WriteFile)", or even more general "Choose copy method", to the Copy/Move dialog?

That way user might use Copy/Move dialog file type filter to copy non-sparsed files with new copy method, and sparsed using old one. This, in current design, would be a 2-step operation: F5 (for non-sparsed using new copy method) - F2 (task to BTM) - F5 (old method for non-sparsed) - F2 (task to BTM), so I guess to make it a 1-step operation a good addition to copy method selector would be its own file type filter.