Very slow file splitting to USB sticks
Moderators: white, Hacker, petermad, Stefan2
Very slow file splitting to USB sticks
Hello all!
I've noticed that file splitting to USB stick, which HAS to be formatted for FAT32, is very, very slow. When I use 7-zip without compression, I get write rate of around 200 MB/sec (which is OK for that stick, it's rather expensive one ), but file splitting to 3 GB limit goes around, ah, 20 MB/sec?
Can something be done about it?
I've noticed that file splitting to USB stick, which HAS to be formatted for FAT32, is very, very slow. When I use 7-zip without compression, I get write rate of around 200 MB/sec (which is OK for that stick, it's rather expensive one ), but file splitting to 3 GB limit goes around, ah, 20 MB/sec?
Can something be done about it?
Re: Very slow file splitting to USB sticks
Yes, split to hard drive and copy the result to USB stick.
Re: Very slow file splitting to USB sticks
That would be workaround. Why is it so slow while working with USB flash drive? Is there possibility for a fix?
Re: Very slow file splitting to USB sticks
USB drives usually have their cache disabled. And file splitting writes probably many small fragments, which is slow without a cache.
Regards
Dalai
Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64
Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
- ghisler(Author)
- Site Admin
- Posts: 48205
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Very slow file splitting to USB sticks
Yes, this happens because I cannot use CopyFileEx to write parts to the stick, I have to use a ReadFile/WriteFile loop. Although I write in multiples of the cluster size, writing is very slow due to the disabled write cache. But you can manually enable it, then it will be much faster. But then you have to unmount the stick via system tray (near the clock) before unplugging.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Very slow file splitting to USB sticks
Damn. I've used same function in Linux in Double Commander, and it works as fast as it should... but I guess there is a big difference between the copy methods in those two OSes.
But how come file splitting with 7-zip without compression works at full speed?
But how come file splitting with 7-zip without compression works at full speed?
Re: Very slow file splitting to USB sticks
7-zip uses local HDD for temp files.
Andrzej P. Wozniak
Polish subforum moderator
Polish subforum moderator
Re: Very slow file splitting to USB sticks
ghisler(Author),
Which buffer size does Split function use? Maybe its buffer could be enlarged?
Which buffer size does Split function use? Maybe its buffer could be enlarged?
- ghisler(Author)
- Site Admin
- Posts: 48205
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Very slow file splitting to USB sticks
Yes, that's a good point. Or maybe try to use file mapping to copy - I think I read somewhere that CopyFileEx uses file mapping.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com