Page 1 of 1

MultiTheaded File Copy

Posted: 2013-01-24, 11:40 UTC
by me.yay
Lately i've seen that Robocopy allows to sepecify the number of parallel transfered files while copying/moving. ( /MT:N, where N is the number of threads)

This Feature speeds up filecopy, especially if lots of small files are involved.
Technicaly i can't explain the reason, but it does!

Instead of copying each file from the copy queue, it takes n-items from the queue and transfers them (almost) at once.

This Feature would be a real gain and makes tc even better for big copy Tasks than it allready is .

Regards,
Metin

Posted: 2013-01-24, 14:02 UTC
by ghisler(Author)
That doesn't make much sense to me, it would cause a lot of jumping of the harddisk heads between the different files. Or maybe you use an SSD (solid state disk)?

Re: MultiTheaded File Copy

Posted: 2023-04-19, 10:13 UTC
by krzychu
Hi

It's a very old topic.

But multi-threaded copying is VERY useful as you're copying something from a corporate NAS over a VPN (Mapped drive). Such a NAS has a lot of performance, but due to the delays caused by the Internet and VPN when copying small files, the transfer is very very low.

The second thing SMB 3.0 has support for multithreading and it also helps when the NAS is connected via 2 x 1Gbps, and our computer 2.5Gbps. Then it is better to have more than one copying thread. But that's probably a separate topic. There are just so many uses when it will make things better (for example QoS on router).

This could work similarly to F2 Queue when copying e.g. F3 Multithread.

Re: MultiTheaded File Copy

Posted: 2023-04-24, 14:23 UTC
by ghisler(Author)
But multi-threaded copying is VERY useful as you're copying something from a corporate NAS over a VPN (Mapped drive).
I'm already getting complaints from time to time that copying with TC saturates the connection and slows down the server. That would be even worse with parallel copying. :-(

For SSDs it would be nice, but there doesn't seem to be an easy way to detect whether a drive is an SSD or HDD.

Re: MultiTheaded File Copy

Posted: 2023-04-25, 15:58 UTC
by hi5
ghisler(Author) wrote: 2023-04-24, 14:23 UTCFor SSDs it would be nice, but there doesn't seem to be an easy way to detect whether a drive is an SSD or HDD.
Perhaps consider a setting, similar to SamePartitions, and introduce AllowMultiThreadedDisks or something like that - when set, allow for an option to be chosen in the copy/move dialog e.g. Use MultiThreading (by default it remains off, but you can choose it for certain actions)

Re: MultiTheaded File Copy

Posted: 2023-04-26, 06:25 UTC
by krzychu
I'm already getting complaints from time to time that copying with TC saturates the connection and slows down the server. That would be even worse with parallel copying. :-(
Interesting complains :lol: Anyway, it could just be an option, disabled by default. It is important that it can work on network resources (without drive leather) and mapped drives. Now at work I got a proposal to use some scripts (Robocopy) for this, and I'd rather just use Total Commander which we paid for.
For SSDs it would be nice, but there doesn't seem to be an easy way to detect whether a drive is an SSD or HDD.
I guess for an SSD it's not a big deal. But if you're mainly working on NAS, with a lag between you and the NAS, it's very annoying to work with small files. Even deleting is slow (this could also be done multi-threaded).

Re: MultiTheaded File Copy

Posted: 2023-05-02, 06:04 UTC
by arko
I support this idea. Make it optional and let users decide.

Re: MultiTheaded File Copy

Posted: 2023-08-07, 11:59 UTC
by krzychu
Any news? Is there a chance for this option?

Re: MultiTheaded File Copy

Posted: 2023-08-07, 12:25 UTC
by Horst.Epp
As a workaround, you can create TC buttons to use FastCopy.
It can run jobs in parallel and also has some settings to control how it's done.

Re: MultiTheaded File Copy

Posted: 2023-08-21, 12:29 UTC
by krzychu
Source: Windows VPN and enterprise cached NAS
Target Local Intel M.2 SSD
Max download speed ~300Mbps
6088 files - 19,1MB (small files)
Total Commander: 39:27 (almost 40 minutes)
FastCopy (configured parallel, buffer 32MB): 07:11
Robocopy /E /MT:32: 00:29
Robocopy /E /MT:64: 00:24

Yes, Robocopy with 64 threads is almost 100 times faster than standard Total Commander copying. That's why I think it's a very useful thing.

Re: MultiTheaded File Copy

Posted: 2024-05-13, 08:22 UTC
by Ankha
Now that SSD are widespread, I think it's a good option to have.

Also, it would be very useful on cloud drives such as Google Drive to use the whole bandwidth.
ghisler(Author) wrote: 2023-04-24, 14:23 UTC For SSDs it would be nice, but there doesn't seem to be an easy way to detect whether a drive is an SSD or HDD.
There are ways to figure it out, but it could simply be an option you can trigger on and off during copy (and it remembers the last choice based based on the source and destination drives for instance).