Enhance Sync on slow drives

Here you can propose new features, make suggestions etc.

Moderators: white, Hacker, petermad, Stefan2

Post Reply
User avatar
MaxX
Power Member
Power Member
Posts: 1024
Joined: 2012-03-23, 18:15 UTC
Location: UA

Enhance Sync on slow drives

Post by *MaxX »

Please, add a switchable option for Sync to scan both drives at the same time.

I've got >500000 small files and folders on my old HDD and slow network drive. Now I have to wait for reading my HDD and then wait for network drive. It would be great to make them work simultaneously.
Also it could make Sync faster on SSD.
Ukrainian Total Commander Translator. Feedback and discuss.
User avatar
MaxX
Power Member
Power Member
Posts: 1024
Joined: 2012-03-23, 18:15 UTC
Location: UA

Re: Enhance Sync on slow drives

Post by *MaxX »

Any feedback, please?
Ukrainian Total Commander Translator. Feedback and discuss.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Enhance Sync on slow drives

Post by *ghisler(Author) »

Unfortunately it's not as simple as you think, because of how sync works: It first reads all directories on one side, and when it reads those on the other side, it directly inserts the results in the list from the left and makes the comparison.

If it would read the two sides at the same time, the two threads would somehow need to merge the results together. The options here aren't very good:
1. Lock the results, search for position, insert entry. This would block the other thread and cause significant slowdowns
2. Create two separate lists for left and right, and merge them later. This would probably be quite a bit slower than the current method
3. Try to read directories on the left and right in sync, e.g. c:\subdir1 and d:\subsir1 at the same time. This could become rather complex when entire subtrees are missing one one or the other side

It would also mean a LOT of combinations to handle, with other directories, archives, ftp, plugins...
Author of Total Commander
https://www.ghisler.com
User avatar
MaxX
Power Member
Power Member
Posts: 1024
Joined: 2012-03-23, 18:15 UTC
Location: UA

Re: Enhance Sync on slow drives

Post by *MaxX »

2ghisler(Author)
I mean only directory scans. Not comparison between them.
Just the first step before file lists are shown.
Ukrainian Total Commander Translator. Feedback and discuss.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Enhance Sync on slow drives

Post by *ghisler(Author) »

The comparison is integrated in the scan of the second side.
Author of Total Commander
https://www.ghisler.com
User avatar
MaxX
Power Member
Power Member
Posts: 1024
Joined: 2012-03-23, 18:15 UTC
Location: UA

Re: Enhance Sync on slow drives

Post by *MaxX »

Ok, I understand. Thanks.
Ukrainian Total Commander Translator. Feedback and discuss.
basil.cinnamon
Junior Member
Junior Member
Posts: 19
Joined: 2008-07-31, 22:19 UTC

Re: Enhance Sync on slow drives

Post by *basil.cinnamon »

The issue isn't reading and comparing the two folders, it is the copy step that is extremely slow. I'm on a fast fiber-optic internet connection, but the copy step from a remote server goes down to 20 Kbs or less. Maybe it's the file-by-file copy method, as opposed to other much faster copy methods. Using File Transfer in various well-known remote apps makes files fly across a lot faster. I don't know what methods they use. The downside of those remote apps is that none of them can sync, they can only copy the files one specifies manually.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: Enhance Sync on slow drives

Post by *ghisler(Author) »

That's strange, Total Commander uses a standard Windows function (CopyFileEx) for copying. Do you get the same slowness when you just copy files with F5?
Author of Total Commander
https://www.ghisler.com
Post Reply