Page 1 of 1

Enhance Sync on slow drives

Posted: 2019-02-15, 06:52 UTC
by MaxX
Please, add a switchable option for Sync to scan both drives at the same time.

I've got >500000 small files and folders on my old HDD and slow network drive. Now I have to wait for reading my HDD and then wait for network drive. It would be great to make them work simultaneously.
Also it could make Sync faster on SSD.

Re: Enhance Sync on slow drives

Posted: 2019-03-10, 12:21 UTC
by MaxX
Any feedback, please?

Re: Enhance Sync on slow drives

Posted: 2019-03-11, 08:36 UTC
by ghisler(Author)
Unfortunately it's not as simple as you think, because of how sync works: It first reads all directories on one side, and when it reads those on the other side, it directly inserts the results in the list from the left and makes the comparison.

If it would read the two sides at the same time, the two threads would somehow need to merge the results together. The options here aren't very good:
1. Lock the results, search for position, insert entry. This would block the other thread and cause significant slowdowns
2. Create two separate lists for left and right, and merge them later. This would probably be quite a bit slower than the current method
3. Try to read directories on the left and right in sync, e.g. c:\subdir1 and d:\subsir1 at the same time. This could become rather complex when entire subtrees are missing one one or the other side

It would also mean a LOT of combinations to handle, with other directories, archives, ftp, plugins...

Re: Enhance Sync on slow drives

Posted: 2019-03-12, 07:28 UTC
by MaxX
2ghisler(Author)
I mean only directory scans. Not comparison between them.
Just the first step before file lists are shown.

Re: Enhance Sync on slow drives

Posted: 2019-03-12, 09:13 UTC
by ghisler(Author)
The comparison is integrated in the scan of the second side.

Re: Enhance Sync on slow drives

Posted: 2019-03-12, 09:50 UTC
by MaxX
Ok, I understand. Thanks.

Re: Enhance Sync on slow drives

Posted: 2021-04-26, 12:35 UTC
by basil.cinnamon
The issue isn't reading and comparing the two folders, it is the copy step that is extremely slow. I'm on a fast fiber-optic internet connection, but the copy step from a remote server goes down to 20 Kbs or less. Maybe it's the file-by-file copy method, as opposed to other much faster copy methods. Using File Transfer in various well-known remote apps makes files fly across a lot faster. I don't know what methods they use. The downside of those remote apps is that none of them can sync, they can only copy the files one specifies manually.

Re: Enhance Sync on slow drives

Posted: 2021-04-27, 13:59 UTC
by ghisler(Author)
That's strange, Total Commander uses a standard Windows function (CopyFileEx) for copying. Do you get the same slowness when you just copy files with F5?