Enhance Sync on slow drives
Moderators: white, Hacker, petermad, Stefan2
Enhance Sync on slow drives
Please, add a switchable option for Sync to scan both drives at the same time.
I've got >500000 small files and folders on my old HDD and slow network drive. Now I have to wait for reading my HDD and then wait for network drive. It would be great to make them work simultaneously.
Also it could make Sync faster on SSD.
I've got >500000 small files and folders on my old HDD and slow network drive. Now I have to wait for reading my HDD and then wait for network drive. It would be great to make them work simultaneously.
Also it could make Sync faster on SSD.
Ukrainian Total Commander Translator. Feedback and discuss.
Re: Enhance Sync on slow drives
Any feedback, please?
Ukrainian Total Commander Translator. Feedback and discuss.
- ghisler(Author)
- Site Admin
- Posts: 48077
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Enhance Sync on slow drives
Unfortunately it's not as simple as you think, because of how sync works: It first reads all directories on one side, and when it reads those on the other side, it directly inserts the results in the list from the left and makes the comparison.
If it would read the two sides at the same time, the two threads would somehow need to merge the results together. The options here aren't very good:
1. Lock the results, search for position, insert entry. This would block the other thread and cause significant slowdowns
2. Create two separate lists for left and right, and merge them later. This would probably be quite a bit slower than the current method
3. Try to read directories on the left and right in sync, e.g. c:\subdir1 and d:\subsir1 at the same time. This could become rather complex when entire subtrees are missing one one or the other side
It would also mean a LOT of combinations to handle, with other directories, archives, ftp, plugins...
If it would read the two sides at the same time, the two threads would somehow need to merge the results together. The options here aren't very good:
1. Lock the results, search for position, insert entry. This would block the other thread and cause significant slowdowns
2. Create two separate lists for left and right, and merge them later. This would probably be quite a bit slower than the current method
3. Try to read directories on the left and right in sync, e.g. c:\subdir1 and d:\subsir1 at the same time. This could become rather complex when entire subtrees are missing one one or the other side
It would also mean a LOT of combinations to handle, with other directories, archives, ftp, plugins...
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Enhance Sync on slow drives
2ghisler(Author)
I mean only directory scans. Not comparison between them.
Just the first step before file lists are shown.
I mean only directory scans. Not comparison between them.
Just the first step before file lists are shown.
Ukrainian Total Commander Translator. Feedback and discuss.
- ghisler(Author)
- Site Admin
- Posts: 48077
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Enhance Sync on slow drives
The comparison is integrated in the scan of the second side.
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com
Re: Enhance Sync on slow drives
Ok, I understand. Thanks.
Ukrainian Total Commander Translator. Feedback and discuss.
-
- Junior Member
- Posts: 19
- Joined: 2008-07-31, 22:19 UTC
Re: Enhance Sync on slow drives
The issue isn't reading and comparing the two folders, it is the copy step that is extremely slow. I'm on a fast fiber-optic internet connection, but the copy step from a remote server goes down to 20 Kbs or less. Maybe it's the file-by-file copy method, as opposed to other much faster copy methods. Using File Transfer in various well-known remote apps makes files fly across a lot faster. I don't know what methods they use. The downside of those remote apps is that none of them can sync, they can only copy the files one specifies manually.
- ghisler(Author)
- Site Admin
- Posts: 48077
- Joined: 2003-02-04, 09:46 UTC
- Location: Switzerland
- Contact:
Re: Enhance Sync on slow drives
That's strange, Total Commander uses a standard Windows function (CopyFileEx) for copying. Do you get the same slowness when you just copy files with F5?
Author of Total Commander
https://www.ghisler.com
https://www.ghisler.com