[Solved] Very slow synchronize dirs over network

[Yves-c]

Hi all!

I sometimes use TC 7.02 to synchronize directories on two laptops, connected with a crossed ethernet cable and GB lan cards. Both computers are under WinXP pro SP2. When comparing directories (including subdirectories) the reading of the directories' content often collapses to a very, very low rate -- it almost hangs. This is visible "live" by the evolution of the total number of read (sub-)directories, at the bottom of the compare window.

Now a strange thing is : when an Explorer window is open that displays some folder of the remote computer, a simple refresh of this window (F5 for example) with speed up enormously the TC synchronization (reading) process. This "trick" lasts for a few seconds, then TC slows down again. On big directories, I have to redo the trick again and again to get reasonable synch times. The tricks always works.

Notes :
- this is the listing of the remote content that is slow (i.e. the first part of the synch process), not the copying of the files, which is between 10 and 20MB/s
- I am not comparing by content
- I usually use NOD32 and ZoneAlarm on both computers (and no Windows firewall), but turning them off doesn't bring any improvement. When doing this test, I check that ZoneAlarm service (TrueVector Internet Monitor) is also off.
- the additional Explorer window used to speed up the process does not have to show the same directory as the one that is being synchronized. It can even show some other partition of the remote computer.
- I think I had the same issue with TC 6.xx version, as far as I remember

Does somebody understand what is going on here? Thanks in advance!

Yves

[roentgen]

I couldn't stop laughing when I saw Nautilus in Linux (a slow app in most regards) browsing samba shares 3 times faster than TC, a native win app. The *same* windows shares, at 3 meters away!!

Somehow unrelated and sorry for that but we all hope that it doesn't take 10 years for Ghisler to move an inch into the right direction.

[ghisler(Author)]

Did you try that with TC running under Samba, or from Windows? Did you turn off all firewalls on the Windows machine when trying this?

[aguirRe]

I don't know if it's related, but I've similar issues since a long time between my old W95 box and my newer XP. I haven't noticed that it's related to TC at all though, any program that reads many directories over the network is slow. And it's not bandwidth limited, it's latency limited.

The first trick I use is to disable the XP QoS Packet Scheduler for the net in question, that makes it about two times faster (unknown why, nothing is using that feature).

The other is to just get some slow TCP/IP transfer running on the XP machine *within* that machine (i.e. via 127.0.0.1), I just script start a small client/server setup. This makes it about 2.5 times faster, again for no obvious reason.

With these tricks, a big file synchronization between the two machines is about five times faster ... always.

I'm sure there's a good explanation for it, but I've never found it despite many hours of searching and investigation.

[Yves-c]

Thanks to both of you for your quick answers! I have made a few more tests (see below).

Did you try that with TC running under Samba, or from Windows? Did you turn off all firewalls on the Windows machine when trying this?

I am under WinXP pro SP2 on both laptops, no samba. All firewalls are off, including Windows one, both on TC computer (computer 1) and on the remotly accessed computer (computer 2).

The first trick I use is to disable the XP QoS Packet Scheduler for the net in question, that makes it about two times faster (unknown why, nothing is using that feature).

I've tried that, I see no noticeable difference.

The other is to just get some slow TCP/IP transfer running on the XP machine *within* that machine (i.e. via 127.0.0.1), I just script start a small client/server setup. This makes it about 2.5 times faster, again for no obvious reason.

I did not try that, however my issue seems different :

I have made some TC synchs on a directory containing 10GB of data in 12600 files / 250 subdirectories. Here are the approximate durations of the "read" process :

- With the above trick (refreshing constantly on computer 1 the content of an Explorer window showing a random folder of the remote computer 2) : 5s
- Without the trick : from 80s to 150s (that is up to 30 times slower!)

It turns out that one get the same 5s performance when one makes repeated disk accesses on computer 2 using "local" calls (refresh of an Explorer window, for example). Alternatively, one can keep a high CPU activity on computer 2 (by moving a big window around, or setting CPU frequency to maximum in power management settings) -- however this latter procedure gives a 11s read time instead of the 5s obtained by repeated disk activity. This is still much better than the usual 80 to 150s.

Things are getting closer, but I still do not see an appropriate solution to the problem. Now it can very well be that this is not a TC issue (not sure though, because the remote browing is quite slow with TC and very fast with explorer). I will check if a dedicated synch software shows the same issue. If you have any idea, your help will be appreciated

[Yves-c]

OK, I have exactly the same annoying issue (and the same tricks work) with SyncBack and UltraCompare, so this is definitively not a TC-specific issue.

Anyway, if somebody knows a convenient workaround, I would be glad to hear it!

[HolgerK]

I don't know if this will work, but may be you can test it:

- Open the "Device Manager" <WIN+pause>
- navigate to "Network adapters"
- select your network device
- properties -> tab "Power Management"
- uncheck "[ ] Allow the computer to turn off this device to save power"

HTH,
Holger

[Yves-c]

Thanks Holger, this is a big improvement : the "remote directory reading" process is now between 12s and 20s. Not quite as fast as with the trick (which still gives 5s), but I think this is now more or less normal, so I will live with that. Thanks again!

[roentgen]

2Yves-c
Maybe this helps: http://www.lvllord.de/?lang=en&url=tools
Use it at your own risk.

[Yves-c]

Just for the record... in fact the solution given by Holger did not work : the improvement I once saw turned out to be not reproducible. The real cause to the problem was the program "RM CPU clock utility" used to manage the CPU state on my Thinkpad T40p (Pentium-M 1,6GHz). I first thought I had got rid of this possibility, but I had not noticed that exiting the program did not turn all CPU handling parameters to the standard ones.

So, I checked that after disabling completely RM CPU Clock utility (reboot with no autostart), and with the standard WinXP power management, the remote directory reading is normal (about 5s for the above mentionned example). I still do not understand the origin of the strange behavior with RM CPU Clock utility, which otherwise works great (note : the scheme used is a "power on demand" type, and the CPU sleep state settings have no influence on the issue, for those who know this program). I still use this useful program, I just switch the CPU scheme to "maximal performance" when I have to make a synchronization over network, which then works OK. The improvements provided by the "tricks" mentionned in the above messages seem to be simply due to an increased CPU activity, which somehow "waked it up" (don't ask me why the directory reading queries over network did not).

Anyway, thanks for your help!

Yves

[8472]

I had the same problem as described here with synchronizing directories over a network. The phase in which the remote machine was scanned was very, very slow. I always thought is was due to the remote machine (which is old: K6-3@375MHz running W2k).
But... with my brand new 2.3GHz dual core (running XP x64) the same problem!
And the trick with opening an explorer to the remote machine and keep F5 pressed also "worked". Confusion! And something to investigate. Unfortunately none of the solutions mentioned above worked for me. By the way the network I use is all Gbit with Jumbo Frames set to 7k.
Clues pointing out that this issue had to be somewhere in Windows:

• - The problem is the same in W2k and XP (x64)
  - Affects the dir sync function of TC as well as other sync programs
  - Generating traffic with explorer speeds up the process

The Solution that works here (on the XP machine that is...) lies in two modifications in the registry:

1) Switching off "last accessed time stamp"

• In HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem add the dword value NtfsDisableLastAccessUpdate.
  Possible values are 0 and 1. Set to 1. This saves the write operations following each read. Not on FAT of course...

2) Switching off delayed ACk on TCP

• This one does 90% of the trick. Downside is that this registry entry was introduced in XP, so it does not work on W2k. See MS KB328890 (since this is my first post I am not allowed to post the link).
  In HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\{Interface GUID} add the dword value TcpAckFrequency.
  Set this to 1 (default is 2).

I spent most of the day searching the web for a ready-to-go solution... but found none. Which surprised me, since this issue/feature exists for at least 8 years! Is it possible that very few people stumbled upon it?

Regards

8472

[ghisler(Author)]

Now if only I knew what the Explorer does (internally) when you press F5, I could do that in TC too...

[8472]

I suspect that Explorer does nothing special under the hood in this case... It only generates traffic!
Thinking about that I opened a second instance of TC, in which I started the same compare as in the first instance. Guess what? Both compares instantly running at full speed!!
This makes some sense with the MS-KB article in mind! (http://support.microsoft.com/kb/328890)
Let's assume that in the compare process there is only one TCP packet sent at a time, and the process also waits for the ACK before sending another.
If the compare is the only process using the TCP connection, it has to wait for the 200ms time-out for every sent packet. Appearently this is a case where the delayed ACK-facility slows things down...

I hope you can use this info to improve TC! (which really is like a Swiss knife to me!)

Regards,

8472

[psyafter]

Hi
I have similar problem.
When I copy files over network (WiFi n series, up to 350-400 Mbps),
my tc was freezed at finishing copy of each file.
So I setted this setting:
"Use compatibility mode for the following drives: ", and added my network drive to list (\Z).

Now I have another problem: My network speed slow down to 0.5-1.2 MByte/s . If I disable this setting back, so my speed back to 50 MByte/s.

and I still not found resolution for it.
I'll try "tips" from ths thread and I'll report if it helped me

thanks

[8472]

This issue may be also have a completely different cause!

Try Google on keywords "usb freeze large file copy ntfs"
I found that on XP 64bit (which is Server 2003 in disguise) the windows file cache is in write back mode; unfortunately I have no clue how to change that to write through. What happens is that ONLY when writing to NTFS, a file is fast written to the cache (in RAM) and then to the storage device in the background. When copying large files this can result in Windows becoming unresponsive until the cache is flushed (half a minute or so).
I have seen several reports on the web about this issue, but no solution. The common thing is the use of a server version of Windows and copying large files to a relatively slow NTFS destination. Mostly USB, but I suspect that a mapped network drive can cause the same.

Succes!

8472