slow directory read of network drive with 1hourdif=1, TC7.50

StatusQuo · Post by *StatusQuo » 2009-09-18, 15:04 UTC

Hi all!

In TC 7.50 Shift+F2 comparing 2 directories on a mapped network drive got much slower compared to TC 7.04a on one of my PCs.
It's a 2,4 GHz Pentium 4 running Win2k SP4 in an NT domain, 8.5 MB/s network throughput to the used server.

Shift+F2 takes about 2 seconds (!) in TC 7.50, while in TC 7.04a it's done after only about 0.5 seconds (with about 800 files on each side).

I tracked it down to the parameter 1hourdif, with an empty INI, only containing

Code: Select all

[Configuration]
1hourdif=1

Removing the line 1hourdif brings back the fast speed of TC 7.04a, putting it back in makes Shift+F2 slow again.

Strangely I can reproduce this only on this PC. The 1hourdif parameter makes no noticeable change on the other tested systems, even my old 500 MHz Pentium 3 does directory reads much faster.

[Edit] Easy method to see when it's done: with identical contents on both panels, before Shift+F2 I mark all files on both sides (with different contents unmark instead) - the marking gets inverted when Shift+F2 is done. [/Edit]

Does somebody have an idea?

Post by *ghisler(Author) » 2009-09-18, 15:41 UTC

I will check that, thanks. Maybe it's because TC 7.5 now uses NTFS-style file times internally, which are much larger numbers than the old FAT32 times (only 32 bit).

StatusQuo · Post by *StatusQuo » 2009-09-18, 18:54 UTC

TIA.

Additionally discovered now: 1hourdif also slows down read of directory contents to about the same extent. This is reproducible also on an XP system here (TC 7.50a pre2).

After Ctrl+r in a large dir (and when entering it) I can watch the refresh travelling down the screen line by line for about 1 second.

Post by *ghisler(Author) » 2009-09-20, 13:51 UTC

I have checked this now: Indeed 1hourdif slows down Shift+F2 due to the more complex calculations needed for NTFS time stamps. But fortunately I found a way to make it almost as fast as with 1hourdif=0. Unfortunately it's not as fast as TC 7.04a with FAT(32) time stamps.

Additionally discovered now: 1hourdif also slows down read of directory contents to about the same extent.

Where? In the normal file lists? 1hourdif isn't used there at all...

After Ctrl+r in a large dir (and when entering it) I can watch the refresh travelling down the screen line by line for about 1 second.

Cannot confirm. And I use a slow Pentium III with 866MHz. The file list is displayed instantaneously with 1hourdif=1 or 1hourdif=0. Maybe a problem with some icons?

StatusQuo · Post by *StatusQuo » 2009-09-20, 22:09 UTC

Great to hear that it will soon be faster, thanks.

The slowdown of Ctrl+R occured in a normal file list, yes (network drive, view mode=Full, Ctrl+F2). The contents were data files (about 3 different types), AFAIR no executables.
I hope to find some time to retest and check what icons they have assigned on monday.

Post by *ghisler(Author) » 2009-09-21, 16:06 UTC

Yes, please let me know if you find out something. I didn't notice any speed differences on the network with 1hourdif=1 and Ctrl+R.

StatusQuo · Post by *StatusQuo » 2009-09-21, 19:38 UTC

Today's test results: indeed 1hourdif made no difference on Ctrl+R rereading the network drive.
I'll report when I find something new in further tests.

Post by *ghisler(Author) » 2009-09-22, 16:14 UTC

Maybe you can use process monitor from www.sysinternals.com to find out why it is slower?

Some possible reasons:
1. Overlay icons: Configuration - Options - Icons
2. Custom colors by file type: Configuration - Options - Colors

These are now refreshed on Ctrl+R.

StatusQuo · Post by *StatusQuo » 2009-09-22, 17:50 UTC

Got it - indeed, IconOverlays=1 caused the slowdown on Ctrl+R (plus flicker while the refresh is travelling down the list). After removing also this one there was almost no delay left.
Unfortunately I like overlay icons for links.

Colors by file type are in use, I'll put checking this and process monitor on my todo list, maybe there are additional things to find...

Post by *ghisler(Author) » 2009-09-23, 15:43 UTC

Well, it was requested by quite a few users (also here in this forum) that overlay icons should be refreshed too on Ctrl+R.

In the next version, I will repaint just the icons so it's a bit less irritating...

StatusQuo · Post by *StatusQuo » 2009-09-24, 15:40 UTC

Hm, if it means that Ctrl+R now got the requested "really reread everything", I think I can live with the flicker and lower speed of Ctrl+R.

StatusQuo · Post by *StatusQuo » 2009-09-25, 22:01 UTC

ghisler(Author) wrote:fortunately I found a way to make it almost as fast as with 1hourdif=0.

Great work, in TC 7.50a it is again fast enough to be ready before starting to annoy (me).
Very nice, thanks a lot.

Post by *ghisler(Author) » 2009-09-27, 09:45 UTC

Thanks for your feedback! The slowdown was indeed caused by the change from FAT32 times (resolution: 2 seconds) to NTFS file time internally (resolution: 1/10'000'000 second).

dantethegreat · Post by *dantethegreat » 2009-10-21, 12:26 UTC

is there any way to disable IconOverlays only on network drives? It really slows down the redraw (on a really slow connection though), but I cannot imagine living without CVS overlay icons. thx

MVV · Post by *MVV » 2009-10-21, 13:46 UTC

dantethegreat wrote:is there any way to disable IconOverlays only on network drives? It really slows down the redraw (on a really slow connection though), but I cannot imagine living without CVS overlay icons. thx

Configuration - Icons, uncheck "Show icons for special folders: &Virtual folders (e.g. Network)", it will disable all icons in Network.