how to solve this issue (large folder list to sync , huge time)?)

English support forum

Moderators: white, Hacker, petermad, Stefan2

Post Reply
Michael REMY
Senior Member
Senior Member
Posts: 202
Joined: 2010-12-12, 10:05 UTC
Location: Amiens

how to solve this issue (large folder list to sync , huge time)?)

Post by *Michael REMY »

hi,

i'm not sure it is a problem/issue from TC but i asked, maybe it can help.

I have a drive with more than 50000 folders i synchronize it every month with an external backup drive.
The reading of the local content take a time : more than 20min to read the local drive, then the extern drive (so more than 30 min, more than 50k folders)

My local drive is unfragmented, ntfs format, No hardware issue i can detect, but it take time.
why ?
It looks like TC check every folder and files whereas i unchecked the "content option" :
Image: https://i.ibb.co/vdCdwG0/Image10.jpg
[img]https://i.ibb.co/vdCdwG0/Image10.jpg[/img]

what is the problem ? why TC check every folders or files ?
Normally, TC should only read the MFT (where date, file size,name,folders are stored).

According to O&O tool (o&o defrag), my MFT size is only 500MB for that 3TB drive, so why more than 30min of 'reading drive ' activity ?
Reading 500MB does not need 30min and does not need to check every folders & files too !? am i wrong ?
i'm an old long user of TC since the windows 3.11 release !, for the history, i came to tc when i migrate from amiga commodore to PC where they are diskmanager and directory opus there.
User avatar
Dalai
Power Member
Power Member
Posts: 9364
Joined: 2005-01-28, 22:17 UTC
Location: Meiningen (Südthüringen)

Re: how to solve this issue (large folder list to sync , huge time)?)

Post by *Dalai »

TC has to read (iterate through) the entire directory structure to determine the files' metadata like size and date. This takes time. As soon as this information is cached by the OS, it won't take as long. If you let the comparison run again, it will work much faster (as long as the system has enough RAM to cache the information). There are other factors like latency involved for external drives, too.

AFAIK user-level programs don't have direct access to the MFT, hence the Everything tool requires either a service or admin rights to work properly.

Regards
Dalai
#101164 Personal licence
Ryzen 5 2600, 16 GiB RAM, ASUS Prime X370-A, Win7 x64

Plugins: Services2, Startups, CertificateInfo, SignatureInfo, LineBreakInfo - Download-Mirror
Michael REMY
Senior Member
Senior Member
Posts: 202
Joined: 2010-12-12, 10:05 UTC
Location: Amiens

Re: how to solve this issue (large folder list to sync , huge time)?)

Post by *Michael REMY »

ok, i understand.

so, only special admin service can read the content of the MFT ... it is so unfair...it will only take a few second to perform such operation instead of reading all the structure.... I guess i have to use SSD now to gain time...

If we install TC with admin right, it is not enough to allow you to add this feature ? i would pay a second licence to have this feature in the next release :-)
i'm an old long user of TC since the windows 3.11 release !, for the history, i came to tc when i migrate from amiga commodore to PC where they are diskmanager and directory opus there.
bobputnam
Junior Member
Junior Member
Posts: 77
Joined: 2003-05-26, 12:31 UTC
Location: Connecticut, USA

Re: how to solve this issue (large folder list to sync , huge time)?)

Post by *bobputnam »

I do something similar, except I use a scheduled batch file using Robocopy that's run every other night. I never have to think about it. (I also use a removable drive in a separate weekly backup.) TCs wonderful, but it's not always the best tool.
Bob P
3-User License 71012
Post Reply