Poor performance of ALT+F7 search

[digitaldog4]

Hello,

I would like to ask for improvement of ALT+F7 search. As the number of files/dirs on disk has grown in recent times, it's become very time-consuming to perform the search.

I think the issue is that Total Commander displays the full directory name of each and every scanned dir down in the status bar. UI updates are expensive but completely unnecessary (if it scans over 1500 dirs per second, what's the use of the information anyway?). I would propose to do the update e.g. 10 times per second, that would be more than enough. I'm talking about searches WITHOUT using Regex or Find text or plugins. Just a simple string search in file/dir names. Not even a wildcard/a glob.

I have a comparison with another two-panel tool, but as a long-time registered user and lover of Total Commander (19 years and counting!), I'd like it to be improved. All users would benefit from the fix. I performed a simple search over my profile directory (containing npm cache, nuget cache, VSCode extensions, Rust and more). The tree contains 757 thousand files and 100 thousand directories. Results are as follows:

Total Commander - 1m03s
another tool - 0m6s

So there's plenty of time to be saved. I repeated the searches a few times to have the data cached completely in memory. Tested on 11.00b6.

Thank you in advance.

[Horst.Epp]

Using TC and the Everything integration this takes only a few seconfs.

[AntonyD]

Yes, that’s right, this tool will help and significantly in this case.
But this does not change the essence of the question. If for some reason another similar program can search as quickly as the same "Everything" does, the question is - what are the super fast functions called there and what is so complicated/long written in Total that it is so slow?
How about if suddenly here we find out that Total not moving too fast AND on other operations?

[tuska]

... the question is - what are the super fast functions called there ...

The answer might be found here: Which software has Everything integrated?

---

Windows 11 Pro (x64) Version 22H2 (OS build Build 22621.1778)
TC 11.00b6 x64/x86 | 'Everything' 1.5.0.1350a (x64)
Search queries: TC <=> 'Everything' & Settings | 'Everything' 1.4.1.1024 (x64)

[AntonyD]

2digitaldog4

I have a comparison with another two-panel tool

What is it?

[digitaldog4]

2digitaldog4

I have a comparison with another two-panel tool

What is it?

Double Commander.

[AntonyD]

2digitaldog4

Double Commander.

And you made a simple filename search? And it was super fast?
Don't you know about some compatibility of this tool with 'Everything'?
Maybe even if it is not listed on provided URL from above - it does indeed use that tool silently?

[digitaldog4]

Please see the attached screencasts for evidence. I'm clicking through all pages so you can see there are no plugins involved. It's the UI update. UI updates are SLOW and EXPENSIVE. And in this case - pointless to be so frequent.

Double Commander: Image: https://i.imgur.com/2Tiyrlg.gif

Total Commander: Image: https://i.imgur.com/5yCscvz.gif

GIF won't show you how often the status bar is updated, but you can try it yourself. Double Commander updates UI only a few times per seconds, saving a lot of unnecessary work.

[tuska]

Search query in TC using tool 'Everything'
602 423 items (485 049 files, 117 374 folders)

Search for:

Code: Select all

ev:files:asdfasdfasdfasdf

Search results:

Code: Select all

[No files found]

Time of scan:
<1 second

---

---

Search directly in Everything 1.5.0.1350a (x64)

Search for:

Code: Select all

files:asdfasdfasdfasdf

Search result:

Code: Select all

0 items (0 files, 0 folders)

Time of scan:
<1 second

[Dalai]

Well, I totally understand your point and agree that GUI updates are slow and expensive in terms of (CPU) resources. But one thing to note is that TC is able to skip the currently searched directory by left-clicking on its name in the status bar. This wouldn't work this well with less frequent updates of the status bar. We'd have to find some balance for this.

Regards
Dalai

[Galizza]

But one thing to note is that TC is able to skip the currently searched directory by left-clicking on its name in the status bar. This wouldn't work this well with less frequent updates of the status bar.

Maybe could be a good idea to make this feature optional, so users that never use it may increase search speed.
Anyway, if Double Commander don't use Everything and is able to search so fast, something should be done to achieve a similar perfomance.

[AntonyD]

As I see in a code DC uses simple PAS-wrapped accordingly FindFirstFileExW() win.function in a loop in a thread-based PAS function WalkAdr.
I have a doubt that TC uses another/ancient win.function and not in a thread-based PAS function
But, unfortunately, the problem may lie in the field of compilers used, which, as a result, will create machine-executable code. Our Total does not shine with support for the most modern Pascal development environments. More precisely, some SINGLE environment/compiler. Because does exist separate support for 32-bit and 64-bit versions

[found this in a help file:]
Total Commander was written utilizing Borland Delphi 1.0 (16 bit) and 2.0 (32 bit) (© 1993-96 by Borland International), and Lazarus/Free Pascal (64 bit)

- and all of that for the sake of the fact that the program must still run on some Win 95, 98, ME, NT4, and 2000? ...
Well, according to today's realities, it's a too big evil, imho.
Just a literal example from today - a thread appeared on the forum: viewtopic.php?t=79290
and the answer uses the facts that there is separately 32-bit Total and 64-bit Total.
And in one, the required functionality is available, and in the other, it is not. That's the whole truth.
It would be very nice if at least and only on the site and only for the purpose of testing (without a guarantee of support - at your own risk) would be posted builds of the program, built in a really modern product. As I see it, given that some parts of the Total project also require C++ support - the only choice imho is Embarcadero RAD Studio in nowadays.

[ghisler(Author)]

I'm also using FindFirstFileExW directly. It's much faster when you repeat a search because it takes data from cache.
If you need extremely fast search, use Everything - there is no way to search an entire disk (with Windows on it) in just 6 seconds if no indexing is used.

[Dalai]

I just searched my Windows directory and it took 9 seconds - without a virus scanner and without Everything. Repeated searches (from cache) take 6 seconds. Admittedly it's Win7 instead of Win10. Still, these search times are nowhere near a minute...

It's possible that a virus scanner slows things down massively. I suggest to temporarily disable the scanner, including Defender. Maybe Double Commander is excluded from the scanner or doesn't trigger it the same way and thus isn't slowed down as much (or at all).

Regards
Dalai

[digitaldog4]

I just searched my Windows directory and it took 9 seconds - without a virus scanner and without Everything. Repeated searches (from cache) take 6 seconds. Admittedly it's Win7 instead of Win10. Still, these search times are nowhere near a minute...

It's possible that a virus scanner slows things down massively. I suggest to temporarily disable the scanner, including Defender. Maybe Double Commander is excluded from the scanner or doesn't trigger it the same way and thus isn't slowed down as much (or at all).

Regards
Dalai

Or maybe TC's UI status bar updates are causing slowdown. It's as simple as that.