Page 1 of 1
Speed up searches with FindFirstFileEx
Posted: 2013-04-23, 16:22 UTC
by MarcinW
Since Windows 7 / Windows Server 2008 R2 (version 6.1) there are new functionalities of FindFirstFileExA/FindFirstFileExW function:
1) fInfoLevelId parameter can have now FindExInfoBasic value - searches don't query short file names, improving filesystem enumeration speed.
2) dwAdditionalFlags parameter can have now FIND_FIRST_EX_LARGE_FETCH flag - so all searches use internally buffer of size 0x10000 instead of 0x1000 (as described in
Crash on VirtualPC thread) - this decreases significantly number of calls to NtQueryDirectoryFile function, and - in consequence - decreases user-kernel-user transitions, when NtQueryDirectoryFile calls a filesystem driver.
You can even use
SwapImportedFunction from PE.pas unit, build a small wrapper, hook all calls to FindFirstFileA/FindFirstFileW and redirect them to FindFirstFileExA/FindFirstFileExW.
Regards
Posted: 2013-04-25, 13:08 UTC
by ghisler(Author)
Thanks for the info!
1) TC needs the short names in various places like file_id.diz files or in case of very long paths, so I cannot use this.
2) I will try that. It should be no problem because my calls to FindFirstFileW are encapsulated in another function.
Btw, are there any differences between FindFirstFileW and FindFirstFileExW I need to be aware of? I don't want to break anything by switching to this function, e.g. access to network drives with non-Microsoft software like NAS devices...
Posted: 2013-04-26, 10:31 UTC
by Hacker
Christian,
1) TC needs the short names in various places like file_id.diz files or in case of very long paths, so I cannot use this.
Could you not only use short file names when there is actually a need for them? Ie. when a long path becomes problematic or when file_id.diz files are turned on?
Roman
Posted: 2013-04-26, 12:54 UTC
by Lefteous
I really hope 8.5 will do something about the search in terms of indexed based search - can't wait...
Posted: 2013-04-26, 14:14 UTC
by Horst.Epp
Lefteous wrote:I really hope 8.5 will do something about the search in terms of indexed based search - can't wait...
I find using Everything or VTfind is enough for that purpose.
Both integrate fine with TC.
You need NTFS volumes of course, for others Locate32 does the job.
Posted: 2013-04-26, 15:07 UTC
by jjk
Unfortunately Everything or UltraSearch and probably VTfind need admin rights to work. But in work environments we have them rarely.
Does anybody know the contribution of FindFirstFileW and FindFirstFileExW in accelerating search ? Can we hope great improvements with their ?
Posted: 2013-04-26, 15:46 UTC
by Horst.Epp
jjk wrote:Unfortunately Everything or UltraSearch and probably VTfind need admin rights to work. But in work environments we have them rarely.
Does anybody know the contribution of FindFirstFileW and FindFirstFileExW in accelerating search ? Can we hope great improvements with their ?
Everything can run as a service.
Then you don't need Admin rights for using it.
You must of course have admin rights to install it as a service
Posted: 2013-04-27, 07:42 UTC
by Flint
Horst.Epp wrote:You need NTFS volumes of course
With latest betas you can add FAT volumes for "non-journal-based" monitoring in Everything (though it will be slower and won't always contain 100% up-to-date data).
Posted: 2013-04-28, 16:59 UTC
by Hacker
Lefteous,
You could perhaps also support the following idea:
Indexed search using WDX plugins
Roman
Posted: 2013-04-28, 19:13 UTC
by Lefteous
2
Horst.Epp
Both integrate fine with TC.
I'm not happy with these solutions. How on earth are they 'integrated fine'?
2
Hacker
You could perhaps also support the following idea
Yes I support this idea.
Posted: 2013-04-29, 07:24 UTC
by Horst.Epp
Lefteous wrote:2
Horst.Epp
Both integrate fine with TC.
I'm not happy with these solutions. How on earth are they 'integrated fine'?
...
For VTfind:
You can directly jump to any file or dir found in a new TC tab.
You can feed to list box the search results.
You can view any file from the search result in TC viewer.
What else do you want ?
Posted: 2013-05-17, 13:57 UTC
by MarcinW
ghisler(Author) wrote:Btw, are there any differences between FindFirstFileW and FindFirstFileExW I need to be aware of? I don't want to break anything by switching to this function, e.g. access to network drives with non-Microsoft software like NAS devices...
Since Windows 2000 (or even Windows NT 4) FindFirstFile is just a wrapper around FindFirstFileEx (this is a common practice for "Ex" functions). So, in fact, Total Commander uses FindFirstFileEx since many years. So I'm sure that there won't be any problems when using it directly.
Code: Select all
.text:7D86ABEB ; Exported entry 162. FindFirstFileW
.text:7D86ABEB
.text:7D86ABEB ; =============== S U B R O U T I N E =======================================
.text:7D86ABEB
.text:7D86ABEB ; Attributes: bp-based frame
.text:7D86ABEB
.text:7D86ABEB ; HANDLE __stdcall FindFirstFileW(LPCWSTR lpFileName, LPWIN32_FIND_DATAW lpFindFileData)
.text:7D86ABEB public FindFirstFileW
.text:7D86ABEB FindFirstFileW proc near
.text:7D86ABEB
.text:7D86ABEB lpFileName = dword ptr 8
.text:7D86ABEB lpFindFileData = dword ptr 0Ch
.text:7D86ABEB
.text:7D86ABEB mov edi, edi
.text:7D86ABED push ebp
.text:7D86ABEE mov ebp, esp
.text:7D86ABF0 xor eax, eax
.text:7D86ABF2 push eax ; dwAdditionalFlags
.text:7D86ABF3 push eax ; lpSearchFilter
.text:7D86ABF4 push eax ; fSearchOp
.text:7D86ABF5 push [ebp+lpFindFileData] ; lpFindFileData
.text:7D86ABF8 push eax ; fInfoLevelId
.text:7D86ABF9 push [ebp+lpFileName] ; lpFileName
.text:7D86ABFC call FindFirstFileExW
.text:7D86AC01 pop ebp
.text:7D86AC02 retn 8
.text:7D86AC02 FindFirstFileW endp
.text:7D86AC02
.text:7D86AC02 ; ---------------------------------------------------------------------------
Regards
Posted: 2013-05-20, 12:50 UTC
by ghisler(Author)
That's great, thanks for investigating!