This forum uses cookies. Click X button to hide this message. What is stored? / Privacy
Total Commander Forum Index Total Commander
Forum - Public Discussion and Support
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

HEX Pattern searching using TC
Goto page Previous  1, 2, 3  Next
 
Post new topic   Reply to topic    Total Commander Forum Index -> Total Commander (English) Printable version
View previous topic :: View next topic  
Author Message
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Wed Mar 28, 2018 7:21 pm    Post subject: Reply with quote

milo1012 wrote:
So is your binary file in fact a doc, odt, pdf, rtf, sxw or xml file? In that case you'd need to disable the filter to make binary search possible.

none of the mentioned above
but if I want it disabled anyway, how do I do it ?

milo1012 wrote:
If this is still not helping, I can only guess that your PCREsearch settings don't fit. You may post the content of your PCREsearch.Sample.ini/PCREsearch.ini somewhere (either directly here as a code block if it's not too big, or on some extern site and link to it).

can't post A URL in any way, but I think its only for the first day, will try tomorrow.

An interesting thing I've just noticed.

the file(s) in question (I'm trying to search for(/in)) are very large files,
I've just created a few small file with that pattern and searched with same exact settings made, and it worked just fine (for the small example files)
but when trying to make it real, and search the actual (large) files
it doesn't work.
what can it be ?
does it has something to do because of being large file(s) ?
Back to top
View user's profile Send private message
milo1012
Power Member
Power Member


Joined: 02 Feb 2012
Posts: 1069

PostPosted: Wed Mar 28, 2018 7:33 pm    Post subject: Reply with quote

devd wrote:
the file(s) in question (I'm trying to search for(/in)) are very large files,

Ah, that might be the culprit. The default limits are 50 MiB for custom columns, which you can only bypass with <spacebar> or the TC commands cm_LoadAllOnDemandFields and cm_LoadSelOnDemandFields.
The 2nd limit is the global file size limit which is 2GiB (2048 MiB) by default.
Just use PCREsearchConfig.exe and set one or both entries to a higher value.
(note that I added these limits as a result of an unresponsive TC when searching large files for content, I also covered this in the readme file)

devd wrote:
but if I want it disabled anyway, how do I do it ?

Fastest way: use PCREsearchConfig.exe - in the lower right corner you will have the filter extension setting. Just remove the fitting file extension entries in the dropdown list where you don't want a text filter (button "-")
_________________
TC plugins: PCREsearch and RegXtract
Back to top
View user's profile Send private message Send e-mail
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Wed Mar 28, 2018 8:03 pm    Post subject: Reply with quote

milo1012 wrote:
The 2nd limit is the global file size limit which is 2GiB (2048 MiB) by default.

the files searched in are more than double in size
does it mean nothing can be done ?
Back to top
View user's profile Send private message
milo1012
Power Member
Power Member


Joined: 02 Feb 2012
Posts: 1069

PostPosted: Wed Mar 28, 2018 8:10 pm    Post subject: Reply with quote

devd wrote:
milo1012 wrote:
The 2nd limit is the global file size limit which is 2GiB (2048 MiB) by default.

the files searched in are more than double in size
does it mean nothing can be done ?

Like I said: just set it to a higher value.
Use PCREsearchConfig.exe and set the "Global" dropdown box (in the "File size limits" section) to sth. like "999999" (you can edit the field, not just select from the dropdown list) and it should work.
_________________
TC plugins: PCREsearch and RegXtract
Back to top
View user's profile Send private message Send e-mail
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Wed Mar 28, 2018 8:13 pm    Post subject: Reply with quote

Yes, just edited the INI file manually, and it seems to work now
thanks A lot

another question
can I use wildcards in it, like:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{[10-100]}\x1f\x88
Back to top
View user's profile Send private message
milo1012
Power Member
Power Member


Joined: 02 Feb 2012
Posts: 1069

PostPosted: Wed Mar 28, 2018 8:16 pm    Post subject: Reply with quote

devd wrote:
can I use wildcards in it, like:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{[10-100]}\x1f\x88

No, this won't work, but this would:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}\x1f\x88

Explanation:
PCRE syntax wrote:
QUANTIFIERS
? 0 or 1, greedy
?+ 0 or 1, possessive
?? 0 or 1, lazy
* 0 or more, greedy
*+ 0 or more, possessive
*? 0 or more, lazy
+ 1 or more, greedy
++ 1 or more, possessive
+? 1 or more, lazy
{n} exactly n
{n,m} at least n, no more than m, greedy
{n,m}+ at least n, no more than m, possessive
{n,m}? at least n, no more than m, lazy
{n,} n or more, greedy
{n,}+ n or more, possessive
{n,}? n or more, lazy

_________________
TC plugins: PCREsearch and RegXtract
Back to top
View user's profile Send private message Send e-mail
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Wed Mar 28, 2018 8:21 pm    Post subject: Reply with quote

thank you very much you have been very helpful
Back to top
View user's profile Send private message
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Wed Mar 28, 2018 8:29 pm    Post subject: Reply with quote

Oh, one more thing please

is there a way to use "or"
Example, search in A file this:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}\x1f\x88

"or" this:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}\x2b\xed
Back to top
View user's profile Send private message
milo1012
Power Member
Power Member


Joined: 02 Feb 2012
Posts: 1069

PostPosted: Wed Mar 28, 2018 8:41 pm    Post subject: Reply with quote

devd wrote:
is there a way to use "or"
Example, search in A file this:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}\x1f\x88

"or" this:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}\x2b\xed

Yes, there are many possibilities for this in RegEx. Sth like this should work:
Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}(?:\x1f\x88|\x2b\xed)

And BTW, no need to set the full range of hex bytes each time, you can use the dot/period placeholder instead:
Code:
\x00\x00\x46\x49\x4c\x45.{10,100}(?:\x1f\x88|\x2b\xed)

_________________
TC plugins: PCREsearch and RegXtract
Back to top
View user's profile Send private message Send e-mail
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Wed Mar 28, 2018 8:51 pm    Post subject: Reply with quote

great, thank you
Back to top
View user's profile Send private message
Stefan2
Power Member
Power Member


Joined: 13 Sep 2007
Posts: 1877
Location: Europa

PostPosted: Thu Mar 29, 2018 11:51 am    Post subject: New user are not allowed to post valid URLs Links Reply with quote

devd wrote:

can't post A URL in any way,


New user are not allowed to post valid URLs
https://ghisler.ch/board/viewtopic.php?p=339358#339358
to prevent spam.


But of course you can post plain text
https: ghisler.ch/board/viewtopic.php?p=339358#339358








 
_________________
Begegnen sich zwei Erden, sagt die Eine zur Anderen: "Mir geht's echt schlecht. Ich habe Homo sapiens!"
Back to top
View user's profile Send private message
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Thu Mar 29, 2018 9:22 pm    Post subject: Reply with quote

I'm getting to many results, and need to narrow it down a bit further

I need to add to end

*(\x01\x02 or \x31\x38)\x31\x36

is the code below correct ?


Code:
\x00\x00\x46\x49\x4c\x45[\x00-\xff]{10,100}\x2b\xed[\x00-\xff]{10,20}(\x01\x02|\x31\x38)\x31\x36


really appreciate it
Back to top
View user's profile Send private message
milo1012
Power Member
Power Member


Joined: 02 Feb 2012
Posts: 1069

PostPosted: Fri Mar 30, 2018 6:01 am    Post subject: Reply with quote

devd wrote:
is the code below correct ?

Yes, seems correct.
BTW, there is a built-in RegEx syntax/error checker in the config tool.
You can also use online tools like https://regex101.com.
_________________
TC plugins: PCREsearch and RegXtract
Back to top
View user's profile Send private message Send e-mail
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Fri Mar 30, 2018 8:39 am    Post subject: Reply with quote

milo1012 wrote:
devd wrote:
is the code below correct ?

Yes, seems correct.


Thanks again

milo1012 wrote:
BTW, there is a built-in RegEx syntax/error checker in the config tool.
You can also use online tools like https://regex101.com.

Great, that's good, very good info
Back to top
View user's profile Send private message
devd
Junior Member
Junior Member


Joined: 28 Mar 2018
Posts: 20

PostPosted: Sun Apr 01, 2018 10:32 am    Post subject: Reply with quote

finally got to that part
milo1012 wrote:
You can even get the file offset(s):
Change field type to "String output".
Replace String:
Code:
\xO0,


very interesting, the code I use is working just fine
but when added/changed to "String output", I'm getting no results

EDIT: do I need to change anything in "OP" or "value"
I've tried every possibility in "OP" but maybe need to add something in "VALUE"

Image: http://i64.tinypic.com/sc6gwn.jpg


milo1012 wrote:
You can also use online tools like https://regex101.com.


is there a way I can use Hex in that online tool ?
I tried, but it interpret it as text
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Total Commander Forum Index -> Total Commander (English) All times are GMT - 6 Hours
Goto page Previous  1, 2, 3  Next
Page 2 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Imprint/Impressum: This site is maintained by Ghisler Software GmbH
Privacy Policy | Datenschutzerklärung | Politique de Confidentialité

Using phpBB © phpBB Group