[FEATURE REQUEST:] select duplicates in search result list!

[chrizoo]

EDIT:
****** THERE IS A POLL FOR THIS ISSUE --> HERE ******

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hi. I'd be very thankful, if someone knows a solution to the following - kind of classic - problem:

There are 6000 files out of which 900 are unique, the rest are files with different file name but same content. I want to preserve the 900 unique files and delete the rest.

ALT+F7 (with the compare-by-content-option), gives us a long list, separated with dotted ligns indicating groups of files with same content. Until now, I did it the stupid way: I selected all of the 6000 files and then I went through them and manually(!) deselected 900 files, which is quite some work of course.
Is there anything that could be done to tell TC to select (or deselect for that matter) one file in all of these groups separated by dotted lines, for example the first file in each group (but any given one would be fine, because the files have all the same content anyway) ?

How can this be done?
Thanks so much for your help

screenshot:
Image: http://img442.imageshack.us/img442/8236/24509328kt8.png

FileName1.txt
FileName2.txt
FileName3.txt
-----
FileName4.txt
FileName5.txt
-----
FileName6.txt
FileName7.txt
FileName8.txt
FileName9.txt

[chrizoo]

PS: Why can't I embed the screenshot?

[petermad]

2chrizoo
Embedding screenshots is disabled in this forum.

You can upload it somewhere else and give us a link to it.

[StatusQuo]

2petermad

You can upload it somewhere else and give us a link to it.

Fortunately there's a link in chrizoo's initial posting:
http://img442.imageshack.us/img442/8236/24509328kt8.png

2chrizoo
For a single dir I'd suggest Shift+F2, Num-* for selecting -
this also works in branch view, but I wouldn't trust in this, as there are multiple files with the same name...

[Peter]

Maybe this plugin: ???

http://www.totalcmd.net/plugring/DupSelector1001.html

Peter

[chrizoo]

Thanks for your help guys.

@StatusQuo, I know Shift+F2 but can't really relate it to my problem, as it's about multiple dirs and Shift+F2 compares the left pane with the right one, no?

@Peter, thanks for the plugin link. I'll have to upgrade to TC7 and then I'll try and see... From what I've read in the description though, it helps finding duplicates. That's not the problem, as I can find dupes with ALT+F7 (comparing content). The problem I have is how to select only one out of all the identical files and this for each group separated by dotted lines (see screenshot above). Doesn't any of TC's core functions (without plugins) provide for a solution ?

EDIT: I tried the plugin, but it complains about a non-existing dll file ... I try to look for the dll somehwere on the net in the next days. Have you been able to install it and if yes, does it help with the issue here ?

[Clo]

2chrizoo

Hello !

• I proposed built-in features in the same painting than you need ages ago, as a mock-up, totally ignored…

 KR
Claude
Clo

[chrizoo]

Salut Clo,
I'm quite astonished that you didn't get any feedback on your proposal!! Not just because you are on of the forum mods (as I have just realized) of C.Ghisler's own website ... but in particular because it is very innovative and provides about all possibly imaginable solutions to the problem.

And sorry, I did search the board before without success ... but when you search for "duplicates", the board is virtually packed with discussions and feature requests. I cannot quite understand why so many people raise this shortcoming and provide answers/solutions which don't get implemented:

http://www.ghisler.ch/board/viewtopic.php?t=4637
http://pagesperso-orange.fr/charries/relais/keepcopy.png
http://www.ghisler.ch/board/viewtopic.php?t=16385
http://www.ghisler.ch/board/viewtopic.php?t=15803
http://www.ghisler.ch/board/viewtopic.php?t=5453
http://www.ghisler.ch/board/viewtopic.php?t=10408
http://www.ghisler.ch/board/viewtopic.php?t=8724

... just to mention a few of them ....

In the first topic listed above ghisler(author) wrote:

Unfortunately no one has proposed a good method WHICH of the duplicates should be selected and which not, therefore I haven't implemented anything yet. Good ideas to solve this are always welcome. Selecting newer files is not an option, since duplicates often also have the same date/time.

In the last mentioned topic, ghisler(author) wrote:

If anyone has another idea of selecting files in this situation, then it's NOW the time to suggest it, because I'm still in the phase of adding new functions.

Sheepdog eloquently replied:

Keep the files that are

1. newest
2. oldest
3. with longest
   • [3.1] filename
     [3.2] full path (path inc. filemname)
     [3.3] Dir path (path without filename)
4. with shortest
   • [4.1] filename
     [4.2] full path (path inc. filemname)
     [4.3] Dir path (path without filename)
5. Export a List of all equal files

It's taken from Clonspy but I think it's a good bunch of choices that should fit most needs.
sheepdog

The simple answer is that TC - or the computer for that matter - cannot know which duplicates you want to keep and which not. It's entirely of the user's discretion. The only thing TC can do is to offer the user as many choices as possible to automate this task (regarding the date of files, path length, file name length, etc.). Personally I ran into this situation so often - and apparently a great deal of users out here too - that I would have been incredibly thankful for ANY solution - even let's say the first file out of each duplicate group or even random picks would have been fine for me. As a matter of fact, they have exactly the same content anyway ....

And I can't really understand people putting the argument forward that there are cases where the date is the same or that it does matter in which directories the duplicates are ...
... yes, but what about those many cases, where it simply does not matter. It's better to have a solution for these, than not having a solution for either any case, right ?

And while I honor C.Ghisler's striving for data security, one has to ask him/herself where to draw the line between data security and user friendliness. If data security is an absolute must, then - put to an extreme - one could argue that TC should not even have a deletion function ...
And hex editors would not exist in this world...
It's like playing football. You can get hurt. Everyone knows, but should we - as a consequence - forbid football ? Or rather believe in the maturity of people being able to decide for themselves ?

And security could also be increased with a warning notice or something similar (like a checkbox in the options or an wincmd.ini setting for advanced users, etc.).

Besides, in what way is it less safe to search for duplicates and ask TC to select them than to synchronize directories and ask TC to delete all files on the left pane ?? That's exactly the same thing. Depending on your settings, the files are identical by name or by name and content. That's also what you get with ALT+F7 in the results list. If you advocate that no automation should be implemented at that point - for data security - then you would also have to advocate for the deletion function in the synchronize directories to be removed.

And besides, why is it more safe to go through that long list and manually select the first file in each group? I have done this a several times and when the list is really long, let's say hundreds or thousands of files, I don't verify the respective paths any more anyway. I just click through it as fast as possible. So we get to the same point as if the selection could be done with one mouseclick ... just that it saves hours ... (if they don't install clonespy or whatever).

I just don't see why the majority of responsible users should be deprived of easily available features just because someone might possibly be able to delete something ... I mean which beginner would install TC, tell TC to look for files with same name and then ignore all warnings and make an autoselection for deletion purposes ? And quite honestly, if one chooses to ignore warnings, he/she should know what he/she is doing...

I want to quote Samuel from the second abovementioned topic in this respect if I may:

This preselection could perhaps bring a noob to waste his OS [...]. But what could prevent a noob to do so, if he just selects the files manually. Mh he would destroy his system slower.

I support this idea! It should be implemented, because it takes very much time to select all duplicates manually in a very very long list.
[...]
Mr. Ghisler please implement something like this.

Besides, if you look for files with the same content and you delete duplicates, then you still have one identical file left even if you accidentally deleted duplicates you would have wanted to keep .... so with the file transaction log enabled, the user can - even in the worst case scenario - see which files have been deleted and restore everything exactly as it was before (since files with exactly the same content still exist).

And after all, we should all have backups of our important files ... hard disks can fail anytime ....

In this discussion I am very much reminded of something I heard in the news yesterday: In London lampposts are to receive a protective cover to prevent people from getting hurt when they bump into them writing SMS on their mobile phones ...
It's not like that, that people will get more responsible for their own actions ...

The bottom line is that until such time as PCs can read the user's mind, why not stick to the most logic thing to do and adopt the second best solution (for which proposals have been made countless times) ?

[Clo]

2chrizoo

Again…

• All is good points, that sums it up exactly!

• From time to time, I unearth some pic¦s from my laaaarge collection of those fallen in the Deep Well of ignored Proposals…

- This one needs an update, though…

 KR
Claude
Clo

[DrShark]

Maybe this threads from wincmd.ru will help
(translated by Google):
DupSelector
Automatically delete duplicate files

[X-Byte]

Very good suggestions from Claude/chrizoo/Sheepdog.
Maybe this thread has a chance to draw Christian Ghisler's attention towards this nice feature suggestion this time...

[chrizoo]

Thanks for the links DrShark. They point to various other links and I'm still reading ...
I found some code there. vbs scripts and at another point what seemed to be code used for TC plugin coding. Has any of you gotten some script to work and actually select files in the search result list?

PS: Also to say that I'm happy my rather critical and (deliberately) controversial posting has (so far not given rise to an argument-lacking-flamewar.

And yes, me too, I hope Mr. Ghisler is paying attention to the various forum members here having raised that issue. And of course it's his program and we should (and I definitely am!) incredibly glad that it exists ... what would the world be without ... ... but still ... TC would be a great deal better with this... Just to point out that we are not complaining here, but trying to provide meaningful solutions.

[Vochomurka]

http://ppro.totalcmd.net/Scripts/MarkStep.htm

[chrizoo]

Thanks Vochomurka! What must I do to invoke the code? In case it's .vbs : I have disabled the Windows Scripting Host (WSH)for security reasons and would like to leave it at that. Is there any way I can still use this script inside TC ?

EDIT:
I also found THIS LINK .
I think this is a basic solution for the problem. But of course the proposals made above are much better since they offer much more choices as to which files to select.
Still the same question: Anybody knows how to use this .vbs script inside TC with the WSH disabled? Can I adapt the script so that it is invoked by TC internally for example, instead of via the WSH ?

[Vochomurka]

Here is some info you need.