| View previous topic :: View next topic |
| Author |
Message |
white
Power Member
Joined: 19 Nov 2003
Posts: 1304
Location: Netherlands
|
 Posted: Thu Jul 26, 2012 6:04 am Post subject: |
 |
|
| Valentino wrote: | (although this optimization is so easy that I would just make it instead of talking about it  )?
|
Cool. How would you do it then? And how would it effect speed and memory usage?
_________________
#16626 Personal licence |
|
| Back to top |
|
 |
MVV
Power Member
Joined: 03 Aug 2008
Posts: 4532
Location: Russian Federation
|
|
| Back to top |
|
|
ghisler(Author)
Site Admin
Joined: 04 Feb 2003
Posts: 24602
Location: Switzerland
|
| Posted: Thu Jul 26, 2012 7:10 am Post subject: |
|
|
OK, I have added some code now to filter out the duplicates when packing to ZIP, TAR and plugins. Unfortunately it cannot be done with external packers (EXE, e.g. RAR) because TC just passes the directories to them and tells them to recurse. But RAR seems to filter out duplicatss by itself.
Currently when packing tu ZIP, just choose "overwrite all".
_________________
Author of Total Commander
http://www.ghisler.com |
|
| Back to top |
|
|
Valentino
Power Member
Joined: 06 Feb 2003
Posts: 665
Location: Ukraine
|
| Posted: Fri Jul 27, 2012 7:25 am Post subject: |
|
|
| MVV wrote: | | white wrote: | | Cool. How would you do it then? And how would it effect speed and memory usage? |
Temporary string array or pointer array (if folders are already in some array). Sort, then check if file starts with path from that array using binary search, it is very fast. Binary search allows to compare up to 10 strings (up to first different character) if we have 1000 selected folders. |
I don't know how to make it in Delphi, and I don't know what structures TC uses to collect files/folders, but in C++ we can use std::set which stores its items sorted, for example: | Code: | typedef std::set<std::string> StringSet;
StringSet col;
col.insert("F:\\TEMP\\_tc1\\111\\222\\333\\bbb.txt");
col.insert("F:\\TEMP\\_tc1\\111\\222\\333");
col.insert("F:\\TEMP\\_tc1\\111\\222");
col.insert("F:\\TEMP\\_tc1\\111\\222\\444\\ccc.txt");
col.insert("F:\\TEMP\\_tc1\\111\\222\\333\\aaa.txt");
std::cout << "Before:\n\n";
boost::copy(col, std::ostream_iterator<std::string>(std::cout, "\n"));
if (col.size() > 1)
{
for (StringSet::const_iterator itEnd = col.end(), it = itEnd; --it != col.begin(); )
{
if (col.find(GetPath(*it)) != itEnd)
it = col.erase(it);
}
}
std::cout << "\nAfter:\n\n";
boost::copy(col, std::ostream_iterator<std::string>(std::cout, "\n")); |
Last edited by Valentino on Fri Jul 27, 2012 8:47 am; edited 1 time in total |
|
| Back to top |
|
|
ghisler(Author)
Site Admin
Joined: 04 Feb 2003
Posts: 24602
Location: Switzerland
|
| Posted: Fri Jul 27, 2012 8:01 am Post subject: |
|
|
I was already using a list of objects for the previous items when an archive was modified, so adding this change wasn't too difficult. Please try it!
_________________
Author of Total Commander
http://www.ghisler.com |
|
| Back to top |
|
|
MVV
Power Member
Joined: 03 Aug 2008
Posts: 4532
Location: Russian Federation
|
|
| Back to top |
|
|
ghisler(Author)
Site Admin
Joined: 04 Feb 2003
Posts: 24602
Location: Switzerland
|
| Posted: Mon Jul 30, 2012 7:05 am Post subject: |
|
|
Could you try it both with ZIP and a plugin like 7zip, please?
_________________
Author of Total Commander
http://www.ghisler.com |
|
| Back to top |
|
|
MVV
Power Member
Joined: 03 Aug 2008
Posts: 4532
Location: Russian Federation
|
|
| Back to top |
|
|
white
Power Member
Joined: 19 Nov 2003
Posts: 1304
Location: Netherlands
|
| Posted: Tue Jul 31, 2012 9:30 am Post subject: |
|
|
I tested using TC 8.01rc5 32-bit, Windows XP 32-bit, 10000 files of 2k, using internal ZIP.
I created a file list with duplicates. Searched the file list and fed the results to listbox. Then I packed all entries in the listbox.
I did not detect any difference in speed. Packed results are the same as when using TC 8.01rc4.
The removal of duplicates does not always work. It looks like there is a initialization error somewhere.
Sometimes I get overwrite dialogs. Usually only the the first time after I started TC and only one per file.
_________________
#16626 Personal licence |
|
| Back to top |
|
|
ghisler(Author)
Site Admin
Joined: 04 Feb 2003
Posts: 24602
Location: Switzerland
|
| Posted: Thu Aug 02, 2012 7:05 am Post subject: |
|
|
| Quote: | | The removal of duplicates does not always work. It looks like there is a initialization error somewhere. |
Sorry, I cannot help you without any details.
_________________
Author of Total Commander
http://www.ghisler.com |
|
| Back to top |
|
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
