network drives' zip extraction takes too long in WinXP-SP2-

English support forum

Moderators: white, Hacker, petermad, Stefan2

KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi Christian...

well, I did something else by now...
Had the chance to test another system with the online update way.
And after online updating that machine the 'problem' occured the same as on all other SP2 machines I could test it on since the beginning!
So it's not _definitely_ different with an online updated system.
Although there will be always a different amount of files which get updated online from computer to computer...as the online way always checks what's really needed in opposite to the 270MB SP2 offline pack...

But that way we won't track down the cause of the problem in general it seems.

Only thing sure is that it _happens_ with any machine and SP2...trying to unpack a .zip file from a network-drive!! :-(

Bye,
Karl
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi Christian...

just curious if there might be any news on that one here meanwhile!?

Thanks for some info...

Bye,
Karl
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi there...

just some little addon for this topic, today:

I just found out that the 'slowness' also occures when unpacking .iso files over the network from an xp-machine (no matter which sp1 or 2) to a local _SP2_ machine!

This means that the problem seems not to be _only_ with zip-files though! :-/

Maybe this info might help as well here...

Bye,
Karl

P.S.: Sorry...I forgot to mention that in this 'test' I did _NOT_ use total commander to unpack the iso-file...but another standalone tool! This also shows that the problem with unpacking over a network does not belong to any kinda 'bug' in total commander though! But maybe there's a chance to 'fix' anything in the code of tcmd to circumvent this 'sp2 bug' in general or something like this...!? Would be _VERY_ handsome for sure!!
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Well, the problem still exists...

So I just wanted to bring the subject up the list again!
Maybe there is someone with _any_ useful news about that problem here!?

Thanks and bye,
Karl
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi Christian...

just wanted to let you know some latest news about this (annoying) problem which still seems not to be solved by anyone yet... :-/

I found out that the slowness of network unpacking does NOT come only from the installation of SP2 with winXP but also occures with SP1 if you have done a _certain_ update via the windows-update site!!

Only problem is still that I hadn't had the chance to trace down _which_ update caused this slowness behaviour after updating yet!!
But at least it should be among those (probably) 'important updates' from within the last 6-9 ones counting back from today...

Hope this info might help tracking it down even better now! It would be sooooooooo great to see any kind of FIX for this prob (or at least a nice 'workaround'...!?)

Thanks and bye,
Karl
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48093
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Sorry, I didn't have the time yet to modify the zip unpacker, it's quite risky to do something wrong. A workaround would be to use RAR with solid archives instead.
Author of Total Commander
https://www.ghisler.com
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi Christian...

sure it's risky to modify your zip-code...just because of an general 'bug' which seems to be coming from any of those windows updates from the last time which seems to have changed anything special concerning network data-transfer which affects mega-slowness of zip-unpacking now, too etc...
;-)

Of course _I_ can use rar-archives instead...but it's not just _me_ here...but also all those thousands of archives you are getting from anywhere on the net...which you might have downloaded to your 'network' computer and want to unpack over your intranet etc... ;-)

So I guess a final solution will only be tracing down the roots of the 'bug' itself in the system...and why it exists _now_ anyways...so that _you_ also don't need to change your TC internal zip-code as well! As this wouldn't really make sense, too I'd say.

Thanks anyways for your reply...maybe someone else here in the forum will read this, too, and might be able to give us all the _final_ hint what is causing all this in the end...
And it might be fixable way easier with that info then...

Bye,
Karl

...*crossing my fingers for that*... ;-)
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi Christian...

I *FINALLY* had the time to look into that problem lots more with lots of testing...

And I finally found out WHAT the problem is caused by:

The reason unpacking over a network slowed down suddenly when updating to SP2 and even if you install certain 'important updates' from the MS update service lays in those *TWO* system-files:

- mrxsmb.sys
- rdbss.sys

Those files are located in "windows/system32/drivers"-folder.

The first update (with a system using still SP1!!) which changed those two files to a new version (and starting slowing down network .zip depacking etc.!) was:

- critical update KB 885835
direct-link: http://support.microsoft.com/kb/885835/EN-US/

Shortly after that one there was another update changing those files again:

- critical update KB 885250
direct-link: http://support.microsoft.com/kb/885250/EN-US/

In SP2 those changed files are (of course) already included so that the problem occurs directly after installing SP2 on any machine!!

As I'm no coder to find out what _really_ changed in those files (codewise) which causes this lame slow down in network unpacking I use my own 'workaround' to bypass this sh** at the moment...
(but it's surely not the perfect way to live with forever!! ;-) )

Workaround:
I simply exchanged those two files with the version _before_ those SP1-updates listed above and/or the files which came with SP2 anyway!!

When I check the windows update site now I get those two updates listed again for download which I simply switched off to be displayed in the update list to not get the files renewed again!

Of course this is not the perfect way as there might be some security bulletin open again in the system here now. But maybe not as the updates exchanged even more files as well which I had not to change back!
So maybe it doesn't really matter for the update-fix itself of the basic security problem.

Anyways, Christian, you are at least more 'coder' than me...so maybe there's a chance now for you to find out the real reason what changed in those two files from the past to 'today' to be able to react with your total commander code to it that zip-(un)packing over a network will speed up again as it used to be...

If you need those original (old) files of those two .sys-files, just let me know and I could send you those for comparing reasons...

Would be _really_ cool if there might be a chance to finally _fix_ this annoying problem which is caused by some weird MS-update and annoys a lot of people since that even more... :-/

Thanks for readind and hope I could help any of the users here a little bit... ;-)

Bye,
Karl
Ithier
Junior Member
Junior Member
Posts: 9
Joined: 2005-03-30, 14:13 UTC

Post by *Ithier »

Hi,

Thanks for your great study. I have done some study too (for my application) from a coder point of view. It appears that the extra time is spent when you close a file and only when you have perform at least 3 reads . It seems very weird but here is a small C++ which exibit the problem (the program is useless, and I know that you can perform the three reads in one function call, but it is just a test program to demonstrate the problem):

Code: Select all

#include <windows.h>
#include <stdio.h>
#include <time.h>

int main(int argc, char* argv[])
{
	srand( (unsigned)time( NULL ) );
	if (argc != 2)
	{
		printf ("Syntax: %s FileName\n", argv[0]);
		exit (0);
	}

	char buffer1[256];
	char* buffer2 = new char[256];
	char* buffer3 = new char[256];
	for (int i=0; i<20; ++i)
	{
		//Open the file
		HANDLE hFile = CreateFile(argv[1], 
								 GENERIC_READ, 
								 FILE_SHARE_WRITE|FILE_SHARE_READ,
								 NULL,
								 OPEN_EXISTING,
								 FILE_ATTRIBUTE_NORMAL,
								 NULL);
		if (hFile == NULL)
		{
			printf ("Cannot open file\n");
			continue;
		}
		DWORD pos = ((double)rand() * GetFileSize(hFile, NULL))/RAND_MAX;
		//Some reading
		DWORD read;
		SetFilePointer(hFile, pos, NULL, FILE_BEGIN);
		ReadFile (hFile, buffer1, 16, &read, NULL);
		ReadFile (hFile, buffer2, 16, &read, NULL);
		ReadFile (hFile, buffer3, 16, &read, NULL);

		//Now the fun part: closing the file !
		DWORD start = GetTickCount();
		CloseHandle (hFile);
		printf ("Close after read at %d: %d\n", pos, GetTickCount() - start);
	}
	return 1;
}

When you execute this program, you will see that the close function takes 1000ms nearly every time (if the file is big enough otherwise some parts are in the cache and it is instantaneous). If you remove a ReadFile, then the close function is instantaneous every time!

I think that I have found a workaround, it is just to create a file mapping (without using it).
So just after openig the file and getting the file handle, I have just added the folowing line:
m_hFileMap = CreateFileMapping((HANDLE)m_hFile, NULL, PAGE_READONLY, 0,0,NULL);
and I just close it just before closing my file:
CloseHandle (m_hFileMap);

Except for this two added lines, my program is not modified and is as fast as before this @#{¤$µ^ patch.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48093
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

That's a really interesting workaround! In the meantime i have already modified my unzip code: I'm now opening the zip only once, and then unzip all files from it with the same file handle. This way the zip is opened only once.
Author of Total Commander
https://www.ghisler.com
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi there Ither and Chrsitian,

glad to see that I could help a little bringing some new life in this really annoying topic here!! :-D

Well, what I just really don't understand is what those damn MS patches have to do with that weird behaviour...
Is that just a side effect which MS even doesn't know about? Or is it something that was forced by some other security hole which could only to be fixed that way?!
Is that weird or what!?

Anyways, christian, when will be that new zip-unpacking modification implemented in TC ? :-)
I'd really like to test it out if it will be as fast as it used to be again before those MS-patches! (or even faster!? ;-D)
(at the moment I'm still using 'my' file-exchange workaround here to avoid it! But of course this might also still keep another security hole open again probably...! ;-) )

So another safe fix would be surely a way better solution in future! :-D

Thanks for keeping me/us updated on any further news about this topic!

Bye,
Karl

...maybe someone could also inform MS themselves about their 'weird' buggy patchwork they have done there!? ;-)
Would also help for future things I guess! But whom to be mailed to anyways??
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48093
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

It will be in TC 7.0, which is planned for around the end of the year now.
Author of Total Commander
https://www.ghisler.com
User avatar
frenky
Senior Member
Senior Member
Posts: 250
Joined: 2005-07-30, 19:36 UTC

Post by *frenky »

Sorry if this has been answered somewhere else, but do you have some feature list in upcomming 7.0 release?
User avatar
ado
Senior Member
Senior Member
Posts: 445
Joined: 2003-02-18, 13:22 UTC
Location: Slovakia, Pezinok

Post by *ado »

frenky wrote:Sorry if this has been answered somewhere else, but do you have some feature list in upcomming 7.0 release?
it is always secret :-)...just read this forum and from time to time there are some leaking information about next version; but do not expect someone will give you list 8)

ado
KarlK
Junior Member
Junior Member
Posts: 13
Joined: 2004-08-26, 21:05 UTC

Post by *KarlK »

Hi everyone...

Just wanted to let you know that THIS topic here is luckily closed now as in the current release version (tested with 6.54a for example) the problem discussed here IS solved finally and unpacking/packing zip-files over a network is fast as hell again! :-D

Thanks for all your reading and help!

And of course MANY THANKS to Christian for fixing this in one of the BEST tools around the world called Total Commander of course! ;-)

Bye,
Karl
Post Reply