(again) Slow FTP transfer

English support forum

Moderators: Stefan2, white, sheep, Hacker

User avatar
hlloyge
Member
Member
Posts: 120
Joined: 2006-11-02, 23:14 UTC

(again) Slow FTP transfer

Post by *hlloyge » 2009-08-06, 13:50 UTC

I can't believe this :D I've read all threads here regarding this subject, made changes to wcx_ftp.ini, entered 115200 as chunk size, and still I get slow FTP transfer. I am on gigabit LAN, and the max transfer speed I get to linux FTP server (vsftpd) is 25 MB/sec. Passive, non passive, default size, 115200... it doesn't matter. When transfering with FileZilla, I get 50-70 MB/sec, which is the max speed of software RAID5 on that server (tested). I know this isn't so high on mr. Ghisler list of priorities, because it works, just slower 8) but please, can you check out what is the source of this? I don't transfer often, but when I do, it is rather large file sets, so speed is somewhat important :D
Is there some mode where this can be debugged? I'd like to help as much as I can, I am not programmer, but I am skillfull with computers, and I'd like to help somehow to solve this problem.

User avatar
hlloyge
Member
Member
Posts: 120
Joined: 2006-11-02, 23:14 UTC

Post by *hlloyge » 2009-08-22, 12:01 UTC

Bumping the thread.

User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 37548
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) » 2009-08-23, 14:17 UTC

Sorry, I cannot say where the bottleneck is in your case. :(
Author of Total Commander
http://www.ghisler.com

User avatar
hlloyge
Member
Member
Posts: 120
Joined: 2006-11-02, 23:14 UTC

Post by *hlloyge » 2009-08-27, 21:10 UTC

Do you use some outside ftp library, or do you have your own written library?

User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 37548
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) » 2009-08-27, 21:25 UTC

I'm not using any outside library, all direct access to wsock32.dll or ws2_32.dll.
Author of Total Commander
http://www.ghisler.com

User avatar
Mjolnir
Junior Member
Junior Member
Posts: 81
Joined: 2005-10-08, 21:03 UTC

Post by *Mjolnir » 2009-09-01, 10:39 UTC

Hum, very interresting, I get same problem.

Here mine :

- Local system : XP SP2 // TC 7.50 RC2
--- wsock32.dll : 32088 oct (vers 5.1.2600.2180)
--- ws2_32.dll : 82944 oct (vers 5.1.2600.2180)
- Distant system : Solaris 10 OS

For a 20Mb file :
-> FTP using TC internal FTP : ~60 to 150 koctets/s
-> FTP using FileZilla : 2.8 MB/s
-> FTP using TC internal FTP (= try again) : ~60 to 150 koctets/s

(and this also occurs wuth same behavior on another colleage system running with XP SP2 & TC7.5PB5)

Any clue ?


Another question : is there away to use an external dll from TC (and continue to use the "ftp GUI popup window" from TC), rather the wsock32 or ws2_32 ?
Longue vie a TC... deja + de 10 ans de bons et loyaux services chez moi !

User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 37548
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) » 2009-09-01, 15:35 UTC

Sorry, no clue, I get > 20 MB/sec with a local server...
Author of Total Commander
http://www.ghisler.com

User avatar
Mjolnir
Junior Member
Junior Member
Posts: 81
Joined: 2005-10-08, 21:03 UTC

Post by *Mjolnir » 2009-09-02, 09:12 UTC

:(

Tried again this morning with XP's ftp.exe and works with same transfert rate that one I get with Filezilla.
I've also tried an old TC : unpack32_656.exe, always same bad transfert rate.

Here my TC log dump for a 2Mo file :

Code: Select all

221-Total traffic for this session was 765859 bytes in 3 transfers.
221-Thank you for using the FTP service on SolarisHost1.
221 Goodbye.
----------
Connect to: (09/02/2009 10:58:05)
hostname=SolarisHost1
username=lsav
startdir=
SolarisHost1=XXXXXXX
220 SolarisHost1 FTP server ready.
USER lsav
331 Password required for lsav.
PASS ***********
230-Please read the file README.txt
230-  it was last modified on Wed Jul 16 16:31:33 2008 - 412 days ago
230 User lsav logged in.
SYST
215 UNIX Type: L8 Version: SUNOS
FEAT
500 'FEAT': command not understood.
CLNT Total Commander (UTF-8)
500 'CLNT Total Commander (UTF-8)': command not understood.
OPTS UTF8 ON
500 'OPTS UTF8 ON': command not understood.
Connect ok!
PWD
257 "/export/home/lsav" is current directory.
Lire le répertoire
TYPE A
200 Type set to A.
PORT XXXXXXX,13,5
200 PORT command successful.
LIST
150 Opening ASCII mode data connection for /bin/ls.
Téléchargé¦s
En attente du serveur...
226 Transfer complete.
TYPE I
200 Type set to I.
PORT XXXXXXX,13,6
200 PORT command successful.
STOR 2Mo.gz
150 Opening BINARY mode data connection for 2Mo.gz.
Envoyés: 1,910,002 octets, 109 koctets/s
226 Transfer complete.
SIZE 2Mo.gz
213 1910002
Copied (09/02/2009 10:58:21): C:\Temp\2Mo.gz -> ftp://SolarisHost1/export/home/lsav/2Mo.gz 1,910,002 octets, 150 koctets/s
Lire le répertoire
TYPE A
200 Type set to A.
PORT XXXXXXX,13,7
200 PORT command successful.
LIST
150 Opening ASCII mode data connection for /bin/ls.
Téléchargé¦s
En attente du serveur...
226 Transfer complete.
And here log for FileZilla (always "faster") :

Code: Select all

Status:	Resolving address of SolarisHost1 
Status:	Connecting to SolarisHost1:21...
Status:	Connection established, waiting for welcome message...
Response:	220 crux.grenoble.ferma FTP server ready.
Command:	USER lsav
Response:	331 Password required for lsav.
Command:	PASS ****
Response:	230-Please read the file README.txt
Response:	230-  it was last modified on Wed Jul 16 16:31:33 2008 - 412 days ago
Response:	230 User lsav logged in.
Command:	SYST
Response:	215 UNIX Type: L8 Version: SUNOS
Command:	FEAT
Response:	500 'FEAT': command not understood.
Status:	Connected
Status:	Retrieving directory listing...
Command:	PWD
Response:	257 "/export/home/lsav" is current directory.
Command:	TYPE I
Response:	200 Type set to I.
Command:	PASV
Response:	227 Entering Passive Mode (XXXXXXX,235,86)
Command:	LIST
Response:	150 Opening BINARY mode data connection for /bin/ls.
Response:	226 Transfer complete.
Status:	Directory listing successful
Status:	Resolving address of SolarisHost1
Status:	Connecting to XXXXXXX:21...
Status:	Connection established, waiting for welcome message...
Response:	220 SolarisHost1 FTP server ready.
Command:	USER lsav
Response:	331 Password required for lsav.
Command:	PASS ****
Response:	230-Please read the file README.txt
Response:	230-  it was last modified on Wed Jul 16 16:31:33 2008 - 412 days ago
Response:	230 User lsav logged in.
Status:	Connected
Status:	Starting upload of C:\Temp\2Mo.gz
Command:	CWD /export/home/lsav
Response:	250 CWD command successful.
Command:	PWD
Response:	257 "/export/home/lsav" is current directory.
Command:	TYPE I
Response:	200 Type set to I.
Command:	PASV
Response:	227 Entering Passive Mode (XXXXXXX,209,3)
Command:	STOR 2Mo.gz
Response:	150 Opening BINARY mode data connection for 2Mo.gz.
Response:	226 Transfer complete.
Status:	File transfer successful
Status:	Retrieving directory listing...
Command:	PASV
Response:	227 Entering Passive Mode (XXXXXXX,96,175)
Command:	LIST
Response:	150 Opening BINARY mode data connection for /bin/ls.
Response:	226 Transfer complete.
Status:	Directory listing successful
Remarks :
- I've changed my IP & real Host Name in the traces for (privacy) by XXXXXXX or SolarisHost1
- from TC log, seems there are little error at "connection", but connection still occurs :

Code: Select all

FEAT
CLNT Total Commander (UTF-8)
OPTS UTF8 ON

Questions (if you know ?) :
- are those errors "normal" (or acceptable) ? // are those error "responsible" of our TC slowness ?
- is the internal TC ftp is "natif" or does TC use a XP dll ? (so that I could change it ?)
- is there a way to use another FTP plugin ? (I already got a sftp plugin, but no ftp plugin...)
- wanna make me perform other tests ? // get other logs ?

Thx !
Longue vie a TC... deja + de 10 ans de bons et loyaux services chez moi !

User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 37548
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) » 2009-09-02, 10:34 UTC

The only difference I see is that you use Passive mode in FileZilla - you may try that in Total Commander too. But it shouldn't have any effect on the transfer rate, except when using some kind of content filtering firewall...
Author of Total Commander
http://www.ghisler.com

User avatar
Mjolnir
Junior Member
Junior Member
Posts: 81
Joined: 2005-10-08, 21:03 UTC

Post by *Mjolnir » 2009-09-02, 12:09 UTC

I did also use passive mode (in FTP generic Configuration, and in the FTP detail connection) without success (still slow) :(
I also activated the passive mode for FileZilla, same high rate.
Longue vie a TC... deja + de 10 ans de bons et loyaux services chez moi !

User avatar
Hacker
Moderator
Moderator
Posts: 11262
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker » 2009-09-02, 21:07 UTC

Mjolnir,
Then perhaps Wireshark could help.

Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.

User avatar
Mjolnir
Junior Member
Junior Member
Posts: 81
Joined: 2005-10-08, 21:03 UTC

Post by *Mjolnir » 2009-09-14, 10:50 UTC

OK, I'm not a LAN expert, but here something interresting : I've used WireShark to check the FTP frames from my PC to the Solaris system I target :

- with FileZilla : frame length is 1460 bytes all over the time, with something like 0.5% of Retransmission

- win TC : here is the strange behavior : ~50% time, frame length is 1460, and the other ~50% time, Frame length is 1024 or 512 or even 76 bytes ?!? (and I got at least 3x to 10x times more "Retransmission" frames that with FileZilla)
Longue vie a TC... deja + de 10 ans de bons et loyaux services chez moi !

User avatar
hlloyge
Member
Member
Posts: 120
Joined: 2006-11-02, 23:14 UTC

Post by *hlloyge » 2009-09-14, 12:54 UTC

Hm, that could explain, as it waits for each packet to get confirmation of arrival from server...

User avatar
Mjolnir
Junior Member
Junior Member
Posts: 81
Joined: 2005-10-08, 21:03 UTC

Post by *Mjolnir » 2009-09-14, 14:43 UTC

hlloyge wrote:Hm, that could explain, as it waits for each packet to get confirmation of arrival from server...
Yep... but question is : why do I got this pb with TC and not with FileZilla ?!?
Longue vie a TC... deja + de 10 ans de bons et loyaux services chez moi !

User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 37548
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) » 2009-09-14, 15:38 UTC

Is this for uploads or downloads? For uploads, you can play with the UploadBlockSize inc wcx_ftp.ini (see help). For downloads, TC shouldn't control the size as the recipient...
Author of Total Commander
http://www.ghisler.com

Post Reply