I really like the 'FTP download from list' feature in TC, but I get the feeling I use it in ways that are subtly different than it was designed for. After living with some of the limitations and complications of it I decided to make a write-up of how I see it could be improved.
Here is a short description of how it works right now (in italics), divided in separate steps, each step followed by some comments on how I could see some improvements, or how I tend to use this feature differently.
(1) You somehow collect a list of URLs to be downloaded (FTP or HTTP urls), possibly by using the 'download later' in the interactive FTP client part of TC. Next you choose "Net|FTP download from list" from the menu.
Most often I generate lists using my own homewritten tools, or simply using an editor.
(2) That will popup a dialog asking you for the list to download from. It prefills this dialog with the most recently used download list. If you want another list, you have to either type the path to the list file yourself or use the ">>" button to get a file selection dialog.
I tend to maintain different lists for different tasks, and quite often make new download lists. So, there are several download lists that I want to be able to select quickly. And new download lists tend to appear close to the last used download list ('close' in the directory hierarchy).
Now here there are some UI issues. There is no MostRecentlyUsed (MRU) dropdown list in this dialog, so, quickly selecting an older list is not an option.
Feature request 1: Add an MRU list to this dialog (changing the EditBox to a ComboBox and maintaining an MRU list).
Once you hit the '>>' button you can start browsing for a list file. However, this dialog starts off in an utterly irrelevant directory, such as the last location where you selected a TC plugin. It would make much more sense to have a separate state for where to start browsing for a download list, and not combining it with things like 'default location to find plugin files'. The two most logical places for starting this 'search for download list' would be either the directory where the last download list was found (or more precisely: the directory indicated by the EditBox/ComboBox in the dialog), or the directory of the 'current panel'.
Feature request 2: Change the start location of the 'browse for download list' file dialog.
(3) Next, TC starts scanning/loading the list.
For large download lists (several thousand entries), this really takes a long time. This may be a target for optimization.
(4) Then it shows a second dialog, asking for a destination directory where to download the files (defaulting to the currently active panel).
No real comments here, except that this second dialog may be combined into the first dialog. I realize that that request may sound far simpler than it is
(5) Each line in the list file can have one of two different formats. In the first format the line contains just the URL of the file to be downloaded (FTP or HTTP). In the second format it also starts with that URL, but additionally contains the string " -> " and the name of the destination file (relative to the destination directory given in step (4) )
I never tried to used absolute paths for the destination, but I notice that it is not possible to have path separators in the destination.
Feature request 3: It would be nice if for instance the following would work:
Code: Select all
http://site.com/path1/path2/file.xml -> path2\file.xml(6) After you click ok, TC happily starts downloading the files. A little window pops up 'in the background' to show you the status of the download operation. Below, I refer to this window as the 'background download window'. It has buttons to add extra files to the list, remove files from the list, pause the transfer or apply a speed limit.
Obviously this window was designed to be compact. Nevertheless, I would prefer some modifications to it. I never use the '+' or '-' buttons, preferring to simply edit the download list in an editor. I miss two features though.
Feature request 4: Add an explicit field telling how many files of how many in total have been processed (successfully or not). You may argue that the vertical scrollbarr tells you that, but it doesn't for even a medium number of files (the scrollbar thumb is at its minimum size for a long time, with no visible progress)
Feature request 5: Make the 'background download window' resizable, at least in the vertical direction.
(7) If you have configured a default Proxy to use, it will use that for all your downloads.
... but you can't override that setting in any of the dialogs shown. In some of my working environments I have to extensively switch between 'proxy on' and 'proxy off', and I notice that TC often looses its default setting for proxy. The case is that in those environments I Must use a proxy for all Http connections, but Must Not use a proxy for Ftp connections. I can live with forcing the same proxy setting for all downloads in a list, but not being able to set it explicitly anywhere for 'FTP from list' is a pain. The only way I have found to enable proxy use after it once more has reset to 'off' is to quit TC, edit the wcx_ftp.ini file manually to specify the default proxy host and restarting TC.
Feature request 6: Add an option to specify the proxy to be used for 'FTP download from list', just like the 'Download from URL' dialog.
That option should be specified in the dialogs mentioned in step (2) or (4). Obviously, that option should be local to that particular 'background download window', and not a global for TC. It should be possible to have a second 'background download window' that uses a different proxy setting.
(8) Each file is now downloaded one by one. If you want multiple simultaneous connections, then split the file and invoke another 'FTP download from list'.
Perfect! Please keep it that way and do not add 'multiple simultaneous downloads from one window'. Restricting the downloads to one at a time really improves download efficiency on slow internet connections.
(9) Each time a file is downloaded succesfully, the first three letters of the URL in the download file are replaced with the letters 'OK-'. IIRC, TC checks at this stage if the download list has changed and reloads it if necessary.
Perfect again! Though it would be nice to know exactly what makes TC rescan the file and to what extent it is safe to edit when a download is ongoing. Does it only scan for additions? Or can I reorder items in the list?
(10) If an error occurs, two different things may happen. If the error is server-side, (e.g. "File not found" or "authorization required") the download instruction is ignored. At the end of all the downloads a dialog is shown that 'an error occurred'. The file is not marked as 'OK-' in the download list. You can look at the download list afterwards to see which file failed (recognizable by the missing 'OK-'). Or you cam just retry the same download list.
Perfect once more...
(11) If the error is client side (target file already exists), a dialog pops up, giving you the option to either Overwrite, Resume or Abort. The download operation stalls until you give an answer to this dialog.
Eh, oops. Not perfect.
This is great for short download lists. It is an absolute horror for large downloads. According to murphy's law, there will be duplicates. The 'giving an option' part is the great thing. The 'stall' part is the horror. Nothing as bad as returning to your computer after a few hours only to discover that nothing was downloaded because the comp is still waiting for user feedback on the third file in the list (of 6000).
But there is another thing first.
Feature request 7: Add a 'Skip' option as the fourth option in this dialog, treating the file the same as in the case of a server side error; i.e. dont download anything.
Now the big problem is how to prevent the 'stall' part without removing the 'giving the option of handling the case' part. I see two solutions. They can be combined, but either of them alone can solve the problem.
Feature request 8a: Add an option to explicitly specify before the download starts (in the dialogs of steps (2) or (4)) how duplicates should be resolved. The default should be 'Ask', but the other options should be 'Overwrite', 'Resume', 'Abort' and 'Skip'.
Feature request 8b: Add a timer to the dialog (timer length configurable). If there is no answer to the dialog within a certain time (e.g. 1 minute), take the safest assumption ('Skip') and continue with the next file.
