Total Updater 0.8.6.9 - Total Commander & plugin updater

Discuss and announce Total Commander plugins, addons and other useful tools here, both their usage and their development.

Moderators: white, Hacker, petermad, Stefan2

Post Reply
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

@petermad: Thanks for the dump list - as far as I can tell there's nothing wrong with it at all, except that some plugin's / addon's online version currently can't be detected (the filename doesn't matches its online id & no WEBID exists in case of them - or some of them are not even uploaded to totalcmd.net). I am going to add some of those plugins in the next versions to the Internal Plugin Database (excuse me but i'll refer to it as "IPD" in the future - wondering why the hell I gave it such a long name :D) - in the meantime you can also add some items to the UserDB if you'd like to, and maybe post it to me / publicly, so they can be added & embedded into the IPD / downloaded and used by other users too.

I have considered checking "pluginst.inf" lines and analyzed a bunch of them, but unfortunately (and surprisingly!) these are the least updated files... so it's not really worth checking them, most of them doesn't even contains version information, or it is forgotten on some initial state (1.0.0 etc; just like the case with much verinfo section inside exe/plugin files) - in addition: older plugins doesn't even have a pluginst.inf file (good ol' manual install)...

I'm not sure how could I check updates for language files... would be nice to implement such feature, but I think its just nearly impossible. I can't put them into any database, since there can be many files with e.g. the name "WCMD_ENG.LNG", and all of them could refer to another and another language pack contribution. What should I check to be able to detect & search updates for them? Unfortunately I really have no clue. :?

@frenky: You can modify the suggested entries Tichij wrote above, however proxy support is in the future to-do, at the time of writing this post the only component which will care about (and i'm not sure if its going to handle it correctly, its really untested!) is the one which does the query of the latest plugin versions from net (which you said is working correctly). Without considering this fact you can still set those values correctly to prepare Total Updater's future versions to be able to work with the proxy just fine. :)

Edit (Feb 20, 2013 00:17): Proxy support (for downloading plugins & querying latest versions) has been added to Beta6.
Last edited by Bluestar on 2013-02-20, 22:00 UTC, edited 1 time in total.
» Developer of Total Updater & extDir utility.
User avatar
EricB
Senior Member
Senior Member
Posts: 355
Joined: 2008-03-25, 22:21 UTC
Location: The Netherlands

Post by *EricB »

Bluestar,

Would it be feasible to use other file properties to determine version of installed plugin which has no version info? I'm thinking of file size and date. It would require additional fields in the database and user input for completion, but it might work for the last dozen of badly identifiable plugins.

Regards, EricB
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

@EricB:
Unfortunately filesize won't help at all - when you add a plugin on totalcmd.net, the author can fill in any size given in KB (so its not detected automatically), and they usually (as I do) give the whole downloadable package's size, which differs from the size of the main wfx/wlx/etc only in itself - thus filesize is definitely out of the game.

Date (last modification date) is currently actively being used when comparing local file versions with the latest ones (in case TU finds newer version on the net from a given plugin), but date alone is not a trustworthy method to decide if an addon is updatable or not. Two reasons: (the evident one is that) last modification date can easily be overwritten by the user with a single click / by various OS calls in given situations / by a virus scanner etc, and the second reason is that in many cases, a developer compiles the project on a day (e.g. on Feb 20, but decides to make it public on totalcmd.net only one / two or more days later) [there are plugins on totalcmd.net which local file modificiation date is 2005.02.27, while it has been uploaded/updated on totalcmd.net on 2010.01.30 - cabwcx).

_________________


Maybe some of you are actually wondering / thinking about how Total Updater works to do its tasks, so I decided to explain it as simple as possible (and with the aim to try not to be extremely boring :)):

01) It detects the files based on the found (or given) wincmd.ini - nothing magic here

02) Then TU tries to detect each plugin's version number based on:
a) the FileVersion field(s) - not an easy task, since many of the developers (respect for the exemptions) forgets to fill out / keep updating the FileVersion field(s) embedded into the PE (executable/dll) files.
Or even 'better', on this picture you can show you a totally general situation: image - as you can see, TotalUpdater always choose the latest found versioninfo from the PE file's sections (when it is possible, since the StringFileInfo field can contain any other garbage strings, and often devs put in texts like "0.5.4 BETA PRO" (image) or "TC plugin 1.3" etc (TU can handle these cases too)...
b) the readme/history/etc files: so the main algorithm failed and we didn't find any usable versioninfo in any section of the PE file -> lets try to use the intelligent version detection algorithm (if its not disabled by the user): it's going to search the actual plugin's directory for the following files (in order):
  • *history*.txt
  • *history*.htm*
  • *readme*en*.txt
  • *read*me*.txt
  • *read*me*.en*
  • *read*me*.ru*
  • {filename}*.txt
If any file has been found, it places them in descending order (by last modification date) in an internal list, and uses the first one to get the versioninfo from (later I may check all the files in the given directory for version informations, and compare them with each other - however this could slowdown the app a lot, in case of many installed utils/plugins without verinfo & with a lot readme/history etc files). E.g. for me it loads 95 items into the list with intelligent version detection off; by enabling it, this number raises to 112 (in total only 6 items are filtered out totally from my list, because of the absence of any valid version information).

03) In case we have found any usable version information using any method, we are going to compare the queried online version with it (by using some tricks, 'cause thats not that simple either - why should it be... :D). A general VS_VERSION_INFO/FileVersion field can only contain numbers and dots - while an online version can contain absolutely anything, which the author thinks would just look so fine & cool. So when a version is "1.0.0.2" in a given file, the author may have filled the online version with a following string: 1.0.2 Extreme Edition. (unfortunately it is really a general thing, I often see plugins with such version numbering).
TU is still able to handle it, and in such cases just assumes that they are in fact the same versions (compares them by removing dots and zeros); yeah, in fact they should never be the same (1.0.2 should be a newer version that 1.0.0.2), but generally we just assume that such thing doesn't happens often when the local version is SO similar to the online version (and it is really a rare case, actually none of my installed matches this special condition). So far so good.

04) If the online version seems to be newer, lets do some extra datecheck (if its not disabled) to avoid false alarms: now we are going to assume that the publication day is always one day after the compile (last modification) date: of course it isn't true in all cases, but will not cause any problem, since in general, the plugins that can be found on totalcmd.net aren't updated daily (you can modify PubDateDiffCorr=1 to any value, or completely turn this extra compare off by setting UpdateDateCheck=False inside [Plugins] section of TotalUpdater.ini) - however its not recommended as its pretty useful, and disabling it results in more false alarms.

05) If the online version still seems to be newer, lets check it in the OverridePlugVer section of the utility's config file: if its found, and the CRC32 value in the config file still matches the actual local file's CRC32 value, display it as "OK." in the list, instead of an updatable item - and this is the actual end of the comparement task.


Regards,
Bluestar
Last edited by Bluestar on 2013-02-20, 13:58 UTC, edited 10 times in total.
» Developer of Total Updater & extDir utility.
User avatar
EricB
Senior Member
Senior Member
Posts: 355
Joined: 2008-03-25, 22:21 UTC
Location: The Netherlands

Post by *EricB »

@Bluestar:

Yes, I understand. My idea was to manually register the properties of the W*X in the totalcmd.net archive (but only for plugins that cannot be identified in another way), so it can be compared to local file properties. In that case file size would work, because not the archive size, but of the contents would be used. I think that TC.writedate is normally static, so that could be used for comparison too, although sometimes the 1 hr difference problem pops up...

Regards, EricB
User avatar
EricB
Senior Member
Senior Member
Posts: 355
Joined: 2008-03-25, 22:21 UTC
Location: The Netherlands

Post by *EricB »

@Bluestar:

Yes, I understand. My idea was to manually register the properties of the W*X in the totalcmd.net archive (but only for plugins that cannot be identified in another way), so it can be compared to local file properties. In that case file size would work, because not the archive size, but of the contents would be used. I think that TC.writedate is normally static, so that could be used for comparison too, although sometimes the 1 hr difference problem pops up...

Regards, EricB

Edit: just after submitting this I thought of using W*X hash as unique identifier. CRC or SHA1 (or any other for that matter) would definitely work for that.
Edit2: Sorry for double post; smtp fault from board...
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

@EricB:
That could be done, but I wonder who would constantly update totalcmd.net database with such informations regarding these special plugins (e.g. the CRC32 value of each problematic plugin's each new version would do the trick). Besides that, I don't have any access to the server, so I would need to ask Flint to prepare the database & query script for such informations, and I don't know if he has any spare time for things like this...

Regards,
Bluestar

@petermad:
petermad wrote:
@petermad: Could you please confirm that the utility works for you & loads all plugins with existing UseIniInProgramDir line in wincmd.ini? From BETA4 there shouldn't be any issues with it.
Sorry, this still doesn't work for me - have you tested with NO Ghisler information in Windows' Registry?
Unfortunately I'm still unable to reproduce your issue. I've renamed both (HKLM&HKCU) "Ghisler" registry entries, set "[Configuration]UseIniInProgramDir=7" in wincmd.ini (in "%APPDATA%\Ghisler\wincmd.ini), then closed TC, restarted it (wincmd.ini file has automatically been created in c:\totalcmd), written "UseIniInProgramDir=7" into that file too, saved it, reloaded TC, and it works just fine - loads the appropriate files from this exact folder.

:idea: One thing which just came into my mind: please check if you have a "wincmd.ini" file in your Windows directory or not. In case no registry entries exists & TU finds it, it is going to be used instead of the one in Total Commander's default directory.

:?: And does the problem persists if you run Total Updater directly from Total Commander (so it can catch its %COMMANDER_INI% environment variable)?
» Developer of Total Updater & extDir utility.
User avatar
petermad
Power Member
Power Member
Posts: 14824
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Post by *petermad »

One thing which just came into my mind: please check if you have a "wincmd.ini" file in your Windows directory or not. In case no registry entries exists & TU finds it, it is going to be used instead of the one in Total Commander's default directory.
That's it - I had a wincmd.ini in C:\windows from testing a different version of TC - removing that wincmd.ini did the trick. :D

I find it a bug though, that TU superceeds C:\windows\wincmd.ini when UseIniInProgramDir=7 is used - TC doesn't, so TU shouldn't either. :(
And does the problem persists if you run Total Updater directly from Total Commander (so it can catch its %COMMANDER_INI% environment variable)?
I always run TU - and almost everything else from TC - I ALWAYS have TC running :wink:
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

petermad wrote:That's it - I had a wincmd.ini in C:\windows from testing a different version of TC - removing that wincmd.ini did the trick. :D

I find it a bug though, that TU superceeds C:\windows\wincmd.ini when UseIniInProgramDir=7 is used - TC doesn't, so TU shouldn't either. :(
Ah, finally! Glad it works this way for you too.

Btw I'm not sure what you mean by "superceeding" wincmd.ini. Are you saying I shouldn't check for "UseIniInProgramDir" key when wincmd.ini is found in %windir%? Or you mean I should just arrange the code a bit by checking {systemdrive}\totalcmd first, and then %windir%?

Now getting the wincmd.ini path goes like this: %COMMANDER_INI% -> HKCU -> HKLM -> %AppData%\Ghisler -> %WinDir% -> {systemdrive}\totalcmd (& TU is checking the "UseIniInProgramDir" key everywhere - of course it takes the first found file for result). Not sure how exactly does TC manage the same thing (especially when theres no AppData folder & registry entries), theres no official documentation about that (except this, which I'm following)...
petermad wrote:I always run TU - and almost everything else from TC - I ALWAYS have TC running :wink:
So just like I do. :D TC is the first thing to start after the Windows has loaded - I could set it to run automatically on startup, but clicking once on TC's icon in the taskbar & seeing the initial 'logo dialog' showing up just gives it a special feeling - if you know what I mean. :)
» Developer of Total Updater & extDir utility.
User avatar
petermad
Power Member
Power Member
Posts: 14824
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Post by *petermad »

Btw I'm not sure what you mean by "superceeding" wincmd.ini. Are you saying I shouldn't check for "UseIniInProgramDir" key when wincmd.ini is found in %windir%? Or you mean I should just arrange the code a bit by checking {systemdrive}\totalcmd first, and then %windir%?

Now getting the wincmd.ini path goes like this: HKCU -> HKLM -> %AppData%\Ghisler -> %WinDir% -> {systemdrive}\totalcmd (& its checking the "UseIniInProgramDir" everywhere).
I mean that UseIniInProgramDir=7 and UseIniInProgramDir=5 should override everything else (except commandline parameter /i= )

So the sequence if TC is running should be:
Commandline parameter /i= (and /f=) -> UseIniInProgramDir=5/7 (ProgramDir = %COMMANDER_PATH%) -> HKCU -> HKLM -> UseIniInProgramDir=1/3 -> %AppData%\Ghisler -> %WinDir% -> {systemdrive}\totalcmd

You should always use the path given by %COMMANDER_INI% if TC is running, which I believe is like I stated here above.

If TC is NOT running, I think you should find the install-path in: HKCU or HKLM - then you should check wincmd.ini in the install path for UseIniInProgramDir=5/7 - if UseIniInProgramDir=5/7 is not present, then use ini-directory from HKCU or HKLM in that order - if no ini-values in Registry, then use %AppData%\Ghisler and finally try %WinDir% then {systemdrive}\totalcmd


BTW - I think it would be good if TU showed the full path to the ini-file it assumes TC is using.
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

Thanks for describing it so detailed, I'm going to revise the current method of catching the wincmd.ini in future versions & make it more stable and accurate to handle all special cases.
petermad wrote:BTW - I think it would be good if TU showed the full path to the ini-file it assumes TC is using.
It is already possible, just try the following method:
Bluestar wrote:[face=tahoma]Total Updater v0.7 Beta 4
Changelog:
* New: Show currently used wincmd.ini's path on hover in the Configuration tab/Detect[...] radiobutton[/face]
» Developer of Total Updater & extDir utility.
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

[face=tahoma]Total Updater v0.7 Beta 6

New version introduces complete proxy support, and it contains some GUI fixes & a new function is implemented as well, which lets you to manually set direct download url for any UserDB entry (you just need to add DirectLink=http://dowload.it.com/myfile.zip under the actual section in userdb.ini - it is going to override "WebsiteID" but only for downloading the actual file from the given link).

Update is strongly recommended!
(read the changelog for more info...)

:arrow: Download | 21-02-2013 | MD5 (exe): 30869e661e290aae2a084d96efbba40b

Changelog:

* New: Chinese Simplified language support (thanks to FeiXJ)
* New: Updated Lithuanian language (thanks to Dalius Guţauskas (Tichij))
* New: You can manually set [PluginsFileName] DirectLink=http://dowload.it.com/myfile.zip in userdb.ini for any entry (where "PluginsFileName" is the name of the section)
* Change: from now on its also possible to download the latest Total Commander version (x32+x64 package)
* Fixed: downloading plugins using proxy is working now (set parameters in TotalUpdater.ini [Configuration] section)
* Fixed: querying latest versions of plugins also works with proxy set in the ini (regardless of IE's settings)
* Fixed: main form repositions itself to the initial state on refreshing list (if the position has been saved previously)
* Fixed: minimizing and then restoring the utility causes to 'forget' its last window size
* Fixed: other minor bugfixes & code tweaks
[/face]
» Developer of Total Updater & extDir utility.
User avatar
petermad
Power Member
Power Member
Posts: 14824
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Post by *petermad »

2Bluestar

I am so sorry :oops: - when I wrote earlier:
I had a wincmd.ini in C:\windows from testing a different version of TC - removing that wincmd.ini did the trick
I was wrong - I had forgotten that I had configured TU to search for other addons, and when I saw a full list, I didn't really look at it and didn't see that it wasn't plugins, just addons.

I have now tried beta 6 - an it still doesn't work with redirection of justfilename.

I tried:
Show currently used wincmd.ini's path on hover in the Configuration tab/Detect[...] radiobutton
and I get the correct path to wincmd.ini - but so I also does with beta 5.


I have made a some tests:

My Total Commander is installed in c:\wc and that is also where my wincmd.ini and plugins.ini is located. TU correctly shows me that it will use c:\wc\wincmd.ini but it cannot figure out to use c:\wc\plugins.ini. when I use RedirectSection=plugins.ini.

If I copy c:\wc\plugins.ini to c:\windows\plugins.ini - then it works (and this time I am sure), so for some reason even though TU knows that wincmd.ini is situated in c:\wc - it looks for plugins.ini in c:\windows !

I also tried to copy plugins.ini to %AppData%\Ghisler - with no effect.

I furthermore made a registry entry:

Code: Select all

[HKEY_CURRENT_USER\Software\Ghisler\Total Commander]
"InstallDir"="c:\\totalcmd"
and then copied plugins.ini to c:\totalcmd - also to no avail.

Finally I tried adding this:

Code: Select all

[HKEY_CURRENT_USER\Software\Ghisler\Total Commander]
"InstallDir"="c:\\wc"
"IniFileName"="c:\\wc\\wincmd.ini"

[HKEY_LOCAL_MACHINE\SOFTWARE\Ghisler\Total Commander]
"InstallDir"="c:\\wc"
"IniFileName"="c:\\wc\\wincmd.ini"
Again with no success.

So to add up: Even though TU knows that wincmd.ini is situated in c:\wc the only place TU seems to look for plugins.ini is in c:\windows (%WinDir%)
Last edited by petermad on 2013-02-21, 02:20 UTC, edited 3 times in total.
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
Bluestar
Senior Member
Senior Member
Posts: 377
Joined: 2007-06-10, 15:26 UTC
Location: Hungary
Contact:

Post by *Bluestar »

@petermad:
Actually no change has been made regarding "UseIniInProgramDir" in the current version (main goal was to make it work using proxies & to fix some annoying window resize bugs after minimizing), and as it seems I need to rethink & rewrite a couple of things for this "UseIniIn.." thing to correctly work in all cases... please use the Configuration tab's "Load the list of plugins from a manually given file" option until this fix arrives, thanks.
» Developer of Total Updater & extDir utility.
User avatar
petermad
Power Member
Power Member
Posts: 14824
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Post by *petermad »

2Bluestar
Actually no change has been made regarding "UseIniInProgramDir" in the current version
Aha - that explains it



Please read my previous post again - I have added some more tests.

Load the list of plugins from a manually given file
That's what I am doing currently ;-)
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
EricB
Senior Member
Senior Member
Posts: 355
Joined: 2008-03-25, 22:21 UTC
Location: The Netherlands

Post by *EricB »

Bluestar wrote:@EricB:
That could be done, but I wonder who would constantly update totalcmd.net database with such informations regarding these special plugins (e.g. the CRC32 value of each problematic plugin's each new version would do the trick). Besides that, I don't have any access to the server, so I would need to ask Flint to prepare the database & query script for such informations, and I don't know if he has any spare time for things like this...

Regards,
Bluestar
Hi Bluestar,

I understand your concerns, I'm merely trying to figure out if there is an easy way to identify plugins which do not have proper version info. We have both reached the conclusion that it would involve some manual work, and we cannot expect Flint to do it all ;-) I wouldn't mind collecting crc/md5/sha1 of such plugins and putting those in a database. Mind, most of the plugins we're talking about are not updated for some time. On the other hand, MVV told me that putting version info in a very small wdx plugin made by him could increase file size from 2.5k to 3.5k (see http://ghisler.ch/board/viewtopic.php?p=261573#261573), which could be considered bloating, if you see the relative size difference. So in such cases the alternative of a crc would be nice.

Regards, EricB
Post Reply