Page 1 of 1

FTP: hostnames with multp. addresses, try all before failing

Posted: 2010-03-29, 16:08 UTC
by Sob
Any hostname can resolve to multiple addresses. Currently TC takes only one of them and if connection fails, TC does not try others. In the past it wasn't very big problem, because multiple addresses were not used very often and if they were, then mostly all of them worked. With introduction of IPv6, multiple addresses (at least two, one IPv4 and one IPv6) are becoming quite common. Also because of still a little experimental nature of IPv6, failures can occur more often.

The proper solution is to use getaddrinfo() function with AF_UNSPEC and try all addresses in returned order until the connection succeeds.

---

If getaddrinfo() is used, IPv6 support could be enabled by default with almost no risk of breaking anything. The nice thing about getaddrinfo() is that it's inteligent and returns the addresses in best order for every machine:

- PC without IPv6 does not get IPv6 addresses at all
- PC with 6to4 IPv6 gets 6to4 IPv6 first, then IPv4 and production IPv6 last
- PC with production IPv6 gets production IPv6, then 6to4 IPv6 and last IPv4
- PC with Teredo IPv6 gets only IPv4 addresses (unless it specifically requests IPv6)

So in most cases the first address will work. There can be exceptions for misconfigured PCs with assigned IPv6 addresses but without actual IPv6 connectivity. But they will still work, because next address will be tried, there will be only slight delay. But after all, they have it broken, so they should suffer or fix it. ;)

Posted: 2010-03-29, 16:21 UTC
by ghisler(Author)
That's an interesting idea indeed. Currently the user has to choose whether he wants to prefer IPv6 or IPv4 if both are returned:

PreferIPv6=-1
-1 IPv6 disabled (default)
1 prefer IPv6 for servers which support both IPv4 and IPv6
0 prefer IPv4 (normal Internet addresses)

Posted: 2010-03-29, 16:30 UTC
by Sob
I currently use PreferIPv6=1. It works great when server has IPv6 address and I'm connected to network that also has IPv6 working. But when I connect the same computer (notebook) to other network without IPv6, it's not so great. TC still tries to connect to IPv6 address (because of PreferIPv6=1) and then does not fall back to IPv4 when IPv6 fails. If getaddrinfo() was used, it would go directly to IPv4 address when connected to IPv4-only network.

Posted: 2010-03-30, 16:46 UTC
by ghisler(Author)
I see - I'm already using getaddrinfo, but not using AF_UNSPEC and not trying all addresses, because the timeout for non-working address can be quite long. I will consider it.

Posted: 2010-03-30, 18:28 UTC
by Sob
Lets split it in two parts.

First trying multiple addresses (regardless of address family). This should definitely be implemented. Usually the first one will work. And in cases it won't, it's much better to wait a while and connect to another than not connect at all.

With IPv6 enabled by default it might not be so clear. The safest way for now is to leave it off by default and let user enable it (as it's now). But this way it won't ever get used, because default install won't work with IPv6 and no one will bother looking for hidden ini option. I also admit that my "fix it of suffer" for misconfigured PCs might not be the best for the real world. :)

So I propose to find inspiration in Windows. Enable IPv6 by default but allow to easily turn it off (one more checkbox for that would easily fit in FTP setting dialog :). Then the default would use getaddrinfo with AF_UNSPEC which would mostly work fine. And with IPv6 off it'd just use AF_INET.
Optionally you could provide some enlightenment for users. If first address was IPv6 and connection failed, TC could show some informational message about possibility of misconfigured IPv6 (preferably with Don't show again checkbox).

Posted: 2011-09-24, 01:50 UTC
by Sob
As much as I'm sure all users would like to see their feature requests implemented, I believe this one is kind of special. It's not just for me, it's for everyone. Everyone will need it eventually, they just don't know it yet. ;)

IPv6 is slowly rising. It will take some time before people will actually have to use it. But it would be nice if TC was prepared in time. And given it's relatively long release cycle, I believe 8.0 is the right version for this change. And since we have those public betas now... :)

Current state:
PreferIPv6=-1 (default) - IPv6 doesn't exist for TC
PreferIPv6=1 - if server has both IPv4 and IPv6 addreses, TC tries IPv6; if it fails, it's complete failure, TC does not try IPv4
PreferIPv6=0 - as above, including the failure, only switched address families

Proposed change:
PreferIPv6=-1 (default for 9x) - IPv6 doesn't exist for TC
PreferIPv6=1 - TC tries all addresses from getaddrinfo(AF_INET6) and then all addresses from getaddrinfo(AF_INET)
PreferIPv6=0 - TC tries all addresses from getaddrinfo(AF_INET) and then all addresses from getaddrinfo(AF_INET6)
PreferIPv6=2 (default for NT) - TC tries all addresses from getaddrinfo(AF_UNSPEC)

Enabling IPv6 by default is safe-enough choice, because system will choose the right address almost always. There are broken systems (system thinks it has IPv6 connectivity, but in fact it doesn't), but the last numbers I've seen were far less than 1%. And after all, those users will notice the problem much sooner before trying TC's FTP client. Many other applications, e.g. all current web browsers behave exactly the same as TC would with proposed PreferIPv6=2.

Posted: 2011-09-25, 20:43 UTC
by ghisler(Author)
I will check how difficult it would be to add.

Posted: 2011-11-26, 16:15 UTC
by Sob
Good, I see some changes in beta 10. The basic functionality, trying more addresses and not failing as soon as the first one doesn't respond, works fine.

One bug: with PreferIPv6=-1 (assuming the meaning didn't change and it's still IPv6 disabled), TC also tries IPv6 addresses (after IPv4).

And second, it seems that my suggested PreferIPv6=2, i.e. try all addresses in order returned by getaddrinfo(AF_UNSPEC), wasn't included, which is bad. Why? Because unless user manually sets PreferIPv6=1 (and none of the regular users will do it), TC will not try IPv6 before IPv4, even if it's the best option for him (in case he has native IPv6 and only NATed IPv4).

Posted: 2011-11-27, 13:37 UTC
by ghisler(Author)
One bug: with PreferIPv6=-1 (assuming the meaning didn't change and it's still IPv6 disabled), TC also tries IPv6 addresses (after IPv4).
This isn't actually a bug, it's a new feature. You can use -2 now to prevent any IPv6 calls.
And second, it seems that my suggested PreferIPv6=2, i.e. try all addresses in order returned by getaddrinfo(AF_UNSPEC), wasn't included, which is bad. Why? Because unless user manually sets PreferIPv6=1 (and none of the regular users will do it), TC will not try IPv6 before IPv4, even if it's the best option for him (in case he has native IPv6 and only NATed IPv4).
I will consider it as an additional option.

Posted: 2011-11-27, 15:39 UTC
by Sob
ghisler(Author) wrote:This isn't actually a bug, it's a new feature. You can use -2 now to prevent any IPv6 calls.
-2 disables multiple addresses handling and only tries one, as TC7 did. It's fine. But what is now the difference between -1 and 0? They both try all IPv4 and then all IPv6 addresses.
I will consider it as an additional option.
It might sound too radical, but it would be best as new default .

The idea is that IPv6 is better than IPv4 (there can be arguments about various technical details, but it's not the point here). Soon even big ISPs who so far were able to give public IPv4 addresses to their users will be forced to deploy NATs. And NAT for thousands of users is a little more complicated than NAT in home router serving three family PCs. So they will also try to deploy native IPv6 and hope that most applications will automatically use it to lower the demands on NATs. On Windows getaddrinfo(AF_UNSPEC) is the mechanism that can make it happen. It's pretty safe, as long as system doesn't have native IPv6 connectivity, it sorts IPv6 addresses after IPv4 ones. The day native IPv6 connectivity arrives, IPv6 becomes preferred. And if it's safe default for web browsers used by people who know nothing about computers, it should be safe also for TC aimed at (at least a little more) advanced users. No pressure of course, just saying... :)

Posted: 2011-11-28, 14:18 UTC
by ghisler(Author)
With the end of the available IPv4 addresses, IPv6 will probably become much more popular soon. That's why I want to enable it by default now.

-1 uses the old wsock32.dll for connections, and wsock2_32.dll only for IPv6 functions, while 0 uses wsock2_32.dll for all calls. I was worried about using wsock2_32.dll for everything, that's why I disabled IPv6 by default. Now it's enabled by default with this trick. With -2, the new getaddrinfo (which is only present in wsock2_32.dll) isn't loaded. Only this function allows to get multiple IP addresses for a single server name. That's why '-2' now behaves like the old '0'.

Posted: 2011-12-13, 00:52 UTC
by Sob
Great, PreferIPv6=2 made it into beta 12 and works fine. Thanks.

It's still not the default, but I take it that you see preferring IPv4 as safer for now.