Detecting loss of ADSL & switchover to 3G

Discussion in 'Broadband' started by Peter, Jul 17, 2013.

  1. Peter

    Peter Guest

    We have an ADSL connection at work, with a 3G backup.

    The ISP (a supposedly high quality specialist business ISP) gives us a
    number of fixed IPs, which are supposed to get rerouted to the 3G if
    the ADSL fails.

    If I power down the ADSL modem, it all works fine. The router detects
    the loss of the WAN1 ("ethernet") service and switches over to WAN1
    ("3G") which is a USB 3G adaptor. When the ADSL has been back up for 2
    minutes, the 3G is shut down and it switches back to the ADSL.

    We pay just £2.50/month for the 3G backup so this is pretty good. We
    pay about £40/month for the whole thing including an 800kbits/sec UP
    speed (8M down; the exchange doesn't do the 20mb).

    However we have had several losses of ADSL where the 3G backup did not
    work. The ISP is pretty arrogant about it (seems to be their in-house
    operating mode) and blame it on our equipment. However on the last-1
    time it happened (weekend as usual) I did go to the office and found
    that the IP had changed... this is OK for outgoing stuff but obviously
    any incoming connections (RDP, VPN, etc) will fail.

    The ISP claims this cannot happen with their equipment because they
    allocate fixed IPs. But this is self evidently bollocks. The IP is
    allocated (DHCP) by the DHCP server in the ISP's router, and customers
    who pay for a fixed IP just happen to get the same IP every time. Or
    maybe not....

    The Q is how could one possibly detect such a scenario i.e. where you
    have one of these cases

    1) the ADSL connection seems OK (how the Draytek router detects the
    failure of WAN1 is undocumented) and the break in connectivity is
    further upstream (usually blamed on British Telecom) or

    2) the ADSL connection works but the IP has been changed

    The router can't be configured to periodically check an external
    website and check it's own IP.

    We could dump this ISP but they all seem to have these problems from
    time to time. The only ISP I know of which has had zero downtime for
    years is ZEN (a really brilliant company) but I can't use them at work
    because we use them at home and that is where the backup server
    (rsynced from the office one every night) is located!

    There are services which can detect external loss of connectivity but
    how can one use that to switch over the router to use the 3G adapter?

    Especially when the router is then not accessible remotely, due to the
    loss of connectivity!

    I imagine there are high-end solutions for this, but what's really
    needed (for cheap people like me - our router is the £300 Draytek
    2955) is some means whereby the router can detect that full
    connectivity has been lost, or the IP has changed.

    I think all the Draytek routers do is they look for a loss of some
    kind of regular keep-alive packet which passes between the ADSL gear
    in the telephone exchange, and the router (or maybe the ADSL modem - a
    Draytek 120). All that will detect is a physical loss of the ADSL
    connection to the exchange.

    One gets a related sort of issue with say a smartphone with the WIFI
    enabled and configured to connect automatically, and also 3G enabled.
    All the time it is picking up unencrypted WIFI access points, it won't
    use the 3G - because it is too stupid to actually check connectivity
    all the way out by e.g. checking if google.com can be accessed. And
    because nearly all the WIFI APs are commercial "pay" ones, nothing
    works... Satellite phones also frequently suffer from a "perfect data
    connection" (you get DHCP etc etc) but no data will come through.

    I can get two different ISPs to give us two ADSL services, and the
    router will switch between the two, but (a) the criteria for the
    fallover won't be any more intelligent and (b) we obviously can't have
    the same IPs from both.

    I wonder how this is solved professionally. An external monitoring
    service, sending an SMS to the router and switching it over? Or maybe
    a clever router which communicates with an external service
    periodically and gets it to check incoming connection?
     
    Peter, Jul 17, 2013
    #1
    1. Advertisements

  2. Peter

    Graham J Guest

    Comments in-line ...
    Is this A&A ???

    How does the ISP know that your ADSL has failed?
    Do you mean WAN2 ????

    Does it also recognise when you unplug the phone wire from the ADSL modem?
    Do you mean the WAN1 IP address for ADSL had changed?
    A fault in the BT network can cause this. Normally the PPP
    authentication is passed through the BT network to a (Radius) server at
    your ISP, where your username and password are recognised and your ISP
    issues a (consistent) IP address. If the BT network has a problem, then
    your ISP's Radius server is not accessible and BT issues your router
    with a substitute IP address. This is normally sufficient to run
    diagnostics but the BT DNS server thereby made available to you
    deliberately does not resolve most names: thereby limiting access to BT
    websites only.

    When this fault occurred, what IP address were you allocated? Was it
    one from the range allocated to your ISP?
    No - surely it should be clear from the configuration of the router!

    and the break in connectivity is
    You could try the Draytek 2830. On this the WAN1 port is an ADSL modem,
    and the failover detection can be configured for either ARP or ping to
    an IP. Normally you would choose ARP, but all this will prove is that
    the ADSL has connectivity. If you choose Ping and specify the static IP
    address that you know the ADSL port should have, then a failure such as
    I describe above would cause failover to 3G.

    Does your 3G backup have the same IP address as the ADSL service? If
    not, how do you use RDP and VPN during failover?

    From what you wrote above, the backup 3G service does use the same IP
    address. So any failover detection mechanism relying on IP addresses
    will fail.
    Multiple ISPs and a router running BGP. Think Cisco and £5,000 upwards.
    This is what ISPs themselves do, and businesses who host websites.

    You could script a telnet session to interrogate the router for its WAN
    IP address, to verify that it is the address it ought to be. Run this
    once every minute, and get it to email/sms if a mismatch is ever seen.
    Make the script write its results to a log file so you have the evidence
    for your ISP.
     
    Graham J, Jul 17, 2013
    #2
    1. Advertisements

  3. at huge cost and complexity.

    What you want is e.g a fixed IP address that you own, that is part of
    the GLOBAL BGP tables, so that its route can be advertised by e.g. two
    different ISPs depending on who has the best route to you.

    Thats a BBC.co.uk type solution, not joe-soap.co.uk


    You are frankly asking a lot to expect established sessions to survive a
    route switch. IF your ISP is offering this as a bundle, then they ought
    to be able to manage it.

    They should be big enough.

    No, do it via the ISP, and if they are not doing it, find someone who will.
     
    The Natural Philosopher, Jul 17, 2013
    #3
  4. Peter

    Peter Guest

    They can tell if I pull the plug on the ADSL modem. I get an SMS
    telling me ADSL has failed. And the router switches over to 3G because
    it detects WAN1 is down.
    Yes; sorry.
    Yes I think so. Normally, to test this, I switch off the ADSL modem.
    Yes. To something random.
    Interesting... I had a recent case where we got full outgoing
    functionality, on the "random" IP.
    I don't recall, but it was very different from what it shoul dbe.
    I cannot find it in the config of the router (2955). Most likely it is
    accessible if you telnet into it and hack around the unix stuff, but
    it isn't apparent in the GUI.

    There is a separate config for a fallover if more bandwidth is needed.

    This is the WAN2 menu (if you can read it)
    What "WAN1 fail" means is unclear.
    Hmmm... the 2955 costs even more than the 2830 I think :)

    That's interesting.
    It does - if the ISP has configured it right. It wasn't done
    initially.
    I think, however, that the 3G is dead if WAN1 is working. I think the
    ISP activates the 3G account, and routes the IPs to it, only when
    *they* are aware the ADSL has failed.

    Now we are onto the question as to how the *ISP* knows ADSL has
    failed!

    If I pull the ADSL modem, they know right away.

    On the last failure (13th July, no 3G failover) I got no SMS from
    them, but they told me they don't send them out if the issue is
    widespread.
    I think there are services that can monitor a website and send an SMS
    if duff, etc.

    I ought to set that up, because we will have an online shop soon also.
     
    Peter, Jul 17, 2013
    #4
  5. Peter

    Graham J Guest

    More comments in-line[snip]
    Generally recognised as the best ...
    Please test this and report back.

    [snip]

    No, the DHCP server recognises the MAC address of your router and issues
    the IP address specifed for that MAC address. Unless their DHCP server
    is faulty ....
    Please make sure you note it down next time the fault occurs. Then use a
    Whois service to find who owns it.

    Report it to A&A - they should be able to interrogate their DHCP server
    and confirm the address they issued matches the one you report.
    I have here a V2910. Basically an earlier and simpler version of your 2955.

    Under WAN - Internet Access - on the screen where you set the PPPoE
    Client mode and ISP Access Setup, the lower part of the screen shows
    "WAN Connection Detection" and you can select either ARP or PING. I
    suspect the V2955 must have a similar screen.

    [snip]
    What do you need from the 2955 that is not provided by the 2830?
    Because A&A use LCP monitoring - this confirms packet exchange between
    you and them, irrespective (I think) of the IP address allocated to your
    router.

    So the issue is the allocation of the "incorrect" IP address. I think
    you need to escalate this within A&A, and ask them to lend you one of
    their routers to confirm whether the fault can then be demonstrated to
    occur.

    If only your equipment demonstrates the problem, complain to Draytek.
     
    Graham J, Jul 17, 2013
    #5
  6. Peter

    Andy Burns Guest

    What "MAC address" would the router use when falling back to the USB
    dongle? How would the ISP recognise this as the same when using the WAN1
    port?
     
    Andy Burns, Jul 18, 2013
    #6
  7. Peter

    Peter Guest

    Will do.
    I don't see how it can be doing that since I have two identically
    configured 2955s (but their MAC addresses can't be the same) which I
    can swap and everything works as before.

    I also have two of the Draytek 120 modems (which have no config) and
    can swap them around. They do sometimes pack up.
    I asked them if you could look at their logs to see why the 3G failed
    on the 13th. The man refused to do anything about it.
    Oh yes!
    Currenly it is set to ARP detect.

    So it looks like I could ping a specific external IP.

    What does the TTL parameter do? Is it the number of seconds between
    pings?

    If I put in the IP we *should* be getting allocated by the ISP, does
    that actually check anything useful? I wonder if it would just do a
    local loopback i.e. useless.
    I can't recall why we bought it.

    SSL VPN perhaps?
    They have offered a box called a "Firebrick" but I don't want to be
    spending yet more of my time (probably days; it usually works out like
    that) setting that up. This is supposed to work...

    The outage with an IP change is very rare but next time I will make a
    note of what it is. It just means I have to go there physically,
    because RDP doesn't work.

    We have used Clara, Eclipse, and now A&A. All have about the same
    downtime rate. Always blamed on BT. Downtime on ZEN has been zero IIRC
    over maybe 10 years. All these are on the same exchange (01273). The
    others used to change the "fixed" IP too. Eclipse used to do it quite
    a lot.
     
    Peter, Jul 18, 2013
    #7
  8. Peter

    Graham J Guest

    I was wrong. What I described is how DHCP reservations work on a LAN.

    I imagine the IP address is looked up from the account parameters when
    the PPP sessions starts.
     
    Graham J, Jul 18, 2013
    #8
  9. Peter

    Graham J Guest

    Comments in-line
    You are correct. What I described is how DHCP reservations work on a LAN.

    I imagine the IP address is looked up from your account parameters when
    the PPP session starts. It really must be a problem at A&A ...

    [snip]
    But that's not what happened, according to your previous explanation.
    What you said was that the ADSL got the "wrong" IP address, so your VPN
    and RDP connections would not work. So far as A&A was concerned, your
    ADSL connection was good, so there was no reason to switch to 3G.
    That's worrying. My impression is that A&A are the best ISP for this
    class of service.

    Clearly if they are not prepared to help, you should ask elsewhere.
    Maybe the people here have a recomendation. But LCP monitoring, SMS
    messages on failure, and automatic switchover to 3G are not the province
    of everyday ISPs.

    You could ask Zen if they have a product to meet your needs ...?
    Google TTL !!!!!!!!!!
    If you've been issued the "wrong" IP then somebody else might have been
    issued yours. So pinging your IP would not be a useful test. A Cisco
    router might allow multiple "beaconing" mechanisms to detect failure.

    Ask A&A for one of their internal IP addresses that is only PINGable
    from your ADSL connection, but not from elswhere on the internet. They
    should at least understand the question!
    That will always be the case - it is the BT network that provides the
    connectivity. But the failover system you have should accommodate this.
    If that's really true you have been very lucky!
     
    Graham J, Jul 18, 2013
    #9
  10. radius servers associate login name and IP address if serving 'static'
    ip addresses.
     
    The Natural Philosopher, Jul 18, 2013
    #10
  11. Peter

    Peter Guest

    Apologies for any confusion. On the 13th I don't know if the wrong IP
    was allocated. I do know the 3G backup failed.
    They are a bit variable. There is a clear culture of "we are the top
    anoraks in the world" and this is reflected in most of their comms.
    For example their emails are not send like normal people send emails
    (by replying to your From: header !!!!). They address their emails
    using BCC. It's quite bizzare and I have never seen that before. They
    seem to be using some funny email package which supports PGP or some
    such, and yes with PGP one tended to get this. Same with using anon
    remailers. I asked them why they don't use normal email but they
    didn't see what I was talking about.

    If I could get the 3G backup elsewhere I would move right away. We pay
    about £40/month but their bizzare pricing model gives us (IIRC) less
    than 10GB/month of data. At home I have ZEN at £20/month and we get
    50GB, unmetered uplink data, and can have the 2x UP speed for an extra
    tenner or so.
    Indeed. We are in the countryside so wire breaks are not that unusual.
    Time to live. Yes I knew that :) But that wasn't my question. Is this
    a straight timer between the pings, do you think?
    OK, but even pinging 8.8.8.8 every few mins might still be better than
    what we now have - no?

    What is ARP detect, in this context?
    OK will do.

    SSL was an interesting experiment but is crippled by the lack of
    clients. Windoze comes with a nice PPTP client which "just works". The
    SSL client for a Draytek is downloaded from the router and they offer
    a Java app and an active-x app, both of which work.

    They also offer a Samba app which is a crude file browser, with a
    stupid bug in that the vertical scroll bar doesn't work, so if you
    view a directory containing more than about 30 files youcan't see the
    rest. I have written to them about this many times, with screenshots,
    but it doesn't seem to register. Frustrating! This would be great
    because it allows you to use your client browser as a file explorer,
    so would work on any smartphone etc.

    In reality, times move on and I increasingly find PPTP works, whereas
    a few years ago many wifi and cellular networks didn't support it.
    That was the main driver for a SSL VPN on 443 - it should always work.

    One irritating thing, off topic now, is that the cellular networks are
    blocking tethering when you are abroad and hoping to use one of the
    special EU-mandated discount data deals. They can detect it
    immediately, without any data even passing, due to apparent complicity
    by the phone manufacturers (unsuprising since the mfgs need the
    networks to flog their phones in their shops!). Nokia tell the network
    that Joikuspot is running. Apple tell the network, by using a
    different APN. Using the phone as a bluetooth modem still works
    though!!! And if they snooped on the data (browser agent strings etc)
    one could use a VPN. A VPN from the laptop I mean; I spent countless
    hours trying to get the only Nokia supported VPN (IPSEC) working, with
    the Drayteks as the VPN terminator...
     
    Peter, Jul 18, 2013
    #11
  12. Peter

    Mike Guest

    As an A&A customer, there have been some recent BT fumbles that
    have affected A&A customers (and other ISPs too). During these
    fumbles, I lose my ADSL link, and when it comes back, along with
    all the other 100,000 customers, BT's authentication servers
    give up and give me a "parking" address. Last time it was
    a 172.something address, not my actual 81.something external
    WAN IP address.

    It's not the ISP allocating in this case, they NEVER SEE your request
    to connect, BT hijack it and dump you in a playpen. If you reboot the
    router, you may be allowed out to talk to A&A, and then you'll get
    your real address.

    The problem you have is that your router thinks it's got a valid
    IP address and has no idea that it's a dud.

    The problem A&A have is that you dropped off the internet and
    didn't come back yet.

    The problem is BT having overloaded systems in the middle.

    I also have a "fixed" IP address that changes when BT cock it up,
    with no fallback system, it's just wait, or reboot now and then.

    I suspect BT here ...
    Seconded. I don't know about "arrogant" but they're usually right about the
    source of problems, and don't mess about hiding behind confidentiality
    clauses that won't allow them to state who broke what.

    As to how to find if the address has been picked up wrong, I continually
    "ping" the correct IP address of the WAN side of the router. When BT
    fumble the authentication, this ping fails, because the WAN side is no
    longer 81.x.x.x and that is easily detected from inside the network
    without digging into the router setup pages.

    I don't know if you can use that with a clever 3G fallback system!
    If you can work out the time and date of when the ADSL went down,
    when the IP address went "wrong" you may correlate it with the various
    BT failures that can trigger this.
     
    Mike, Jul 18, 2013
    #12
  13. Peter

    Graham J Guest

    Peter wrote:
    [snip]
    No you don't understand !!!!

    If you had Googled it you will know it has nothing to do with time! It
    measures the number of router hops allowed before the packet expires and
    is not replied to.

    So it is perhaps possible to specify a low number so that it will see
    replies for traffic leaving your router and going via A&A ADSL, but
    traffic going via 3G and the 3G provider's routers will exceed the TTL
    value and therefore not get replies.
     
    Graham J, Jul 18, 2013
    #13
  14. Peter

    Peter Guest

    Just tried it.

    Pulling out the BT cable from the ADSL modem does cause the fallover
    to 3G but it takes longer - about 1 minute - whereas depowering the
    modem causes the fallover within seconds.

    Plugging the cable back in recovers the ADSL about a minute later, and
    the 3G shuts down.

    So a break in the BT copper appears to be covered OK.
     
    Peter, Jul 18, 2013
    #14
  15. Peter

    Peter Guest

    I am sorry but I am unclear as to how I could use this.

    Are you suggesting I should set it up to ping say A&A's DNS server
    which is 217.169.20.20 and .21 ? And use a TTL of just 1?
    What is the TTL of 61 mean?
    So maybe a TTL of 3 ?

    But what will that give me? Perhaps you mean setting this up on some
    other system, to detect if there has been a fallover to 3G and then I
    could be notified?

    I found this re the ARP detect business

    http://www.draytek.com/.upload/pdffiles/dc7e482cf3b61f804a24f83f0b525536.pdf

    Sounds like the ping is every 1 second. So I need to use an IP which
    is happy to get pinged every 1 second... I suppose I could ping the
    backup server (on ZEN).
    What TTL should I use for that?
    Tracert to that IP fails, which is not unusual.
     
    Peter, Jul 18, 2013
    #15
  16. I THINK the way that ADSL works is that all users get passed to BTs
    radius servers which are SUPPOSED to consult with the ISPs own radius
    servers to get the DHCP information.

    If BT mangles the user identity for example, you might still get a
    session, but not the one you were expecting.

    The problem arises because BT doesn't know who to route the frame relay
    connection to, until its been authenticated.
    And BTs authentication is not always foolproof.

    I do not know how a 3G auth system works at all, but if anyone does, id
    be interested to know.
     
    The Natural Philosopher, Jul 18, 2013
    #16
  17. Peter

    Graham J Guest

    In principle, yes; but A&A might have more than one DNS server to allow
    for failures. You need them to suggest an address in their network
    which will guarantee replies if their network is good, and not reply if
    something is wrong.

    Looking at the router of a client using A&A you might try their gateway
    IP = 81.187.81.187 - your router should show the current gateway IP next
    to your own IP.
    No, the TTL started at 64 and was decremented by one for each hop in
    your traceroute report. So it would go through a further 61 routers
    before it failed.

    If you used this for the failure detection, set TTL=2 because the
    traffic originates in your router and should go only via:

    c.gormless.thn.aa.net.uk [90.155.53.53]
    dns1.aa.net.uk [217.169.20.20]
    If you ping something outside the A&A network then a fault elsewhere on
    the internet will cause a failover. Not useful. If there is a fault
    elsewhere on the internet then the only logical resolution is a separate
    ISP and BGP - that way your traffic could be routed around a failure.
    This is how the internet is designed to achieve resilience.
     
    Graham J, Jul 18, 2013
    #17
  18. Peter

    Graham J Guest


    Exactly as I would expect. The modem has to recognise that its
    connection has failed, and tell the router probably by changing the
    behaviour of its Ethernet interface (in some way that will be documented
    in the modem manual) - so this takes time.

    If you power off the modem the router sees immediatley that the WAN
    interface has failed.
     
    Graham J, Jul 18, 2013
    #18
  19. Peter

    Graham J Guest

    It's even worse than that. BT's system might have given you the IP
    address of another A&A customer, so A&A doesn't know that anything is wrong!
     
    Graham J, Jul 18, 2013
    #19
  20. Peter

    John Weston Guest

    To prevent this, reconfigure your router to use a manual,
    locally configured address rather than using DHCP. You can
    do this with AAISP, since they provide fixed addresses for
    your account. If BT answer your PPP login request, then your
    asking them to use your AAISP address will fail and they
    reject the request. Your router keeps repeating the logon
    until it sees AAISP again. This works for me - on Saturday,
    I saw 8, 4 & 6 minute outages whilst BT were messing about
    and never got connected to their test network.

    For info about this BT outage, the OP should look at the
    AAISP status report. For the problem on Saturday, see:
    http://status.aa.net.uk/apost.cgi?incident=1819
     
    John Weston, Jul 18, 2013
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.