Load-balancing across four T1's on 2 routers

Discussion in 'Cisco' started by Sean-Usenet, Aug 31, 2006.

  1. Sean-Usenet

    Sean-Usenet Guest

    I am setting up the following:

    - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    - Those 2 routers are also connected to a 100mb layer-3 switch.
    - Our firewall will also connected to that layer-3 switch.
    - The firewall's' default gateway will be that layer-3 switch.
    - The workstations are behind the firewall, and will use the firewall
    as their default gateway

    - OSPF will be running on the 2 routers and the layer-3 switch, and
    also on 2 routers on the ISP's site.
    - The OSPF area will be Totally Stubby, thus the ISP's routers will be
    advertising default routes into our network.

    As long as all four T-1's are up, everything should work fine:

    - The workstations will route outbound packets to the firewall
    - The firewall will route the packets to the layer-3 switch
    - The layer-3 switch is running OSPF and will see two equal cost
    default routes, and will load-balance traffic between our two routers
    - The routers will in turn also have two defaults routes (1 route
    through each T-1), and load-balance traffic across each T-1

    My problem is what happens when one T-1 goes down? Our layer-3 switch
    will still see equal cost routes and split the traffic across the two
    routers, even though one router has 1/2 the bandwidth.

    Can someone help me with this problem? Please let me know if you have
    any questions on what I explained above! Here is a diagram of the
    setup, i hope it looks ok:

    R1 R2 (ISP Routers)
    || ||
    || || (4 total T-1s)
    || ||
    R1 R2 (Our Routers)
    | |
    \ /
    \ /
    \ /

    Sean-Usenet, Aug 31, 2006
    1. Advertisements

  2. Sean-Usenet

    Merv Guest

    How many FastEthernet ports on your 2800 routers ?
    Merv, Aug 31, 2006
    1. Advertisements

  3. Sean-Usenet

    Igor Mamuzic Guest

    Igor Mamuzic, Aug 31, 2006
  4. Sean-Usenet

    James Guest

    I am 99% sure that your layer three switch will see four equal cost
    routes not two, when one T1 goes down it will then see three routes.
    Your layer three switch will take this into consideration when making
    its balancing decision.

    James, Aug 31, 2006
  5. Sean-Usenet

    Merv Guest

    There may be some additional things to consider ...

    What happens if an ISP upstream router becomes partitiononed from the
    rest of the ISP network - the T1 will stay up but your traffic will be
    blackholed - believe it happens.

    Also what approach is being planned to load balance the traffic across
    each of the pairs of T1s ?
    Merv, Aug 31, 2006
  6. Sean-Usenet

    Merv Guest

    That depends.

    It would be true if the T1s are not bundled and a default route is
    configured to point to next hop on each of the two T1's

    However if MLPPP we used to bundle the T1's for load balanicng then
    there would only be one default route per 2800 and thus only two in
    total seen by the layer 3 switch.
    Merv, Aug 31, 2006
  7. Sean-Usenet

    Sean-Usenet Guest


    There are 2 FE ports on each router, with only 1 FE port in use.
    Sean-Usenet, Aug 31, 2006
  8. Sean-Usenet

    Sean-Usenet Guest

    We do not plan on using MLPPP. As I understand, our layer-3 switch
    will only have 2 default route entires - 1 for each router, not 4
    default route entries - 2 from each router. Isn't that not correct?
    Sean-Usenet, Aug 31, 2006
  9. Sean-Usenet

    Merv Guest

    What is the origin of default route on each 2800 ?

    Is it provided by the ISP via a dynamic routing protocol ?

    Or is it via static routes configured on the 2800 ?
    Merv, Aug 31, 2006
  10. Sean-Usenet

    Merv Guest

    That being the case and assuming you will be using CEF, be aware that
    the two T1 will not be evenly load balanced in real time as CEF does
    per destination load balancing.
    Merv, Aug 31, 2006
  11. Sean-Usenet

    Sean-Usenet Guest

    The ISP's routers will be ABRs, and our area will be configured as a
    totally stubby network. Because of that the ABR will automatically
    inject the default routes into our area.
    Sean-Usenet, Aug 31, 2006
  12. Sean-Usenet

    Sean-Usenet Guest

    Hi again Merv, thanks for helping me out with this.

    Yea, i understand that by default CEF is per desination-source, but
    there is an option to switch it to per packet, which we may use.
    Sean-Usenet, Aug 31, 2006
  13. Well, if I were setting this up, I'm not sure I would need to use the
    multilayer capabilities of the layer-3 switch. Is the Cisco 2800
    capable of GLBP? If so, I would set up GLBP on both of the routers, and
    make the load-balanced gateway address the default route for the
    firewall. And then the routers can weigh their traffic capabilities and
    load balance themselves.

    Merv does bring up a good point about needing to mitigate the effects
    of the ISP losing connectivity.
    Nathan Harmon, Aug 31, 2006
  14. Sean-Usenet

    Merv Guest

    BTW is it one ISP or two ?
    Merv, Aug 31, 2006
  15. Sean-Usenet

    Sean-Usenet Guest

    Hi Nathan

    I looked a little at using GLBP, but I was concerned about how well it
    would load-balance, since all traffic is going through the firewall.

    - When the firewall receives its first packet, it will ARP for the mac
    of the default gateway
    - The GLBP AVG will respond to the arp request with the virtual mac of
    itself or the other router
    - Then the firewall will add this arp response it its arp cache and
    forward the data packet
    - Since the arp response is now stored in the firewall's arp cache, it
    will not arp again until it expires, thus it will continue to use the
    same router

    In other words, GLBP load-balances on a per source host basis, and
    unfortunetly becaues of the firewall there is only 1 host.

    Does that make sense, or is my logic off somewhere?
    Sean-Usenet, Aug 31, 2006
  16. Sean-Usenet

    Sean-Usenet Guest

    It is the same ISP
    Sean-Usenet, Aug 31, 2006
  17. Sean-Usenet

    Sean-Usenet Guest

    If one of the two ISP routers does come partitioned (eg. its FE port
    fails) won't it stop sending a default route down the T1s to us?

    Since the ISPs routers are configured as ABR and our area is a totally
    stubby area, the ISPs routers will send a default route to us
    automatically. Will the ISP's router continue to send a default route
    even though all its other interfaces are down?

    The traffic will be load-balanced across the pair of T1s via equal-cost
    load-balancing because of OSPF
    Sean-Usenet, Aug 31, 2006
  18. Sean-Usenet

    nakhmanson Guest


    I am sorry for stupid question, but I just can't resist. WHY all that
    hustle with 4 T1's without MLPP, 2 routers + OSPF, if you have just ONE
    provider. As far as I understand, you are trying to "invent the wheel",
    which is design "indestructible" Internet access, or am I wrong? If
    not, then WHY you want a SINGLE L3 switch (which you don't need) +
    SINGLE firewall?

    nakhmanson, Aug 31, 2006
  19. Sean-Usenet

    Sean-Usenet Guest

    Haha, well I dumbed down the full setup a little bit for simplicity

    It is actually not a single L3 switch, it is two L3 switches with
    redundant 32Gbps interconnects between them. With 1 router going to
    each switch. The L3 switch is needed because the firewall is not setup
    to run OSPF.

    The firewall is not a single firewall, it is an active/passive firewall
    cluster. One firewall connects to one of the above L3 switches and one
    firewall connects to the other L3 switch.

    Here is the reason for not using MLPP:

    If all four T1s are up, everything would work fine with MLPP. Each
    router would see a 3Mb connection. The L3 switch would have 2 default
    routes in its routing table, and perform equal-cost load-balancing.
    The traffic would also load-balance very nicely across the T1s because
    of MLPP.

    The problem is if we lose one T1. At that point, one router has a
    1.5Mb connection and the other still has a 3Mb connection. The L3
    switch will then see 2 UN-equal cost default routes in its routing
    table. Because they are not equal-cost routes, all traffic would be
    directed to the router that has two operational T1s. The end result is
    the same as loosing two T1s even though we only lost 1. OSPF only
    performs equal-cost load-balancing.

    Without using MLPP, the L3 switch will still see two equal-cost default
    routes and route traffic to both routers. Unless of course a router
    looses both T1s, then it won't receive any traffic since it won't be
    passing along the default route from the ABR any longer.

    The reason for using a L3 switch and not GLBP on the routers is because
    GLBP load-balances on a per source-host basis. Since the source host
    is always the firewall, the traffic will always go through the same

    Now that I answered your question, any help with mine? :)

    Sean-Usenet, Sep 1, 2006
  20. Sean-Usenet

    Sean-Usenet Guest

    Hi James

    Thanks for the response.

    Actually the L3 switch will only see 2 equal-cost routes. The L3
    switch will show the 1 default route with a next hop of 1 router and a
    2nd default route with a next hop of the other router.

    I mocked this up in my lab to be 100% sure. Although, it would have
    been great if the L3 switch did see 4 routes!

    Sean-Usenet, Sep 1, 2006
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.