Load-balancing across four T1's on 2 routers

Discussion in 'Cisco' started by Sean-Usenet, Aug 31, 2006.

  1. Sean-Usenet

    Sean-Usenet Guest

    I am setting up the following:

    - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    - Those 2 routers are also connected to a 100mb layer-3 switch.
    - Our firewall will also connected to that layer-3 switch.
    - The firewall's' default gateway will be that layer-3 switch.
    - The workstations are behind the firewall, and will use the firewall
    as their default gateway

    - OSPF will be running on the 2 routers and the layer-3 switch, and
    also on 2 routers on the ISP's site.
    - The OSPF area will be Totally Stubby, thus the ISP's routers will be
    advertising default routes into our network.

    As long as all four T-1's are up, everything should work fine:

    - The workstations will route outbound packets to the firewall
    - The firewall will route the packets to the layer-3 switch
    - The layer-3 switch is running OSPF and will see two equal cost
    default routes, and will load-balance traffic between our two routers
    - The routers will in turn also have two defaults routes (1 route
    through each T-1), and load-balance traffic across each T-1


    My problem is what happens when one T-1 goes down? Our layer-3 switch
    will still see equal cost routes and split the traffic across the two
    routers, even though one router has 1/2 the bandwidth.

    Can someone help me with this problem? Please let me know if you have
    any questions on what I explained above! Here is a diagram of the
    setup, i hope it looks ok:

    R1 R2 (ISP Routers)
    || ||
    || || (4 total T-1s)
    || ||
    R1 R2 (Our Routers)
    | |
    \ /
    \ /
    \ /
    Layer-3
    Switch
    |
    |
    |
    Firewall
    |
    |
    |
    Layer-2
    Switch
    |
    |
    |
    |
    Workstations


    Thanks!
    Sean
    Sean-Usenet, Aug 31, 2006
    #1
    1. Advertising

  2. Sean-Usenet

    Merv Guest

    How many FastEthernet ports on your 2800 routers ?
    Merv, Aug 31, 2006
    #2
    1. Advertising

  3. Sean-Usenet

    Igor Mamuzic Guest

    Maybe you could find solution on one of these links:
    - if you have 12.3 IOS see:
    http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a00801d1e95.html
    or
    - if you have 12.4 IOS see:
    http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a00801d1e95.html


    B.R.
    Igor



    "Sean-Usenet" <> wrote in message
    news:...
    >I am setting up the following:
    >
    > - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    > - Those 2 routers are also connected to a 100mb layer-3 switch.
    > - Our firewall will also connected to that layer-3 switch.
    > - The firewall's' default gateway will be that layer-3 switch.
    > - The workstations are behind the firewall, and will use the firewall
    > as their default gateway
    >
    > - OSPF will be running on the 2 routers and the layer-3 switch, and
    > also on 2 routers on the ISP's site.
    > - The OSPF area will be Totally Stubby, thus the ISP's routers will be
    > advertising default routes into our network.
    >
    > As long as all four T-1's are up, everything should work fine:
    >
    > - The workstations will route outbound packets to the firewall
    > - The firewall will route the packets to the layer-3 switch
    > - The layer-3 switch is running OSPF and will see two equal cost
    > default routes, and will load-balance traffic between our two routers
    > - The routers will in turn also have two defaults routes (1 route
    > through each T-1), and load-balance traffic across each T-1
    >
    >
    > My problem is what happens when one T-1 goes down? Our layer-3 switch
    > will still see equal cost routes and split the traffic across the two
    > routers, even though one router has 1/2 the bandwidth.
    >
    > Can someone help me with this problem? Please let me know if you have
    > any questions on what I explained above! Here is a diagram of the
    > setup, i hope it looks ok:
    >
    > R1 R2 (ISP Routers)
    > || ||
    > || || (4 total T-1s)
    > || ||
    > R1 R2 (Our Routers)
    > | |
    > \ /
    > \ /
    > \ /
    > Layer-3
    > Switch
    > |
    > |
    > |
    > Firewall
    > |
    > |
    > |
    > Layer-2
    > Switch
    > |
    > |
    > |
    > |
    > Workstations
    >
    >
    > Thanks!
    > Sean
    >
    Igor Mamuzic, Aug 31, 2006
    #3
  4. Sean-Usenet

    James Guest

    I am 99% sure that your layer three switch will see four equal cost
    routes not two, when one T1 goes down it will then see three routes.
    Your layer three switch will take this into consideration when making
    its balancing decision.

    James


    Igor Mamuzic wrote:
    > Maybe you could find solution on one of these links:
    > - if you have 12.3 IOS see:
    > http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a00801d1e95.html
    > or
    > - if you have 12.4 IOS see:
    > http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a00801d1e95.html
    >
    >
    > B.R.
    > Igor
    >
    >
    >
    > "Sean-Usenet" <> wrote in message
    > news:...
    > >I am setting up the following:
    > >
    > > - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    > > - Those 2 routers are also connected to a 100mb layer-3 switch.
    > > - Our firewall will also connected to that layer-3 switch.
    > > - The firewall's' default gateway will be that layer-3 switch.
    > > - The workstations are behind the firewall, and will use the firewall
    > > as their default gateway
    > >
    > > - OSPF will be running on the 2 routers and the layer-3 switch, and
    > > also on 2 routers on the ISP's site.
    > > - The OSPF area will be Totally Stubby, thus the ISP's routers will be
    > > advertising default routes into our network.
    > >
    > > As long as all four T-1's are up, everything should work fine:
    > >
    > > - The workstations will route outbound packets to the firewall
    > > - The firewall will route the packets to the layer-3 switch
    > > - The layer-3 switch is running OSPF and will see two equal cost
    > > default routes, and will load-balance traffic between our two routers
    > > - The routers will in turn also have two defaults routes (1 route
    > > through each T-1), and load-balance traffic across each T-1
    > >
    > >
    > > My problem is what happens when one T-1 goes down? Our layer-3 switch
    > > will still see equal cost routes and split the traffic across the two
    > > routers, even though one router has 1/2 the bandwidth.
    > >
    > > Can someone help me with this problem? Please let me know if you have
    > > any questions on what I explained above! Here is a diagram of the
    > > setup, i hope it looks ok:
    > >
    > > R1 R2 (ISP Routers)
    > > || ||
    > > || || (4 total T-1s)
    > > || ||
    > > R1 R2 (Our Routers)
    > > | |
    > > \ /
    > > \ /
    > > \ /
    > > Layer-3
    > > Switch
    > > |
    > > |
    > > |
    > > Firewall
    > > |
    > > |
    > > |
    > > Layer-2
    > > Switch
    > > |
    > > |
    > > |
    > > |
    > > Workstations
    > >
    > >
    > > Thanks!
    > > Sean
    > >
    James, Aug 31, 2006
    #4
  5. Sean-Usenet

    Merv Guest

    There may be some additional things to consider ...

    What happens if an ISP upstream router becomes partitiononed from the
    rest of the ISP network - the T1 will stay up but your traffic will be
    blackholed - believe it happens.

    Also what approach is being planned to load balance the traffic across
    each of the pairs of T1s ?
    Merv, Aug 31, 2006
    #5
  6. Sean-Usenet

    Merv Guest


    > I am 99% sure that your layer three switch will see four equal cost routes not two


    That depends.

    It would be true if the T1s are not bundled and a default route is
    configured to point to next hop on each of the two T1's

    However if MLPPP we used to bundle the T1's for load balanicng then
    there would only be one default route per 2800 and thus only two in
    total seen by the layer 3 switch.
    Merv, Aug 31, 2006
    #6
  7. Sean-Usenet

    Sean-Usenet Guest

    Hello

    There are 2 FE ports on each router, with only 1 FE port in use.


    Merv wrote:
    > How many FastEthernet ports on your 2800 routers ?
    Sean-Usenet, Aug 31, 2006
    #7
  8. Sean-Usenet

    Sean-Usenet Guest

    We do not plan on using MLPPP. As I understand, our layer-3 switch
    will only have 2 default route entires - 1 for each router, not 4
    default route entries - 2 from each router. Isn't that not correct?

    Merv wrote:
    > > I am 99% sure that your layer three switch will see four equal cost routes not two

    >
    > That depends.
    >
    > It would be true if the T1s are not bundled and a default route is
    > configured to point to next hop on each of the two T1's
    >
    > However if MLPPP we used to bundle the T1's for load balanicng then
    > there would only be one default route per 2800 and thus only two in
    > total seen by the layer 3 switch.
    Sean-Usenet, Aug 31, 2006
    #8
  9. Sean-Usenet

    Merv Guest

    What is the origin of default route on each 2800 ?

    Is it provided by the ISP via a dynamic routing protocol ?

    Or is it via static routes configured on the 2800 ?
    Merv, Aug 31, 2006
    #9
  10. Sean-Usenet

    Merv Guest


    > We do not plan on using MLPPP.


    That being the case and assuming you will be using CEF, be aware that
    the two T1 will not be evenly load balanced in real time as CEF does
    per destination load balancing.
    Merv, Aug 31, 2006
    #10
  11. Sean-Usenet

    Sean-Usenet Guest

    The ISP's routers will be ABRs, and our area will be configured as a
    totally stubby network. Because of that the ABR will automatically
    inject the default routes into our area.

    Merv wrote:
    > What is the origin of default route on each 2800 ?
    >
    > Is it provided by the ISP via a dynamic routing protocol ?
    >
    > Or is it via static routes configured on the 2800 ?
    Sean-Usenet, Aug 31, 2006
    #11
  12. Sean-Usenet

    Sean-Usenet Guest

    Hi again Merv, thanks for helping me out with this.

    Yea, i understand that by default CEF is per desination-source, but
    there is an option to switch it to per packet, which we may use.

    Merv wrote:
    > > We do not plan on using MLPPP.

    >
    > That being the case and assuming you will be using CEF, be aware that
    > the two T1 will not be evenly load balanced in real time as CEF does
    > per destination load balancing.
    Sean-Usenet, Aug 31, 2006
    #12
  13. Sean-Usenet wrote:
    > I am setting up the following:
    >
    > - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    > - Those 2 routers are also connected to a 100mb layer-3 switch.
    > - Our firewall will also connected to that layer-3 switch.
    > - The firewall's' default gateway will be that layer-3 switch.
    > - The workstations are behind the firewall, and will use the firewall
    > as their default gateway


    Well, if I were setting this up, I'm not sure I would need to use the
    multilayer capabilities of the layer-3 switch. Is the Cisco 2800
    capable of GLBP? If so, I would set up GLBP on both of the routers, and
    make the load-balanced gateway address the default route for the
    firewall. And then the routers can weigh their traffic capabilities and
    load balance themselves.

    Merv does bring up a good point about needing to mitigate the effects
    of the ISP losing connectivity.
    Nathan Harmon, Aug 31, 2006
    #13
  14. Sean-Usenet

    Merv Guest

    BTW is it one ISP or two ?
    Merv, Aug 31, 2006
    #14
  15. Sean-Usenet

    Sean-Usenet Guest

    Hi Nathan

    I looked a little at using GLBP, but I was concerned about how well it
    would load-balance, since all traffic is going through the firewall.

    - When the firewall receives its first packet, it will ARP for the mac
    of the default gateway
    - The GLBP AVG will respond to the arp request with the virtual mac of
    itself or the other router
    - Then the firewall will add this arp response it its arp cache and
    forward the data packet
    - Since the arp response is now stored in the firewall's arp cache, it
    will not arp again until it expires, thus it will continue to use the
    same router

    In other words, GLBP load-balances on a per source host basis, and
    unfortunetly becaues of the firewall there is only 1 host.

    Does that make sense, or is my logic off somewhere?


    Nathan Harmon wrote:
    > Sean-Usenet wrote:
    > > I am setting up the following:
    > >
    > > - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    > > - Those 2 routers are also connected to a 100mb layer-3 switch.
    > > - Our firewall will also connected to that layer-3 switch.
    > > - The firewall's' default gateway will be that layer-3 switch.
    > > - The workstations are behind the firewall, and will use the firewall
    > > as their default gateway

    >
    > Well, if I were setting this up, I'm not sure I would need to use the
    > multilayer capabilities of the layer-3 switch. Is the Cisco 2800
    > capable of GLBP? If so, I would set up GLBP on both of the routers, and
    > make the load-balanced gateway address the default route for the
    > firewall. And then the routers can weigh their traffic capabilities and
    > load balance themselves.
    >
    > Merv does bring up a good point about needing to mitigate the effects
    > of the ISP losing connectivity.
    Sean-Usenet, Aug 31, 2006
    #15
  16. Sean-Usenet

    Sean-Usenet Guest

    It is the same ISP

    Merv wrote:
    > BTW is it one ISP or two ?
    Sean-Usenet, Aug 31, 2006
    #16
  17. Sean-Usenet

    Sean-Usenet Guest

    If one of the two ISP routers does come partitioned (eg. its FE port
    fails) won't it stop sending a default route down the T1s to us?

    Since the ISPs routers are configured as ABR and our area is a totally
    stubby area, the ISPs routers will send a default route to us
    automatically. Will the ISP's router continue to send a default route
    even though all its other interfaces are down?


    The traffic will be load-balanced across the pair of T1s via equal-cost
    load-balancing because of OSPF


    Merv wrote:
    > There may be some additional things to consider ...
    >
    > What happens if an ISP upstream router becomes partitiononed from the
    > rest of the ISP network - the T1 will stay up but your traffic will be
    > blackholed - believe it happens.
    >
    > Also what approach is being planned to load balance the traffic across
    > each of the pairs of T1s ?
    Sean-Usenet, Aug 31, 2006
    #17
  18. Sean-Usenet

    Guest

    Sean-Usenet wrote:
    > It is the same ISP
    >
    > Merv wrote:
    > > BTW is it one ISP or two ?


    Sean

    I am sorry for stupid question, but I just can't resist. WHY all that
    hustle with 4 T1's without MLPP, 2 routers + OSPF, if you have just ONE
    provider. As far as I understand, you are trying to "invent the wheel",
    which is design "indestructible" Internet access, or am I wrong? If
    not, then WHY you want a SINGLE L3 switch (which you don't need) +
    SINGLE firewall?

    Roman
    , Aug 31, 2006
    #18
  19. Sean-Usenet

    Sean-Usenet Guest

    Haha, well I dumbed down the full setup a little bit for simplicity
    sake.

    It is actually not a single L3 switch, it is two L3 switches with
    redundant 32Gbps interconnects between them. With 1 router going to
    each switch. The L3 switch is needed because the firewall is not setup
    to run OSPF.

    The firewall is not a single firewall, it is an active/passive firewall
    cluster. One firewall connects to one of the above L3 switches and one
    firewall connects to the other L3 switch.

    Here is the reason for not using MLPP:

    If all four T1s are up, everything would work fine with MLPP. Each
    router would see a 3Mb connection. The L3 switch would have 2 default
    routes in its routing table, and perform equal-cost load-balancing.
    The traffic would also load-balance very nicely across the T1s because
    of MLPP.

    The problem is if we lose one T1. At that point, one router has a
    1.5Mb connection and the other still has a 3Mb connection. The L3
    switch will then see 2 UN-equal cost default routes in its routing
    table. Because they are not equal-cost routes, all traffic would be
    directed to the router that has two operational T1s. The end result is
    the same as loosing two T1s even though we only lost 1. OSPF only
    performs equal-cost load-balancing.

    Without using MLPP, the L3 switch will still see two equal-cost default
    routes and route traffic to both routers. Unless of course a router
    looses both T1s, then it won't receive any traffic since it won't be
    passing along the default route from the ABR any longer.

    The reason for using a L3 switch and not GLBP on the routers is because
    GLBP load-balances on a per source-host basis. Since the source host
    is always the firewall, the traffic will always go through the same
    router.
    http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122newft/122t/122t15/ft_glbp.htm


    Now that I answered your question, any help with mine? :)

    Thanks
    Sean


    wrote:
    > Sean-Usenet wrote:
    > > It is the same ISP
    > >
    > > Merv wrote:
    > > > BTW is it one ISP or two ?

    >
    > Sean
    >
    > I am sorry for stupid question, but I just can't resist. WHY all that
    > hustle with 4 T1's without MLPP, 2 routers + OSPF, if you have just ONE
    > provider. As far as I understand, you are trying to "invent the wheel",
    > which is design "indestructible" Internet access, or am I wrong? If
    > not, then WHY you want a SINGLE L3 switch (which you don't need) +
    > SINGLE firewall?
    >
    > Roman
    Sean-Usenet, Sep 1, 2006
    #19
  20. Sean-Usenet

    Sean-Usenet Guest

    Hi James

    Thanks for the response.

    Actually the L3 switch will only see 2 equal-cost routes. The L3
    switch will show the 1 default route with a next hop of 1 router and a
    2nd default route with a next hop of the other router.

    I mocked this up in my lab to be 100% sure. Although, it would have
    been great if the L3 switch did see 4 routes!

    Sean

    James wrote:
    > I am 99% sure that your layer three switch will see four equal cost
    > routes not two, when one T1 goes down it will then see three routes.
    > Your layer three switch will take this into consideration when making
    > its balancing decision.
    >
    > James
    >
    >
    > Igor Mamuzic wrote:
    > > Maybe you could find solution on one of these links:
    > > - if you have 12.3 IOS see:
    > > http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a00801d1e95.html
    > > or
    > > - if you have 12.4 IOS see:
    > > http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_feature_guide09186a00801d1e95.html
    > >
    > >
    > > B.R.
    > > Igor
    > >
    > >
    > >
    > > "Sean-Usenet" <> wrote in message
    > > news:...
    > > >I am setting up the following:
    > > >
    > > > - 2 Cisco 2800 series routers, each has two T-1 internet connections.
    > > > - Those 2 routers are also connected to a 100mb layer-3 switch.
    > > > - Our firewall will also connected to that layer-3 switch.
    > > > - The firewall's' default gateway will be that layer-3 switch.
    > > > - The workstations are behind the firewall, and will use the firewall
    > > > as their default gateway
    > > >
    > > > - OSPF will be running on the 2 routers and the layer-3 switch, and
    > > > also on 2 routers on the ISP's site.
    > > > - The OSPF area will be Totally Stubby, thus the ISP's routers will be
    > > > advertising default routes into our network.
    > > >
    > > > As long as all four T-1's are up, everything should work fine:
    > > >
    > > > - The workstations will route outbound packets to the firewall
    > > > - The firewall will route the packets to the layer-3 switch
    > > > - The layer-3 switch is running OSPF and will see two equal cost
    > > > default routes, and will load-balance traffic between our two routers
    > > > - The routers will in turn also have two defaults routes (1 route
    > > > through each T-1), and load-balance traffic across each T-1
    > > >
    > > >
    > > > My problem is what happens when one T-1 goes down? Our layer-3 switch
    > > > will still see equal cost routes and split the traffic across the two
    > > > routers, even though one router has 1/2 the bandwidth.
    > > >
    > > > Can someone help me with this problem? Please let me know if you have
    > > > any questions on what I explained above! Here is a diagram of the
    > > > setup, i hope it looks ok:
    > > >
    > > > R1 R2 (ISP Routers)
    > > > || ||
    > > > || || (4 total T-1s)
    > > > || ||
    > > > R1 R2 (Our Routers)
    > > > | |
    > > > \ /
    > > > \ /
    > > > \ /
    > > > Layer-3
    > > > Switch
    > > > |
    > > > |
    > > > |
    > > > Firewall
    > > > |
    > > > |
    > > > |
    > > > Layer-2
    > > > Switch
    > > > |
    > > > |
    > > > |
    > > > |
    > > > Workstations
    > > >
    > > >
    > > > Thanks!
    > > > Sean
    > > >
    Sean-Usenet, Sep 1, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Darren Green
    Replies:
    5
    Views:
    738
    Hansang Bae
    Jun 24, 2004
  2. Jason
    Replies:
    2
    Views:
    2,434
    Jason
    Oct 22, 2004
  3. Matt
    Replies:
    1
    Views:
    650
    Vincent C Jones
    Oct 17, 2005
  4. Matt
    Replies:
    1
    Views:
    4,034
  5. Big Phil
    Replies:
    3
    Views:
    1,705
    NetExpert
    May 1, 2007
Loading...

Share This Page