OSPF & CEF next-hops with regards to load balancing.

Discussion in 'Cisco' started by tmcgov05, Feb 9, 2009.

  1. tmcgov05

    tmcgov05 Guest

    My current topology is as follows. We have an MSFC that is the default
    gateway for several internal VLANs. The MSFC also has a "WAN Access"
    VLAN where two WAN routers (call them RouterA and RouterB) sit. Each
    of these routers has a T3 to a single router at our remote site. We
    are running OSPF on the "WAN Access" VLAN, and the two routers and
    MSFC have an established adjacency. When packets hit the MSFC destined
    for subnets at the remote site, they have an OSPF equal cost path
    using either RouterA or RouterB as the next hop. However, CEF is also
    coming into play. For specific source/destination pairs CEF stores one
    of the next-hops as its preferred path and naturally always uses this.
    The problem is that during periods of heavy backup traffic between a
    particular source/destination, the "preferred" path causes one of the
    T3s to become fully utilized while the other can remain very
    underutilized. Can anyone recommend a way to even this traffic out ?
    Is it even possible given the topology and use of OSPF with CEF ?

    Router A ----(T3 WAN)--------|
    | |
    | |
    MSFC--(Eth)---- Remote Router C
    | |
    | |
    Router B ----(T3 WAN)--------|
    tmcgov05, Feb 9, 2009
    1. Advertisements

  2. tmcgov05

    bod43 Guest

    You need to enable CEF per-packet load-balancing.

    Here is the first description I came across -
    I recommend that you read Cisco's own material.

    It may be that not all platforms actually support
    this. I some ways it is best just to try it if you have
    some down-time since this sort of thing (specific hardware
    support) can be rather trying to figure out with 100%
    certainty from the documentation.

    I am pretty sure the router will be OK but the 6500
    is the one I am worried about.
    bod43, Feb 9, 2009
    1. Advertisements

  3. tmcgov05

    tmcgov05 Guest

    I thought that per packet load sharing only applies where there are
    two links on the same router that is making the routing decision. In
    my topology, the MFSC does not have the WAN links....its next-hops are
    the routers on the ethernet segment. Will this still work ?
    tmcgov05, Feb 9, 2009
  4. tmcgov05

    bod43 Guest

    I should have mentioned that the possible out of order
    packets could affect the backup system or other
    applications adversely. You need to check that it
    does not get too unhappy. For example
    you could get a *huge* number of unnecessarily
    re-transmitted packets.

    You might consider as an alternative configuring
    QoS to prioritise your non-backup traffic.

    Finally, remember that TCP in particular is very
    robust in the presence of oversubscription
    and my initial reaction would be to consider
    leaving it as it is unless symptoms (user complaints)
    were present.
    bod43, Feb 9, 2009
  5. tmcgov05

    bod43 Guest

    Well it applies wherever any router has two
    equal cost (or otherwise too) paths.
    It doesn't care how many hops there are.
    Your MSFC seems to conform.
    bod43, Feb 9, 2009
  6. tmcgov05

    tmcgov05 Guest

    Well it applies wherever any router has two
    I see..so even though the command "ip load-sharing per-packet" is
    applied on the interface, it really load balances across routing table
    next-hops via that interface, and doesn't necessarily need multiple
    directly connected interfaces as next-hops to function. Is that
    correct ?

    tmcgov05, Feb 10, 2009
  7. tmcgov05

    Thrill5 Guest

    This is a very common "problem", but not easy to resolve. You can try per
    packet load-balancing as described in the other thread, but this can result
    in other problems which is why it is not the default. Per-packet
    load-balancing is used when you have equal-cost paths to the same
    destination, it doesn't matter if they are directly connected or through
    other routers.

    Their are several ways to solve the problem but they all have downsides or
    differnet levels of complexity. You need to determine which is the most
    1) Do nothing. Unless the backups are impacting other traffic its not a

    2) Implement QoS so that backup traffic is a lower priority other traffic.
    That way if the the backup traffic won't affect your other traffic during
    times of congestion. This is good idea even if you choose another solution
    because if one of the T3's goes down you will be in the same boat.

    3) Upgrade the bandwidth on the circuits so that each one can carry the full
    load. This is the simplest and best solution, but the downside is the extra
    cost. You should look into this as a solution because I have found that
    100MB WAN connections are the same (and sometimes cheaper) than a T3. WAN
    bandwidth costs have been dropping like a rock over the years, and my
    company is currently getting 1GB/s pipes for less than what a T3 cost 3
    years ago.

    4) If you have several destination networks at the remote side, you can play
    with routing metrics sent by Router C so that Router A is prefered on some
    destinations and Router B is preferred on others. This is complicated and
    will require you to manually tune the routers. Not a good idea.

    5) Implement per-packet or per-destination load-balancing. This could
    cause other problems because packets can arrive out of order. (This is
    supported on the 6500 in hardware (if you have the right supervisor and IOS
    version). If you implement this, you need to keep in mind that weird
    problems may result, and its not in any way obvious that out-of-order
    packets is the culprit. Symptoms could be retransmissions, slow data rates,
    reset connections, and any number of other issues. Not saying it will
    happen, but you need to be aware of it. It might be that only a single
    applicatation out of hundreds is affected.
    Thrill5, Feb 10, 2009
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.