OSPF & CEF next-hops with regards to load balancing.

My current topology is as follows. We have an MSFC that is the default
gateway for several internal VLANs. The MSFC also has a "WAN Access"
VLAN where two WAN routers (call them RouterA and RouterB) sit. Each
of these routers has a T3 to a single router at our remote site. We
are running OSPF on the "WAN Access" VLAN, and the two routers and
MSFC have an established adjacency. When packets hit the MSFC destined
for subnets at the remote site, they have an OSPF equal cost path
using either RouterA or RouterB as the next hop. However, CEF is also
coming into play. For specific source/destination pairs CEF stores one
of the next-hops as its preferred path and naturally always uses this.
The problem is that during periods of heavy backup traffic between a
particular source/destination, the "preferred" path causes one of the
T3s to become fully utilized while the other can remain very
underutilized. Can anyone recommend a way to even this traffic out ?
Is it even possible given the topology and use of OSPF with CEF ?


Router A ----(T3 WAN)--------|
| |
| |
MSFC--(Eth)---- Remote Router C
| |
| |
Router B ----(T3 WAN)--------|

 

You need to enable CEF per-packet load-balancing.

Here is the first description I came across -
http://www.web-articles.info/e/a/title/Load-Balancing-in-CEF/
I recommend that you read Cisco's own material.

It may be that not all platforms actually support
this. I some ways it is best just to try it if you have
some down-time since this sort of thing (specific hardware
support) can be rather trying to figure out with 100%
certainty from the documentation.

I am pretty sure the router will be OK but the 6500
is the one I am worried about.

 

I thought that per packet load sharing only applies where there are
two links on the same router that is making the routing decision. In
my topology, the MFSC does not have the WAN links....its next-hops are
the routers on the ethernet segment. Will this still work ?

 

I should have mentioned that the possible out of order
packets could affect the backup system or other
applications adversely. You need to check that it
does not get too unhappy. For example
you could get a *huge* number of unnecessarily
re-transmitted packets.

You might consider as an alternative configuring
QoS to prioritise your non-backup traffic.

Finally, remember that TCP in particular is very
robust in the presence of oversubscription
and my initial reaction would be to consider
leaving it as it is unless symptoms (user complaints)
were present.

 

Well it applies wherever any router has two
equal cost (or otherwise too) paths.
It doesn't care how many hops there are.
Your MSFC seems to conform.

 

Well it applies wherever any router has two

I see..so even though the command "ip load-sharing per-packet" is
applied on the interface, it really load balances across routing table
next-hops via that interface, and doesn't necessarily need multiple
directly connected interfaces as next-hops to function. Is that
correct ?





..

 

This is a very common "problem", but not easy to resolve. You can try per
packet load-balancing as described in the other thread, but this can result
in other problems which is why it is not the default. Per-packet
load-balancing is used when you have equal-cost paths to the same
destination, it doesn't matter if they are directly connected or through
other routers.

Their are several ways to solve the problem but they all have downsides or
differnet levels of complexity. You need to determine which is the most
palatable.
1) Do nothing. Unless the backups are impacting other traffic its not a
problem.

2) Implement QoS so that backup traffic is a lower priority other traffic.
That way if the the backup traffic won't affect your other traffic during
times of congestion. This is good idea even if you choose another solution
because if one of the T3's goes down you will be in the same boat.

3) Upgrade the bandwidth on the circuits so that each one can carry the full
load. This is the simplest and best solution, but the downside is the extra
cost. You should look into this as a solution because I have found that
100MB WAN connections are the same (and sometimes cheaper) than a T3. WAN
bandwidth costs have been dropping like a rock over the years, and my
company is currently getting 1GB/s pipes for less than what a T3 cost 3
years ago.

4) If you have several destination networks at the remote side, you can play
with routing metrics sent by Router C so that Router A is prefered on some
destinations and Router B is preferred on others. This is complicated and
will require you to manually tune the routers. Not a good idea.

5) Implement per-packet or per-destination load-balancing. This could
cause other problems because packets can arrive out of order. (This is
supported on the 6500 in hardware (if you have the right supervisor and IOS
version). If you implement this, you need to keep in mind that weird
problems may result, and its not in any way obvious that out-of-order
packets is the culprit. Symptoms could be retransmissions, slow data rates,
reset connections, and any number of other issues. Not saying it will
happen, but you need to be aware of it. It might be that only a single
applicatation out of hundreds is affected.