«

Hello everyone,

We have a complete switched network with VTP domain with many VLAN's and
having 2x Cisco 6509 and 34x 3524 inline power which are connected in
a chain 3 till 5 Cisco 3524's between the 6509 switches. The 6509
switches are connected with an etherchannel of 2 GigaBps.

Spontaneous at a completely random time we have a complete overall
network failure for 1 or 2 seconds. Enough to disconnect telephone calls
(Cisco VoIP) and data Cluster failures. (and complaining colleagues).

I try to figure out what the problem is by sending all logging from all
switches to a UNIX syslog. But I cannot find a logging which leads to
this problem. Almost every network counters of all ports seams to be
allright, no drop, overrun etc...

What could it possible be the problem? Can someone give me a hint were
to look for or how to trace?

Kind regards Michel.

Just a wild guess, with out the topology and configs it is very difficult to answer. Do you have
spanning tree turned-on on all your switched interfaces where there is a potential loop ? Check for
any interface flaps in the debugs which might create a broadcast storm making the network
unreachable for second or so.

Hope this helps,
Vidyaranya Maddi.

Michel wrote:

> Hello everyone,
>
> We have a complete switched network with VTP domain with many VLAN's and
> having 2x Cisco 6509 and 34x 3524 inline power which are connected in a
> chain 3 till 5 Cisco 3524's between the 6509 switches. The 6509 switches
> are connected with an etherchannel of 2 GigaBps.
>
> Spontaneous at a completely random time we have a complete overall
> network failure for 1 or 2 seconds. Enough to disconnect telephone calls
> (Cisco VoIP) and data Cluster failures. (and complaining colleagues).
>
> I try to figure out what the problem is by sending all logging from all
> switches to a UNIX syslog. But I cannot find a logging which leads to
> this problem. Almost every network counters of all ports seams to be
> allright, no drop, overrun etc...
>
> What could it possible be the problem? Can someone give me a hint were
> to look for or how to trace?
>
> Kind regards Michel.
>
>
>

> Just a wild guess, with out the topology and configs it is very
> difficult to answer. Do you have spanning tree turned-on on all your
> switched interfaces where there is a potential loop ? Check for any
> interface flaps in the debugs which might create a broadcast storm
> making the network unreachable for second or so.

Yes I use Spanning-tree.. On every Ethernet port I user "Spanning-tree
portfast". But, I have a couple of ports, users... which the port is
flapping. I see these in my log:

%RTD-1-LINK_FLAP: FastEthernet 0/1 link down/up 5 times per min\par

So this could be the cause of my problems? I will hunt them down

thanks! michel.

You might wnat to just shut this guy down and see if the problem goes away. If yes, you fould the
culprit. Let me know how it goes.

-Maddi.

Michel wrote:

>
>
>> Just a wild guess, with out the topology and configs it is very
>> difficult to answer. Do you have spanning tree turned-on on all your
>> switched interfaces where there is a potential loop ? Check for any
>> interface flaps in the debugs which might create a broadcast storm
>> making the network unreachable for second or so.
>
>
> Yes I use Spanning-tree.. On every Ethernet port I user "Spanning-tree
> portfast". But, I have a couple of ports, users... which the port is
> flapping. I see these in my log:
>
> %RTD-1-LINK_FLAP: FastEthernet 0/1 link down/up 5 times per min\par
>
> So this could be the cause of my problems? I will hunt them down
>
> thanks! michel.
>

Michel <> writes:
>
> We have a complete switched network with VTP domain with many VLAN's
> and having 2x Cisco 6509 and 34x 3524 inline power which are connected
> in a chain 3 till 5 Cisco 3524's between the 6509 switches. The 6509
> switches are connected with an etherchannel of 2 GigaBps.

Sounds familar.

> Spontaneous at a completely random time we have a complete overall
> network failure for 1 or 2 seconds. Enough to disconnect telephone
> calls (Cisco VoIP) and data Cluster failures. (and complaining
> colleagues).

The strange thing is that you mention just 1 to 2 seconds. If it would
be significantly longer, I would opt for an STP topology change. But
such a short disruption would only occur with rapid STP, and that is
not available with the 35xxXL series.

> I try to figure out what the problem is by sending all logging from
> all switches to a UNIX syslog. But I cannot find a logging which leads
> to this problem. Almost every network counters of all ports seams to
> be allright, no drop, overrun etc...

That's the sort of error we all like most...

> What could it possible be the problem? Can someone give me a hint were
> to look for or how to trace?

Is your switched network just one single broadcast domain or are the
6509s by chance routing between VLANs? If they do, what is your ARP
timeout on the vlanX EtherSVI interfaces?

I have found a strange problem where a very tight ARP timeout of 120s
on a 6509s vlanX interfaces caused intermittend loss of supposed to
be routed packets. It seemed the 6509 (SUP2+MSFC2+PFC2 running inte-
grated IOS) was unable to refresh the ARP entries for up to 20s, but
not always. I had a TAC Case on this, but Cisco couldn't recreate the
problem. The easy workaround was to increase the ARP timeout to 300s
which is considered tight enough in a somewhat larger broadcast domain
anyway. Just note that the HSRP-peering backup routers of the 6509 in
question, 3550-12Gs, didn't even remotely have any problem with routing
and a 120s ARP timeout.

--
The _S_anta _C_laus _O_peration
or "how to turn a complete illusion into a neverending money source"

-> Andre "ABPSoft" Beck +++ ABP-RIPE +++ Dresden, Germany, Spacetime <-