Spontaneous network failure 1 or 2 seconds

Discussion in 'Cisco' started by Michel, Nov 13, 2003.

  1. Michel

    Michel Guest

    Hello everyone,

    We have a complete switched network with VTP domain with many VLAN's and
    having 2x Cisco 6509 and 34x 3524 inline power which are connected in
    a chain 3 till 5 Cisco 3524's between the 6509 switches. The 6509
    switches are connected with an etherchannel of 2 GigaBps.

    Spontaneous at a completely random time we have a complete overall
    network failure for 1 or 2 seconds. Enough to disconnect telephone calls
    (Cisco VoIP) and data Cluster failures. (and complaining colleagues).

    I try to figure out what the problem is by sending all logging from all
    switches to a UNIX syslog. But I cannot find a logging which leads to
    this problem. Almost every network counters of all ports seams to be
    allright, no drop, overrun etc...

    What could it possible be the problem? Can someone give me a hint were
    to look for or how to trace?

    Kind regards Michel.

    Michel, Nov 13, 2003
    #1
    1. Advertising

  2. Just a wild guess, with out the topology and configs it is very difficult to answer. Do you have
    spanning tree turned-on on all your switched interfaces where there is a potential loop ? Check for
    any interface flaps in the debugs which might create a broadcast storm making the network
    unreachable for second or so.

    Hope this helps,
    Vidyaranya Maddi.

    Michel wrote:

    > Hello everyone,
    >
    > We have a complete switched network with VTP domain with many VLAN's and
    > having 2x Cisco 6509 and 34x 3524 inline power which are connected in a
    > chain 3 till 5 Cisco 3524's between the 6509 switches. The 6509 switches
    > are connected with an etherchannel of 2 GigaBps.
    >
    > Spontaneous at a completely random time we have a complete overall
    > network failure for 1 or 2 seconds. Enough to disconnect telephone calls
    > (Cisco VoIP) and data Cluster failures. (and complaining colleagues).
    >
    > I try to figure out what the problem is by sending all logging from all
    > switches to a UNIX syslog. But I cannot find a logging which leads to
    > this problem. Almost every network counters of all ports seams to be
    > allright, no drop, overrun etc...
    >
    > What could it possible be the problem? Can someone give me a hint were
    > to look for or how to trace?
    >
    > Kind regards Michel.
    >
    >
    >
    Vidyaranya Maddi, Nov 13, 2003
    #2
    1. Advertising

  3. Michel

    Michel Guest


    > Just a wild guess, with out the topology and configs it is very
    > difficult to answer. Do you have spanning tree turned-on on all your
    > switched interfaces where there is a potential loop ? Check for any
    > interface flaps in the debugs which might create a broadcast storm
    > making the network unreachable for second or so.


    Yes I use Spanning-tree.. On every Ethernet port I user "Spanning-tree
    portfast". But, I have a couple of ports, users... which the port is
    flapping. I see these in my log:

    %RTD-1-LINK_FLAP: FastEthernet 0/1 link down/up 5 times per min\par

    So this could be the cause of my problems? I will hunt them down :)

    thanks! michel.
    Michel, Nov 14, 2003
    #3
  4. You might wnat to just shut this guy down and see if the problem goes away. If yes, you fould the
    culprit. Let me know how it goes.

    -Maddi.

    Michel wrote:

    >
    >
    >> Just a wild guess, with out the topology and configs it is very
    >> difficult to answer. Do you have spanning tree turned-on on all your
    >> switched interfaces where there is a potential loop ? Check for any
    >> interface flaps in the debugs which might create a broadcast storm
    >> making the network unreachable for second or so.

    >
    >
    > Yes I use Spanning-tree.. On every Ethernet port I user "Spanning-tree
    > portfast". But, I have a couple of ports, users... which the port is
    > flapping. I see these in my log:
    >
    > %RTD-1-LINK_FLAP: FastEthernet 0/1 link down/up 5 times per min\par
    >
    > So this could be the cause of my problems? I will hunt them down :)
    >
    > thanks! michel.
    >
    Vidyaranya Maddi, Nov 14, 2003
    #4
  5. Michel

    Andre Beck Guest

    Michel <> writes:
    >
    > We have a complete switched network with VTP domain with many VLAN's
    > and having 2x Cisco 6509 and 34x 3524 inline power which are connected
    > in a chain 3 till 5 Cisco 3524's between the 6509 switches. The 6509
    > switches are connected with an etherchannel of 2 GigaBps.


    Sounds familar.

    > Spontaneous at a completely random time we have a complete overall
    > network failure for 1 or 2 seconds. Enough to disconnect telephone
    > calls (Cisco VoIP) and data Cluster failures. (and complaining
    > colleagues).


    The strange thing is that you mention just 1 to 2 seconds. If it would
    be significantly longer, I would opt for an STP topology change. But
    such a short disruption would only occur with rapid STP, and that is
    not available with the 35xxXL series.

    > I try to figure out what the problem is by sending all logging from
    > all switches to a UNIX syslog. But I cannot find a logging which leads
    > to this problem. Almost every network counters of all ports seams to
    > be allright, no drop, overrun etc...


    That's the sort of error we all like most...

    > What could it possible be the problem? Can someone give me a hint were
    > to look for or how to trace?


    Is your switched network just one single broadcast domain or are the
    6509s by chance routing between VLANs? If they do, what is your ARP
    timeout on the vlanX EtherSVI interfaces?

    I have found a strange problem where a very tight ARP timeout of 120s
    on a 6509s vlanX interfaces caused intermittend loss of supposed to
    be routed packets. It seemed the 6509 (SUP2+MSFC2+PFC2 running inte-
    grated IOS) was unable to refresh the ARP entries for up to 20s, but
    not always. I had a TAC Case on this, but Cisco couldn't recreate the
    problem. The easy workaround was to increase the ARP timeout to 300s
    which is considered tight enough in a somewhat larger broadcast domain
    anyway. Just note that the HSRP-peering backup routers of the 6509 in
    question, 3550-12Gs, didn't even remotely have any problem with routing
    and a 120s ARP timeout.

    --
    The _S_anta _C_laus _O_peration
    or "how to turn a complete illusion into a neverending money source"

    -> Andre "ABPSoft" Beck +++ ABP-RIPE +++ Dresden, Germany, Spacetime <-
    Andre Beck, Nov 15, 2003
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael T. Anderson
    Replies:
    0
    Views:
    2,817
    Michael T. Anderson
    Jun 23, 2005
  2. George

    Re: Spontaneous Re-boot (Need Info)

    George, Aug 4, 2003, in forum: Computer Support
    Replies:
    0
    Views:
    505
    George
    Aug 4, 2003
  3. Lag every 60 seconds on wireless network

    , Nov 20, 2006, in forum: Wireless Networking
    Replies:
    20
    Views:
    5,028
    =?Utf-8?B?bmVpbDJqbw==?=
    Jan 31, 2007
  4. Replies:
    19
    Views:
    782
  5. Spontaneous loss of network connectivity

    , Nov 3, 2008, in forum: Wireless Networking
    Replies:
    3
    Views:
    424
Loading...

Share This Page