Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Computing > Cisco > How to prevent network loop?

Reply
Thread Tools

How to prevent network loop?

 
 
aquarius
Guest
Posts: n/a
 
      02-02-2007
We had a network problem that was caused by a debug command given in one
switch. After the debug command the CPU load of the switch was 100% and the
whole network went down.

The network is a star-type network that has 20 pieces of Cisco 3750
connected to one cisco 7609 router. The connection is done via gigabit fiber
ports and the ports are configured as L2 trunk ports.

There is MSTP with few instances running in the network but it did not
prevent the network from the loop. My question is that is there a way to
prevent the network from this kind of problem?

-Sami


 
Reply With Quote
 
 
 
 
Bjarke Andersen
Guest
Posts: n/a
 
      02-02-2007
"aquarius" <(E-Mail Removed)> crashed Echelon writing
news:_GEwh.47656$(E-Mail Removed):

> There is MSTP with few instances running in the network but it did not
> prevent the network from the loop. My question is that is there a way to
> prevent the network from this kind of problem?


Spanning-tree and BPDU-guard on endtail links (meaning no BPDU-guard on
links which serves as backup line e.g. 3 core switches linked together in a
circle)

--
Bjarke Andersen
 
Reply With Quote
 
 
 
 
aquarius
Guest
Posts: n/a
 
      02-02-2007

"Bjarke Andersen" <(E-Mail Removed)> wrote in message
news:Xns98CB7D457B0D5bjoegdk@130.225.247.90...
> "aquarius" <(E-Mail Removed)> crashed Echelon writing
> news:_GEwh.47656$(E-Mail Removed):
>
> > There is MSTP with few instances running in the network but it did not
> > prevent the network from the loop. My question is that is there a way to
> > prevent the network from this kind of problem?

>
> Spanning-tree and BPDU-guard on endtail links (meaning no BPDU-guard on
> links which serves as backup line e.g. 3 core switches linked together in

a
> circle)
>
> --
> Bjarke Andersen


I didn't quite undesrtand your answer. There are 2x24 port gigabit interface
cards in the 7609 and every 3750 has 2 gigabit trunk interfaces connected to
7609 and one of these interfaces is blocked by MSTP. Do you mean that the
Spanning-tree and BPDU-guard should not be configured to these blocked
interfaces?

-Sami


 
Reply With Quote
 
Bod43@hotmail.co.uk
Guest
Posts: n/a
 
      02-02-2007
On 2 Feb, 12:06, "aquarius" <(E-Mail Removed)>
wrote:
> "Bjarke Andersen" <(E-Mail Removed)> wrote in message
>
> news:Xns98CB7D457B0D5bjoegdk@130.225.247.90...
>
> > "aquarius" <(E-Mail Removed)> crashed Echelon writing
> >news:_GEwh.47656$(E-Mail Removed):

>
> > > There is MSTP with few instances running in the network but it did not
> > > prevent the network from the loop. My question is that is there a way to
> > > prevent the network from this kind of problem?

>
> > Spanning-tree and BPDU-guard on endtail links (meaning no BPDU-guard on
> > links which serves as backup line e.g. 3 core switches linked together in

> a
> > circle)

>
> > --
> > Bjarke Andersen

>
> I didn't quite undesrtand your answer. There are 2x24 port gigabit interface
> cards in the 7609 and every 3750 has 2 gigabit trunk interfaces connected to
> 7609 and one of these interfaces is blocked by MSTP. Do you mean that the
> Spanning-tree and BPDU-guard should not be configured to these blocked
> interfaces?
>


The smart-ass answer is that Cisco do not recommend the use
of the debug command in production networks.

Really! How could it be otherwise?

Some debugs are suitable for use in production but it is pretty
much left up to the Network Engineer to evaluate each
use of debug themselves.

Did you follow the debug guidelines?

no logg console ! or logg level < debugging at least

If I thought there was the slightest risk add:
no logg monitor ! or logg level < debugging at least
no logg trap ! or logg level < debugging at least

!! also don't log to snmp whatever command that is.

so we log only to the buffer, the cheapest option.

logg buffer debb
logg buffer 50000 ! if we have enough memory

Then consider just how your debug may affect the router.

Use available tools to limit the number of log messages.

It the end of the day, if a router or switch has not got enough
CPU to process spanning tree messages then the network
may well break. Debug is designed to be as good for
troubleshooting as possible, this means that it HAS to be
more important for the CPU to do debug than it is to do
spanning tree. We are back full circle, debug is not
supported in production.

You could manually shutdown the interfaces that
could cause loops.


 
Reply With Quote
 
stephen
Guest
Posts: n/a
 
      02-02-2007
"Bjarke Andersen" <(E-Mail Removed)> wrote in message
news:Xns98CB7D457B0D5bjoegdk@130.225.247.90...
> "aquarius" <(E-Mail Removed)> crashed Echelon writing
> news:_GEwh.47656$(E-Mail Removed):
>
> > There is MSTP with few instances running in the network but it did not
> > prevent the network from the loop. My question is that is there a way to
> > prevent the network from this kind of problem?

>
> Spanning-tree and BPDU-guard on endtail links (meaning no BPDU-guard on
> links which serves as backup line e.g. 3 core switches linked together in

a
> circle)


MST will fall back to spanning tree under some conditions - usually when the
2 switch ports decide they are not both MST compatible.

"raw" spanning tree doesnt block ports with some 1 way link problems - and a
1 way loop is just as good a broadcast overload generator as the ordinary
kind.

Also if you manage to build a working loop and the load makes the CPU hit
100% it may end up stuck in that state - although Cisco IOS /CatOS seems
better than some other switch code for this.

My preference is to use L3 design to get rid of spanning tree.

If you have to put up with it for some reason, uni directional link
detection (UDLD) can get rid of the most common 1 way faults. Note - Cisco
special so not feasible if you have other manufacturers kit.

But it is worth remembering that spanning tree mostly works well, but it
works by actively suppressing loops when only when all ports around the loop
work in both directions, so if something goes wrong the failure is likely to
cause an active loop
>
> --
> Bjarke Andersen

--
Regards

http://www.velocityreviews.com/forums/(E-Mail Removed) - replace xyz with ntl


 
Reply With Quote
 
aquarius
Guest
Posts: n/a
 
      02-03-2007

"stephen" <(E-Mail Removed)> wrote in message
news:q1Owh.61964$(E-Mail Removed)...
> MST will fall back to spanning tree under some conditions - usually when
> the
> 2 switch ports decide they are not both MST compatible.
>
> "raw" spanning tree doesnt block ports with some 1 way link problems - and
> a
> 1 way loop is just as good a broadcast overload generator as the ordinary
> kind.
>
> Also if you manage to build a working loop and the load makes the CPU hit
> 100% it may end up stuck in that state - although Cisco IOS /CatOS seems
> better than some other switch code for this.
>
> My preference is to use L3 design to get rid of spanning tree.


Yes, that would solve the problem, but it is not possible at the moment.

>
> If you have to put up with it for some reason, uni directional link
> detection (UDLD) can get rid of the most common 1 way faults. Note - Cisco
> special so not feasible if you have other manufacturers kit.



Yes, we have the UDLD in use for "broken wire" cases.

>
> But it is worth remembering that spanning tree mostly works well, but it
> works by actively suppressing loops when only when all ports around the
> loop
> work in both directions, so if something goes wrong the failure is likely
> to
> cause an active loop
>>


And this time the cause of the loop was CPU overload. I will try the
loop-guard,
as soon as I know how and where should I configure it.

>> --
>> Bjarke Andersen

> --
> Regards
>
> (E-Mail Removed) - replace xyz with ntl
>


Thanks for your answer !

-Sami


 
Reply With Quote
 
Thrill5
Guest
Posts: n/a
 
      02-06-2007
The cause of the CPU problem was your debug command. Please see the
previous post. If you spike the CPU because of debug command flooding the
console with debug messages, or because the debug command you entered
requires significant CPU to process, you will still bring that device down,
layer 2 or layer 3 switch/routing not withstanding.

Scott


"aquarius" <(E-Mail Removed)> wrote in message
newsP%wh.2957$(E-Mail Removed) i...
>
> "stephen" <(E-Mail Removed)> wrote in message
> news:q1Owh.61964$(E-Mail Removed)...
>> MST will fall back to spanning tree under some conditions - usually when
>> the
>> 2 switch ports decide they are not both MST compatible.
>>
>> "raw" spanning tree doesnt block ports with some 1 way link problems -
>> and a
>> 1 way loop is just as good a broadcast overload generator as the ordinary
>> kind.
>>
>> Also if you manage to build a working loop and the load makes the CPU hit
>> 100% it may end up stuck in that state - although Cisco IOS /CatOS seems
>> better than some other switch code for this.
>>
>> My preference is to use L3 design to get rid of spanning tree.

>
> Yes, that would solve the problem, but it is not possible at the moment.
>
>>
>> If you have to put up with it for some reason, uni directional link
>> detection (UDLD) can get rid of the most common 1 way faults. Note -
>> Cisco
>> special so not feasible if you have other manufacturers kit.

>
>
> Yes, we have the UDLD in use for "broken wire" cases.
>
>>
>> But it is worth remembering that spanning tree mostly works well, but it
>> works by actively suppressing loops when only when all ports around the
>> loop
>> work in both directions, so if something goes wrong the failure is likely
>> to
>> cause an active loop
>>>

>
> And this time the cause of the loop was CPU overload. I will try the
> loop-guard,
> as soon as I know how and where should I configure it.
>
>>> --
>>> Bjarke Andersen

>> --
>> Regards
>>
>> (E-Mail Removed) - replace xyz with ntl
>>

>
> Thanks for your answer !
>
> -Sami
>



 
Reply With Quote
 
aquarius
Guest
Posts: n/a
 
      02-06-2007
Yes, that's clear, but the question was that is there a way to protect the
rest of the network in case of one switch's CPU overload. I have now
configured spanning-tree loopguard to all root- and alternative ports and
debug-commands will not be used any more...

-Sami
"Thrill5" <(E-Mail Removed)> wrote in message
news(E-Mail Removed)...
> The cause of the CPU problem was your debug command. Please see the
> previous post. If you spike the CPU because of debug command flooding the
> console with debug messages, or because the debug command you entered
> requires significant CPU to process, you will still bring that device

down,
> layer 2 or layer 3 switch/routing not withstanding.
>
> Scott
>
>
> "aquarius" <(E-Mail Removed)> wrote in message
> newsP%wh.2957$(E-Mail Removed) i...
> >
> > "stephen" <(E-Mail Removed)> wrote in message
> > news:q1Owh.61964$(E-Mail Removed)...
> >> MST will fall back to spanning tree under some conditions - usually

when
> >> the
> >> 2 switch ports decide they are not both MST compatible.
> >>
> >> "raw" spanning tree doesnt block ports with some 1 way link problems -
> >> and a
> >> 1 way loop is just as good a broadcast overload generator as the

ordinary
> >> kind.
> >>
> >> Also if you manage to build a working loop and the load makes the CPU

hit
> >> 100% it may end up stuck in that state - although Cisco IOS /CatOS

seems
> >> better than some other switch code for this.
> >>
> >> My preference is to use L3 design to get rid of spanning tree.

> >
> > Yes, that would solve the problem, but it is not possible at the moment.
> >
> >>
> >> If you have to put up with it for some reason, uni directional link
> >> detection (UDLD) can get rid of the most common 1 way faults. Note -
> >> Cisco
> >> special so not feasible if you have other manufacturers kit.

> >
> >
> > Yes, we have the UDLD in use for "broken wire" cases.
> >
> >>
> >> But it is worth remembering that spanning tree mostly works well, but

it
> >> works by actively suppressing loops when only when all ports around the
> >> loop
> >> work in both directions, so if something goes wrong the failure is

likely
> >> to
> >> cause an active loop
> >>>

> >
> > And this time the cause of the loop was CPU overload. I will try the
> > loop-guard,
> > as soon as I know how and where should I configure it.
> >
> >>> --
> >>> Bjarke Andersen
> >> --
> >> Regards
> >>
> >> (E-Mail Removed) - replace xyz with ntl
> >>

> >
> > Thanks for your answer !
> >
> > -Sami
> >

>
>



 
Reply With Quote
 
stephen
Guest
Posts: n/a
 
      02-06-2007
"aquarius" <(E-Mail Removed)> wrote in message
news:aKVxh.47783$(E-Mail Removed)...
> Yes, that's clear, but the question was that is there a way to protect the
> rest of the network in case of one switch's CPU overload. I have now
> configured spanning-tree loopguard to all root- and alternative ports and
> debug-commands will not be used any more...


there is a clear answer - but you wont like it, because it sounds like you
cannot use it. The flip side to that is it may be a good reason to prise
some money loose to fix it......

Design out L2 loops.

dont use spanning tree to suppress loops in normal operation - use spanning
tree as a safety feature in case someone adds a loop by accident.

because of the protocol design, every device / link within any loop has to
run to protocol correctly within the timeout window to keep the loop from
acting as a broadcast replicator.
>
> -Sami
> "Thrill5" <(E-Mail Removed)> wrote in message
> news(E-Mail Removed)...
> > The cause of the CPU problem was your debug command. Please see the
> > previous post. If you spike the CPU because of debug command flooding

the
> > console with debug messages, or because the debug command you entered
> > requires significant CPU to process, you will still bring that device

> down,
> > layer 2 or layer 3 switch/routing not withstanding.
> >
> > Scott
> >
> >
> > "aquarius" <(E-Mail Removed)> wrote in message
> > newsP%wh.2957$(E-Mail Removed) i...
> > >
> > > "stephen" <(E-Mail Removed)> wrote in message
> > > news:q1Owh.61964$(E-Mail Removed)...
> > >> MST will fall back to spanning tree under some conditions - usually

> when
> > >> the
> > >> 2 switch ports decide they are not both MST compatible.
> > >>
> > >> "raw" spanning tree doesnt block ports with some 1 way link

problems -
> > >> and a
> > >> 1 way loop is just as good a broadcast overload generator as the

> ordinary
> > >> kind.
> > >>
> > >> Also if you manage to build a working loop and the load makes the CPU

> hit
> > >> 100% it may end up stuck in that state - although Cisco IOS /CatOS

> seems
> > >> better than some other switch code for this.
> > >>
> > >> My preference is to use L3 design to get rid of spanning tree.
> > >
> > > Yes, that would solve the problem, but it is not possible at the

moment.
> > >
> > >>
> > >> If you have to put up with it for some reason, uni directional link
> > >> detection (UDLD) can get rid of the most common 1 way faults. Note -
> > >> Cisco
> > >> special so not feasible if you have other manufacturers kit.
> > >
> > >
> > > Yes, we have the UDLD in use for "broken wire" cases.
> > >
> > >>
> > >> But it is worth remembering that spanning tree mostly works well, but

> it
> > >> works by actively suppressing loops when only when all ports around

the
> > >> loop
> > >> work in both directions, so if something goes wrong the failure is

> likely
> > >> to
> > >> cause an active loop
> > >>>
> > >
> > > And this time the cause of the loop was CPU overload. I will try the
> > > loop-guard,
> > > as soon as I know how and where should I configure it.
> > >
> > >>> --
> > >>> Bjarke Andersen
> > >> --
> > >> Regards
> > >>
> > >> (E-Mail Removed) - replace xyz with ntl
> > >>
> > >
> > > Thanks for your answer !
> > >
> > > -Sami
> > >

--
Regards

(E-Mail Removed) - replace xyz with ntl


 
Reply With Quote
 
aquarius
Guest
Posts: n/a
 
      02-07-2007
You guys have confirmed me that the best solution is to redesign the network
from L2 to L3.

-Sami
"stephen" <(E-Mail Removed)> wrote in message
news:Kb4yh.148$(E-Mail Removed)...
> "aquarius" <(E-Mail Removed)> wrote in message
> news:aKVxh.47783$(E-Mail Removed)...
> > Yes, that's clear, but the question was that is there a way to protect

the
> > rest of the network in case of one switch's CPU overload. I have now
> > configured spanning-tree loopguard to all root- and alternative ports

and
> > debug-commands will not be used any more...

>
> there is a clear answer - but you wont like it, because it sounds like you
> cannot use it. The flip side to that is it may be a good reason to prise
> some money loose to fix it......
>
> Design out L2 loops.
>
> dont use spanning tree to suppress loops in normal operation - use

spanning
> tree as a safety feature in case someone adds a loop by accident.
>
> because of the protocol design, every device / link within any loop has to
> run to protocol correctly within the timeout window to keep the loop from
> acting as a broadcast replicator.
> >
> > -Sami
> > "Thrill5" <(E-Mail Removed)> wrote in message
> > news(E-Mail Removed)...
> > > The cause of the CPU problem was your debug command. Please see the
> > > previous post. If you spike the CPU because of debug command flooding

> the
> > > console with debug messages, or because the debug command you entered
> > > requires significant CPU to process, you will still bring that device

> > down,
> > > layer 2 or layer 3 switch/routing not withstanding.
> > >
> > > Scott
> > >
> > >
> > > "aquarius" <(E-Mail Removed)> wrote in message
> > > newsP%wh.2957$(E-Mail Removed) i...
> > > >
> > > > "stephen" <(E-Mail Removed)> wrote in message
> > > > news:q1Owh.61964$(E-Mail Removed)...
> > > >> MST will fall back to spanning tree under some conditions - usually

> > when
> > > >> the
> > > >> 2 switch ports decide they are not both MST compatible.
> > > >>
> > > >> "raw" spanning tree doesnt block ports with some 1 way link

> problems -
> > > >> and a
> > > >> 1 way loop is just as good a broadcast overload generator as the

> > ordinary
> > > >> kind.
> > > >>
> > > >> Also if you manage to build a working loop and the load makes the

CPU
> > hit
> > > >> 100% it may end up stuck in that state - although Cisco IOS /CatOS

> > seems
> > > >> better than some other switch code for this.
> > > >>
> > > >> My preference is to use L3 design to get rid of spanning tree.
> > > >
> > > > Yes, that would solve the problem, but it is not possible at the

> moment.
> > > >
> > > >>
> > > >> If you have to put up with it for some reason, uni directional link
> > > >> detection (UDLD) can get rid of the most common 1 way faults.

Note -
> > > >> Cisco
> > > >> special so not feasible if you have other manufacturers kit.
> > > >
> > > >
> > > > Yes, we have the UDLD in use for "broken wire" cases.
> > > >
> > > >>
> > > >> But it is worth remembering that spanning tree mostly works well,

but
> > it
> > > >> works by actively suppressing loops when only when all ports around

> the
> > > >> loop
> > > >> work in both directions, so if something goes wrong the failure is

> > likely
> > > >> to
> > > >> cause an active loop
> > > >>>
> > > >
> > > > And this time the cause of the loop was CPU overload. I will try the
> > > > loop-guard,
> > > > as soon as I know how and where should I configure it.
> > > >
> > > >>> --
> > > >>> Bjarke Andersen
> > > >> --
> > > >> Regards
> > > >>
> > > >> (E-Mail Removed) - replace xyz with ntl
> > > >>
> > > >
> > > > Thanks for your answer !
> > > >
> > > > -Sami
> > > >

> --
> Regards
>
> (E-Mail Removed) - replace xyz with ntl
>
>



 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
Prevent Access to Network Using MAC Addresses kware@netexperts.co.uk Cisco 2 07-10-2005 07:16 AM
How to prevent windows xp sp2 from autoconnecting to a network? Mario Wireless Networking 3 05-02-2005 10:15 PM
Prevent access to unsecured wireless network =?Utf-8?B?SmF3cw==?= Wireless Networking 3 05-01-2005 04:31 AM
Can I prevent the kids connecting to any available network ? anthony Wireless Networking 4 04-26-2005 11:50 AM



Advertisments