Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Computing > Cisco > Mystery Network Problem

Reply
Thread Tools

Mystery Network Problem

 
 
Aaron
Guest
Posts: n/a
 
      02-18-2008
So I'm tossing this out there to see if anyone might have some
thoughts on where to look because I just find this baffling.

We had a Cisco 3750 Catalyst switch set up with a number of hosts
connected to it. All of these hosts were configured to do NIC teaming
with etherchannels set up on the switch. We then added a second 3750
to create a stack. We re-configured the etherchannels on the existing
servers so that each one had one connection to each switch. We then
added a number of new servers with each one using etherchannels (one
connection to each switch). Everything seemed fine. All of the
servers were available on the network, etc. But, we noticed that of
the 12 new servers, two of them had this odd issue. All of the
servers are in the same VLAN and IP's in the same subnet. All of them
can get out to the internet and can be reached by *most* computers in
the same VLAN and in other VLANS. BUT...the two oddballs can't talk
to the other servers connected to the same 3750.

I turned off the NIC teaming and disabled the secondary NIC port. We
removed the switch ports from the etherchannel so it was just a
normal, plain jane port. Still had the problem. Moving the server IP
information from the primary to the secondary NIC port cleared up the
problem...at first. While trying various permutations of the NIC
teaming (trying to track down the problem) things got the point where
if the server was configured to use NIC1, it could get out to the
internet and other various servers, but couldn't talk (no ping, no
thing..they don't even see ARP requests) to the other servers on the
switch (EXCEPT those in a different VLAN) or some other random
machines on that same VLAN.). Setting the server to use NIC2 resulted
in it being able to talk to servers it previously couldn't talk to,
but it suddenly couldn't talk to the gateway and so couldn't reach
anything else. It was previously able to do so.

What this FEELS like is that the switch has somehow declared these two
server's MAC addresses as persona non-grata and won't route traffic
from or to it except to seemingly random exceptions.

Anyone have any thoughts on where to look as I'm pretty much stumped
at this point.
 
Reply With Quote
 
 
 
 
Thrill5
Guest
Posts: n/a
 
      02-18-2008
The 3750 only has a limited amount of TCAM that must be used for routing
(CEF entries) switching (for MAC entries), multicast and qos queues.
Sometimes the "default" allocation of TCAM isn't right and you have to
adjust it.

Do the MAC entries for the servers exist in "show mac-address-table", if
not you probably have an TCAM issue. Also do a "show mac-address-table
count" and see how many entries are left for MAC addresses (this will be the
last line of the output). If you are doing layer 3 on the switch then you
probably don't have enough TCAM allocated for CEF entries, and the reason
you are having issues.

The "show sdm prefer" command will show you what is currently configured.
The default is "default" but we have had to change this on 3750's that we
are doing layer 3 switching on. To change it, issue a "sdm prefer
<template>" command, and you will need to reboot for it to take effect.


See this article for more information on TCAM and the 3750.
http://www.enterprisenetworkingplane...le.php/3527301

"Aaron" <> wrote in message
news:27d46f52-5c8c-4752-baec-...
> So I'm tossing this out there to see if anyone might have some
> thoughts on where to look because I just find this baffling.
>
> We had a Cisco 3750 Catalyst switch set up with a number of hosts
> connected to it. All of these hosts were configured to do NIC teaming
> with etherchannels set up on the switch. We then added a second 3750
> to create a stack. We re-configured the etherchannels on the existing
> servers so that each one had one connection to each switch. We then
> added a number of new servers with each one using etherchannels (one
> connection to each switch). Everything seemed fine. All of the
> servers were available on the network, etc. But, we noticed that of
> the 12 new servers, two of them had this odd issue. All of the
> servers are in the same VLAN and IP's in the same subnet. All of them
> can get out to the internet and can be reached by *most* computers in
> the same VLAN and in other VLANS. BUT...the two oddballs can't talk
> to the other servers connected to the same 3750.
>
> I turned off the NIC teaming and disabled the secondary NIC port. We
> removed the switch ports from the etherchannel so it was just a
> normal, plain jane port. Still had the problem. Moving the server IP
> information from the primary to the secondary NIC port cleared up the
> problem...at first. While trying various permutations of the NIC
> teaming (trying to track down the problem) things got the point where
> if the server was configured to use NIC1, it could get out to the
> internet and other various servers, but couldn't talk (no ping, no
> thing..they don't even see ARP requests) to the other servers on the
> switch (EXCEPT those in a different VLAN) or some other random
> machines on that same VLAN.). Setting the server to use NIC2 resulted
> in it being able to talk to servers it previously couldn't talk to,
> but it suddenly couldn't talk to the gateway and so couldn't reach
> anything else. It was previously able to do so.
>
> What this FEELS like is that the switch has somehow declared these two
> server's MAC addresses as persona non-grata and won't route traffic
> from or to it except to seemingly random exceptions.
>
> Anyone have any thoughts on where to look as I'm pretty much stumped
> at this point.



 
Reply With Quote
 
 
 
 
Aaron
Guest
Posts: n/a
 
      02-19-2008
We ended up tracking down the problem to how the servers NIC teaming
was set up. The company that did the initial build and install set
the NIC teaming up to use Broadcom's proprietary "Smart Load
Balancing". Apparently, this was butting heads something fierce with
the Fast Etherchannel settings on the switch. Changing that NIC
teaming type to just "Generic Trunking" cleared up all the problems!

On Feb 18, 6:14 pm, "Thrill5" <nos...@somewhere.com> wrote:
> The 3750 only has a limited amount of TCAM that must be used for routing
> (CEF entries) switching (for MAC entries), multicast and qos queues.
> Sometimes the "default" allocation of TCAM isn't right and you have to
> adjust it.
>
> Do the MAC entries for the servers exist in "show mac-address-table", if
> not you probably have an TCAM issue. Also do a "show mac-address-table
> count" and see how many entries are left for MAC addresses (this will be the
> last line of the output). If you are doing layer 3 on the switch then you
> probably don't have enough TCAM allocated for CEF entries, and the reason
> you are having issues.
>
> The "show sdm prefer" command will show you what is currently configured.
> The default is "default" but we have had to change this on 3750's that we
> are doing layer 3 switching on. To change it, issue a "sdm prefer
> <template>" command, and you will need to reboot for it to take effect.
>
> See this article for more information on TCAM and the 3750.http://www.enterprisenetworkingplane...le.php/3527301
>



 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Mystery "Logging on to network" (dial up) takes long time Brad Computer Information 3 09-29-2006 02:42 PM
UPDATE: Mystery "Logging on to network" (dial up) takes long time Brad Computer Information 0 09-28-2006 05:59 PM
unicode mystery/problem Petr Jakes Python 4 09-22-2006 02:09 PM
ZAP 0.0.0.0/0.0.0.0 mystery network Ockerr Computer Support 3 05-29-2006 12:21 PM
Network Config Mystery Tony Sperling Windows 64bit 24 05-01-2006 08:30 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57