Cisco 4507R and Broadcom NetXtreme NICs

Discussion in 'Cisco' started by Inthemix88, Jun 6, 2007.

  1. Inthemix88

    Inthemix88 Guest

    Good morning all,
    We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    the NICs reports that it is connected at 100 Mb instead of 1000. We
    are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    (20). I know this is not the latest version but all our switches run
    the same IOS and we are having this problem only on 3 out of 8
    buildings. Our servers are IBM 336 with the previously mentioned NIC
    cards. We have replaced the motherboard on the IBM servers. We updated
    the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    updated one server to sp2, problem still exists). We have updated the
    NIC card driver to the latest version. We swapped the cables out using
    cat 5e and 6. We also tried different ports on the blade and switch
    (we have 2 48 port gigabit blades).
    If anyone can give us some feedback, it would be greatly appreciated.

    Thanks in advanced,
    Martin Reyes
    LHRIC
    Inthemix88, Jun 6, 2007
    #1
    1. Advertising

  2. Inthemix88

    Trendkill Guest

    On Jun 6, 9:05 am, Inthemix88 <> wrote:
    > Good morning all,
    > We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    > the NICs reports that it is connected at 100 Mb instead of 1000. We
    > are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    > (20). I know this is not the latest version but all our switches run
    > the same IOS and we are having this problem only on 3 out of 8
    > buildings. Our servers are IBM 336 with the previously mentioned NIC
    > cards. We have replaced the motherboard on the IBM servers. We updated
    > the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    > updated one server to sp2, problem still exists). We have updated the
    > NIC card driver to the latest version. We swapped the cables out using
    > cat 5e and 6. We also tried different ports on the blade and switch
    > (we have 2 48 port gigabit blades).
    > If anyone can give us some feedback, it would be greatly appreciated.
    >
    > Thanks in advanced,
    > Martin Reyes
    > LHRIC


    Not to suggest the obvious, but have you hard coded both sides (switch
    & server) to 1000/full per cisco best practices. While auto/auto on
    both sides is also acceptable (for gig only), you should really stick
    to a plan of hard coding both sides as it makes for better long term
    support, documentation, and maintenance.
    Trendkill, Jun 6, 2007
    #2
    1. Advertising

  3. Inthemix88

    stephen Guest

    "jw" <> wrote in message
    news:46670847$0$4332$...
    > Ibm servers frequently have issues with auto negotiation.
    > Check ALL your ibm servers for sp/dplx harcoding and errors on those

    ports.
    > Trendkill wrote:
    > > On Jun 6, 9:05 am, Inthemix88 <> wrote:
    > >> Good morning all,
    > >> We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    > >> the NICs reports that it is connected at 100 Mb instead of 1000. We
    > >> are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    > >> (20). I know this is not the latest version but all our switches run
    > >> the same IOS and we are having this problem only on 3 out of 8
    > >> buildings. Our servers are IBM 336 with the previously mentioned NIC
    > >> cards. We have replaced the motherboard on the IBM servers. We updated
    > >> the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    > >> updated one server to sp2, problem still exists). We have updated the
    > >> NIC card driver to the latest version. We swapped the cables out using
    > >> cat 5e and 6. We also tried different ports on the blade and switch
    > >> (we have 2 48 port gigabit blades).
    > >> If anyone can give us some feedback, it would be greatly appreciated.
    > >>
    > >> Thanks in advanced,
    > >> Martin Reyes
    > >> LHRIC

    > >
    > > Not to suggest the obvious, but have you hard coded both sides (switch
    > > & server) to 1000/full per cisco best practices.


    Not sure this is true for GigE.

    although a lot of devices had problems with 100 Base-TX, the IEEE standard
    says auto is mandatory for 1000 Base-T, since it is used to sort out master
    / slave etc.
    http://www.ethermanage.com/ethernet/autoneg.html

    Even on GigE fibre where the 2 devices dont have to negotiate, some devices
    will not work unless auto- negotiate is enabled on the other device
    - I stumbled across this on a Foundry L3 switch recently, connecting to a
    GigE on an SDH mux.

    While auto/auto on
    > > both sides is also acceptable (for gig only), you should really stick
    > > to a plan of hard coding both sides as it makes for better long term
    > > support, documentation, and maintenance.
    > >


    i think with GigE you are likely to find at least as many places where
    disabling auto is going to cause problems rather than solve them.

    --
    Regards

    - replace xyz with ntl
    stephen, Jun 6, 2007
    #3
  4. Inthemix88

    Trendkill Guest

    On Jun 6, 3:33 pm, "stephen" <> wrote:
    > "jw" <> wrote in message
    >
    > news:46670847$0$4332$...
    >
    >
    >
    > > Ibm servers frequently have issues with auto negotiation.
    > > Check ALL your ibm servers for sp/dplx harcoding and errors on those

    > ports.
    > > Trendkill wrote:
    > > > On Jun 6, 9:05 am, Inthemix88 <> wrote:
    > > >> Good morning all,
    > > >> We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    > > >> the NICs reports that it is connected at 100 Mb instead of 1000. We
    > > >> are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    > > >> (20). I know this is not the latest version but all our switches run
    > > >> the same IOS and we are having this problem only on 3 out of 8
    > > >> buildings. Our servers are IBM 336 with the previously mentioned NIC
    > > >> cards. We have replaced the motherboard on the IBM servers. We updated
    > > >> the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    > > >> updated one server to sp2, problem still exists). We have updated the
    > > >> NIC card driver to the latest version. We swapped the cables out using
    > > >> cat 5e and 6. We also tried different ports on the blade and switch
    > > >> (we have 2 48 port gigabit blades).
    > > >> If anyone can give us some feedback, it would be greatly appreciated.

    >
    > > >> Thanks in advanced,
    > > >> Martin Reyes
    > > >> LHRIC

    >
    > > > Not to suggest the obvious, but have you hard coded both sides (switch
    > > > & server) to 1000/full per cisco best practices.

    >
    > Not sure this is true for GigE.
    >
    > although a lot of devices had problems with 100 Base-TX, the IEEE standard
    > says auto is mandatory for 1000 Base-T, since it is used to sort out master
    > / slave etc.http://www.ethermanage.com/ethernet/autoneg.html
    >
    > Even on GigE fibre where the 2 devices dont have to negotiate, some devices
    > will not work unless auto- negotiate is enabled on the other device
    > - I stumbled across this on a Foundry L3 switch recently, connecting to a
    > GigE on an SDH mux.
    >
    > While auto/auto on
    >
    > > > both sides is also acceptable (for gig only), you should really stick
    > > > to a plan of hard coding both sides as it makes for better long term
    > > > support, documentation, and maintenance.

    >
    > i think with GigE you are likely to find at least as many places where
    > disabling auto is going to cause problems rather than solve them.
    >
    > --
    > Regards
    >
    > - replace xyz with ntl


    I agree with you, although if he's having issues with auto, he might
    as well try hard-coding. There is no one answer with gig, but you
    have to set both sides to the same regardless of which way you go. At
    my previous employer, we had a lot of issues with IBM boxes on boot
    and auto-mdix. The command was hidden, but we had tier 3 at TAC and
    IBM on the phone and no one knew enough to own up. All I can tell you
    is those damn Regattas were sending some kind of pulse that caused
    auto-mdix to trip, and we eventually had to set auto-mdix to disable
    on all our switch ports for these boxes. Fixed the problem right
    away, although never got an official explanation.
    Trendkill, Jun 6, 2007
    #4
  5. Inthemix88

    jw Guest

    Ibm servers frequently have issues with auto negotiation.
    Check ALL your ibm servers for sp/dplx harcoding and errors on those ports.
    Trendkill wrote:
    > On Jun 6, 9:05 am, Inthemix88 <> wrote:
    >> Good morning all,
    >> We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    >> the NICs reports that it is connected at 100 Mb instead of 1000. We
    >> are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    >> (20). I know this is not the latest version but all our switches run
    >> the same IOS and we are having this problem only on 3 out of 8
    >> buildings. Our servers are IBM 336 with the previously mentioned NIC
    >> cards. We have replaced the motherboard on the IBM servers. We updated
    >> the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    >> updated one server to sp2, problem still exists). We have updated the
    >> NIC card driver to the latest version. We swapped the cables out using
    >> cat 5e and 6. We also tried different ports on the blade and switch
    >> (we have 2 48 port gigabit blades).
    >> If anyone can give us some feedback, it would be greatly appreciated.
    >>
    >> Thanks in advanced,
    >> Martin Reyes
    >> LHRIC

    >
    > Not to suggest the obvious, but have you hard coded both sides (switch
    > & server) to 1000/full per cisco best practices. While auto/auto on
    > both sides is also acceptable (for gig only), you should really stick
    > to a plan of hard coding both sides as it makes for better long term
    > support, documentation, and maintenance.
    >
    jw, Jun 6, 2007
    #5
  6. Inthemix88

    jw Guest

    I have never seen an issue where hardcoding was a problem with gige or not.
    I always hardcode sp/dpx, and I have seen at least 4 ibm servers where
    throughput was down to about one meg when auto was used.


    stephen wrote:
    > "jw" <> wrote in message
    > news:46670847$0$4332$...
    >> Ibm servers frequently have issues with auto negotiation.
    >> Check ALL your ibm servers for sp/dplx harcoding and errors on those

    > ports.
    >> Trendkill wrote:
    >>> On Jun 6, 9:05 am, Inthemix88 <> wrote:
    >>>> Good morning all,
    >>>> We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    >>>> the NICs reports that it is connected at 100 Mb instead of 1000. We
    >>>> are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    >>>> (20). I know this is not the latest version but all our switches run
    >>>> the same IOS and we are having this problem only on 3 out of 8
    >>>> buildings. Our servers are IBM 336 with the previously mentioned NIC
    >>>> cards. We have replaced the motherboard on the IBM servers. We updated
    >>>> the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    >>>> updated one server to sp2, problem still exists). We have updated the
    >>>> NIC card driver to the latest version. We swapped the cables out using
    >>>> cat 5e and 6. We also tried different ports on the blade and switch
    >>>> (we have 2 48 port gigabit blades).
    >>>> If anyone can give us some feedback, it would be greatly appreciated.
    >>>>
    >>>> Thanks in advanced,
    >>>> Martin Reyes
    >>>> LHRIC
    >>> Not to suggest the obvious, but have you hard coded both sides (switch
    >>> & server) to 1000/full per cisco best practices.

    >
    > Not sure this is true for GigE.
    >
    > although a lot of devices had problems with 100 Base-TX, the IEEE standard
    > says auto is mandatory for 1000 Base-T, since it is used to sort out master
    > / slave etc.
    > http://www.ethermanage.com/ethernet/autoneg.html
    >
    > Even on GigE fibre where the 2 devices dont have to negotiate, some devices
    > will not work unless auto- negotiate is enabled on the other device
    > - I stumbled across this on a Foundry L3 switch recently, connecting to a
    > GigE on an SDH mux.
    >
    > While auto/auto on
    >>> both sides is also acceptable (for gig only), you should really stick
    >>> to a plan of hard coding both sides as it makes for better long term
    >>> support, documentation, and maintenance.
    >>>

    >
    > i think with GigE you are likely to find at least as many places where
    > disabling auto is going to cause problems rather than solve them.
    >
    jw, Jun 7, 2007
    #6
  7. Inthemix88

    Sam Wilson Guest

    In article <m_D9i.13522$>,
    "stephen" <> wrote:

    > "jw" <> wrote in message
    > news:46670847$0$4332$...
    > > Ibm servers frequently have issues with auto negotiation.
    > > Check ALL your ibm servers for sp/dplx harcoding and errors on those

    > ports.
    > > Trendkill wrote:
    > > > Not to suggest the obvious, but have you hard coded both sides (switch
    > > > & server) to 1000/full per cisco best practices.

    >
    > Not sure this is true for GigE.
    >
    > although a lot of devices had problems with 100 Base-TX, the IEEE standard
    > says auto is mandatory for 1000 Base-T, since it is used to sort out master
    > / slave etc.
    > http://www.ethermanage.com/ethernet/autoneg.html
    >
    > Even on GigE fibre where the 2 devices dont have to negotiate, some devices
    > will not work unless auto- negotiate is enabled on the other device
    > - I stumbled across this on a Foundry L3 switch recently, connecting to a
    > GigE on an SDH mux.


    I had understood (though I can't find a reference just now) that
    matching negotiation stances was required in all kinds of GigE, thus
    avoiding the duplex mismatch problem you get with 10/100 ethernet. That
    is, if one end tries to negotiate and the other won't then the link
    won't come up, though that does lead to the weird-seeming situation
    where the hard-set end of the link comes up and the negotiating one
    doesn't.

    > While auto/auto on
    > > > both sides is also acceptable (for gig only), you should really stick
    > > > to a plan of hard coding both sides as it makes for better long term
    > > > support, documentation, and maintenance.
    > > >

    >
    > i think with GigE you are likely to find at least as many places where
    > disabling auto is going to cause problems rather than solve them.


    Even on 10/100 we're beginning to find more problems with disabling auto
    than with setting it - we're probably going to change to auto by default
    for all our servers.

    Sam
    Sam Wilson, Jun 7, 2007
    #7
  8. Inthemix88

    Inthemix88 Guest

    Hello all,
    Thanks for replying. There is no way to hard set gigabite on the cards
    or the Cisco 4507R. If you issue the speed command on the cisco box to
    configure the port to gigabit ethernet, It will default to AUTO. If
    you look at the properties of the NIC, there is no place to choose
    1000 / FULL Duplex except to pick auto. This is why my problem is so
    weird. I have the same setup in another 5 buildings and have no
    issues. All hardware is the same, all firmware is the same, all cisco
    IOS is the same.
    Thanks for the replies.
    Martin Reyes

    On Jun 7, 4:52 am, Sam Wilson <> wrote:
    > In article <m_D9i.13522$>,
    >
    >
    >
    > "stephen" <> wrote:
    > > "jw" <> wrote in message
    > >news:46670847$0$4332$...
    > > > Ibm servers frequently have issues with auto negotiation.
    > > > Check ALL your ibm servers for sp/dplx harcoding and errors on those

    > > ports.
    Inthemix88, Jun 8, 2007
    #8
  9. Inthemix88

    Inthemix88 Guest

    Hello all,
    Thanks for replying. There is no way to hard set gigabite on the cards
    or the Cisco 4507R. If you issue the speed command on the cisco box to
    configure the port to gigabit ethernet, It will default to AUTO. If
    you look at the properties of the NIC, there is no place to choose
    1000 / FULL Duplex except to pick auto. This is why my problem is so
    weird. I have the same setup in another 5 buildings and have no
    issues. All hardware is the same, all firmware is the same, all cisco
    IOS is the same.
    Thanks for the replies.
    Martin Reyes

    On Jun 7, 4:52 am, Sam Wilson <> wrote:
    > In article <m_D9i.13522$>,
    >
    >
    >
    > "stephen" <> wrote:
    > > "jw" <> wrote in message
    > >news:46670847$0$4332$...
    > > > Ibm servers frequently have issues with auto negotiation.
    > > > Check ALL your ibm servers for sp/dplx harcoding and errors on those

    > > ports.
    Inthemix88, Jun 8, 2007
    #9
  10. Inthemix88

    Merv Guest

    Suggest posting the output of show module and show diag from a problem-
    free switch and same froma troubled switch
    Merv, Jun 9, 2007
    #10
  11. Inthemix88

    Guest

    On 9 Jun, 22:40, Merv <> wrote:
    > Suggest posting the output of show module and show diag from a problem-
    > free switch and same froma troubled switch


    I know that you have checked/changed the cables
    but I thought I would mention that GBE uses all 4
    pairs in CATx and 100M uses only two.

    Bad cables WILL cause this exact symptom.
    I would want to very carefully check
    cabling end to end for correct pairing for a start.
    Easiset way for me would be to use known "good"
    GBE devices.
    , Jun 10, 2007
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Richard Graves

    Catalyst switches and Broadcom NICs

    Richard Graves, May 13, 2005, in forum: Cisco
    Replies:
    2
    Views:
    3,692
    PLANCKAERT Nicolas
    May 17, 2005
  2. Andreas Eckert
    Replies:
    2
    Views:
    1,767
    Andreas Eckert
    May 16, 2005
  3. Ned
    Replies:
    2
    Views:
    3,254
  4. Teros
    Replies:
    7
    Views:
    6,708
    jimjawn
    Sep 30, 2011
  5. stumpz
    Replies:
    5
    Views:
    3,303
    Andrey Tarasov
    Jan 27, 2006
Loading...

Share This Page