Cisco 4507R and Broadcom NetXtreme NICs

Discussion in 'Cisco' started by Inthemix88, Jun 6, 2007.

  1. Inthemix88

    Inthemix88 Guest

    Good morning all,
    We are having an issue with Broabcom Gigabit NetXtreme Nics. One of
    the NICs reports that it is connected at 100 Mb instead of 1000. We
    are running a Cisco 4507R on the backend. Cisco IOS Version is 12.2
    (20). I know this is not the latest version but all our switches run
    the same IOS and we are having this problem only on 3 out of 8
    buildings. Our servers are IBM 336 with the previously mentioned NIC
    cards. We have replaced the motherboard on the IBM servers. We updated
    the firmware on the NIC cards. Our NOS is Windows 2003 sp1. (We
    updated one server to sp2, problem still exists). We have updated the
    NIC card driver to the latest version. We swapped the cables out using
    cat 5e and 6. We also tried different ports on the blade and switch
    (we have 2 48 port gigabit blades).
    If anyone can give us some feedback, it would be greatly appreciated.

    Thanks in advanced,
    Martin Reyes
    LHRIC
     
    Inthemix88, Jun 6, 2007
    #1
    1. Advertisements

  2. Inthemix88

    Trendkill Guest

    Not to suggest the obvious, but have you hard coded both sides (switch
    & server) to 1000/full per cisco best practices. While auto/auto on
    both sides is also acceptable (for gig only), you should really stick
    to a plan of hard coding both sides as it makes for better long term
    support, documentation, and maintenance.
     
    Trendkill, Jun 6, 2007
    #2
    1. Advertisements

  3. Inthemix88

    stephen Guest

    Not sure this is true for GigE.

    although a lot of devices had problems with 100 Base-TX, the IEEE standard
    says auto is mandatory for 1000 Base-T, since it is used to sort out master
    / slave etc.
    http://www.ethermanage.com/ethernet/autoneg.html

    Even on GigE fibre where the 2 devices dont have to negotiate, some devices
    will not work unless auto- negotiate is enabled on the other device
    - I stumbled across this on a Foundry L3 switch recently, connecting to a
    GigE on an SDH mux.

    While auto/auto on
    i think with GigE you are likely to find at least as many places where
    disabling auto is going to cause problems rather than solve them.
     
    stephen, Jun 6, 2007
    #3
  4. Inthemix88

    Trendkill Guest

    I agree with you, although if he's having issues with auto, he might
    as well try hard-coding. There is no one answer with gig, but you
    have to set both sides to the same regardless of which way you go. At
    my previous employer, we had a lot of issues with IBM boxes on boot
    and auto-mdix. The command was hidden, but we had tier 3 at TAC and
    IBM on the phone and no one knew enough to own up. All I can tell you
    is those damn Regattas were sending some kind of pulse that caused
    auto-mdix to trip, and we eventually had to set auto-mdix to disable
    on all our switch ports for these boxes. Fixed the problem right
    away, although never got an official explanation.
     
    Trendkill, Jun 6, 2007
    #4
  5. Inthemix88

    jw Guest

    Ibm servers frequently have issues with auto negotiation.
    Check ALL your ibm servers for sp/dplx harcoding and errors on those ports.
     
    jw, Jun 6, 2007
    #5
  6. Inthemix88

    jw Guest

    I have never seen an issue where hardcoding was a problem with gige or not.
    I always hardcode sp/dpx, and I have seen at least 4 ibm servers where
    throughput was down to about one meg when auto was used.
     
    jw, Jun 7, 2007
    #6
  7. Inthemix88

    Sam Wilson Guest

    I had understood (though I can't find a reference just now) that
    matching negotiation stances was required in all kinds of GigE, thus
    avoiding the duplex mismatch problem you get with 10/100 ethernet. That
    is, if one end tries to negotiate and the other won't then the link
    won't come up, though that does lead to the weird-seeming situation
    where the hard-set end of the link comes up and the negotiating one
    doesn't.
    Even on 10/100 we're beginning to find more problems with disabling auto
    than with setting it - we're probably going to change to auto by default
    for all our servers.

    Sam
     
    Sam Wilson, Jun 7, 2007
    #7
  8. Inthemix88

    Inthemix88 Guest

    Hello all,
    Thanks for replying. There is no way to hard set gigabite on the cards
    or the Cisco 4507R. If you issue the speed command on the cisco box to
    configure the port to gigabit ethernet, It will default to AUTO. If
    you look at the properties of the NIC, there is no place to choose
    1000 / FULL Duplex except to pick auto. This is why my problem is so
    weird. I have the same setup in another 5 buildings and have no
    issues. All hardware is the same, all firmware is the same, all cisco
    IOS is the same.
    Thanks for the replies.
    Martin Reyes
     
    Inthemix88, Jun 8, 2007
    #8
  9. Inthemix88

    Inthemix88 Guest

    Hello all,
    Thanks for replying. There is no way to hard set gigabite on the cards
    or the Cisco 4507R. If you issue the speed command on the cisco box to
    configure the port to gigabit ethernet, It will default to AUTO. If
    you look at the properties of the NIC, there is no place to choose
    1000 / FULL Duplex except to pick auto. This is why my problem is so
    weird. I have the same setup in another 5 buildings and have no
    issues. All hardware is the same, all firmware is the same, all cisco
    IOS is the same.
    Thanks for the replies.
    Martin Reyes
     
    Inthemix88, Jun 8, 2007
    #9
  10. Inthemix88

    Merv Guest

    Suggest posting the output of show module and show diag from a problem-
    free switch and same froma troubled switch
     
    Merv, Jun 9, 2007
    #10
  11. Inthemix88

    Bod43 Guest

    I know that you have checked/changed the cables
    but I thought I would mention that GBE uses all 4
    pairs in CATx and 100M uses only two.

    Bad cables WILL cause this exact symptom.
    I would want to very carefully check
    cabling end to end for correct pairing for a start.
    Easiset way for me would be to use known "good"
    GBE devices.
     
    Bod43, Jun 10, 2007
    #11
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.