Catalysts trouble

Discussion in 'Cisco' started by grzybek, Feb 2, 2004.

  1. grzybek

    grzybek Guest

    Hi,
    We have problem in our inside extensive network built by Cisco 3350-12 G,
    3550 XL, 3508 , 6006
    This network in base on 2 Laver with one central router Cat 6006.
    From Sunday in our switches on Gigabit interfaces links are forced to down
    on any switch.

    Logs from switches:
    1y11w: %SYS-2-MALLOCFAIL: Memory allocation of 1680 bytes failed from
    0x13CC08,
    pool I/O, alignment 0

    1y11w: %ETHCNTR-3-LOOP_BACK_DETECTED: Loop-back detected on
    GigabitEthernet0/xx.
    The port is forced to linkdown.

    1y11w: %SYS-3-CPUHOG: Task ran for 4476 msec (2481/52), process = Net Input,
    PC
    = 1529CC.

    Any bug ?

    I only enable udlp on the switches.

    Thanks in advance.
    grzybek
     
    grzybek, Feb 2, 2004
    #1
    1. Advertisements

  2. grzybek

    Paul Stewart Guest

    Have you upgraded into the last IOS/CatOS code? I noticed your uptime
    as 1 Year, 11 weeks.. that's what made me think of this....

    Others may disagree but that's a very long time for a switch to be
    online without any code upgrades... we upgrade all our switches as
    soon as upgrades come out and it hasn't come back to haunt us yet...:)
     
    Paul Stewart, Feb 2, 2004
    #2
    1. Advertisements

  3. grzybek

    Gert Doering Guest

    This is a somewhat philosophical debate. You're inducing a down time for
    *every* upgrade - which adds up.

    My main core switch (Cat 5500) has an uptime of...

    Uptime is 737 days, 17 hours, 25 minutes

    .... which means "*no* down time for over two years". I like that.

    Of course we have the most recent software available, and we're reading
    the release notes to watch out for things that could bite us. So if the
    switch would start acting up, we're prepared.

    gert
     
    Gert Doering, Feb 14, 2004
    #3
  4. grzybek

    bartje29 Guest

    Hi,

    We had a similar problem in our datacenter yesterday.
    We lost all layer-2 connectivity during about 3 minutes. ( looks like
    spanning-tree was recalculated during three minutes , impact on all
    trunks ).
    Just before the event we had a similar error in one of our 3550-XL
    catalysts :

    Nov 23 11:48:38.711: %SYS-2-MALLOCFAIL: Memory allocation of 1680 bytes
    failed from 0x177858, alignment 0

    Pool: I/O Free: 2796 Cause: Memory fragmentation

    Alternate Pool: None Free: 0 Cause: No Alternate pool



    -Process= "Pool Manager", ipl= 0, pid= 5

    -Traceback= 1BFE34 1C1488 17785C 1D064C 1D0894 1E0C44 1E423C


    Nov 23 11:49:08.740: %SYS-2-MALLOCFAIL: Memory allocation of 1680 bytes
    failed from 0x177858, alignment 0

    Pool: I/O Free: 2796 Cause: Memory fragmentation

    Alternate Pool: None Free: 0 Cause: No Alternate pool



    -Process= "Pool Manager", ipl= 0, pid= 5

    -Traceback= 1BFE34 1C1488 17785C 1D064C 1D07BC 1E0C44 1E423C

    Nov 23 11:49:38.789: %SYS-2-MALLOCFAIL: Memory allocation of 1680 bytes
    failed from 0x177858, alignment 0

    Pool: I/O Free: 2796 Cause: Memory fragmentation

    Alternate Pool: None Free: 0 Cause: No Alternate pool



    -Process= "Pool Manager", ipl= 0, pid= 5

    -Traceback= 1BFE34 1C1488 17785C 1D064C 1D07BC 1E0C44 1E423C





    Can I get some more info about the problem you had , and how did you
    solve it. I guess IOS upgrade.
    Allthough we are not 100% sure that this caused our downtime. Just a
    possible cause.

    Bart
     
    bartje29, Nov 24, 2004
    #4
  5. grzybek

    Ivan Ostreš Guest

    A process wanted 1680 bytes of memory but there wasn't any free memory.
    It was from I/O poll which was fragmented.
    Pool Manager was doing allocation.


    So, what can you do?

    1) open Bug navigator on CCO and look for memory leak bugs. If you found
    a bug and it is RESOLVED, then upgrade to image that has fix

    2) open a TAC case and if they do not have image that fixes that problem
    (unlikely) they willl create fix
     
    Ivan Ostreš, Nov 24, 2004
    #5
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.