Gigabit adapters in HP Proliant Server running Windows 2003

Discussion in 'Cisco' started by The Sabos, Oct 5, 2007.

  1. The Sabos

    The Sabos Guest

    We have an 6513 Cisco router and we have some HP Proliant servers running
    Windows 2003 with gigabit adapters. Ports are set to 1000 on the router
    and the server are set to 1000/Full with flow control at auto. But when
    we do a windows copy from server to server we are only getting 230 MBps at
    the highest throughput rate. Should we be getting more throughput than
    this?



    Any suggestion would be appreciated.

    Thanks in Advanced.
    Eric Sabo
     
    The Sabos, Oct 5, 2007
    #1
    1. Advertising

  2. The Sabos

    Trendkill Guest

    On Oct 5, 4:10 pm, "The Sabos" <> wrote:
    > We have an 6513 Cisco router and we have some HP Proliant servers running
    > Windows 2003 with gigabit adapters. Ports are set to 1000 on the router
    > and the server are set to 1000/Full with flow control at auto. But when
    > we do a windows copy from server to server we are only getting 230 MBps at
    > the highest throughput rate. Should we be getting more throughput than
    > this?
    >
    > Any suggestion would be appreciated.
    >
    > Thanks in Advanced.
    > Eric Sabo


    All depends on the server, backplane, protocol, etc. Are the source
    and destination on different switches? Are they in the same network?
    How much b/w do you have across your backbone if it is being routed or
    pushed across switches? How much utilization do you have across those
    links? If all of that checks out, how fast are your drives, what is
    the utilization of the server itself during that transfer?
    Additionally, windows copy is terrible from my experience, have you
    tried ftp? How does that work for you?
     
    Trendkill, Oct 5, 2007
    #2
    1. Advertising

  3. The Sabos

    alexd Guest

    The Sabos wrote:

    > We have an 6513 Cisco router and we have some HP Proliant servers running
    > Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    > router
    > and the server are set to 1000/Full with flow control at auto. But when
    > we do a windows copy from server to server we are only getting 230 MBps at
    > the highest throughput rate.


    What do you get with those two servers on a different gigabit switch?

    > Should we be getting more throughput than this?


    It's a pretty difficult question to answer, given how many variables there
    are. If you're just interested in the raw throughput of the switch from the
    servers, disregarding disk speed etc, try testing it with iperf:

    http://dast.nlanr.net/Projects/Iperf/

    I suggest you avoid the Java version, as it tends to max out your CPU before
    your network. Getting 93Mbps on a switched 100Mbit network here, btw.

    --
    <http://ale.cx/> (AIM:troffasky) ()
    21:19:47 up 4 days, 23:07, 2 users, load average: 0.17, 0.19, 0.17
    09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0
     
    alexd, Oct 5, 2007
    #3
  4. The Sabos

    The Sabos Guest

    All servers are in the same VLAN.

    How does one find how much utilization between the links?

    Yes, thanks for all the information. I understand that Windows Copy is
    very bad but we haven't tried FTP yet?



    "Trendkill" <> wrote in message
    news:...
    > On Oct 5, 4:10 pm, "The Sabos" <> wrote:
    >> We have an 6513 Cisco router and we have some HP Proliant servers running
    >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    >> router
    >> and the server are set to 1000/Full with flow control at auto. But
    >> when
    >> we do a windows copy from server to server we are only getting 230 MBps
    >> at
    >> the highest throughput rate. Should we be getting more throughput than
    >> this?
    >>
    >> Any suggestion would be appreciated.
    >>
    >> Thanks in Advanced.
    >> Eric Sabo

    >
    > All depends on the server, backplane, protocol, etc. Are the source
    > and destination on different switches? Are they in the same network?
    > How much b/w do you have across your backbone if it is being routed or
    > pushed across switches? How much utilization do you have across those
    > links? If all of that checks out, how fast are your drives, what is
    > the utilization of the server itself during that transfer?
    > Additionally, windows copy is terrible from my experience, have you
    > tried ftp? How does that work for you?
    >
     
    The Sabos, Oct 6, 2007
    #4
  5. The Sabos

    Trendkill Guest

    On Oct 5, 7:02 pm, "The Sabos" <> wrote:
    > All servers are in the same VLAN.
    >
    > How does one find how much utilization between the links?
    >
    > Yes, thanks for all the information. I understand that Windows Copy is
    > very bad but we haven't tried FTP yet?
    >
    > "Trendkill" <> wrote in message
    >
    > news:...
    >
    > > On Oct 5, 4:10 pm, "The Sabos" <> wrote:
    > >> We have an 6513 Cisco router and we have some HP Proliant servers running
    > >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    > >> router
    > >> and the server are set to 1000/Full with flow control at auto. But
    > >> when
    > >> we do a windows copy from server to server we are only getting 230 MBps
    > >> at
    > >> the highest throughput rate. Should we be getting more throughput than
    > >> this?

    >
    > >> Any suggestion would be appreciated.

    >
    > >> Thanks in Advanced.
    > >> Eric Sabo

    >
    > > All depends on the server, backplane, protocol, etc. Are the source
    > > and destination on different switches? Are they in the same network?
    > > How much b/w do you have across your backbone if it is being routed or
    > > pushed across switches? How much utilization do you have across those
    > > links? If all of that checks out, how fast are your drives, what is
    > > the utilization of the server itself during that transfer?
    > > Additionally, windows copy is terrible from my experience, have you
    > > tried ftp? How does that work for you?


    A SNMP monitoring tool will work. Not sure of any free ones right
    offhand, but I'm sure some others can recommend some. However, 200+
    mbps is not bad in my opinion for windows copy, so I would try ftp
    first and see how you fare. I have seen etherchannel gig to servers
    run at 2-3 gigs per second, but for most windows based single-gig
    boxes, several hundred meg is probably par for the course. Not to say
    a well tuned box can't get to 8-900 mbps, but does take a powerful,
    well-tuned box. And to answer your question, a proper server should
    push over 90% utilization of its link, but when you get to gig or
    multi-gig, there are a lot more thresholds you start to hit, many of
    which are hardware.

    In short, try the ftp and let us know.
     
    Trendkill, Oct 6, 2007
    #5
  6. The Sabos

    The Sabos Guest

    How does one do a simple test with IPerf?


    "alexd" <> wrote in message
    news:...
    > The Sabos wrote:
    >
    >> We have an 6513 Cisco router and we have some HP Proliant servers running
    >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    >> router
    >> and the server are set to 1000/Full with flow control at auto. But
    >> when
    >> we do a windows copy from server to server we are only getting 230 MBps
    >> at
    >> the highest throughput rate.

    >
    > What do you get with those two servers on a different gigabit switch?
    >
    >> Should we be getting more throughput than this?

    >
    > It's a pretty difficult question to answer, given how many variables there
    > are. If you're just interested in the raw throughput of the switch from
    > the
    > servers, disregarding disk speed etc, try testing it with iperf:
    >
    > http://dast.nlanr.net/Projects/Iperf/
    >
    > I suggest you avoid the Java version, as it tends to max out your CPU
    > before
    > your network. Getting 93Mbps on a switched 100Mbit network here, btw.
    >
    > --
    > <http://ale.cx/> (AIM:troffasky) ()
    > 21:19:47 up 4 days, 23:07, 2 users, load average: 0.17, 0.19, 0.17
    > 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0
    >
     
    The Sabos, Oct 6, 2007
    #6
  7. The Sabos

    Brian V Guest

    "The Sabos" <> wrote in message
    news:6ozNi.61$C2.19@trnddc02...
    > All servers are in the same VLAN.
    >
    > How does one find how much utilization between the links?
    >
    > Yes, thanks for all the information. I understand that Windows Copy
    > is very bad but we haven't tried FTP yet?
    >
    >
    >
    > "Trendkill" <> wrote in message
    > news:...
    >> On Oct 5, 4:10 pm, "The Sabos" <> wrote:
    >>> We have an 6513 Cisco router and we have some HP Proliant servers
    >>> running
    >>> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    >>> router
    >>> and the server are set to 1000/Full with flow control at auto. But
    >>> when
    >>> we do a windows copy from server to server we are only getting 230 MBps
    >>> at
    >>> the highest throughput rate. Should we be getting more throughput
    >>> than
    >>> this?
    >>>
    >>> Any suggestion would be appreciated.
    >>>
    >>> Thanks in Advanced.
    >>> Eric Sabo

    >>
    >> All depends on the server, backplane, protocol, etc. Are the source
    >> and destination on different switches? Are they in the same network?
    >> How much b/w do you have across your backbone if it is being routed or
    >> pushed across switches? How much utilization do you have across those
    >> links? If all of that checks out, how fast are your drives, what is
    >> the utilization of the server itself during that transfer?
    >> Additionally, windows copy is terrible from my experience, have you
    >> tried ftp? How does that work for you?
    >>

    >


    Are you sure it's big "B"? Thats MegaBYTES per second. Switches are rated in
    bits (little b) per second. If it truely is bytes per second, you've
    exceeded gigabit speed (bytes X 8 = bits) 2.2mbps. Where are you getting
    your numbers from?
     
    Brian V, Oct 6, 2007
    #7
  8. The Sabos

    The Sabos Guest

    I figure out how to use the program.

    From my computer to the server I am getting 93.2 MB

    From a server (gigabit connect) to server (gigabit connect) - 333 MB

    With this numbers is this okay for the CISCO 6513 Router?

    I would have thought it would have been about 400-500 MB at least.

    Is there anything we can do to improve the speed? TCP windows size was 8
    KBytes


    "alexd" <> wrote in message
    news:...
    > The Sabos wrote:
    >
    >> We have an 6513 Cisco router and we have some HP Proliant servers running
    >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    >> router
    >> and the server are set to 1000/Full with flow control at auto. But
    >> when
    >> we do a windows copy from server to server we are only getting 230 MBps
    >> at
    >> the highest throughput rate.

    >
    > What do you get with those two servers on a different gigabit switch?
    >
    >> Should we be getting more throughput than this?

    >
    > It's a pretty difficult question to answer, given how many variables there
    > are. If you're just interested in the raw throughput of the switch from
    > the
    > servers, disregarding disk speed etc, try testing it with iperf:
    >
    > http://dast.nlanr.net/Projects/Iperf/
    >
    > I suggest you avoid the Java version, as it tends to max out your CPU
    > before
    > your network. Getting 93Mbps on a switched 100Mbit network here, btw.
    >
    > --
    > <http://ale.cx/> (AIM:troffasky) ()
    > 21:19:47 up 4 days, 23:07, 2 users, load average: 0.17, 0.19, 0.17
    > 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0
    >
     
    The Sabos, Oct 6, 2007
    #8
  9. The Sabos

    Trendkill Guest

    On Oct 5, 8:14 pm, "The Sabos" <> wrote:
    > I figure out how to use the program.
    >
    > From my computer to the server I am getting 93.2 MB
    >
    > From a server (gigabit connect) to server (gigabit connect) - 333 MB
    >
    > With this numbers is this okay for the CISCO 6513 Router?
    >
    > I would have thought it would have been about 400-500 MB at least.
    >
    > Is there anything we can do to improve the speed? TCP windows size was 8
    > KBytes
    >
    > "alexd" <> wrote in message
    >
    > news:...
    >
    > > The Sabos wrote:

    >
    > >> We have an 6513 Cisco router and we have some HP Proliant servers running
    > >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    > >> router
    > >> and the server are set to 1000/Full with flow control at auto. But
    > >> when
    > >> we do a windows copy from server to server we are only getting 230 MBps
    > >> at
    > >> the highest throughput rate.

    >
    > > What do you get with those two servers on a different gigabit switch?

    >
    > >> Should we be getting more throughput than this?

    >
    > > It's a pretty difficult question to answer, given how many variables there
    > > are. If you're just interested in the raw throughput of the switch from
    > > the
    > > servers, disregarding disk speed etc, try testing it with iperf:

    >
    > >http://dast.nlanr.net/Projects/Iperf/

    >
    > > I suggest you avoid the Java version, as it tends to max out your CPU
    > > before
    > > your network. Getting 93Mbps on a switched 100Mbit network here, btw.

    >
    > > --
    > > <http://ale.cx/> (AIM:troffasky) ()
    > > 21:19:47 up 4 days, 23:07, 2 users, load average: 0.17, 0.19, 0.17
    > > 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0


    Unless there is some architectural component I am not aware of, this
    must be a source/destination server issue. I have 6500 series
    switches that consistently push between 2 and 8 gigs per second
    without batting an eye. As for single server bandwidth, I have seen
    2-3 gigs per second for tivoli backup boxes (4 gig etherchannel to the
    server itself), and for single-gig connections, I have seen 8-900 megs
    fairly regularly. Granted these are almost all non-wintel boxes, and
    are usually very large IBM nodes/system complexes or another flavor of
    unix. As for a windows server that is tuned and has some good
    hardware, i have seen 3-400 meg, but have never really watched them
    too closely. All in all, I am guessing you have hit a threshold on
    your server or with whatever copy program you are using, but your
    performance seems within my expectations.
     
    Trendkill, Oct 6, 2007
    #9
  10. The Sabos

    The Sabos Guest

    I think I have found what the problem is, I think I need to up the size of
    the TCPWindowSize.

    The problem is we are trying to back up over the network and the more speed
    we get the fast it will go.

    Thanks for all your help and input on this issue.


    "Trendkill" <> wrote in message
    news:...
    > On Oct 5, 8:14 pm, "The Sabos" <> wrote:
    >> I figure out how to use the program.
    >>
    >> From my computer to the server I am getting 93.2 MB
    >>
    >> From a server (gigabit connect) to server (gigabit connect) - 333 MB
    >>
    >> With this numbers is this okay for the CISCO 6513 Router?
    >>
    >> I would have thought it would have been about 400-500 MB at least.
    >>
    >> Is there anything we can do to improve the speed? TCP windows size was
    >> 8
    >> KBytes
    >>
    >> "alexd" <> wrote in message
    >>
    >> news:...
    >>
    >> > The Sabos wrote:

    >>
    >> >> We have an 6513 Cisco router and we have some HP Proliant servers
    >> >> running
    >> >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    >> >> router
    >> >> and the server are set to 1000/Full with flow control at auto. But
    >> >> when
    >> >> we do a windows copy from server to server we are only getting 230
    >> >> MBps
    >> >> at
    >> >> the highest throughput rate.

    >>
    >> > What do you get with those two servers on a different gigabit switch?

    >>
    >> >> Should we be getting more throughput than this?

    >>
    >> > It's a pretty difficult question to answer, given how many variables
    >> > there
    >> > are. If you're just interested in the raw throughput of the switch from
    >> > the
    >> > servers, disregarding disk speed etc, try testing it with iperf:

    >>
    >> >http://dast.nlanr.net/Projects/Iperf/

    >>
    >> > I suggest you avoid the Java version, as it tends to max out your CPU
    >> > before
    >> > your network. Getting 93Mbps on a switched 100Mbit network here, btw.

    >>
    >> > --
    >> > <http://ale.cx/> (AIM:troffasky) ()
    >> > 21:19:47 up 4 days, 23:07, 2 users, load average: 0.17, 0.19, 0.17
    >> > 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0

    >
    > Unless there is some architectural component I am not aware of, this
    > must be a source/destination server issue. I have 6500 series
    > switches that consistently push between 2 and 8 gigs per second
    > without batting an eye. As for single server bandwidth, I have seen
    > 2-3 gigs per second for tivoli backup boxes (4 gig etherchannel to the
    > server itself), and for single-gig connections, I have seen 8-900 megs
    > fairly regularly. Granted these are almost all non-wintel boxes, and
    > are usually very large IBM nodes/system complexes or another flavor of
    > unix. As for a windows server that is tuned and has some good
    > hardware, i have seen 3-400 meg, but have never really watched them
    > too closely. All in all, I am guessing you have hit a threshold on
    > your server or with whatever copy program you are using, but your
    > performance seems within my expectations.
    >
     
    The Sabos, Oct 6, 2007
    #10
  11. The Sabos

    The Sabos Guest

    Brian,

    You are correct, it is MBytes. My initial numbers were from the network
    utility from HP but the later numbers are from the Iperf stats that I
    collected.



    "Brian V" <> wrote in message
    news:...
    >
    > "The Sabos" <> wrote in message
    > news:6ozNi.61$C2.19@trnddc02...
    >> All servers are in the same VLAN.
    >>
    >> How does one find how much utilization between the links?
    >>
    >> Yes, thanks for all the information. I understand that Windows Copy
    >> is very bad but we haven't tried FTP yet?
    >>
    >>
    >>
    >> "Trendkill" <> wrote in message
    >> news:...
    >>> On Oct 5, 4:10 pm, "The Sabos" <> wrote:
    >>>> We have an 6513 Cisco router and we have some HP Proliant servers
    >>>> running
    >>>> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    >>>> router
    >>>> and the server are set to 1000/Full with flow control at auto. But
    >>>> when
    >>>> we do a windows copy from server to server we are only getting 230 MBps
    >>>> at
    >>>> the highest throughput rate. Should we be getting more throughput
    >>>> than
    >>>> this?
    >>>>
    >>>> Any suggestion would be appreciated.
    >>>>
    >>>> Thanks in Advanced.
    >>>> Eric Sabo
    >>>
    >>> All depends on the server, backplane, protocol, etc. Are the source
    >>> and destination on different switches? Are they in the same network?
    >>> How much b/w do you have across your backbone if it is being routed or
    >>> pushed across switches? How much utilization do you have across those
    >>> links? If all of that checks out, how fast are your drives, what is
    >>> the utilization of the server itself during that transfer?
    >>> Additionally, windows copy is terrible from my experience, have you
    >>> tried ftp? How does that work for you?
    >>>

    >>

    >
    > Are you sure it's big "B"? Thats MegaBYTES per second. Switches are rated
    > in bits (little b) per second. If it truely is bytes per second, you've
    > exceeded gigabit speed (bytes X 8 = bits) 2.2mbps. Where are you getting
    > your numbers from?
     
    The Sabos, Oct 6, 2007
    #11
  12. The Sabos

    Guest

    On 6 Oct, 03:47, "The Sabos" <> wrote:
    > I think I have found what the problem is, I think I need to up the size of
    > the TCPWindowSize.
    >
    > The problem is we are trying to back up over the network and the more speed
    > we get the fast it will go.
    >
    > Thanks for all your help and input on this issue.
    >
    > "Trendkill" <> wrote in message
    >
    > news:...
    >
    >
    >
    > > On Oct 5, 8:14 pm, "The Sabos" <> wrote:
    > >> I figure out how to use the program.

    >
    > >> From my computer to the server I am getting 93.2 MB

    >
    > >> From a server (gigabit connect) to server (gigabit connect) - 333 MB

    >
    > >> With this numbers is this okay for the CISCO 6513 Router?

    >
    > >> I would have thought it would have been about 400-500 MB at least.

    >
    > >> Is there anything we can do to improve the speed? TCP windows size was
    > >> 8
    > >> KBytes

    >
    > >> "alexd" <> wrote in message

    >
    > >>news:...

    >
    > >> > The Sabos wrote:

    >
    > >> >> We have an 6513 Cisco router and we have some HP Proliant servers
    > >> >> running
    > >> >> Windows 2003 with gigabit adapters. Ports are set to 1000 on the
    > >> >> router
    > >> >> and the server are set to 1000/Full with flow control at auto. But
    > >> >> when
    > >> >> we do a windows copy from server to server we are only getting 230
    > >> >> MBps
    > >> >> at
    > >> >> the highest throughput rate.

    >
    > >> > What do you get with those two servers on a different gigabit switch?

    >
    > >> >> Should we be getting more throughput than this?

    >
    > >> > It's a pretty difficult question to answer, given how many variables
    > >> > there
    > >> > are. If you're just interested in the raw throughput of the switch from
    > >> > the
    > >> > servers, disregarding disk speed etc, try testing it with iperf:

    >
    > >> >http://dast.nlanr.net/Projects/Iperf/

    >
    > >> > I suggest you avoid the Java version, as it tends to max out your CPU
    > >> > before
    > >> > your network. Getting 93Mbps on a switched 100Mbit network here, btw.

    >
    > >> > --
    > >> > <http://ale.cx/> (AIM:troffasky) ()
    > >> > 21:19:47 up 4 days, 23:07, 2 users, load average: 0.17, 0.19, 0.17
    > >> > 09 f9 11 02 9d 74 e3 5b d8 41 56 c5 63 56 88 c0

    >
    > > Unless there is some architectural component I am not aware of, this
    > > must be a source/destination server issue. I have 6500 series
    > > switches that consistently push between 2 and 8 gigs per second
    > > without batting an eye. As for single server bandwidth, I have seen
    > > 2-3 gigs per second for tivoli backup boxes (4 gig etherchannel to the
    > > server itself), and for single-gig connections, I have seen 8-900 megs
    > > fairly regularly. Granted these are almost all non-wintel boxes, and
    > > are usually very large IBM nodes/system complexes or another flavor of
    > > unix. As for a windows server that is tuned and has some good
    > > hardware, i have seen 3-400 meg, but have never really watched them
    > > too closely. All in all, I am guessing you have hit a threshold on
    > > your server or with whatever copy program you are using, but your
    > > performance seems within my expectations.- Hide quoted text -


    For maximun backup throughput you may need to adjust
    max tcp receive window
    backup software - network block size
    backup software - tape block size
    backupo software - buffer, number of blocks

    Depending on your exact server model and
    backup hardware 1000Gbps may be approachable
    or not.

    The switch itself is not going to be
    a limitation.

    A KEY, let me repeat, KEY, issue is
    the block/window sizes versus the round trip
    time between the machines.
    The throughput is limited to

    RTT * block size.

    At 1 ms with a windows copy (absolute
    max block size of 64k). The throughput
    will be constrained to

    64,000,000 Bytes per sec.

    This is called the Bandwidth Delay product.

    If you change the block size in iperf
    you will be able to drive the network pretty hard.

    iperf -c -l 100000 x.x.x.x
    iperf -s -l 100000

    The default is 8k which I wold guess
    would not usually
    allow a 1G network to be saturated unless
    maybe you had a super machine (two:).

    The easiest way to fond the limit of the network hardware
    is to add iperf sessions

    iperf -s -l ...
    ipers -s -p 5002 -l ...
    iperf -s -p 5003 -l ...

    iperf -c -l ... x.x.x.x
    iperf -c -p -5002 -l .... x.x.x.x
    ........

    Until the aggregate throughput stops increasing.
    You have then eliminated individual iperf settings
    or behaviours as an issue. Something will
    truely, let me repeat TRUELY, be full up.

    Even ping can be used to generate high bandwidths
    if you have enough of them.

    c:\> for /l %i in (1, 1, 100) do start cmd /c
    fping x.x.x.x -s 1400 -t 0

    roughly

    fping.exe from http://www.kwakkelflap.com/

    The software is better than the name which
    I can never remember.
     
    , Oct 6, 2007
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. samul888@vip.163.com
    Replies:
    0
    Views:
    863
    samul888@vip.163.com
    Nov 10, 2005
  2. Replies:
    5
    Views:
    1,232
  3. Replies:
    7
    Views:
    551
  4. Replies:
    2
    Views:
    1,269
  5. Replies:
    0
    Views:
    540
Loading...

Share This Page