Re: TCP Window sizing issue

Discussion in 'Cisco' started by Stephen, May 7, 2010.

  1. Stephen

    Stephen Guest

    On Fri, 07 May 2010 11:31:38 -0700, Aaron Leonard <>
    wrote:

    >On Fri, 7 May 2010 06:44:46 -0700 (PDT), jonny2shoes <> wrote:
    >
    >~ I'm seeing some interesting behavior across our WAN. We I transfer TCP
    >~ traffic (ex. 1mb file) it will take about 1 minute. When I transfer
    >~ the same file via FTP it takes 5 seconds.
    >
    >?
    >
    >FTP just uses TCP. What protocol are you using when it takes 1 minute?
    >
    >SMB? If so, then please note that SMB (V1 at any rate) has notoriously
    >lousy performance on high latency links.


    SMB uses TCP, but the application file transfer moves "blocks" of data
    - up to 64K when it gets going with the right settings.

    so for every 64 Kbytes the file transfer "waits" for 1 round trip time
    across your WAN - that can dominate the transfer speed even with
    relatively low round trip delays on am 11 Mbps link.

    servers / PCs with fast LAN interfaces will dump that entire 64K lump
    transfer at LAN speed - so if the routers / switches or whatever
    facing the LAN cannot buffer most of that at the speed mismatch you
    may also lose packets in each block.

    SMB will just slectively resend the missing packets - but some TCP
    offload cards have been known to muck up the logic and add a timeout
    every time - in 1 case this slowed the transfer across a 10M link from
    5 to 6 Mbps to 600 Kbps.
    >


    >~ My WAN link is about 11 megs
    >~ and I'm not even close to pegging it with TCP file transfers. I think
    >~ it might have to do with TCP window sizing, when doing a packet dump I
    >~ see a bunch of checksum errors and it appears to retransmit a bit.
    >
    >Any errors / retransmissions will definitely slow you down a ton.
    >Get the errors fixed.


    You may find FTP is using a different packet size if the errors only
    occur in 1 type of transfer.
    >
    >~ The
    >~ files transfer fine across the LAN with no errors, i'm wondering if
    >~ there is some adjustments I should be making to the WAN egress
    >~ interfaces of the routers.
    >
    >Well, if you are losing/corrupting data, you'll definitely want to
    >fix the problem.
    >
    >Aaron

    --
    Regards

    - replace xyz with ntl
    Stephen, May 7, 2010
    #1
    1. Advertising

  2. Stephen

    bod43 Guest

    On 7 May, 20:03, Stephen <> wrote:
    > On Fri, 07 May 2010 11:31:38 -0700, Aaron Leonard <>
    > wrote:
    >
    > >On Fri, 7 May 2010 06:44:46 -0700 (PDT), jonny2shoes <> wrote:

    >
    > >~ I'm seeing some interesting behavior across our WAN. We I transfer TCP
    > >~ traffic (ex. 1mb file) it will take about 1 minute. When I transfer
    > >~ the same file via FTP it takes 5 seconds.

    >
    > >?

    >
    > >FTP just uses TCP.  What protocol are you using when it takes 1 minute?

    >
    > >SMB?  If so, then please note that SMB (V1 at any rate) has notoriously
    > >lousy performance on high latency links.

    >
    > SMB uses TCP, but the application file transfer moves "blocks" of data
    > - up to 64K when it gets going with the right settings.
    >
    > so for every 64 Kbytes the file transfer "waits" for 1 round trip time
    > across your WAN - that can dominate the transfer speed even with
    > relatively low round trip delays on am 11 Mbps link.
    >
    > servers / PCs with fast LAN interfaces will dump that entire 64K lump
    > transfer at LAN speed - so if the routers / switches or whatever
    > facing the LAN cannot buffer most of that at the speed mismatch you
    > may also lose packets in each block.
    >
    > SMB will just slectively resend the missing packets - but some TCP
    > offload cards have been known to muck up the logic and add a timeout
    > every time - in 1 case this slowed the transfer across a 10M link from
    > 5 to 6 Mbps to 600 Kbps.
    >
    >
    >
    > >~ My WAN link is about 11 megs
    > >~ and I'm not even close to pegging it with TCP file transfers. I think
    > >~ it might have to do with TCP window sizing, when doing a packet dump I
    > >~ see a bunch of checksum errors and it appears to retransmit a bit.

    >
    > >Any errors / retransmissions will definitely slow you down a ton.
    > >Get the errors fixed.

    >
    > You may find FTP is using a different packet size if the errors only
    > occur in 1 type of transfer.
    >
    > >~ The
    > >~ files transfer fine across the LAN with no errors, i'm wondering if
    > >~ there is some adjustments I should be making to the WAN egress
    > >~ interfaces of the routers.

    >
    > >Well, if you are losing/corrupting data, you'll definitely want to
    > >fix the problem.


    One slightly divergent view - SMB in my experience averages
    32k per round trip. I have never seen anything different from this
    and I have done *lots* of looking. I have very often observed
    60k then 4k then 60k then 4k ... etc. blocks. Occasionally
    32k, 32k, 32k .... however either results in an average
    block size of 32k.

    Say between London and NY this results in a maximum
    data rate of 32k bytes per 80ms,
    400k bytes per sec
    3.2M bits per second.

    No matter if you have a 10G private network you can
    only ever get at best 3.2M bps with SMB.

    As already mentioned TCP on its own does not do file
    transfers since it not an application layer protocol.

    FTP is an application layer protocol as is SMB (in effect
    the Microsoft Windows network file system). By the way
    this was utterly bizarrely renamed from Server Message
    Block to Common Internet File System (CIFS).

    This was in my opinion bizarre since it is not suitable as a
    WAN file system, is almost never used on the Internet,
    and has disasterous performance problems when used over
    any distance over a couple of hundered miles.

    A more descriptive term might have been -
    Uncommon and Completely Useless Internetless
    Filish System.

    By the way the protocol is proprietary and secret -
    but has been reverse engineered by the SABMA
    team ( I suspect).

    CIFS
    Common - no way Jose
    Internet - not a chance
    File - well it does have some from time to time
    System - barely

    SMB v2 has I believe been made available on Vista and
    server 2008. It is suggested that some of these dire
    issues have been addressed. Unfortunately I have not
    yet been able to evaluate its behaviour.


    Further reading, google [bandwidth delay product].
    bod43, May 8, 2010
    #2
    1. Advertising

  3. Stephen

    alexd Guest

    On 08/05/10 03:14, bod43 wrote:

    > FTP is an application layer protocol as is SMB (in effect
    > the Microsoft Windows network file system). By the way
    > this was utterly bizarrely renamed from Server Message
    > Block to Common Internet File System (CIFS).


    Windows [XP at least] supports WebDAV natively as a network file system.
    This may be better than SMB in WANs as it uses HTTP(S).

    --
    <http://ale.cx/> (AIM:troffasky) ()
    20:21:13 up 14 days, 20:00, 2 users, load average: 0.45, 0.44, 0.32
    It is better to have been wasted and then sober
    than to never have been wasted at all
    alexd, May 12, 2010
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kevin
    Replies:
    1
    Views:
    767
    Walter Roberson
    Nov 10, 2004
  2. DJ Chiro
    Replies:
    1
    Views:
    3,227
    Rowdy Yates
    Nov 7, 2003
  3. G. W. Clay

    Window sizing

    G. W. Clay, Jul 12, 2003, in forum: Computer Support
    Replies:
    4
    Views:
    732
    G. W. Clay
    Jul 13, 2003
  4. Ron Patterson

    Control Window Sizing

    Ron Patterson, Sep 4, 2003, in forum: Computer Support
    Replies:
    3
    Views:
    2,582
    trout
    Sep 6, 2003
  5. jill

    Sizing firefox window.

    jill, Mar 12, 2006, in forum: Computer Support
    Replies:
    5
    Views:
    540
    Paul_B
    Mar 12, 2006
Loading...

Share This Page