router to router latency question

Discussion in 'Cisco' started by garlic, Dec 22, 2005.

  1. garlic

    garlic Guest

    hi,

    I have two 2600 routers connected to two different ISPs with different
    line speeds
    one is using frame relay with 1mbps connection, the other is using
    leased line 256kbps connection with hdlc encapsulation.

    router to router latency with no utilization for the frame-relay is
    about 1ms average
    router to router latency with no utilization for the leased line
    connection is about 15-22ms average

    when we saturate the links to 96%-100% for both connections, the frame
    relay router to router latency basically stays the same but the leased
    line connection's router to router latency jumps to a whopping 900ms
    average....

    my question is they are both maxed out in the utilization (100%) but
    why one stays basically stays consistent in the router to router
    latency and the other does not? does it really have to do with the line
    speed even though both are at 100 % percent util?

    thanks for any explanation
    garlic, Dec 22, 2005
    #1
    1. Advertising

  2. garlic

    karateD Guest

    Garlic,

    You have a lot of variables here but here goes ... 1) Frame Relay is a
    superior protocol, specifically designed for high-speed switching at
    layer 2. 2) When you use HDLC over a lease-line, frames are being
    encapsulated - more tear down at remote end, etc. 3) There is more
    error checking over the leased line. ... if you had 2 crappy lines,
    one to each of the ISP's, then the leased line might be a better
    solution.
    karateD, Dec 22, 2005
    #2
    1. Advertising

  3. In article <>,
    garlic <> wrote:
    >when we saturate the links to 96%-100% for both connections, the frame
    >relay router to router latency basically stays the same but the leased
    >line connection's router to router latency jumps to a whopping 900ms
    >average....


    >my question is they are both maxed out in the utilization (100%) but
    >why one stays basically stays consistent in the router to router
    >latency and the other does not?


    Is it possible that the frame relay circuit is full duplex
    but the leased line is half?

    On half-duplex circuits, as utilization increases, the "cost"
    of turning around the line to send an ACK increases greatly...
    and if one of the sources happens to be sending pure UDP then
    there might not be a convenient time to turn the line around.
    --
    If you lie to the compiler, it will get its revenge. -- Henry Spencer
    Walter Roberson, Dec 22, 2005
    #3
  4. In article <>,
    garlic <> wrote:
    >hi,
    >
    >I have two 2600 routers connected to two different ISPs with different
    >line speeds
    >one is using frame relay with 1mbps connection, the other is using
    >leased line 256kbps connection with hdlc encapsulation.
    >
    >router to router latency with no utilization for the frame-relay is
    >about 1ms average
    >router to router latency with no utilization for the leased line
    >connection is about 15-22ms average
    >
    >when we saturate the links to 96%-100% for both connections, the frame
    >relay router to router latency basically stays the same but the leased
    >line connection's router to router latency jumps to a whopping 900ms
    >average....
    >
    >my question is they are both maxed out in the utilization (100%) but
    >why one stays basically stays consistent in the router to router
    >latency and the other does not? does it really have to do with the line
    >speed even though both are at 100 % percent util?
    >
    >thanks for any explanation


    Insufficient information to make an informed guess, but my initial
    reaction is that you are not measuring what you think you are
    measuring. A few key questions:

    1- How are you measuring latency?

    2- From what router to what router?

    3- How are you loading down the links?

    4- How are you measuring the loading?

    Assuming the answer to #1 is "pinging the other end of the link"
    your measurements are inconsistent. You can't ping the other end of
    a frame relay link because frame relay is a layer two protocol. And
    your no load measurements for the 256Kbps link are very high. Round
    trip serialization delay for a 64 byte ping packet at 256Kbps is
    only 4 ms, so you're looking at 10-20 ms of processing delay in
    the routers.

    A quick look at queueing theory tells me that you are filling
    the 256K link to 98%. On the frame link, I suspect you are pinging
    yourself rather than the other end of the link. If you were actually
    seeing queueing delays at full loading, you're "fully loaded" frame
    link is still only at less than 66% load (best case ping at 64
    bytes would be 0.6ms, and you're seeing less than 2ms at full load).

    Good luck and good hunting.
    --
    Vincent C Jones, Consultant Expert advice and a helping hand
    Networking Unlimited, Inc. for those who want to manage and
    Tenafly, NJ Phone: 201 568-7810 control their networking destiny
    http://www.networkingunlimited.com
    Vincent C Jones, Dec 22, 2005
    #4
  5. In article <>,
    karateD <> wrote:
    >Garlic,
    >
    >You have a lot of variables here but here goes ... 1) Frame Relay is a
    >superior protocol, specifically designed for high-speed switching at
    >layer 2. 2) When you use HDLC over a lease-line, frames are being
    >encapsulated - more tear down at remote end, etc. 3) There is more
    >error checking over the leased line. ... if you had 2 crappy lines,
    >one to each of the ISP's, then the leased line might be a better
    >solution.


    KarateD, you need to hit the books and learn a bit about how frame
    relay actually works. The above may be what the marketing weenies
    want you to believe, but it has no basis in fact. To begin with,
    frame relay frames are also encapsulated in HDLC framing, so if
    anything, there is more overhead to frame relay, not less. The error
    checking is identical between a frame relay access link and a Cisco
    default HDLC point-to-point link. Both use a 16 bit CRC and discard
    errored frames.

    Nobody knows you're a dog on the Internet...
    --
    Vincent C Jones, Consultant Expert advice and a helping hand
    Networking Unlimited, Inc. for those who want to manage and
    Tenafly, NJ Phone: 201 568-7810 control their networking destiny
    http://www.networkingunlimited.com
    Vincent C Jones, Dec 22, 2005
    #5
  6. garlic

    karateD Guest

    Vincent C Jones wrote:
    >Insufficient information to make an informed guess


    .... but apparently you have enough to be an arrogant F#. The guy was
    just asking for some help explaining this, and you "first reaction" is
    to be insulting.F# ...

    As I wrote earlier, "There are a lot of variables here ..." - including
    the Frame Relay encapsulation type, the routers at each end, duplex,
    quality of the line itself, ...

    Everyone knows when you are arrogant ...
    karateD, Dec 22, 2005
    #6
  7. garlic

    Hansang Bae Guest

    karateD wrote:
    snip[vincent's reply]
    > ... but apparently you have enough to be an arrogant F#. The guy was
    > just asking for some help explaining this, and you "first reaction" is
    > to be insulting.F# ...
    >
    > As I wrote earlier, "There are a lot of variables here ..." -
    > including the Frame Relay encapsulation type, the routers at each
    > end, duplex, quality of the line itself, ...
    >
    > Everyone knows when you are arrogant ...



    Not if you have the goods to back it up. Let's see, Vincent,
    literally, wrote the book on fault tolerant network design. He's a
    Ph.D. He's a real Prof Engineer. I've personally seen some of his
    work in action. They are quite elegant (and complex). It took me a
    bit to absorb what he was trying to accomplish and then later support
    it (I was responsible for moving the data center where he did some work)

    He was also replying to your comments so he didn't address the original
    posters comments in his reply to your post. So your first sentence is
    completely off the mark. And I didn't think he was insulting in the
    least.

    Finally, one advice I can give you is "best way out of a hole is to
    stop digging." For example, why would duplex matter for FR/P2P links?

    Hang around the group for a bit and try to learn from those who know
    what they are saying. Don't get some damn defensive for cryin out loud.


    --

    hsb


    "Somehow I imagined this experience would be more rewarding" Calvin
    **************************ROT13 MY ADDRESS*************************
    Due to the volume of email that I receive, I may not be able to
    reply to emails sent to my account. Please post a followup instead.
    ********************************************************************
    Hansang Bae, Dec 23, 2005
    #7
  8. In article <fVLqf.13437$-nyc.rr.com>,
    Hansang Bae <> wrote:

    >Finally, one advice I can give you is "best way out of a hole is to
    >stop digging." For example, why would duplex matter for FR/P2P links?


    I was the one who asked whether the duplexes are the same for both
    devices. If I recall correctly, some P2P links are implemented
    as 2-wire half-duplex. When you have asymmetric data volumes,
    full duplex can potentially keep streaming because the ACKs can
    flow back while data is being sent; but with half duplex the
    line turn-arounds have significant impact on the queuing theory.

    --
    "It is important to remember that when it comes to law, computers
    never make copies, only human beings make copies. Computers are given
    commands, not permission. Only people can be given permission."
    -- Brad Templeton
    Walter Roberson, Dec 23, 2005
    #8
  9. garlic

    Guest

    Hi,

    > ... arrogant


    I have been reading, and perhaps contributing
    to this group for a number of years and
    for me Vincent is probably the most
    valuable contributor in terms
    of the amount that it is possible to learn.
    I see no sign of arrogance at all.

    His writing is always of the highest
    quality with carefully constructed sentences
    and he always avoids personal comments
    of any kind.

    My first cut would most certainly be to believe
    every last word, letter, full stop that he writes.

    Clearly from time to time there will be differences in
    preferences for this or that however
    he is most likely exactly correct.

    Anyway, have a nice holiday, In the UK we are off
    'til Wednesday:)

    Need to do CCDA for employer's cisco relationship
    so off to buy myself a christmas present.

    I fancy the Diane Teare Cisco press one.
    , Dec 24, 2005
    #9
  10. garlic

    garlic Guest

    Ayon kay Vincent C Jones:
    > In article <>,
    > garlic <> wrote:
    > >hi,
    > >
    > >I have two 2600 routers connected to two different ISPs with different
    > >line speeds
    > >one is using frame relay with 1mbps connection, the other is using
    > >leased line 256kbps connection with hdlc encapsulation.
    > >
    > >router to router latency with no utilization for the frame-relay is
    > >about 1ms average
    > >router to router latency with no utilization for the leased line
    > >connection is about 15-22ms average
    > >
    > >when we saturate the links to 96%-100% for both connections, the frame
    > >relay router to router latency basically stays the same but the leased
    > >line connection's router to router latency jumps to a whopping 900ms
    > >average....
    > >
    > >my question is they are both maxed out in the utilization (100%) but
    > >why one stays basically stays consistent in the router to router
    > >latency and the other does not? does it really have to do with the line
    > >speed even though both are at 100 % percent util?
    > >
    > >thanks for any explanation

    >
    > Insufficient information to make an informed guess, but my initial
    > reaction is that you are not measuring what you think you are
    > measuring. A few key questions:
    >
    > 1- How are you measuring latency?


    -you are correct, by pinging the other end of the link using my router
    at default of 100 bytes
    >
    > 2- From what router to what router?
    > My router to the next hop router, which I believe is the provider's router on the other side of the link
    > 3- How are you loading down the links?
    > My router has two serial interfaces, connected to a switch (default vlan 1), and two workstations connected to that switch (default vlan1 also). I put in global IPs for the workstations provided by my ISPs and created a route-map policy so the workstation with the appropriate IP passes by the correct serial interface to the correct provider.


    > 4- How are you measuring the loading?
    > by simultanenous ftping, bittorrent, etc on the workstations


    > Assuming the answer to #1 is "pinging the other end of the link"
    > your measurements are inconsistent. You can't ping the other end of
    > a frame relay link because frame relay is a layer two protocol. And
    > your no load measurements for the 256Kbps link are very high. Round
    > trip serialization delay for a 64 byte ping packet at 256Kbps is
    > only 4 ms, so you're looking at 10-20 ms of processing delay in
    > the routers.


    Isnt HDLC a layer two protocol too like frame relay?
    >
    > A quick look at queueing theory tells me that you are filling
    > the 256K link to 98%. On the frame link, I suspect you are pinging
    > yourself rather than the other end of the link. If you were actually
    > seeing queueing delays at full loading, you're "fully loaded" frame
    > link is still only at less than 66% load (best case ping at 64
    > bytes would be 0.6ms, and you're seeing less than 2ms at full load).
    >
    > Good luck and good hunting.
    > --
    > Vincent C Jones, Consultant Expert advice and a helping hand
    > Networking Unlimited, Inc. for those who want to manage and
    > Tenafly, NJ Phone: 201 568-7810 control their networking destiny
    > http://www.networkingunlimited.com
    garlic, Jan 2, 2006
    #10
  11. In article <>,
    garlic <> wrote:
    >
    >Ayon kay Vincent C Jones:
    >> In article <>,
    >> garlic <> wrote:
    >> >hi,
    >> >
    >> >I have two 2600 routers connected to two different ISPs with different
    >> >line speeds one is using frame relay with 1mbps connection, the other
    >> >is using leased line 256kbps connection with hdlc encapsulation.
    >> >
    >> >router to router latency with no utilization for the frame-relay is
    >> >about 1ms average router to router latency with no utilization for
    >> >the leased line connection is about 15-22ms average
    >> >
    >> >when we saturate the links to 96%-100% for both connections, the frame
    >> >relay router to router latency basically stays the same but the leased
    >> >line connection's router to router latency jumps to a whopping 900ms
    >> >average....
    >> >
    >> >my question is they are both maxed out in the utilization (100%) but
    >> >why one stays basically stays consistent in the router to router
    >> >latency and the other does not? does it really have to do with the line
    >> >speed even though both are at 100 % percent util?
    >> >
    >> >thanks for any explanation

    >>
    >> Insufficient information to make an informed guess, but my initial
    >> reaction is that you are not measuring what you think you are
    >> measuring. A few key questions:
    >>
    >> 1- How are you measuring latency?

    >
    >-you are correct, by pinging the other end of the link using my router
    >at default of 100 bytes


    Close enough to my original assumption of 64 byte ping packets...

    > My router to the next hop router, which I believe is the provider's
    > router on the other side of the link


    This is one place where the inconsistent results could come from...
    (i.e., you are not pinging what you think you are). However, there
    is an alternate explanation, see below.

    >> 3- How are you loading down the links?


    > My router has two serial interfaces, connected to a switch (default
    > vlan 1), and two workstations connected to that switch (default
    > vlan1 also). I put in global IPs for the workstations provided
    > by my ISPs and created a route-map policy so the workstation with
    > the appropriate IP passes by the correct serial interface to the
    > correct provider.


    OK.

    >> 4- How are you measuring the loading?

    > by simultanenous ftping, bittorrent, etc on the workstations


    OK. So the pings are going to the ISP's router while the loading traffic
    is going through the ISP's router.

    >> Assuming the answer to #1 is "pinging the other end of the link"
    >> your measurements are inconsistent. You can't ping the other end of
    >> a frame relay link because frame relay is a layer two protocol. And
    >> your no load measurements for the 256Kbps link are very high. Round
    >> trip serialization delay for a 64 byte ping packet at 256Kbps is
    >> only 4 ms, so you're looking at 10-20 ms of processing delay in
    >> the routers.

    >
    >Isnt HDLC a layer two protocol too like frame relay?


    Yes, but HDLC only gets you to the "other end of the physical link"
    whereas frame relay (which uses HDLC) gets you to the "other end
    of the virtual circuit" which could be an arbitrary number of
    hops. However, this is probably a moot point, my suspicion is that
    your ISP terminates your physical frame link and your logical frame
    link on the same router so there is no meaningful difference between
    the two links from a protocol viewpoint, only the data rates and
    the router at the other end are different.

    >> A quick look at queueing theory tells me that you are filling
    >> the 256K link to 98%. On the frame link, I suspect you are pinging
    >> yourself rather than the other end of the link. If you were actually
    >> seeing queueing delays at full loading, you're "fully loaded" frame
    >> link is still only at less than 66% load (best case ping at 64
    >> bytes would be 0.6ms, and you're seeing less than 2ms at full load).
    >>
    >> Good luck and good hunting.
    >> --
    >> Vincent C Jones, Consultant


    Two conclusions (really questions for you), as there are two independent
    inconsistencies in the performance you are seeing:

    1 - You are not seeing any impact of loading on the response time of the
    frame relay link. This implies to me that the link is configured to use
    "fair queueing" or the equivalent, so that the traffic hog and your test
    traffic are being independently queued. This is exactly what "fair
    queueing" is intended to accomplish. It is also the default on serial
    links on Cisco routers.

    2 - You are seeing abysmal performance on your 256Kbps link.
    The use of an underpowered, brain dead router is implied by the
    lack of "fair queueing", your response time degradation looks like
    simple FIFO queueing. But even an antique sloth router like a 25xx
    only requires 3-4 ms to forward a packet, nowhere near the 10-15
    ms unaccounted for delays you are seeing at no load. This could
    be because the router is overloaded and unable to respond to your
    traffic, or it could be that you are paying for 256 Kbps and only
    using 64 Kbps. Check what bandwidth you can get with an unopposed
    FTP or BitTorrent.

    DISCLAIMER: You get what you pay for. The conclusions above are based
    on the limited information provided, much conjecture, and minimal
    effort. While they are the most likely explanation of the behaviors
    you are reporting, they are not the only explanation and further
    testing would be required to verify or disprove each hypothesis.

    --
    Vincent C Jones, Consultant Expert advice and a helping hand
    Networking Unlimited, Inc. for those who want to manage and
    Tenafly, NJ Phone: 201 568-7810 control their networking destiny
    http://www.networkingunlimited.com
    Vincent C Jones, Jan 10, 2006
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Williams
    Replies:
    2
    Views:
    812
    clubfoot
    Apr 25, 2006
  2. Replies:
    2
    Views:
    8,861
    Michael Newbery
    Jun 19, 2006
  3. Replies:
    6
    Views:
    764
    Martin Gallagher
    Feb 14, 2007
  4. Replies:
    0
    Views:
    544
  5. Replies:
    0
    Views:
    583
Loading...

Share This Page