Question about big Gig E rings

Discussion in 'Cisco' started by thcollicutt, May 31, 2007.

  1. thcollicutt

    thcollicutt Guest

    We are looking at building a province wide fiber network for
    connection of various government offices.

    4 hubs - partially meshed
    56 pops- one connection to nearest hub, backup connection to next
    nearest hub
    2-300 sites - connected to neares pop

    The hubs and pops are to be connected with gigabit fiber (not low
    dispersion). Sites will be connected to the pops by whatever method
    makes the most sense for the site. All fiber links are <90Km, and
    most <80Km.

    Hubs are connected as follows

    B ==> 6 fibers via North route ==> C
    B ==> 6 fibers via South route ==> C
    D ==> 6 fibers via North route ==> C
    D ==> 6 fibers via South route ==> C
    D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    A ==> 6 fibers via North route ==> B
    A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km

    (North and south routes connecting to D are different routes than
    connecting to A or C, as all are on different rings)

    We are considering small server farms at the hub sites, with backup
    servers at othe hubs. VoIP might be considered in the near future.

    It has been suggested that SONET is the only option that is open to
    us, due to the fast redirect of traffic if there is a fiber failure.
    I have seen various vendors advertise equipment that would allow
    Ethernet to be used, but with what is probably proprietory protocols
    or implimentations.

    We are looking to combine core government, health care, and the school
    system on the same infrastructure. We are looking at the possibility
    of VLANs, QinQ, VRF-lite, perhaps CWDM.

    Is SONET the solution of choice, or am I correct in thinking that this
    can be accomplished using Ethernet? Has anyone used these vendor
    solutions for providing resiliency?
    thcollicutt, May 31, 2007
    #1
    1. Advertising

  2. thcollicutt

    Thrill5 Guest

    SONET is the way to go if you want extremely fast failover detection and
    recovery. SONET is will do this in micro-seconds, Ethernet would do this
    in less than a minute. Since you have "North" and "South" routes built out,
    it looks to me that SONET was the choice when the fiber routes were
    engineered. If the plan was using Ethernet, the fiber build-out plan would
    be different. For Ethernet, you would have different paths and fewer fibers
    would be required, but it would not be a resilient and fail-over would not
    be anywhere near as fast. SONET provides virtually no packet loss due to a
    fiber break or equipment failure.

    Scott

    "thcollicutt" <> wrote in message
    news:...
    > We are looking at building a province wide fiber network for
    > connection of various government offices.
    >
    > 4 hubs - partially meshed
    > 56 pops- one connection to nearest hub, backup connection to next
    > nearest hub
    > 2-300 sites - connected to neares pop
    >
    > The hubs and pops are to be connected with gigabit fiber (not low
    > dispersion). Sites will be connected to the pops by whatever method
    > makes the most sense for the site. All fiber links are <90Km, and
    > most <80Km.
    >
    > Hubs are connected as follows
    >
    > B ==> 6 fibers via North route ==> C
    > B ==> 6 fibers via South route ==> C
    > D ==> 6 fibers via North route ==> C
    > D ==> 6 fibers via South route ==> C
    > D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    > A ==> 6 fibers via North route ==> B
    > A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km
    >
    > (North and south routes connecting to D are different routes than
    > connecting to A or C, as all are on different rings)
    >
    > We are considering small server farms at the hub sites, with backup
    > servers at othe hubs. VoIP might be considered in the near future.
    >
    > It has been suggested that SONET is the only option that is open to
    > us, due to the fast redirect of traffic if there is a fiber failure.
    > I have seen various vendors advertise equipment that would allow
    > Ethernet to be used, but with what is probably proprietory protocols
    > or implimentations.
    >
    > We are looking to combine core government, health care, and the school
    > system on the same infrastructure. We are looking at the possibility
    > of VLANs, QinQ, VRF-lite, perhaps CWDM.
    >
    > Is SONET the solution of choice, or am I correct in thinking that this
    > can be accomplished using Ethernet? Has anyone used these vendor
    > solutions for providing resiliency?
    >
    Thrill5, May 31, 2007
    #2
    1. Advertising

  3. thcollicutt

    thcollicutt Guest

    On May 31, 2:06 pm, "Thrill5" <> wrote:
    > SONET is the way to go if you want extremely fast failover detection and
    > recovery. SONET is will do this in micro-seconds, Ethernet would do this
    > in less than a minute. Since you have "North" and "South" routes built out,
    > it looks to me that SONET was the choice when the fiber routes were
    > engineered. If the plan was using Ethernet, the fiber build-out plan would
    > be different. For Ethernet, you would have different paths and fewer fibers
    > would be required, but it would not be a resilient and fail-over would not
    > be anywhere near as fast. SONET provides virtually no packet loss due to a
    > fiber break or equipment failure.
    >
    > Scott


    The North/South routes were more to keep the network from getting cut
    in half in case someone decides to DUI and takes out a poll, and then
    can't get his medical records because the hospital is offline.

    Extra fibers were put in so we wouldn't need to go back if we needed
    more bandwidth than was available on a single pair.
    thcollicutt, May 31, 2007
    #3
  4. "thcollicutt" <> ha scritto nel messaggio
    news:...
    > We are looking at building a province wide fiber network for
    > connection of various government offices.
    >
    > 4 hubs - partially meshed
    > 56 pops- one connection to nearest hub, backup connection to next
    > nearest hub
    > 2-300 sites - connected to neares pop
    >
    > The hubs and pops are to be connected with gigabit fiber (not low
    > dispersion). Sites will be connected to the pops by whatever method
    > makes the most sense for the site. All fiber links are <90Km, and
    > most <80Km.
    >
    > Hubs are connected as follows
    >
    > B ==> 6 fibers via North route ==> C
    > B ==> 6 fibers via South route ==> C
    > D ==> 6 fibers via North route ==> C
    > D ==> 6 fibers via South route ==> C
    > D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    > A ==> 6 fibers via North route ==> B
    > A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km
    >
    > (North and south routes connecting to D are different routes than
    > connecting to A or C, as all are on different rings)
    >
    > We are considering small server farms at the hub sites, with backup
    > servers at othe hubs. VoIP might be considered in the near future.
    >
    > It has been suggested that SONET is the only option that is open to
    > us, due to the fast redirect of traffic if there is a fiber failure.
    > I have seen various vendors advertise equipment that would allow
    > Ethernet to be used, but with what is probably proprietory protocols
    > or implimentations.
    >
    > We are looking to combine core government, health care, and the school
    > system on the same infrastructure. We are looking at the possibility
    > of VLANs, QinQ, VRF-lite, perhaps CWDM.
    >
    > Is SONET the solution of choice, or am I correct in thinking that this
    > can be accomplished using Ethernet? Has anyone used these vendor
    > solutions for providing resiliency?
    >


    Hi,

    SONET is good but legacy; if you do not need to reuse a SONET network, you
    have better alternatives ( well, in fact you could have either SONET and
    other technologies at the same time if you need to )
    Look at the Multi Service Equipment from Cisco ( ONS 15xxx ). You coulhave
    have SONET,RPR,Ethernet,old PDH ( T1/E1,etc. ) and also CWMA and DWMA.

    Regards,
    Gabriele
    Gabriele Beltrame, May 31, 2007
    #4
  5. thcollicutt

    stephen Guest

    "thcollicutt" <> wrote in message
    news:...
    > We are looking at building a province wide fiber network for
    > connection of various government offices.
    >
    > 4 hubs - partially meshed
    > 56 pops- one connection to nearest hub, backup connection to next
    > nearest hub
    > 2-300 sites - connected to neares pop
    >
    > The hubs and pops are to be connected with gigabit fiber (not low
    > dispersion). Sites will be connected to the pops by whatever method
    > makes the most sense for the site. All fiber links are <90Km, and
    > most <80Km.
    >
    > Hubs are connected as follows
    >
    > B ==> 6 fibers via North route ==> C
    > B ==> 6 fibers via South route ==> C
    > D ==> 6 fibers via North route ==> C
    > D ==> 6 fibers via South route ==> C
    > D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    > A ==> 6 fibers via North route ==> B
    > A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km
    >
    > (North and south routes connecting to D are different routes than
    > connecting to A or C, as all are on different rings)
    >
    > We are considering small server farms at the hub sites, with backup
    > servers at othe hubs. VoIP might be considered in the near future.
    >
    > It has been suggested that SONET is the only option that is open to
    > us, due to the fast redirect of traffic if there is a fiber failure.
    > I have seen various vendors advertise equipment that would allow
    > Ethernet to be used, but with what is probably proprietory protocols
    > or implimentations.


    you can do fast rerouting with Ethernet pipes - as long as your route at the
    nodal points.

    We have shown OSPF based rerouting on Cisco Cat 6500s at sub 50 mSec in the
    lab - but it takes a fair amount of tuning to get there.

    you need the GigE links to give "loss of light" on a fault so the equipment
    can react immediately - shouldnt be a problem if you are driving dark fibre
    directly.

    we are just commissioning a similar network for a customer - dual central
    hubs with GigE to each location.

    However we are using "legacy" modern SDH to drive the plumbing at STM-64 /
    9.8 Gbps and carving the GigEs out of that since we need non IP services as
    well.

    >
    > We are looking to combine core government, health care, and the school
    > system on the same infrastructure. We are looking at the possibility
    > of VLANs, QinQ, VRF-lite, perhaps CWDM.


    MPLS is probably the best way to do this, if all you need to do is move
    packets.

    Fast reroute on MPLS equipment converges as quickly as SDH (european flavour
    of SONET used here in UK).

    Although MPLS can support lots of VPN structures "on top", the boring
    conventional service is routed IP VPNs - which means these are the ones
    which are well known, well behaved and that a lot of trained engineers know
    how to build and look after.

    I have worked on Cisco MPLS a lot - good, loads of features, but expensive
    and you need to lab test to find out which code versions will work with the
    features you need. Just about all the other router manufacturers also
    support it, as well as traditional telecoms suppliers (Alcatel specifically
    seems to make reliable MPLS stuff)

    What you have said implies that routed IP is going to do what you need - so
    start with that as the design baseline.

    MPLS allows each of your logical overlays to choose their own addressing,
    QoS, topology and various other things, so will reduce the complexities of
    glueing several networks together and making them play nice.
    >
    > Is SONET the solution of choice, or am I correct in thinking that this
    > can be accomplished using Ethernet? Has anyone used these vendor
    > solutions for providing resiliency?


    SONET / SDH equipment has benefited from the same improvements in
    performance / chip scale as anything else made of complex electronics.

    The problem you may have is the kit is almost always optimised for a telco
    and use on a large scale, so needs expensive management tools and so on, and
    you may find maintenance etc from a 3rd party "harder" than routing style
    equipment.

    If you go down the SONET route then you are going to be acting like a telco,
    and be supplying pipes rather than IP networks to each (unless you build
    that as another layer) - that may be a good design tradeoff depending on
    what you want.
    >

    --
    Regards

    - replace xyz with ntl
    stephen, Jun 1, 2007
    #5
  6. thcollicutt

    thcollicutt Guest

    On Jun 1, 2:57 pm, "stephen" <> wrote:
    > "thcollicutt" <> wrote in message
    >
    > news:...
    >
    >
    >
    >
    >
    > > We are looking at building a province wide fiber network for
    > > connection of various government offices.

    >
    > > 4 hubs - partially meshed
    > > 56 pops- one connection to nearest hub, backup connection to next
    > > nearest hub
    > > 2-300 sites - connected to neares pop

    >
    > > The hubs and pops are to be connected with gigabit fiber (not low
    > > dispersion). Sites will be connected to the pops by whatever method
    > > makes the most sense for the site. All fiber links are <90Km, and
    > > most <80Km.

    >
    > > Hubs are connected as follows

    >
    > > B ==> 6 fibers via North route ==> C
    > > B ==> 6 fibers via South route ==> C
    > > D ==> 6 fibers via North route ==> C
    > > D ==> 6 fibers via South route ==> C
    > > D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    > > A ==> 6 fibers via North route ==> B
    > > A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km

    >
    > > (North and south routes connecting to D are different routes than
    > > connecting to A or C, as all are on different rings)

    >
    > > We are considering small server farms at the hub sites, with backup
    > > servers at othe hubs. VoIP might be considered in the near future.

    >
    > > It has been suggested that SONET is the only option that is open to
    > > us, due to the fast redirect of traffic if there is a fiber failure.
    > > I have seen various vendors advertise equipment that would allow
    > > Ethernet to be used, but with what is probably proprietory protocols
    > > or implimentations.

    >
    > you can do fast rerouting with Ethernet pipes - as long as your route at the
    > nodal points.
    >
    > We have shown OSPF based rerouting on Cisco Cat 6500s at sub 50 mSec in the
    > lab - but it takes a fair amount of tuning to get there.
    >
    > you need the GigE links to give "loss of light" on a fault so the equipment
    > can react immediately - shouldnt be a problem if you are driving dark fibre
    > directly.
    >
    > we are just commissioning a similar network for a customer - dual central
    > hubs with GigE to each location.
    >
    > However we are using "legacy" modern SDH to drive the plumbing at STM-64 /
    > 9.8 Gbps and carving the GigEs out of that since we need non IP services as
    > well.
    >
    >
    >
    > > We are looking to combine core government, health care, and the school
    > > system on the same infrastructure. We are looking at the possibility
    > > of VLANs, QinQ, VRF-lite, perhaps CWDM.

    >
    > MPLS is probably the best way to do this, if all you need to do is move
    > packets.
    >
    > Fast reroute on MPLS equipment converges as quickly as SDH (european flavour
    > of SONET used here in UK).
    >
    > Although MPLS can support lots of VPN structures "on top", the boring
    > conventional service is routed IP VPNs - which means these are the ones
    > which are well known, well behaved and that a lot of trained engineers know
    > how to build and look after.
    >
    > I have worked on Cisco MPLS a lot - good, loads of features, but expensive
    > and you need to lab test to find out which code versions will work with the
    > features you need. Just about all the other router manufacturers also
    > support it, as well as traditional telecoms suppliers (Alcatel specifically
    > seems to make reliable MPLS stuff)
    >
    > What you have said implies that routed IP is going to do what you need - so
    > start with that as the design baseline.
    >
    > MPLS allows each of your logical overlays to choose their own addressing,
    > QoS, topology and various other things, so will reduce the complexities of
    > glueing several networks together and making them play nice.
    >
    >
    >
    > > Is SONET the solution of choice, or am I correct in thinking that this
    > > can be accomplished using Ethernet? Has anyone used these vendor
    > > solutions for providing resiliency?

    >
    > SONET / SDH equipment has benefited from the same improvements in
    > performance / chip scale as anything else made of complex electronics.
    >
    > The problem you may have is the kit is almost always optimised for a telco
    > and use on a large scale, so needs expensive management tools and so on, and
    > you may find maintenance etc from a 3rd party "harder" than routing style
    > equipment.
    >
    > If you go down the SONET route then you are going to be acting like a telco,
    > and be supplying pipes rather than IP networks to each (unless you build
    > that as another layer) - that may be a good design tradeoff depending on
    > what you want.
    >
    > --
    > Regards
    >
    > - replace xyz with ntl- Hide quoted text -
    >
    > - Show quoted text -


    50 ms or less failover is good, but currently we have a network of
    rented lines with no failover and a 4 hour repair time. I think
    failover of a minute or two may fail the desired time for VoIP, but it
    is significantly faster than anything we have now.

    I must say that the carriers we have been using have been quite good
    at getting these lines back up and running on the rare occasion that
    they go down, so I don't want to give the impression that the lines go
    down all the time, or take a really long time to repair. I am just
    noting the difference between SONET and current repair times.

    I am not ready to jump into SONET for a new network, and I don't want
    a 400+ km of fiber being controlled by Spanning tree. So, I am
    looking at having multiple areas, and routing between them. The
    server group has suggested that the central ring, which connects all
    the hub sites, to be layer 2 so they can have the same IP subnet
    present at each hub site for them to do their server redundancy. This
    looks like it may complicate the design a bit. My experience with
    MPLS is knowing what it is.

    I'm going to have to go back and reread my stuff on OSPF and Spanning
    tree. I was just curious, since I have a vendor coming in to explain
    some optoelectronics they have for sale, whether my wish to remain
    with ethernet based solutions is a reasonable one, and whether others
    have been able to do someting similar.
    thcollicutt, Jun 4, 2007
    #6
  7. thcollicutt

    stephen Guest

    "thcollicutt" <> wrote in message
    news:...
    > On Jun 1, 2:57 pm, "stephen" <> wrote:
    > > "thcollicutt" <> wrote in message
    > >
    > > news:...
    > >
    > >
    > >
    > >
    > >
    > > > We are looking at building a province wide fiber network for
    > > > connection of various government offices.

    > >
    > > > 4 hubs - partially meshed
    > > > 56 pops- one connection to nearest hub, backup connection to next
    > > > nearest hub
    > > > 2-300 sites - connected to neares pop

    > >
    > > > The hubs and pops are to be connected with gigabit fiber (not low
    > > > dispersion). Sites will be connected to the pops by whatever method
    > > > makes the most sense for the site. All fiber links are <90Km, and
    > > > most <80Km.

    > >
    > > > Hubs are connected as follows

    > >
    > > > B ==> 6 fibers via North route ==> C
    > > > B ==> 6 fibers via South route ==> C
    > > > D ==> 6 fibers via North route ==> C
    > > > D ==> 6 fibers via South route ==> C
    > > > D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    > > > A ==> 6 fibers via North route ==> B
    > > > A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km

    > >
    > > > (North and south routes connecting to D are different routes than
    > > > connecting to A or C, as all are on different rings)

    > >
    > > > We are considering small server farms at the hub sites, with backup
    > > > servers at othe hubs. VoIP might be considered in the near future.

    > >
    > > > It has been suggested that SONET is the only option that is open to
    > > > us, due to the fast redirect of traffic if there is a fiber failure.
    > > > I have seen various vendors advertise equipment that would allow
    > > > Ethernet to be used, but with what is probably proprietory protocols
    > > > or implimentations.

    > >
    > > you can do fast rerouting with Ethernet pipes - as long as your route at

    the
    > > nodal points.
    > >
    > > We have shown OSPF based rerouting on Cisco Cat 6500s at sub 50 mSec in

    the
    > > lab - but it takes a fair amount of tuning to get there.
    > >
    > > you need the GigE links to give "loss of light" on a fault so the

    equipment
    > > can react immediately - shouldnt be a problem if you are driving dark

    fibre
    > > directly.
    > >
    > > we are just commissioning a similar network for a customer - dual

    central
    > > hubs with GigE to each location.
    > >
    > > However we are using "legacy" modern SDH to drive the plumbing at STM-64

    /
    > > 9.8 Gbps and carving the GigEs out of that since we need non IP services

    as
    > > well.
    > >
    > >
    > >
    > > > We are looking to combine core government, health care, and the school
    > > > system on the same infrastructure. We are looking at the possibility
    > > > of VLANs, QinQ, VRF-lite, perhaps CWDM.

    > >
    > > MPLS is probably the best way to do this, if all you need to do is move
    > > packets.
    > >
    > > Fast reroute on MPLS equipment converges as quickly as SDH (european

    flavour
    > > of SONET used here in UK).
    > >
    > > Although MPLS can support lots of VPN structures "on top", the boring
    > > conventional service is routed IP VPNs - which means these are the ones
    > > which are well known, well behaved and that a lot of trained engineers

    know
    > > how to build and look after.
    > >
    > > I have worked on Cisco MPLS a lot - good, loads of features, but

    expensive
    > > and you need to lab test to find out which code versions will work with

    the
    > > features you need. Just about all the other router manufacturers also
    > > support it, as well as traditional telecoms suppliers (Alcatel

    specifically
    > > seems to make reliable MPLS stuff)
    > >
    > > What you have said implies that routed IP is going to do what you need -

    so
    > > start with that as the design baseline.
    > >
    > > MPLS allows each of your logical overlays to choose their own

    addressing,
    > > QoS, topology and various other things, so will reduce the complexities

    of
    > > glueing several networks together and making them play nice.
    > >
    > >
    > >
    > > > Is SONET the solution of choice, or am I correct in thinking that this
    > > > can be accomplished using Ethernet? Has anyone used these vendor
    > > > solutions for providing resiliency?

    > >
    > > SONET / SDH equipment has benefited from the same improvements in
    > > performance / chip scale as anything else made of complex electronics.
    > >
    > > The problem you may have is the kit is almost always optimised for a

    telco
    > > and use on a large scale, so needs expensive management tools and so on,

    and
    > > you may find maintenance etc from a 3rd party "harder" than routing

    style
    > > equipment.
    > >
    > > If you go down the SONET route then you are going to be acting like a

    telco,
    > > and be supplying pipes rather than IP networks to each (unless you build
    > > that as another layer) - that may be a good design tradeoff depending on
    > > what you want.
    > >
    > > --
    > > Regards
    > >
    > > - replace xyz with ntl- Hide quoted text -
    > >
    > > - Show quoted text -

    >
    > 50 ms or less failover is good, but currently we have a network of
    > rented lines with no failover and a 4 hour repair time. I think
    > failover of a minute or two may fail the desired time for VoIP, but it
    > is significantly faster than anything we have now.
    >
    > I must say that the carriers we have been using have been quite good
    > at getting these lines back up and running on the rare occasion that
    > they go down, so I don't want to give the impression that the lines go
    > down all the time, or take a really long time to repair. I am just
    > noting the difference between SONET and current repair times.
    >
    > I am not ready to jump into SONET for a new network, and I don't want
    > a 400+ km of fiber being controlled by Spanning tree. So, I am
    > looking at having multiple areas, and routing between them. The
    > server group has suggested that the central ring, which connects all
    > the hub sites, to be layer 2 so they can have the same IP subnet
    > present at each hub site for them to do their server redundancy.


    In theory from a server perspective this is great, but very problematic for
    the plumbing when L2 goes between sites.

    they really need to think about using a server resilience scheme that they
    can use at layer 3 or higher.

    This
    > looks like it may complicate the design a bit.


    this is serous understatement - since L2 links between sites seem to fail in
    various ways - most of which i only found when they happened....

    The problem is that any such failure where there are devices on the 2 sites
    subnet will end up with the "split subnet" problem.

    this is where there are 2 bits of the same subnet which "should" stay
    connected dont. This leads to big connectivity holes in a notionally
    resilient network.

    Routers then deliver packets to which ever section of the subnet is
    closest - because the implicit assumption for the routers is that subnets
    are "atomic" and dont get carved into bits.

    The same can happen with router to router links - but the routers use a
    routing protocol to detect such transit path problems and will reroute
    around the subnet.

    My experience with
    > MPLS is knowing what it is.
    >
    > I'm going to have to go back and reread my stuff on OSPF and Spanning
    > tree.


    OSPF would be my preferred way to do something like this.

    I was just curious, since I have a vendor coming in to explain
    > some optoelectronics they have for sale, whether my wish to remain
    > with ethernet based solutions is a reasonable one, and whether others
    > have been able to do someting similar.
    >

    --
    Regards

    - replace xyz with ntl
    stephen, Jun 4, 2007
    #7
  8. thcollicutt

    thcollicutt Guest

    On Jun 4, 6:04 pm, "stephen" <> wrote:
    > "thcollicutt" <> wrote in message
    >
    > news:...
    >
    >
    >
    >
    >
    > > On Jun 1, 2:57 pm, "stephen" <> wrote:
    > > > "thcollicutt" <> wrote in message

    >
    > > >news:...

    >
    > > > > We are looking at building a province wide fiber network for
    > > > > connection of various government offices.

    >
    > > > > 4 hubs - partially meshed
    > > > > 56 pops- one connection to nearest hub, backup connection to next
    > > > > nearest hub
    > > > > 2-300 sites - connected to neares pop

    >
    > > > > The hubs and pops are to be connected with gigabit fiber (not low
    > > > > dispersion). Sites will be connected to the pops by whatever method
    > > > > makes the most sense for the site. All fiber links are <90Km, and
    > > > > most <80Km.

    >
    > > > > Hubs are connected as follows

    >
    > > > > B ==> 6 fibers via North route ==> C
    > > > > B ==> 6 fibers via South route ==> C
    > > > > D ==> 6 fibers via North route ==> C
    > > > > D ==> 6 fibers via South route ==> C
    > > > > D ==> 6 fibers ==> B : Regenerated to keep < 90Km
    > > > > A ==> 6 fibers via North route ==> B
    > > > > A ==> 6 fibers via South route ==> C : Regenerated to keep < 90Km

    >
    > > > > (North and south routes connecting to D are different routes than
    > > > > connecting to A or C, as all are on different rings)

    >
    > > > > We are considering small server farms at the hub sites, with backup
    > > > > servers at othe hubs. VoIP might be considered in the near future.

    >
    > > > > It has been suggested that SONET is the only option that is open to
    > > > > us, due to the fast redirect of traffic if there is a fiber failure.
    > > > > I have seen various vendors advertise equipment that would allow
    > > > > Ethernet to be used, but with what is probably proprietory protocols
    > > > > or implimentations.

    >
    > > > you can do fast rerouting with Ethernet pipes - as long as your route at

    > the
    > > > nodal points.

    >
    > > > We have shown OSPF based rerouting on Cisco Cat 6500s at sub 50 mSec in

    > the
    > > > lab - but it takes a fair amount of tuning to get there.

    >
    > > > you need the GigE links to give "loss of light" on a fault so the

    > equipment
    > > > can react immediately - shouldnt be a problem if you are driving dark

    > fibre
    > > > directly.

    >
    > > > we are just commissioning a similar network for a customer - dual

    > central
    > > > hubs with GigE to each location.

    >
    > > > However we are using "legacy" modern SDH to drive the plumbing at STM-64

    > /
    > > > 9.8 Gbps and carving the GigEs out of that since we need non IP services

    > as
    > > > well.

    >
    > > > > We are looking to combine core government, health care, and the school
    > > > > system on the same infrastructure. We are looking at the possibility
    > > > > of VLANs, QinQ, VRF-lite, perhaps CWDM.

    >
    > > > MPLS is probably the best way to do this, if all you need to do is move
    > > > packets.

    >
    > > > Fast reroute on MPLS equipment converges as quickly as SDH (european

    > flavour
    > > > of SONET used here in UK).

    >
    > > > Although MPLS can support lots of VPN structures "on top", the boring
    > > > conventional service is routed IP VPNs - which means these are the ones
    > > > which are well known, well behaved and that a lot of trained engineers

    > know
    > > > how to build and look after.

    >
    > > > I have worked on Cisco MPLS a lot - good, loads of features, but

    > expensive
    > > > and you need to lab test to find out which code versions will work with

    > the
    > > > features you need. Just about all the other router manufacturers also
    > > > support it, as well as traditional telecoms suppliers (Alcatel

    > specifically
    > > > seems to make reliable MPLS stuff)

    >
    > > > What you have said implies that routed IP is going to do what you need -

    > so
    > > > start with that as the design baseline.

    >
    > > > MPLS allows each of your logical overlays to choose their own

    > addressing,
    > > > QoS, topology and various other things, so will reduce the complexities

    > of
    > > > glueing several networks together and making them play nice.

    >
    > > > > Is SONET the solution of choice, or am I correct in thinking that this
    > > > > can be accomplished using Ethernet? Has anyone used these vendor
    > > > > solutions for providing resiliency?

    >
    > > > SONET / SDH equipment has benefited from the same improvements in
    > > > performance / chip scale as anything else made of complex electronics.

    >
    > > > The problem you may have is the kit is almost always optimised for a

    > telco
    > > > and use on a large scale, so needs expensive management tools and so on,

    > and
    > > > you may find maintenance etc from a 3rd party "harder" than routing

    > style
    > > > equipment.

    >
    > > > If you go down the SONET route then you are going to be acting like a

    > telco,
    > > > and be supplying pipes rather than IP networks to each (unless you build
    > > > that as another layer) - that may be a good design tradeoff depending on
    > > > what you want.

    >
    > > > --
    > > > Regards

    >
    > > > - replace xyz with ntl- Hide quoted text -

    >
    > > > - Show quoted text -

    >
    > > 50 ms or less failover is good, but currently we have a network of
    > > rented lines with no failover and a 4 hour repair time. I think
    > > failover of a minute or two may fail the desired time for VoIP, but it
    > > is significantly faster than anything we have now.

    >
    > > I must say that the carriers we have been using have been quite good
    > > at getting these lines back up and running on the rare occasion that
    > > they go down, so I don't want to give the impression that the lines go
    > > down all the time, or take a really long time to repair. I am just
    > > noting the difference between SONET and current repair times.

    >
    > > I am not ready to jump into SONET for a new network, and I don't want
    > > a 400+ km of fiber being controlled by Spanning tree. So, I am
    > > looking at having multiple areas, and routing between them. The
    > > server group has suggested that the central ring, which connects all
    > > the hub sites, to be layer 2 so they can have the same IP subnet
    > > present at each hub site for them to do their server redundancy.

    >
    > In theory from a server perspective this is great, but very problematic for
    > the plumbing when L2 goes between sites.
    >
    > they really need to think about using a server resilience scheme that they
    > can use at layer 3 or higher.
    >
    > This
    >
    > > looks like it may complicate the design a bit.

    >
    > this is serous understatement - since L2 links between sites seem to fail in
    > various ways - most of which i only found when they happened....
    >
    > The problem is that any such failure where there are devices on the 2 sites
    > subnet will end up with the "split subnet" problem.
    >
    > this is where there are 2 bits of the same subnet which "should" stay
    > connected dont. This leads to big connectivity holes in a notionally
    > resilient network.
    >
    > Routers then deliver packets to which ever section of the subnet is
    > closest - because the implicit assumption for the routers is that subnets
    > are "atomic" and dont get carved into bits.
    >
    > The same can happen with router to router links - but the routers use a
    > routing protocol to detect such transit path problems and will reroute
    > around the subnet.
    >
    > My experience with
    >
    > > MPLS is knowing what it is.

    >
    > > I'm going to have to go back and reread my stuff on OSPF and Spanning
    > > tree.

    >
    > OSPF would be my preferred way to do something like this.
    >
    > I was just curious, since I have a vendor coming in to explain> some optoelectronics they have for sale, whether my wish to remain
    > > with ethernet based solutions is a reasonable one, and whether others
    > > have been able to do someting similar.

    >
    > --
    > Regards
    >
    > - replace xyz with ntl- Hide quoted text -
    >
    > - Show quoted text -


    Thanks. This pretty much sums up what my thoughts were when I sat
    down to figure out how to do this.
    thcollicutt, Jun 5, 2007
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. just ask
    Replies:
    3
    Views:
    478
    ┬░Mike┬░
    Jun 12, 2004
  2. aeQea
    Replies:
    7
    Views:
    4,996
    Jordan Lund
    Apr 27, 2004
  3. Gary

    Linksys GIG v's Cisco Gig

    Gary, Oct 14, 2006, in forum: Cisco
    Replies:
    3
    Views:
    1,693
    Walter Roberson
    Oct 15, 2006
  4. grappletech
    Replies:
    2
    Views:
    522
    grappletech
    Mar 9, 2007
  5. Great Deals
    Replies:
    1
    Views:
    782
    Trendkill
    Sep 10, 2007
Loading...

Share This Page