Routing for Verizon FIOS -- Reward for answer

Discussion in 'Cisco' started by Fletcher James, May 3, 2008.

  1. This is your opportunity to be a Cisco hero (and to earn a tin of incredible
    cookies, with the first correct answer.) I suspect that the right person
    can solve this problem in a snap, but the solution has been eluding us for
    over a month.

    We have been assigned a block of 64 static IP addresses (actually, 61) by
    Verizon, for our Business FIOS network. Let's call our addresses

    We wish to place a Cisco 1841 directly on the FIOS connection, and then have
    a handful of devices inside (perimeter network), connected by a simple
    Ethernet switch. Most of the addresses will be handled by an ISA server
    (firewall/NAT, which protects our LAN and a separate Web Server zone), but a
    few other devices will be independent (e.g. a videoteleconference unit which
    doesn't play well inside the firewall, a wireless router for untrusted
    devices, etc.)

    For many reasons, it would be best if we were simply routing our traffic to
    the inside of the Cisco, so that our 70.x.x.64/26 subnet is on the INSIDE of
    the 1841.

    The problem we have is this: Verizon's gateway is 70.x.x.1. Unlike our
    other ISPs, they have NOT assigned us a separate 30-bit subnet with an
    address for our router (in this case, that would be 70.x.x.2). I think
    Verizon just expected us to NAT everything immediately after their
    interface, the way that residential customers do with their Actiontec
    router/firewall units.

    So the problem is: What do we use as an address for the outside interface
    of our router, which will allow it to route traffic to the gateway, OR, how
    do we otherwise deal with this problem?

    To demonstrate: If we assign our router's outside to .66 (they've told us
    not to use .65) then we need a netmask of so that we can
    route outbound through the gateway. Unfortunately, that then defines ALL of
    our public addresses as being on the outside of the router. We've looked at
    a long list of solutions, and none of them are very good:

    OPTION A: Currently, we have declared our outside interface as
    70.x.x.126/24. We then force all of our inbound traffic to the inside with
    a long list of entries such as:

    ip route 70.x.x.69 FastEthernet0/0

    This works, but poorly -- I suspect there's a lot of unnecessary ARPing
    going on.

    OPTION B: We could keep the public addresses on the outside, and then NAT
    them to private addresses between the Cisco and the perimeter network (e.g.
    70.x.x.69 --> and then NAT them a second time in the ISA server.

    OPTION C: We could "steal" the address 70.x.x.2/30 for our outside
    interface,and hope that it never causes a problem (We've tried this, but
    have had inconsistent results -- it works, and then when we re-boot our
    router it mysteriously fails.)

    OPTION D: We could assign a PRIVATE address to the outside of our router --
    say, But then, how would we direct traffic to our gateway? If we
    provide a default route just by interface

    ip route FastEthernet0/1)

    then it's got to ARP for every single outbound address. QUESTION: would the
    following solve that problem:

    ip route 70.x.x.1

    ip route 70.x.x.1 FastEthernet0/1

    You're the genius. Tell us Option E.

    I would very much appreciate it if you could cc me directly on any reply.


    Fletcher James
    Levit & James, Inc.
    Fletcher James, May 3, 2008
    1. Advertisements

  2. Fletcher James

    Trendkill Guest

    Can you get verizon to sell you another separate /31 (yes a /31 works,
    we use them all the time for point to point routing adjacencies) and
    then setup routing on both sides? Else I don't see why you can't
    carve out the /30 as you have said, and ensure that Verizon and you
    are advertising properly on both sides. Or install a switch between
    them and you for external hosts and NAT for anything going internal.
    Not sure you can do that with FIOS, never had to deal with it.
    Trendkill, May 3, 2008
    1. Advertisements

  3. Fletcher James

    Darren Green Guest

    Out of interest, could you not assign the ip address to the LAN side
    of the router and on the WAN port use ip unnumbered to the LAN

    int Fa0/X
    ip address 70.x.x.X /26
    int Serial X/X
    ip unnumbered Fa0/X

    I seem to recall I have have done this in the past mainly on ADSL
    boxes where the client wanted to present a public IP address


    Darren Green, May 4, 2008
  4. Fletcher James

    Merv Guest

    1. It appears that the router in question is using a Fast Ethernet
    interface facing the FIOS ONT.
    If this is in fact the case then an ip unnumbered command will be
    rejected on a multi-access interface.
    ip unnumbered can only be used on point-to-point serial interfaces.

    2. The following static routing construct is viable:
    ip classless
    ip route 70.x.x.1
    ip route 70.x.x.1 FastEthernet0/1

    Option D would probably work but it is definitely a cludge.

    The suggestion by other responders to approach Verizon to see if they
    will provide a /30 or /31 for the FIOS link would definitely be
    worthwhile pursuing.

    However be forewarned that if Verizon has not planned for this
    "feature", it will impact their service provisioning systems and they
    will less inclined to provide what you are looking for - i.e a /30 for
    the FIOS link.
    Merv, May 4, 2008
  5. You said:
    One month ago: After 30 minutes on hold & several intermediary stops, the
    very 1st tech I spoke with said, "Oh, yeah we just need to get you another
    IP address." He then transferred me to the business office and jumped off
    the line. The business office said sure, the would be able to ... expand my
    block to 128 or 256 (leaving me with the same problem) but had NO policy
    under which they could sell a discontiguous IP address! I then spent
    another 45 minutes going in circles, until I found another tech who said
    "Oh, yeah we just need to get you another IP address. Let me transfer you
    to the Business Office..."

    You also said:
    I hope that's true -- it would seem to be our only serious alternative to
    double NATing. I agree it's unusual -- do you know whether there's some
    specific downside, from the standpoint of performance or reliability? Do
    you think we will need arp proxy to make this work (which our current config
    Fletcher James, May 5, 2008
  6. Fletcher James

    Merv Guest

    if you configured say 70.x.x.2 on the "inside" FE router interface and
    made that the default gateway for all the devices, then proxy ARP
    could be disabled on the inside interface

    Proxy ARP would need to be enabled on the outside interface to answer
    ARP request for 70.x.x.y coming from the Verizon router @ 70.x.x.1

    You could try the following to see if it works for you from both a
    connectivity and a performance perspective, save your current config
    before doing so.

    int fa 0/0
    description inside LAN interface
    ip addr 70.x.x.2 ! default gateway for devices on

    int fa 0/1
    description outside interface facing Verizon FIOS ONT
    ip addr <any IP address / some mask>
    ip proxy-arp ! in order to be able to answer ARP requests from

    ip classless
    ip route 70.x.x.1
    ip route 70.x.x.1 FastEthernet0/1
    Merv, May 5, 2008
  7. Fletcher James

    Trendkill Guest

    I think the issue is that his range is ..64/26, and he was told to
    use .66 as his router w/ a 24 bit mask (so he can hit .1) and not to
    use anything else. Therefore I'm not sure he can put a different
    address on that external interface, and there is no easy way to subnet
    without cutting out half of his assigned addresses. If he uses the
    first set to assign a /30, then he can't get to Verizon's router, and
    if he could, then he can't easily subnet whats left without vlan'ing
    it into a /30, a /29, a /28, and a /27. What a pain, I'd be screaming
    at verizon for a /30, even if its not public.
    Trendkill, May 5, 2008
  8. Fletcher James

    Trendkill Guest

    I wonder if he could ask Verizon to put a secondary IP on the VLAN of
    interface on their router, say .67. Assign a /31 point to point for
    the router uplink, then have the router split up the /30, /29, /28, /
    27 (presuming his switch is not a L3 switch), and trunk it to the
    switch. He will lose some addressing for network/broadcast addresses,
    but this could work. He would then be effectively giving Verizon one
    of his own addresses. Would that work?
    Trendkill, May 5, 2008
  9. Fletcher James

    Merv Guest

    Oh I would definitely pick up the phone and call again ...

    I wonder if this would work:

    int fa 0/0
    description inside LAN interface
    ip addr 70.x.x.66 ! default gateway

    int fa 0/1
    description outside interface facing Verizon FIOS ONT
    ip addr <any IP address / some mask>
    ip proxy-arp ! to answer ARP requests from 70.x.x.1

    ip classless
    ip route 70.x.x.1
    ip route 70.x.x.1 FastEthernet0/1
    Merv, May 5, 2008
  10. Merv,

    That's exactly what we're planning on trying next.

    As to calling Verizon: our problem is that FIOS was originally designed as a
    consumer service, and the support & sales staff reflects that. There is no
    such thing as specifying how a line is to be provisioned, or even asking a
    support person how their network behaves. There is only three options when
    you order a FIOS line: speed, static IP, and multiple IPs. You can only get
    one block. That's what's programmed into the sales staff's system, and they
    have no way of putting anything else into the computer.

    When you ask tech questions, everyone says "well it works if you use the
    Actiontec router we gave you." If you say "but we have a Cisco", they tell
    you that they would be beyond their "legal boundaries" to discuss how it
    might be programmed.

    Why are we putting up with this? We are getting 30 MBit down, 15 MBit up
    (and we do get that) and 61 IPs for $359.99/mo (vs our T1 which costs about
    $750/mo.) We're the serious real Business FIOS location in our city, and
    we're willing to do a bit of bleeding-edge work in order to get the
    Fletcher James, May 6, 2008
  11. Fletcher James

    Merv Guest

    For that kind of speed I would be willing to figure out how to make
    water flow uphill ...
    Merv, May 6, 2008
  12. Fletcher James

    Trendkill Guest

    If Verizon is willing to put a secondary IP on their router/vlan
    interface, I think you may have a not-so-ugly solution. If not, not
    sure what else to tell you.
    Trendkill, May 6, 2008
  13. Fletcher James

    Dan Lanciani Guest

    | OPTION D: We could assign a PRIVATE address to the outside of our router --
    | say, But then, how would we direct traffic to our gateway? If we
    | provide a default route just by interface
    | ip route FastEthernet0/1
    | then it's got to ARP for every single outbound address. QUESTION: would the
    | following solve that problem:
    | ip route 70.x.x.1
    | ip route 70.x.x.1 FastEthernet0/1

    Although it isn't often seen, you can specify the next hop directly on a
    route to an interface:

    ip route FastEthernet0/1 70.x.x.1

    That's the approach I'd take, with proxy ARP enabled on the interface
    to handle incoming traffic and an arbitrary address as you suggest in
    option D. Depending on how you set up the inside routing you might
    accidentally respond to ARPs for a few extra addresses, but then
    Verizon shouldn't be ARPing for those addresses in the first place.

    Dan Lanciani
    [email protected]*com
    Dan Lanciani, May 6, 2008
  14. Fletcher James

    stephen Guest

    1 suggestion is rig up Wireshark to look at what is actually happening on
    the outside interface.

    some real info is worth a lot of guessing.
    you can NAT an address to be the same address after it is translated (or at
    least you can on a PIX) - any use?
    use up a small amount of addrssing from your block for a /30 to have a link
    to the FIOS next hop.

    So your outside would be

    then you have an inside subnet for the full block, and let proxy ARP fix up
    the next hop stuff for FIOS.

    Cost is you lose some addresses for the reserved part of the /30 (and the
    complexity and debugging of course).

    this depends on the 1800 allowing you to set up overlapping subnets - used
    to work a few years back, but i havent tried recently........
    stephen, May 6, 2008
  15. Fletcher James

    Merv Guest

    use up a small amount of addrssing from your block for a /30 to have a link

    I believe that Trendkill has already pointed out the negative impact
    of attempting this approach:

    "then he can't easily subnet whats left without vlan'ing it into a /
    30, a /29, a /28, and a /27"
    Merv, May 6, 2008
  16. Fletcher James

    Trendkill Guest

    Yes, he can't use .2. He has been given .66 - .126, and .2 is off
    limits. If he takes .68/30 (he was told to avoid .65 for some reason
    as well), then he has to vlan/subnet the remaining portion of the /26
    into a /28 and /27. Not all that bad, but just a pain to have to deal
    with and he loses several addresses. The problem here is Verizon has
    not come to the table and said they will do anything, and if he uses .
    68/30, he can't get to their .1 interface anymore (they want him to
    use the full /24 and only use .65 - .126.

    This is why I was saying that if Verizon's FIOS router has a /24 and a
    IP of .1, why can't they turn up a secondary virtual interface on it
    (secondary IP) of .65 or .69? Then he can do the above to use a /30
    and vlan/trunk the rest which is a pain but will work. If Verizon is
    not willing to do that, then he will have to attempt one of the above
    solutions with default gateway-next hop interface, etc.
    Trendkill, May 6, 2008
  17. Fletcher James

    Trendkill Guest

    Although on second thought, this would work on the way out, but not
    the way back in, as the Verizon router still thinks its a .24 and
    would most likely not route to the next hop to get to the .64/26
    network. The only way to make this clean is to get them to assign a /
    31 or /30 that is not in the /26 they have assigned you or the bigger /
    24 that holds it that they insist on forcing upon you. Even if it was
    a private IP'ed point to point, that would be fine, provided their
    routing is correct, but I'll bet they don't want to split up that /
    24. Have you asked them if there is another block that they can
    assign you that will allow the usage of a /30 or /31 for this point to
    point purpose?
    Trendkill, May 6, 2008
  18. Fletcher James

    Merv Guest

    What is bit strange here is that of having a service were Verizon
    dishes out a /26 which is beyond anything a normal residence would
    require unless it is a multi-unit dwelling. This really is a an
    business Internet service offering but it certainly appears that it is
    not being supported as such.

    Unfortunately teh OP does not have access to a Verizon SE or account
    manager. If I were him I would write to the President of Verizon and
    request to be directed to someone in the Verizon organization who can
    address this business issue.
    Merv, May 6, 2008
  19. Fletcher James


    Apr 30, 2008
    Likes Received:
    Not sure about an 1841 but on a 7206 I would use subinterfaces for each wan address. I'm not sure if an 1841 is capable of this.

    ip address 70.x.x.66
    default gateway 70.x.x.1

    ip address 70.x.x.67
    default gateway 70.x.x.1


    Then use QoS or static routing to choose which subinterface you want traffic routed to.
    FLEngineer, May 6, 2008
  20. It is really a pretty wild situation. There appears to be not a single
    person in the FIOS business office or tech staff who knows ANYTHING. They
    can only read scripts and fill in forms which have been provided to them. I
    will definitely be trying to find some less-formal way way into the
    organization, or looking to see if there's anyone else who has dealt with
    this problem.

    Based on numerous tests, I have come to the conclusion that the
    router/switch on Verizon's side is totally spoof protected: It will not
    respond to an ARP query unless the source address is one of the 61 addresses
    assigned to us (we've tried .2 and also tried assigning to the
    outside of the router, and giving a static route to our gateway.) In one
    case, .2 worked briefly -- probably because we had just swiched over, and it
    had an association of our MAC address with a valid IP. In that
    configuration, we could route in/out of our network, but couldn't ping from
    the router to the gateway. Once that timed out, we were blocked, again.

    We have one remaining option ("OPTION B", in my original post) which is
    where I think we will end up for the medium term: We will route & NAT
    70.x.x.66-.126 on the outside to on the inside. Then, we
    will NAT again in our ISA server to our LAN and Service Zone addresses. Our
    VideoTeleconference units, outside the ISA server, can be told that their
    inside a NAT which is not H.323-aware, and they will work just fine.

    While we may need to make additional adjustments to a couple of other
    devices, there is ONE upside advantage to this: once we've got it working,
    none of the inside hosts will need to be aware of their public IP addresses.
    It also means that if the router falls back to the T1 line, or we switch
    ISPs, this will be almost totally transparent to everybody (the VTCs will
    need to have us adjust "IP Address to Show in Global Phone Book.")
    Fletcher James, May 6, 2008
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.