Site-to-site tunnel w/NAT, return packets decap but not routed?

Discussion in 'Cisco' started by mercutio.viz@gmail.com, Dec 14, 2006.

  1. Guest

    Question: has anyone experienced this before? I'm hoping the solution
    is out there. I've searched the Web already and I can't seem to find
    this particular issue.

    We've got a site-to-site tunnel between our PIX 515UR and our partner's
    Cisco router. We've established the tunnel and traffic seems to flow
    out just fine. We see encaps and decaps both when we send traffic.
    For example, I attempt a telnet to a host on their network and our PIX
    shows 3 encaps and 3 decaps. Their end shows 3 decaps and 3 encaps.
    Their telnet server shows a 'successful' connection attempt - that is,
    their server sees our connection attempt and sends back what I'm
    assuming is an ACK. The problem is that when the packets decap on our
    end they don't seem to go anywhere - it's like they die at the PIX.
    (Thus, I cannot say for certain what their telnet server is sending
    back, but it is sending back something because we see our decaps
    matching lockstep with our encaps.) The only goofy thing about our
    connection is that we are required to NAT our traffic before sending it
    to the tunnel, which we let the PIX do.

    Before I post all of my settings I wanted to find out if you've dealt
    with this before and what you did. Did you have to play with the NAT
    settings? What troubleshooting tips do you have?

    I'll be happy to supply my relevant PIX settings and any console
    debugging output.

    Thanks,
    Michael
    , Dec 14, 2006
    #1
    1. Advertising

  2. In article <>,
    <> wrote:
    >Question: has anyone experienced this before?


    Yes. If the decapsulated return packets meet both ends rules for IPSec
    transmission, then the packets are matched against the outside ACL (if
    sysopt connection permit-ipsec is NOT configured), and are then
    processed through NAT (even if only nat (inside) 0 access-list), and
    then routing is attempted. If your nat statements are incorrect for the
    situation, then the destination might come out as an outside IP and the
    packet would be discarded [assuming PIX 6, or PIX 7 without the
    intra-interface option configured.]

    >The problem is that when the packets decap on our
    >end they don't seem to go anywhere - it's like they die at the PIX.


    I've seen it happen, yes.

    >(Thus, I cannot say for certain what their telnet server is sending
    >back, but it is sending back something because we see our decaps
    >matching lockstep with our encaps.)


    Remember the "capture" command.

    >The only goofy thing about our
    >connection is that we are required to NAT our traffic before sending it
    >to the tunnel, which we let the PIX do.


    It is easy to get that wrong.
    Walter Roberson, Dec 14, 2006
    #2
    1. Advertising

  3. Guest

    Walter Roberson wrote:
    > In article <>,
    > <> wrote:
    > >Question: has anyone experienced this before?

    >
    > Yes. If the decapsulated return packets meet both ends rules for IPSec
    > transmission, then the packets are matched against the outside ACL (if
    > sysopt connection permit-ipsec is NOT configured), and are then
    > processed through NAT (even if only nat (inside) 0 access-list), and
    > then routing is attempted. If your nat statements are incorrect for the
    > situation, then the destination might come out as an outside IP and the
    > packet would be discarded [assuming PIX 6, or PIX 7 without the
    > intra-interface option configured.]


    This sounds like a possibility. We have PIX 6.3.5. I've put the most
    relevant parts of the config at the end of this post.

    >
    > >The problem is that when the packets decap on our
    > >end they don't seem to go anywhere - it's like they die at the PIX.

    >
    > I've seen it happen, yes.
    >
    > >(Thus, I cannot say for certain what their telnet server is sending
    > >back, but it is sending back something because we see our decaps
    > >matching lockstep with our encaps.)

    >
    > Remember the "capture" command.


    I am able to capture the outbound packets heading into the tunnel, but
    nothing coming out of the tunnel. My guess is that the 'die' (i.e. are
    discarded) prior to the capture command grabbing them.

    >
    > >The only goofy thing about our
    > >connection is that we are required to NAT our traffic before sending it
    > >to the tunnel, which we let the PIX do.

    >
    > It is easy to get that wrong.


    I am now convinced that we've got it wrong. We have a Cisco tech (from
    Cisco themselves, not some 3rd party) who set this up. Unfortunately
    he's gone until Monday. I thought I'd do some more legwork myself
    before then, which is why I'm posting here.

    Walter, I have seen some of your posts and I have quickly come to
    respect your knowledge and experience. If you don't mind taking a peek
    at our configs, I'd appreciate it if you would look for any glaring
    issues.

    Just one heads up: we have 2 different site-to-site VPN tunnels. The
    first one is working, which is access list kmc1. There is no NATing
    for this connection. The access-lists crypto_map_45 and 121 are the
    two access lists which are supposed to handle the tunnel traffic and
    the NATing.

    Thanks!
    -Michael

    My config, with irrelevant or private bits removed. The first 3 octets
    of our public IP's have been replaced with "XX.YY.ZZ"...

    pixfirewall# sh run
    : Saved
    :
    PIX Version 6.3(5)
    interface ethernet0 10baset
    interface ethernet1 100basetx
    interface ethernet2 100full
    nameif ethernet0 outside security0
    nameif ethernet1 inside security100
    nameif ethernet2 VPNINT security50
    access-list acl_outside permit icmp any any echo-reply
    access-list acl_outside permit icmp any host FTPSERVER echo
    access-list acl_outside permit icmp any host INTERPROSE echo
    access-list acl_outside permit tcp any host FTPSERVER eq ftp
    access-list acl_outside permit tcp any host INTERPROSE eq ssh
    access-list acl_outside permit tcp any host FTPSERVER eq ftp-data
    access-list acl_outside permit icmp any host EXCHANGE echo
    access-list acl_outside permit tcp any host EXCHANGE eq smtp
    access-list acl_outside permit icmp any host XX.YY.ZZ.44 echo
    access-list acl_outside permit tcp any host XX.YY.ZZ.44 eq pop3
    access-list acl_outside permit tcp any host XX.YY.ZZ.44 eq smtp
    access-list acl_outside permit tcp any any eq citrix-ica
    access-list acl_outside permit tcp any host EXCHANGE eq https
    access-list acl_outside permit tcp any host INTERPROSE eq www
    access-list acl_outside permit tcp any host FTPSERVER eq www
    access-list acl_outside permit ip host 144.160.133.6 any
    access-list acl_outside permit ip 144.154.0.0 255.255.0.0 any
    access-list 101 permit ip 192.168.50.0 255.255.255.0 172.31.0.0
    255.255.255.0
    access-list 101 permit ip 192.168.51.0 255.255.255.0 172.31.0.0
    255.255.255.0
    access-list 101 permit ip 192.168.52.0 255.255.255.0 172.31.0.0
    255.255.255.0
    access-list 101 permit ip 192.168.49.0 255.255.255.0 172.31.0.0
    255.255.255.0
    access-list 101 permit ip 172.23.0.0 255.255.0.0 172.31.0.0
    255.255.255.0
    access-list VMSys permit ip host 192.168.50.244 host 172.31.1.1
    access-list VMSys permit ip host 192.168.50.244 host 172.31.1.2
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.100
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.101
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.102
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.103
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.104
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.105
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.132.106
    access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
    129.47.250.50
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.100
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.101
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.102
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.103
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.104
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.105
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.132.106
    access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
    129.47.250.50
    access-list NONAT permit ip 192.168.50.0 255.255.255.0 172.29.1.0
    255.255.255.0
    access-list NONAT permit ip 192.168.51.0 255.255.255.0 172.29.1.0
    255.255.255.0
    access-list NONAT permit ip 192.168.52.0 255.255.255.0 172.29.1.0
    255.255.255.0
    access-list NONAT permit ip 192.168.50.0 255.255.255.0 129.47.0.0
    255.255.0.0
    access-list NONAT permit ip 192.168.51.0 255.255.255.0 129.47.0.0
    255.255.0.0
    access-list NONAT permit ip 192.168.49.0 255.255.255.0 129.47.0.0
    255.255.0.0
    access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 host
    209.78.140.25
    access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 155.179.0.0
    255.255.0.0
    access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 144.155.0.0
    255.255.0.0
    access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 144.154.0.0
    255.255.0.0
    access-list 121 permit ip 192.168.50.0 255.255.255.0 155.179.0.0
    255.255.0.0
    access-list 121 permit ip 192.168.50.0 255.255.255.0 host 209.78.140.25
    access-list 121 permit ip 192.168.50.0 255.255.255.0 144.155.0.0
    255.255.0.0
    access-list 121 permit ip 192.168.50.0 255.255.255.0 144.154.0.0
    255.255.0.0
    access-list 121 permit ip 172.31.0.0 255.255.0.0 144.154.0.0
    255.255.0.0
    access-list capture permit ip host 192.168.50.21 host 144.155.34.27
    access-list capture permit ip host 144.155.34.27 host 192.168.50.21
    access-list capture permit ip host 192.168.50.21 host 144.154.23.10
    access-list capture permit ip host 144.154.23.10 host 192.168.50.21
    ip audit info action alarm
    ip audit attack action alarm
    no failover
    failover timeout 0:00:00
    failover poll 15
    no failover ip address outside
    no failover ip address inside
    no failover ip address VPNINT
    pdm history enable
    arp timeout 14400
    global (outside) 1 interface
    global (VPNINT) 1 XX.YY.ZZ.46
    global (VPNINT) 2 interface
    nat (inside) 0 access-list NONAT
    nat (inside) 1 access-list 121 0 0
    nat (inside) 1 0.0.0.0 0.0.0.0 0 0
    access-group acl_outside in interface VPNINT
    route VPNINT 0.0.0.0 0.0.0.0 XX.YY.ZZ.254 1
    route inside 10.0.0.0 255.0.0.0 172.16.0.1 1
    route inside 172.16.0.0 255.255.0.0 172.16.0.1 1
    route inside 172.18.1.0 255.255.255.0 172.16.0.1 1
    route inside 172.23.0.0 255.255.0.0 172.16.0.1 1
    route inside 192.168.0.0 255.255.0.0 172.16.0.1 1
    floodguard enable
    sysopt connection permit-ipsec
    sysopt connection permit-pptp
    sysopt ipsec pl-compatible
    crypto ipsec transform-set vpnset esp-3des esp-sha-hmac
    crypto ipsec transform-set strong esp-des esp-sha-hmac
    crypto ipsec transform-set atntvpnset esp-3des esp-sha-hmac
    crypto dynamic-map vpnclient 10 set transform-set vpnset
    crypto map vpnmap 45 ipsec-isakmp
    crypto map vpnmap 45 match address inside_cryptomap_45
    crypto map vpnmap 45 set pfs group2
    crypto map vpnmap 45 set peer 144.160.133.6
    crypto map vpnmap 45 set transform-set atntvpnset
    crypto map vpnmap 45 set security-association lifetime seconds 28800
    kilobytes 3600
    crypto map vpnmap client configuration address initiate
    crypto map vpnmap client configuration address respond
    crypto map vpnmap interface VPNINT
    isakmp enable VPNINT
    isakmp key ******** address 144.160.133.6 netmask 255.255.255.255
    no-xauth no-config-mode
    isakmp identity address
    isakmp nat-traversal 45
    isakmp policy 45 authentication pre-share
    isakmp policy 45 encryption 3des
    isakmp policy 45 hash sha
    isakmp policy 45 group 2
    isakmp policy 45 lifetime 86400
    , Dec 14, 2006
    #3
  4. In article <>,
    <> wrote:

    [PIX 515UR]

    >Just one heads up: we have 2 different site-to-site VPN tunnels. The
    >first one is working, which is access list kmc1. There is no NATing
    >for this connection. The access-lists crypto_map_45 and 121 are the
    >two access lists which are supposed to handle the tunnel traffic and
    >the NATing.


    Looks like that should be inside_cryptomap_45 .

    Looks like you trimmed kmc1 out of the config except for the
    access-list for it.

    access-list VMSys and 101 has no apparent purpose in that config extract.


    >PIX Version 6.3(5)


    >interface ethernet0 10baset
    >interface ethernet1 100basetx


    ethernet0 and ethernet1 are both forced to half duplex ? And so forced
    on whatever is connected to them? Forcing speed and duplex can be
    a good thing for infrastructure, but it can lead to slow slow slow
    transmissions with lots of errors if the other side is set to
    autonegotiate (or autonegotiates even when told not to!)


    >interface ethernet2 100full


    >sysopt connection permit-pptp
    >sysopt ipsec pl-compatible


    Get rid of those two sysopt lines, they are just confusing things.
    You aren't running pptp, and you aren't trying to talk to a PIX 4 running
    a Private Link (pre-IPSec) tunnel.

    >nameif ethernet0 outside security0
    >nameif ethernet1 inside security100
    >nameif ethernet2 VPNINT security50


    >access-list acl_outside permit icmp any any echo-reply
    >access-list acl_outside permit icmp any host FTPSERVER echo
    >access-list acl_outside permit icmp any host INTERPROSE echo
    >access-list acl_outside permit tcp any host FTPSERVER eq ftp
    >access-list acl_outside permit tcp any host INTERPROSE eq ssh
    >access-list acl_outside permit tcp any host FTPSERVER eq ftp-data
    >access-list acl_outside permit icmp any host EXCHANGE echo
    >access-list acl_outside permit tcp any host EXCHANGE eq smtp
    >access-list acl_outside permit icmp any host XX.YY.ZZ.44 echo
    >access-list acl_outside permit tcp any host XX.YY.ZZ.44 eq pop3
    >access-list acl_outside permit tcp any host XX.YY.ZZ.44 eq smtp
    >access-list acl_outside permit tcp any any eq citrix-ica
    >access-list acl_outside permit tcp any host EXCHANGE eq https
    >access-list acl_outside permit tcp any host INTERPROSE eq www
    >access-list acl_outside permit tcp any host FTPSERVER eq www
    >access-list acl_outside permit ip host 144.160.133.6 any
    >access-list acl_outside permit ip 144.154.0.0 255.255.0.0 any


    >sysopt connection permit-ipsec


    You have acl_outside applied to VPNINT, which is presumably reserved
    for VPN traffic and which presumably should not have any normal
    traffic. But you have configured sysopt connection permit-ipsec,
    which exempts VPN traffic from interface ACLs (in both directions),
    so VPN traffic would not be affected by acl_outside and you might
    as well remove anything in acl_outside that does not have to do with
    simple link maintainence. Unless, that is, that VPNINT is named
    misleadingly and your main internet traffic is hitting it instead of
    hitting the "outside" interface? I do note that in the extract you
    posted, you have no access-group against the interface "outside".
    (If acl_outside is applied to the outside interface as well as VPNINT,
    use seperate ACLs, or Things Will Break.)

    Based upon what you posted, I cannot be certain if the lines are
    useful or not, and I cannot tell whether you have used the correct
    IP addresses as the destinations. Hmmm, maybe you use the "outside"
    interface for your other VPN connection?

    >global (outside) 1 interface
    >global (VPNINT) 1 XX.YY.ZZ.46
    >global (VPNINT) 2 interface
    >nat (inside) 0 access-list NONAT
    >nat (inside) 1 access-list 121 0 0
    >nat (inside) 1 0.0.0.0 0.0.0.0 0 0


    That's confusing. You are using policy static to control the nat of
    your traffic to the VPN, but you are using the same policy number, 1,
    for all other traffic except that designated by NONAT, and you are
    using policy number 1 for both global (outside) and global (VPNINT).
    You also have a global (VPNINT) 2 but no corresponding nat 2 policy
    statement. And anything that matches access-list 121 from the "inside"
    but is destined for the "outside" is going to the same global (outside) 1
    as is being used by nat (inside) 0.0.0.0 0.0.0.0 . That suggests that
    you should eliminate anything in access-list 121 that doesn't have
    to do with translation to the VPN, and that you should use a different
    policy number than 1 (and 2?). Even if it all "works out", using
    distinct policy numbers will make it much more obvious to the reader
    what is going on. Policy numbers other than 0 are arbitrary and
    unordered, by the way, so use any convenient non-0 policy number.

    >access-list NONAT permit ip 192.168.50.0 255.255.255.0 172.29.1.0 255.255.255.0
    >access-list NONAT permit ip 192.168.51.0 255.255.255.0 172.29.1.0 255.255.255.0
    >access-list NONAT permit ip 192.168.52.0 255.255.255.0 172.29.1.0 255.255.255.0
    >access-list NONAT permit ip 192.168.50.0 255.255.255.0 129.47.0.0 255.255.0.0
    >access-list NONAT permit ip 192.168.51.0 255.255.255.0 129.47.0.0 255.255.0.0
    >access-list NONAT permit ip 192.168.49.0 255.255.255.0 129.47.0.0 255.255.0.0


    >access-list 121 permit ip 192.168.50.0 255.255.255.0 155.179.0.0 255.255.0.0
    >access-list 121 permit ip 192.168.50.0 255.255.255.0 host 209.78.140.25
    >access-list 121 permit ip 192.168.50.0 255.255.255.0 144.155.0.0 255.255.0.0
    >access-list 121 permit ip 192.168.50.0 255.255.255.0 144.154.0.0 255.255.0.0
    >access-list 121 permit ip 172.31.0.0 255.255.0.0 144.154.0.0 255.255.0.0


    Okay, good, there's no overlap between NONAT and 121.

    >access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 host 209.78.140.25
    >access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 155.179.0.0 255.255.0.0
    >access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 144.155.0.0 255.255.0.0
    >access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 144.154.0.0 255.255.0.0


    If I read this correctly, you want inside traffic in
    192.168.50/24 and 172.31/16 to be NAT'd to XXX.YYY.ZZ.46 and sent over
    the tunnel when certain remote destinations are involved. The access-list
    121 and inside_cryptomap_45 are consistant for that flow; there is no
    obvious error in them.

    >access-group acl_outside in interface VPNINT


    >route VPNINT 0.0.0.0 0.0.0.0 XX.YY.ZZ.254 1


    Hmmm, the default route is to VPNINT instead of the typical outside
    interface. Unusual. You trust the kmc1 VPN more than you trust
    random hosts on the 'net, don't you? So if kmc1 is hanging off the
    other ("outside") interface, the interface it is hanging off of
    would normally have a higher security level than that which public
    traffic an arrive on.

    >route inside 10.0.0.0 255.0.0.0 172.16.0.1 1
    >route inside 172.16.0.0 255.255.0.0 172.16.0.1 1
    >route inside 172.18.1.0 255.255.255.0 172.16.0.1 1
    >route inside 172.23.0.0 255.255.0.0 172.16.0.1 1
    >route inside 192.168.0.0 255.255.0.0 172.16.0.1 1


    The bulk of your ACLs for the problem VPN involve 192.168.50/24 .
    You don't show the IP addresses for the interfaces, but that
    that last route (for 192.168/16) together with the destinations for
    the routes suggest that your inside interface is configured in
    172.16.0.x . But if that is so, then the route inside 172.16/16
    becomes problematic as it would appear to overlap the route
    that the PIX would always add if the interface itself is in 172.16/16 .

    The non 192.168.50/24 traffic that is to go over the link is
    172.31/16, but I notice that there is no route inside 172.31/16 .
    If we speculate that the inside interface is in 172.31/16 itself,
    then the route statements would not work, as the destination of
    a route statement must be in the same IP range as the interface IP.

    I would say tentatively that your major problem might be a lack
    of a 172.31/16 route.


    >crypto ipsec transform-set vpnset esp-3des esp-sha-hmac
    >crypto ipsec transform-set strong esp-des esp-sha-hmac
    >crypto ipsec transform-set atntvpnset esp-3des esp-sha-hmac


    There is nothing in what you posted that obviously uses the 'strong'
    transform-set, but be wary: by the time of PIX 6.3, DES SHA
    is not supported, requring DES MD5 instead [if my memory serves me.]

    >crypto dynamic-map vpnclient 10 set transform-set vpnset


    You do not import vpnclient into any regular crypto map.

    >crypto map vpnmap 45 ipsec-isakmp
    >crypto map vpnmap 45 match address inside_cryptomap_45
    >crypto map vpnmap 45 set pfs group2
    >crypto map vpnmap 45 set peer 144.160.133.6
    >crypto map vpnmap 45 set transform-set atntvpnset
    >crypto map vpnmap 45 set security-association lifetime seconds 28800 kilobytes 3600


    The kilobytes limit used to be non-functional; I don't recall if they
    ever got around to implementing it.

    >crypto map vpnmap client configuration address initiate
    >crypto map vpnmap client configuration address respond


    That implies that in something not shown, you are importing the
    dynamic-map vpnclient into vpnmap, and implies that you have a
    an ip pool not shown, or that you have vpn group statements not shown.
    (If you do have vpn group statements not shown, then that could
    explain the sysopt connection permit-pptp ).

    >crypto map vpnmap interface VPNINT


    >isakmp enable VPNINT


    >isakmp key ******** address 144.160.133.6 netmask 255.255.255.255 no-xauth no-config-mode


    >isakmp identity address


    >isakmp policy 45 authentication pre-share
    >isakmp policy 45 encryption 3des
    >isakmp policy 45 hash sha
    >isakmp policy 45 group 2
    >isakmp policy 45 lifetime 86400


    >isakmp nat-traversal 45


    The isakmp policy numbers are independant of the crypto map policy
    numbers -- they don't both have to be 45 (and normally you'd provide
    multiple transforms for the cypto map, and some fallback transforms
    for isakmp, to make it easier to transition to new equipment and new
    software releases.)

    The number for isakmp nat-traversal is a time, usually 20, and
    does not need to have anything to do with the crypto map or isakmp
    policy numbers.
    Walter Roberson, Dec 14, 2006
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. a.nonny mouse
    Replies:
    2
    Views:
    1,052
  2. Replies:
    0
    Views:
    2,660
  3. Replies:
    3
    Views:
    6,480
  4. Trouble
    Replies:
    0
    Views:
    541
    Trouble
    Aug 4, 2006
  5. lidram
    Replies:
    0
    Views:
    355
    lidram
    Mar 26, 2008
Loading...

Share This Page