Site-to-site tunnel w/NAT, return packets decap but not routed?

Question: has anyone experienced this before? I'm hoping the solution
is out there. I've searched the Web already and I can't seem to find
this particular issue.

We've got a site-to-site tunnel between our PIX 515UR and our partner's
Cisco router. We've established the tunnel and traffic seems to flow
out just fine. We see encaps and decaps both when we send traffic.
For example, I attempt a telnet to a host on their network and our PIX
shows 3 encaps and 3 decaps. Their end shows 3 decaps and 3 encaps.
Their telnet server shows a 'successful' connection attempt - that is,
their server sees our connection attempt and sends back what I'm
assuming is an ACK. The problem is that when the packets decap on our
end they don't seem to go anywhere - it's like they die at the PIX.
(Thus, I cannot say for certain what their telnet server is sending
back, but it is sending back something because we see our decaps
matching lockstep with our encaps.) The only goofy thing about our
connection is that we are required to NAT our traffic before sending it
to the tunnel, which we let the PIX do.

Before I post all of my settings I wanted to find out if you've dealt
with this before and what you did. Did you have to play with the NAT
settings? What troubleshooting tips do you have?

I'll be happy to supply my relevant PIX settings and any console
debugging output.

Thanks,
Michael

 

Yes. If the decapsulated return packets meet both ends rules for IPSec
transmission, then the packets are matched against the outside ACL (if
sysopt connection permit-ipsec is NOT configured), and are then
processed through NAT (even if only nat (inside) 0 access-list), and
then routing is attempted. If your nat statements are incorrect for the
situation, then the destination might come out as an outside IP and the
packet would be discarded [assuming PIX 6, or PIX 7 without the
intra-interface option configured.]

I've seen it happen, yes.

Remember the "capture" command.

It is easy to get that wrong.

 

This sounds like a possibility. We have PIX 6.3.5. I've put the most
relevant parts of the config at the end of this post.

I am able to capture the outbound packets heading into the tunnel, but
nothing coming out of the tunnel. My guess is that the 'die' (i.e. are
discarded) prior to the capture command grabbing them.

I am now convinced that we've got it wrong. We have a Cisco tech (from
Cisco themselves, not some 3rd party) who set this up. Unfortunately
he's gone until Monday. I thought I'd do some more legwork myself
before then, which is why I'm posting here.

Walter, I have seen some of your posts and I have quickly come to
respect your knowledge and experience. If you don't mind taking a peek
at our configs, I'd appreciate it if you would look for any glaring
issues.

Just one heads up: we have 2 different site-to-site VPN tunnels. The
first one is working, which is access list kmc1. There is no NATing
for this connection. The access-lists crypto_map_45 and 121 are the
two access lists which are supposed to handle the tunnel traffic and
the NATing.

Thanks!
-Michael

My config, with irrelevant or private bits removed. The first 3 octets
of our public IP's have been replaced with "XX.YY.ZZ"...

pixfirewall# sh run
: Saved
:
PIX Version 6.3(5)
interface ethernet0 10baset
interface ethernet1 100basetx
interface ethernet2 100full
nameif ethernet0 outside security0
nameif ethernet1 inside security100
nameif ethernet2 VPNINT security50
access-list acl_outside permit icmp any any echo-reply
access-list acl_outside permit icmp any host FTPSERVER echo
access-list acl_outside permit icmp any host INTERPROSE echo
access-list acl_outside permit tcp any host FTPSERVER eq ftp
access-list acl_outside permit tcp any host INTERPROSE eq ssh
access-list acl_outside permit tcp any host FTPSERVER eq ftp-data
access-list acl_outside permit icmp any host EXCHANGE echo
access-list acl_outside permit tcp any host EXCHANGE eq smtp
access-list acl_outside permit icmp any host XX.YY.ZZ.44 echo
access-list acl_outside permit tcp any host XX.YY.ZZ.44 eq pop3
access-list acl_outside permit tcp any host XX.YY.ZZ.44 eq smtp
access-list acl_outside permit tcp any any eq citrix-ica
access-list acl_outside permit tcp any host EXCHANGE eq https
access-list acl_outside permit tcp any host INTERPROSE eq www
access-list acl_outside permit tcp any host FTPSERVER eq www
access-list acl_outside permit ip host 144.160.133.6 any
access-list acl_outside permit ip 144.154.0.0 255.255.0.0 any
access-list 101 permit ip 192.168.50.0 255.255.255.0 172.31.0.0
255.255.255.0
access-list 101 permit ip 192.168.51.0 255.255.255.0 172.31.0.0
255.255.255.0
access-list 101 permit ip 192.168.52.0 255.255.255.0 172.31.0.0
255.255.255.0
access-list 101 permit ip 192.168.49.0 255.255.255.0 172.31.0.0
255.255.255.0
access-list 101 permit ip 172.23.0.0 255.255.0.0 172.31.0.0
255.255.255.0
access-list VMSys permit ip host 192.168.50.244 host 172.31.1.1
access-list VMSys permit ip host 192.168.50.244 host 172.31.1.2
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.100
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.101
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.102
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.103
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.104
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.105
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.132.106
access-list kmc1 permit ip 192.168.50.0 255.255.255.0 host
129.47.250.50
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.100
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.101
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.102
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.103
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.104
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.105
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.132.106
access-list kmc1 permit ip 192.168.49.0 255.255.255.0 host
129.47.250.50
access-list NONAT permit ip 192.168.50.0 255.255.255.0 172.29.1.0
255.255.255.0
access-list NONAT permit ip 192.168.51.0 255.255.255.0 172.29.1.0
255.255.255.0
access-list NONAT permit ip 192.168.52.0 255.255.255.0 172.29.1.0
255.255.255.0
access-list NONAT permit ip 192.168.50.0 255.255.255.0 129.47.0.0
255.255.0.0
access-list NONAT permit ip 192.168.51.0 255.255.255.0 129.47.0.0
255.255.0.0
access-list NONAT permit ip 192.168.49.0 255.255.255.0 129.47.0.0
255.255.0.0
access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 host
209.78.140.25
access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 155.179.0.0
255.255.0.0
access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 144.155.0.0
255.255.0.0
access-list inside_cryptomap_45 permit ip host XX.YY.ZZ.46 144.154.0.0
255.255.0.0
access-list 121 permit ip 192.168.50.0 255.255.255.0 155.179.0.0
255.255.0.0
access-list 121 permit ip 192.168.50.0 255.255.255.0 host 209.78.140.25
access-list 121 permit ip 192.168.50.0 255.255.255.0 144.155.0.0
255.255.0.0
access-list 121 permit ip 192.168.50.0 255.255.255.0 144.154.0.0
255.255.0.0
access-list 121 permit ip 172.31.0.0 255.255.0.0 144.154.0.0
255.255.0.0
access-list capture permit ip host 192.168.50.21 host 144.155.34.27
access-list capture permit ip host 144.155.34.27 host 192.168.50.21
access-list capture permit ip host 192.168.50.21 host 144.154.23.10
access-list capture permit ip host 144.154.23.10 host 192.168.50.21
ip audit info action alarm
ip audit attack action alarm
no failover
failover timeout 0:00:00
failover poll 15
no failover ip address outside
no failover ip address inside
no failover ip address VPNINT
pdm history enable
arp timeout 14400
global (outside) 1 interface
global (VPNINT) 1 XX.YY.ZZ.46
global (VPNINT) 2 interface
nat (inside) 0 access-list NONAT
nat (inside) 1 access-list 121 0 0
nat (inside) 1 0.0.0.0 0.0.0.0 0 0
access-group acl_outside in interface VPNINT
route VPNINT 0.0.0.0 0.0.0.0 XX.YY.ZZ.254 1
route inside 10.0.0.0 255.0.0.0 172.16.0.1 1
route inside 172.16.0.0 255.255.0.0 172.16.0.1 1
route inside 172.18.1.0 255.255.255.0 172.16.0.1 1
route inside 172.23.0.0 255.255.0.0 172.16.0.1 1
route inside 192.168.0.0 255.255.0.0 172.16.0.1 1
floodguard enable
sysopt connection permit-ipsec
sysopt connection permit-pptp
sysopt ipsec pl-compatible
crypto ipsec transform-set vpnset esp-3des esp-sha-hmac
crypto ipsec transform-set strong esp-des esp-sha-hmac
crypto ipsec transform-set atntvpnset esp-3des esp-sha-hmac
crypto dynamic-map vpnclient 10 set transform-set vpnset
crypto map vpnmap 45 ipsec-isakmp
crypto map vpnmap 45 match address inside_cryptomap_45
crypto map vpnmap 45 set pfs group2
crypto map vpnmap 45 set peer 144.160.133.6
crypto map vpnmap 45 set transform-set atntvpnset
crypto map vpnmap 45 set security-association lifetime seconds 28800
kilobytes 3600
crypto map vpnmap client configuration address initiate
crypto map vpnmap client configuration address respond
crypto map vpnmap interface VPNINT
isakmp enable VPNINT
isakmp key ******** address 144.160.133.6 netmask 255.255.255.255
no-xauth no-config-mode
isakmp identity address
isakmp nat-traversal 45
isakmp policy 45 authentication pre-share
isakmp policy 45 encryption 3des
isakmp policy 45 hash sha
isakmp policy 45 group 2
isakmp policy 45 lifetime 86400

 

[PIX 515UR]

Looks like that should be inside_cryptomap_45 .

Looks like you trimmed kmc1 out of the config except for the
access-list for it.

access-list VMSys and 101 has no apparent purpose in that config extract.

ethernet0 and ethernet1 are both forced to half duplex ? And so forced
on whatever is connected to them? Forcing speed and duplex can be
a good thing for infrastructure, but it can lead to slow slow slow
transmissions with lots of errors if the other side is set to
autonegotiate (or autonegotiates even when told not to!)

Get rid of those two sysopt lines, they are just confusing things.
You aren't running pptp, and you aren't trying to talk to a PIX 4 running
a Private Link (pre-IPSec) tunnel.

You have acl_outside applied to VPNINT, which is presumably reserved
for VPN traffic and which presumably should not have any normal
traffic. But you have configured sysopt connection permit-ipsec,
which exempts VPN traffic from interface ACLs (in both directions),
so VPN traffic would not be affected by acl_outside and you might
as well remove anything in acl_outside that does not have to do with
simple link maintainence. Unless, that is, that VPNINT is named
misleadingly and your main internet traffic is hitting it instead of
hitting the "outside" interface? I do note that in the extract you
posted, you have no access-group against the interface "outside".
(If acl_outside is applied to the outside interface as well as VPNINT,
use seperate ACLs, or Things Will Break.)

Based upon what you posted, I cannot be certain if the lines are
useful or not, and I cannot tell whether you have used the correct
IP addresses as the destinations. Hmmm, maybe you use the "outside"
interface for your other VPN connection?

That's confusing. You are using policy static to control the nat of
your traffic to the VPN, but you are using the same policy number, 1,
for all other traffic except that designated by NONAT, and you are
using policy number 1 for both global (outside) and global (VPNINT).
You also have a global (VPNINT) 2 but no corresponding nat 2 policy
statement. And anything that matches access-list 121 from the "inside"
but is destined for the "outside" is going to the same global (outside) 1
as is being used by nat (inside) 0.0.0.0 0.0.0.0 . That suggests that
you should eliminate anything in access-list 121 that doesn't have
to do with translation to the VPN, and that you should use a different
policy number than 1 (and 2?). Even if it all "works out", using
distinct policy numbers will make it much more obvious to the reader
what is going on. Policy numbers other than 0 are arbitrary and
unordered, by the way, so use any convenient non-0 policy number.

Okay, good, there's no overlap between NONAT and 121.

If I read this correctly, you want inside traffic in
192.168.50/24 and 172.31/16 to be NAT'd to XXX.YYY.ZZ.46 and sent over
the tunnel when certain remote destinations are involved. The access-list
121 and inside_cryptomap_45 are consistant for that flow; there is no
obvious error in them.

Hmmm, the default route is to VPNINT instead of the typical outside
interface. Unusual. You trust the kmc1 VPN more than you trust
random hosts on the 'net, don't you? So if kmc1 is hanging off the
other ("outside") interface, the interface it is hanging off of
would normally have a higher security level than that which public
traffic an arrive on.

The bulk of your ACLs for the problem VPN involve 192.168.50/24 .
You don't show the IP addresses for the interfaces, but that
that last route (for 192.168/16) together with the destinations for
the routes suggest that your inside interface is configured in
172.16.0.x . But if that is so, then the route inside 172.16/16
becomes problematic as it would appear to overlap the route
that the PIX would always add if the interface itself is in 172.16/16 .

The non 192.168.50/24 traffic that is to go over the link is
172.31/16, but I notice that there is no route inside 172.31/16 .
If we speculate that the inside interface is in 172.31/16 itself,
then the route statements would not work, as the destination of
a route statement must be in the same IP range as the interface IP.

I would say tentatively that your major problem might be a lack
of a 172.31/16 route.

There is nothing in what you posted that obviously uses the 'strong'
transform-set, but be wary: by the time of PIX 6.3, DES SHA
is not supported, requring DES MD5 instead [if my memory serves me.]

You do not import vpnclient into any regular crypto map.

The kilobytes limit used to be non-functional; I don't recall if they
ever got around to implementing it.

That implies that in something not shown, you are importing the
dynamic-map vpnclient into vpnmap, and implies that you have a
an ip pool not shown, or that you have vpn group statements not shown.
(If you do have vpn group statements not shown, then that could
explain the sysopt connection permit-pptp ).

The isakmp policy numbers are independant of the crypto map policy
numbers -- they don't both have to be 45 (and normally you'd provide
multiple transforms for the cypto map, and some fallback transforms
for isakmp, to make it easier to transition to new equipment and new
software releases.)

The number for isakmp nat-traversal is a time, usually 20, and
does not need to have anything to do with the crypto map or isakmp
policy numbers.