PIX 506e Connection Problems
 

Hello,

My client has a PIX 506e configured as their primary firewall. They have 4
external IP addresses that are statically mapped to 4 internal servers - one
for SMTP, HTTP, HTTPS, and FTP. Access from the outside world to these
servers works fine. However, what we are finding is that the static
mappings are interfering with internal access to these servers. They
"vanish" momentarily but then reappear. These connection blips happen
continuously but go away if I break the static mapping on the router.

All inside systems are on one subnet and use the PIX as their default
gateway to the outside world. But, inside traffic shouldn't be affected by
the PIX should it? That's what appears to be happening.

Has anyone seen this before? I'm guessing I have something configured
improperly.

Thanks in advance for your help.

Re: PIX 506e Connection Problems
 

In article <8_Sdnam76pYeKP_cRVn-ug@rogers.com>,
John Balch <john@jmbalch.com> wrote:
;All inside systems are on one subnet and use the PIX as their default
;gateway to the outside world. But, inside traffic shouldn't be affected by
;the PIX should it? That's what appears to be happening.

What IP address are they attempting to access the serves from internally?
The IP as known internally, or the IP as known externally? Or are those
the same?

If the internal and external versions of the IP are different and if
the inside users attempt to connect via the external IP, then that
is not supported on the PIX and the PIX will drop the packets
(in all PIX software versions that have been released, it is not allowed
for a packet to come in one [logical] interface and go back out the
same [logical] interface, even if there are translations or VPNs involved.)

If the accesses are via the external IP and you find that most of the
time it works, then I would suspect that your clients are using
MS NT or newer MS Windows, which broadcasts arp requests via the
all-1's "all systems" IP address instead of via the subnet broadcast
IP address. If the destination systems happened to be configured to
listen on the external IP as well as the internal IP, then they would
respond directly to the sender, even though the two are on different
IP subnets. The sending system would then be able to communicate
with that system until that ARP entry timed out in its tables.

I can't think of exactly how it might happen at the moment, but
plausibly there might be circumstances under which the PIX and the
destination Windows box "raced" to reply; if the destination Windows
box happened to reply first, then all would be well, but if the PIX
packet happened to arrive first, then because the PIX doesn't allow
that kind of traversal, the PIX packet would be an ICMP unreachable
or TCP RST packet packet; the latter especially would interfere with
the bypass communications. Furthermore, if the PIX packet replied using
the target IP address as the source, then the MAC of the PIX might
get written into the ARP tables as the MAC for that IP. If that happened,
then the next packets would be sent to the PIX, which would refuse them.
Possibly the IP stack would interpret the ICMP refusal as indicating
that the ARP entry should be flushed; if so, then the time unreachable
would be fairly short, but if the PIX's MAC stayed in the table,
the blip without communications would last until the ARP entry timed
out, and then it would be off to the races again.

I wouldn't want to have to replicate this hypothesis in a lab ;-)
And it's a not a useful hypothesis if the inside users always
access the internal servers through the internal IP.
--
And the wind keeps blowing the angel / Backwards into the future /
And this wind, this wind / Is called / Progress.
-- Laurie Anderson

Re: PIX 506e Connection Problems
 

Thanks for your reply, Walter. But, the internal users are accessing the
internal systems via their internal IP addresses - not the external IP
addresses.

One peculiarity about our configuration is that we use the 128.1.x.x network
internally. I know this is not a standard internal subnet but it was set up
long before I got on board by someone who didn't understand that. I know
someone else has the 128.1.x.x address somewhere out there on the Internet.
Could that be causing this complication? I have the 128.1.x.x defined as
being "internal" and the internal interface on the PIX is 128.1.0.1. I
found some documentation talking about using the "alias" command if you have
internal IP addresses that may overlap with external... but, when I tried
that, the PDM really squawked about it being an obsolete command.

The strange thing is that it seems to work most of the time... when these
failures start to appear, it is typically in mid-afternoon when network
traffic is at its peak. When the failures start, if I remove the static
mapping for the affected servers, the problem immediately goes away and
stays away until I put the static mappings back in.

Any other ideas? Thanks.


"Walter Roberson" <roberson@ibd.nrc-cnrc.gc.ca> wrote in message
news:cjv018$t6n$1@canopus.cc.umanitoba.ca...
> In article <8_Sdnam76pYeKP_cRVn-ug@rogers.com>,
> John Balch <john@jmbalch.com> wrote:
> ;All inside systems are on one subnet and use the PIX as their default
> ;gateway to the outside world. But, inside traffic shouldn't be affected
> by
> ;the PIX should it? That's what appears to be happening.
>
> What IP address are they attempting to access the serves from internally?
> The IP as known internally, or the IP as known externally? Or are those
> the same?
>
> If the internal and external versions of the IP are different and if
> the inside users attempt to connect via the external IP, then that
> is not supported on the PIX and the PIX will drop the packets
> (in all PIX software versions that have been released, it is not allowed
> for a packet to come in one [logical] interface and go back out the
> same [logical] interface, even if there are translations or VPNs
> involved.)
>
> If the accesses are via the external IP and you find that most of the
> time it works, then I would suspect that your clients are using
> MS NT or newer MS Windows, which broadcasts arp requests via the
> all-1's "all systems" IP address instead of via the subnet broadcast
> IP address. If the destination systems happened to be configured to
> listen on the external IP as well as the internal IP, then they would
> respond directly to the sender, even though the two are on different
> IP subnets. The sending system would then be able to communicate
> with that system until that ARP entry timed out in its tables.
>
> I can't think of exactly how it might happen at the moment, but
> plausibly there might be circumstances under which the PIX and the
> destination Windows box "raced" to reply; if the destination Windows
> box happened to reply first, then all would be well, but if the PIX
> packet happened to arrive first, then because the PIX doesn't allow
> that kind of traversal, the PIX packet would be an ICMP unreachable
> or TCP RST packet packet; the latter especially would interfere with
> the bypass communications. Furthermore, if the PIX packet replied using
> the target IP address as the source, then the MAC of the PIX might
> get written into the ARP tables as the MAC for that IP. If that happened,
> then the next packets would be sent to the PIX, which would refuse them.
> Possibly the IP stack would interpret the ICMP refusal as indicating
> that the ARP entry should be flushed; if so, then the time unreachable
> would be fairly short, but if the PIX's MAC stayed in the table,
> the blip without communications would last until the ARP entry timed
> out, and then it would be off to the races again.
>
> I wouldn't want to have to replicate this hypothesis in a lab ;-)
> And it's a not a useful hypothesis if the inside users always
> access the internal servers through the internal IP.
> --
> And the wind keeps blowing the angel / Backwards into the future /
> And this wind, this wind / Is called / Progress.
> -- Laurie Anderson

Re: PIX 506e Connection Problems
 

In article <gfCdnWOQGZkC-f7cRVn-pQ@rogers.com>,
John Balch <john@jmbalch.com> wrote:
:Thanks for your reply, Walter. But, the internal users are accessing the
:internal systems via their internal IP addresses - not the external IP
:addresses.

Had to check -- the pix-won't-route-back problem catches a -lot- of
users.


:One peculiarity about our configuration is that we use the 128.1.x.x network
:internally. I know this is not a standard internal subnet but it was set up
:long before I got on board by someone who didn't understand that. I know
:someone else has the 128.1.x.x address somewhere out there on the Internet.

Looks like 128.1/16 is allocated to BBN.

:Could that be causing this complication?

No, it wouldn't cause the service problems you are seeing -- you just
wouldn't be able to reach hosts on 128.1/16 .

: I have the 128.1.x.x defined as
:being "internal" and the internal interface on the PIX is 128.1.0.1. I
:found some documentation talking about using the "alias" command if you have
:internal IP addresses that may overlap with external... but, when I tried
:that, the PDM really squawked about it being an obsolete command.

Yeah, PDM does not handle 'alias', which really has two or three different
meanings. The equivilent is "reverse nat", either through a nat/global
pair against the opposite interfaces of normal, or through a 'static'
command with the interface order reversed from normal. You would have
to assign an IP address range to "stand in" for the -real- 128.1/16,
such as 10.1/16, and then set it up so that traffic sent to 10.1.x.x
was redirected to 128.1.x.x. It's a bit of a nuisance but it works
if set up correctly. It isn't something you want to volunteer to configure
just for fun, though.


:The strange thing is that it seems to work most of the time... when these
:failures start to appear, it is typically in mid-afternoon when network
:traffic is at its peak. When the failures start, if I remove the static
:mapping for the affected servers, the problem immediately goes away and
:stays away until I put the static mappings back in.

What happens if, instead, you clear xlate ? Or reboot the 506E?
The next time the problem happens, try showing the cpu load and memory
and see if you are running out of steam on the 506e. That's
"show cpu usage" and "show memory" or "show memory detail".
--
I wrote a hack in microcode,
with a goto on each line,
it runs as fast as Superman,
but not quite every time! -- Don Libes et al.

Re: PIX 506e Connection Problems
 

John Balch wrote:
> Hello,
>
> My client has a PIX 506e configured as their primary firewall. They have 4
> external IP addresses that are statically mapped to 4 internal servers - one
> for SMTP, HTTP, HTTPS, and FTP. Access from the outside world to these
> servers works fine. However, what we are finding is that the static
> mappings are interfering with internal access to these servers. They
> "vanish" momentarily but then reappear. These connection blips happen
> continuously but go away if I break the static mapping on the router.
>
> All inside systems are on one subnet and use the PIX as their default
> gateway to the outside world. But, inside traffic shouldn't be affected by
> the PIX should it? That's what appears to be happening.
>
> Has anyone seen this before? I'm guessing I have something configured
> improperly.
>
> Thanks in advance for your help.
>
>

Could it be an ARP issue caused by alias or outside nat? If the hosts
are on the same subnet, then that seems to be the only thing neccessary
for the hosts to communicate. When internal users loose connectivity
to the servers, check the ARP table on the host to see if it has the
correct MAC address for the server ("arp -a" on most OS's). If it does
not, find out what MAC address it does have and identify why that device
is proxy arp'ing for the host(s).

Rik Bain

Re: PIX 506e Connection Problems
 

Thanks... I'll try that.

"Rik Bain" <rbain@nospam.bainz.org> wrote in message
news:41647670$0$55132$ec3e2dad@news.usenetmonster. com...
> John Balch wrote:
>> Hello,
>>
>> My client has a PIX 506e configured as their primary firewall. They have
>> 4 external IP addresses that are statically mapped to 4 internal
>> servers - one for SMTP, HTTP, HTTPS, and FTP. Access from the outside
>> world to these servers works fine. However, what we are finding is that
>> the static mappings are interfering with internal access to these
>> servers. They "vanish" momentarily but then reappear. These connection
>> blips happen continuously but go away if I break the static mapping on
>> the router.
>>
>> All inside systems are on one subnet and use the PIX as their default
>> gateway to the outside world. But, inside traffic shouldn't be affected
>> by the PIX should it? That's what appears to be happening.
>>
>> Has anyone seen this before? I'm guessing I have something configured
>> improperly.
>>
>> Thanks in advance for your help.
>
> Could it be an ARP issue caused by alias or outside nat? If the hosts are
> on the same subnet, then that seems to be the only thing neccessary for
> the hosts to communicate. When internal users loose connectivity to the
> servers, check the ARP table on the host to see if it has the correct MAC
> address for the server ("arp -a" on most OS's). If it does not, find out
> what MAC address it does have and identify why that device is proxy
> arp'ing for the host(s).
>
> Rik Bain

Re: PIX 506e Connection Problems
 

Another strange thing.... if I change the static commands to include the
port specifications... the problem goes away. So, I've done that for now
just to get around that problem and, so far, it got through today without
the problem appearing (and that's a first).

Thanks - I'll try your suggestions next time it happens. I appreciate your
suggestions... trying to around this problem has been a very humbling
experience!


"Walter Roberson" <roberson@ibd.nrc-cnrc.gc.ca> wrote in message
news:ck16f3$ilr$1@canopus.cc.umanitoba.ca...
> In article <gfCdnWOQGZkC-f7cRVn-pQ@rogers.com>,
> John Balch <john@jmbalch.com> wrote:
> :Thanks for your reply, Walter. But, the internal users are accessing
> the
> :internal systems via their internal IP addresses - not the external IP
> :addresses.
>
> Had to check -- the pix-won't-route-back problem catches a -lot- of
> users.
>
>
> :One peculiarity about our configuration is that we use the 128.1.x.x
> network
> :internally. I know this is not a standard internal subnet but it was set
> up
> :long before I got on board by someone who didn't understand that. I
> know
> :someone else has the 128.1.x.x address somewhere out there on the
> Internet.
>
> Looks like 128.1/16 is allocated to BBN.
>
> :Could that be causing this complication?
>
> No, it wouldn't cause the service problems you are seeing -- you just
> wouldn't be able to reach hosts on 128.1/16 .
>
> : I have the 128.1.x.x defined as
> :being "internal" and the internal interface on the PIX is 128.1.0.1. I
> :found some documentation talking about using the "alias" command if you
> have
> :internal IP addresses that may overlap with external... but, when I tried
> :that, the PDM really squawked about it being an obsolete command.
>
> Yeah, PDM does not handle 'alias', which really has two or three different
> meanings. The equivilent is "reverse nat", either through a nat/global
> pair against the opposite interfaces of normal, or through a 'static'
> command with the interface order reversed from normal. You would have
> to assign an IP address range to "stand in" for the -real- 128.1/16,
> such as 10.1/16, and then set it up so that traffic sent to 10.1.x.x
> was redirected to 128.1.x.x. It's a bit of a nuisance but it works
> if set up correctly. It isn't something you want to volunteer to configure
> just for fun, though.
>
>
> :The strange thing is that it seems to work most of the time... when these
> :failures start to appear, it is typically in mid-afternoon when network
> :traffic is at its peak. When the failures start, if I remove the static
> :mapping for the affected servers, the problem immediately goes away and
> :stays away until I put the static mappings back in.
>
> What happens if, instead, you clear xlate ? Or reboot the 506E?
> The next time the problem happens, try showing the cpu load and memory
> and see if you are running out of steam on the 506e. That's
> "show cpu usage" and "show memory" or "show memory detail".
> --
> I wrote a hack in microcode,
> with a goto on each line,
> it runs as fast as Superman,
> but not quite every time! -- Don Libes et al.