Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Computing > Cisco > PIX 515E CPU util at 98% with a compromised system on inside network

Reply
Thread Tools

PIX 515E CPU util at 98% with a compromised system on inside network

 
 
shahidsheikh....com
Guest
Posts: n/a
 
      12-30-2005
Hi all,

I am looking for recommendation on how to prevent a repeat of a recent
episode on one of the networks I support.

Pix 515E running 6.3(3) unrestricted. 3 inside networks. DMZ1, DMZ2,
and Inside. All inside networks use a 172.16.x.x ip address space. The
physical medium for all inside networks is switched 100 Mbps ethernet.
Outside connects to a dual bonded T1 via a cisco 2600.

Inside has the highest security level. Then DMZ1 and DMZ2.

An old linux web server on DMZ2 not under our direct control (and
probably never will be because of political reasons) gets compromised
thru a php hack and causes a syn flood. That results in the Pix with
98% CPU util and it stops responding to requests from other inside
networks resulting in a DoS to the other inside networks. Packets were
being dumped at the DMZ2 interface at 70~80 Mbps.

The embryonic connection limit feature doesn't help here because it
cannot be applied for outside NAT. And even if it did work it wouldn't
help in this case because the embryonic connection limit does nothing
to protect the Pix against high CPU util.

What are my options if all I have is this one Pix to work with. Force
the DMZ2 interface to 10baseT. That may help with the CPU utilization
but what about my 3 Mbps WAN pipe? Any other way to do rate limiting?
Is PIX os 7 the answer?

Thanks,

Shahid

 
Reply With Quote
 
 
 
 
Mark Lar
Guest
Posts: n/a
 
      12-30-2005
shahidsheikh....com wrote:

> I am looking for recommendation on how to prevent a repeat of a recent
> episode on one of the networks I support.

<snip tale of woe>
Simple... turn the Linux box off.

Seriously, if you've got a known vulnerable web server that nobody has
the guts to fix, it's time to either turn it off or pack up and ****
off, because you aren't being allowed to do your job.
 
Reply With Quote
 
 
 
 
Martin Bilgrav
Guest
Posts: n/a
 
      12-30-2005

"shahidsheikh....com" <(E-Mail Removed)> wrote in message
news:(E-Mail Removed) oups.com...
> Hi all,
>
> I am looking for recommendation on how to prevent a repeat of a recent
> episode on one of the networks I support.
>
> Pix 515E running 6.3(3) unrestricted. 3 inside networks. DMZ1, DMZ2,
> and Inside. All inside networks use a 172.16.x.x ip address space. The
> physical medium for all inside networks is switched 100 Mbps ethernet.
> Outside connects to a dual bonded T1 via a cisco 2600.
>
> Inside has the highest security level. Then DMZ1 and DMZ2.
>
> An old linux web server on DMZ2 not under our direct control (and
> probably never will be because of political reasons) gets compromised
> thru a php hack and causes a syn flood. That results in the Pix with
> 98% CPU util and it stops responding to requests from other inside
> networks resulting in a DoS to the other inside networks. Packets were
> being dumped at the DMZ2 interface at 70~80 Mbps.
>
> The embryonic connection limit feature doesn't help here because it
> cannot be applied for outside NAT. And even if it did work it wouldn't
> help in this case because the embryonic connection limit does nothing
> to protect the Pix against high CPU util.
>
> What are my options if all I have is this one Pix to work with. Force
> the DMZ2 interface to 10baseT. That may help with the CPU utilization
> but what about my 3 Mbps WAN pipe? Any other way to do rate limiting?
> Is PIX os 7 the answer?



could it be that log-rates are killing you ?
If so apply log rate limit

And in general terms: Kill the server - it's compromised - You can not risk
getting your own IP's getting abuse repported on the outside world, just
because you have a shitty sys-admin
What would you think your Management will say, if they got word that the
company name was reportted together with hacking abuses ??

HTH'
Martin



>
> Thanks,
>
> Shahid
>



 
Reply With Quote
 
slim
Guest
Posts: n/a
 
      12-30-2005
Martin Bilgrav wrote:
> "shahidsheikh....com" <(E-Mail Removed)> wrote in message
> news:(E-Mail Removed) oups.com...
>
>>Hi all,
>>
>>I am looking for recommendation on how to prevent a repeat of a recent
>>episode on one of the networks I support.
>>
>>Pix 515E running 6.3(3) unrestricted. 3 inside networks. DMZ1, DMZ2,
>>and Inside. All inside networks use a 172.16.x.x ip address space. The
>>physical medium for all inside networks is switched 100 Mbps ethernet.
>>Outside connects to a dual bonded T1 via a cisco 2600.
>>
>>Inside has the highest security level. Then DMZ1 and DMZ2.
>>
>>An old linux web server on DMZ2 not under our direct control (and
>>probably never will be because of political reasons) gets compromised
>>thru a php hack and causes a syn flood. That results in the Pix with
>>98% CPU util and it stops responding to requests from other inside
>>networks resulting in a DoS to the other inside networks. Packets were
>>being dumped at the DMZ2 interface at 70~80 Mbps.
>>
>>The embryonic connection limit feature doesn't help here because it
>>cannot be applied for outside NAT. And even if it did work it wouldn't
>>help in this case because the embryonic connection limit does nothing
>>to protect the Pix against high CPU util.
>>
>>What are my options if all I have is this one Pix to work with. Force
>>the DMZ2 interface to 10baseT. That may help with the CPU utilization
>>but what about my 3 Mbps WAN pipe? Any other way to do rate limiting?
>>Is PIX os 7 the answer?

>
>
>
> could it be that log-rates are killing you ?
> If so apply log rate limit
>
> And in general terms: Kill the server - it's compromised - You can not risk
> getting your own IP's getting abuse repported on the outside world, just
> because you have a shitty sys-admin
> What would you think your Management will say, if they got word that the
> company name was reportted together with hacking abuses ??
>
> HTH'
> Martin
>
>
>
>
>>Thanks,
>>
>>Shahid
>>

>
>
>


Another thought might be that you're exhausting your NAT translation
entry limit. I had a host get infected with a worm and because it was
spawning so many connections so fast, the logs indicated that additional
NAT flows couldn't be built. This killed all legitimate use.

If Walter Roberson chimes in, I'd be interested to hear his take on your
and my experience. I don't know if there's much that can be done in this
scenario. I would have thought that the PIX OS would provide some sort
of SYN limiting that would prevent this from happening, so I'm wondering
if I missed a nerd-knob to tweak.
 
Reply With Quote
 
shahidsheikh....com
Guest
Posts: n/a
 
      12-30-2005
I thought about Logging. Disabled it completely and still get the same
results when I blast one of the inside interfaces with a syn flood.

As for the NAT limit being exhasted, in my case the three way handshake
is never complete. The IRC server out on the internet that the exploit
is trying to get to doesn't exist. So each connection is still in
embryonic state. I don't think defining max connections on the static
for these servers (I don't have control over) will help but I haven't
tried it.

I'm on the same page as you are looking for a way to limit/drop syn
packets if they come at a pace faster than a defined threshold.

Thanks,

Shahid

 
Reply With Quote
 
farmerc farmerc is offline
Junior Member
Join Date: Nov 2006
Posts: 2
 
      09-21-2007
You might also want to check that there are no Debugs running on the box...

I know its not what your looking at currently but ive seen worse.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Network Compromised? =?Utf-8?B?aW5jb3Bhc3M=?= Wireless Networking 5 11-22-2006 08:31 PM
Wireless Network Compromised marcavaj General Computer Support 0 11-13-2006 04:05 PM
Cisco System PIX 515E - Memory and PIX OS upgrade Speed3ple Cisco 0 04-04-2006 10:24 AM
Allow printing traffic from DMZ(Lower Security interface) to inside network on PIX 515E jywu1@hotmail.com Cisco 4 12-15-2005 11:27 AM
MAKING YOUR COMPUTER SYSTEM SECURE AFTER IT'S BEEN COMPROMISED Jene Keller Computer Security 4 11-20-2003 10:37 PM



Advertisments