spam statistics. Commercial software Sophos v Spam Assassin.

Our old system is Spam Assassin, new is Sophos

A sample of tagging correctly over the last 8 days,

Sophos
54%
61%
61%
75%
81%
61%
61%
62%
515%
64% - Average

Spam A
77%
78%
75%
79%
78%
75%
74%
64%
600%
75% - Average

I am sort of perplexed, Sophos is updated every hour automatically, Spam
Assassin 2 or 3 times a week manually....

Ever since we put the new system in Spam Assassin seems to be
consistantly beating it (Sophos)......yet Sophos is costing us an arm
and a leg.....

Is there anybody else who has experienced this sort of thing? Is there
anything that could be skewing the statistics unfairly?

regards

Thing

 

What platform are you using? Sendmail?

Steve

 

Your statistics could be misleading you since you appear to be averaging a
percentage. Return to the base data and draw the end percentage directly
from that. I would show the message counts and 'hits' followed by %, then
sum the counts and hits then show then newly calculated %.

The top list may have had 81% of a very large volume on one day, but because
the volume does not come into the end result its significance may be lost
statistically. It could just as easily be a very quiet day and the products
are as you observe...

- Tim

 

I've seen that(paid for product sucks) here at work where mailmarshal is
crap at finding spam until you wind it up to "Im not going to let
anything through" mode, at which point it catches too much legit mail to
be useful, and so it remains letting a fair bitthrough, although getting
less as I refine the filters.

I havent used it(SA) in a while... does it learn yet?
maybe it's trained itself to be better.

 

Sendmail for Sophos, postfix for Spam Assassin.

regards

Thing

 

Just for info
8300 blocked spam
250 blocked virus
12000 legit
~2800 users
so ~3 legit:2 spam

crap... 40% thats low compared with what I've read elsewhere.

 

If your sendmail is on linux or solaris, you might want to try this Kiwi
product, rather than that british stuff. What do they know about *our*
spam, eh?

http://sourceforge.net/projects/mailwasher/

(somewhat cheaper, too!)

Steve

 

Yes, Linux.

I think we have had this discussion........

1) I have been reading up on mailwasher, I notice the comments on the
performance hit on the server, it is noticable, Sophos also has this
penalty, so on the face of it, no advantage either way.

2) The mail washer's site lacks any depth on explaining the product and
its components or if it is there I could not find it yesterday.

3) Like Sophos I see no evidence it is robust and scalable.

4) Maturity, Sophos's system has been around a while.....mail washer is
how old?

I am trying to build up a case to dump Sophos, I would be shot if I
installed something else as bad or worse....and I like my job....

I also am getting very tired of being some company's software guinee
pig......and getting a whipping as a result when it goes wrong.....

regards

Thing

 

this any help?
http://oss.firetrust.com/home/specs.php

 

50-60% correctly found spam? That seems VERY VERY low (even for
spamassassin). Spam Assassin should give normally out of the box about 95%.

Other products like the open source dspam
(http://www.nuclearelephant.com/projects/dspam/) will give 99.99%
correctness.

Thanks
Craig

 

I think individual users can set it up to do Bayesian learning on their
own e-mail. In its default, systemwide configuration, it doesn't do
this. At least, not the way I have it set up.

 

Maybe a similar principle applies with this as with cryptography: the
open, public approach beats the closed, secretive one simply because the
former can bring a wider range of minds to bear to find weaknesses and
plug them, leading to a stronger product.