Xtra spam filter -- false positives.

Discussion in 'NZ Computing' started by Ralph Fox, Jul 2, 2004.

  1. Ralph Fox

    Ralph Fox Guest

    After 195 days and an estimated 9000 spams, I have just
    had my first false positive from Xtra's spam filter.

    Not bad going, although it does suggest that the rate of
    false positives might possibly be higher than the
    1 in 1,000,000 quoted on Xtra's website:
    http://www.xtra.co.nz/products/0,,11195,00.html#cost3


    --
    Cheers,
    Ralph

    Change is inevitable. Progress is optional.
     
    Ralph Fox, Jul 2, 2004
    #1
    1. Advertising

  2. Ralph Fox

    ~misfit~ Guest

    Ralph Fox wrote:
    > After 195 days and an estimated 9000 spams, I have just
    > had my first false positive from Xtra's spam filter.
    >
    > Not bad going, although it does suggest that the rate of
    > false positives might possibly be higher than the
    > 1 in 1,000,000 quoted on Xtra's website:
    > http://www.xtra.co.nz/products/0,,11195,00.html#cost3


    Statistically it doesn't suggest that at all. Your representative sample
    isn't big enough. You may have just got your 'one in a million', the next
    991,000 could all be fine.
    --
    ~misfit~
     
    ~misfit~, Jul 3, 2004
    #2
    1. Advertising

  3. Ralph Fox

    Howard Guest

    ~misfit~ wrote:
    > Ralph Fox wrote:
    >> After 195 days and an estimated 9000 spams, I have just
    >> had my first false positive from Xtra's spam filter.
    >>
    >> Not bad going, although it does suggest that the rate of
    >> false positives might possibly be higher than the
    >> 1 in 1,000,000 quoted on Xtra's website:
    >> http://www.xtra.co.nz/products/0,,11195,00.html#cost3

    >
    > Statistically it doesn't suggest that at all. Your representative
    > sample isn't big enough. You may have just got your 'one in a
    > million', the next 991,000 could all be fine.


    I'm not a good enough statistician to be able to to supply the confidence
    level at which one can say "the false positives in Xtra's emails filtering
    is worse than 1 in 1 milllion" but a finding of 1 false postive in 9000
    certainly does suggest the statement is more likely true than false. I think
    I'll ask my statistician friend precisely what the confidence level is, and
    report back.

    But anyway, Xtra/Brightmail does have very good filtering compared to Orcon.
    With Xtra I typically had 3 or 4 false negatives per day (out of approx 200
    spams per day) and no observed false positives. With Orcon I had no better
    than 50% filtered as SPAM, and I had a false postive at least once per day.
    Orcon's was worse than useless, and I have fallen back to Mozilla's
    filtering and am looking at setting up SpamAssasin.
     
    Howard, Jul 3, 2004
    #3
  4. Ralph Fox

    Brett Cooper Guest

    On Sat, 3 Jul 2004 15:02:33 +1200, "Howard" <>
    wrote:

    >~misfit~ wrote:
    >> Ralph Fox wrote:
    >>> After 195 days and an estimated 9000 spams, I have just
    >>> had my first false positive from Xtra's spam filter.
    >>>
    >>> Not bad going, although it does suggest that the rate of
    >>> false positives might possibly be higher than the
    >>> 1 in 1,000,000 quoted on Xtra's website:
    >>> http://www.xtra.co.nz/products/0,,11195,00.html#cost3

    >>
    >> Statistically it doesn't suggest that at all. Your representative
    >> sample isn't big enough. You may have just got your 'one in a
    >> million', the next 991,000 could all be fine.

    >
    >I'm not a good enough statistician to be able to to supply the confidence
    >level at which one can say "the false positives in Xtra's emails filtering
    >is worse than 1 in 1 milllion" but a finding of 1 false postive in 9000
    >certainly does suggest the statement is more likely true than false. I think
    >I'll ask my statistician friend precisely what the confidence level is, and
    >report back.
    >
    >But anyway, Xtra/Brightmail does have very good filtering compared to Orcon.
    >With Xtra I typically had 3 or 4 false negatives per day (out of approx 200
    >spams per day) and no observed false positives. With Orcon I had no better
    >than 50% filtered as SPAM, and I had a false postive at least once per day.
    >Orcon's was worse than useless, and I have fallen back to Mozilla's
    >filtering and am looking at setting up SpamAssasin.
    >


    With misfits account of 1 in a million being spread of the whole customer
    base of xtra, would mean is xtra has 100,000 subscribers
    then 1 in ten emails would be a false positive.

    Brett
     
    Brett Cooper, Jul 3, 2004
    #4
  5. Ralph Fox

    ~misfit~ Guest

    Brett Cooper wrote:
    > On Sat, 3 Jul 2004 15:02:33 +1200, "Howard" <>
    > wrote:
    >
    >> ~misfit~ wrote:
    >>> Ralph Fox wrote:
    >>>> After 195 days and an estimated 9000 spams, I have just
    >>>> had my first false positive from Xtra's spam filter.
    >>>>
    >>>> Not bad going, although it does suggest that the rate of
    >>>> false positives might possibly be higher than the
    >>>> 1 in 1,000,000 quoted on Xtra's website:
    >>>> http://www.xtra.co.nz/products/0,,11195,00.html#cost3
    >>>
    >>> Statistically it doesn't suggest that at all. Your representative
    >>> sample isn't big enough. You may have just got your 'one in a
    >>> million', the next 991,000 could all be fine.

    >>
    >> I'm not a good enough statistician to be able to to supply the
    >> confidence level at which one can say "the false positives in Xtra's
    >> emails filtering is worse than 1 in 1 milllion" but a finding of 1
    >> false postive in 9000 certainly does suggest the statement is more
    >> likely true than false. I think I'll ask my statistician friend
    >> precisely what the confidence level is, and report back.
    >>
    >> But anyway, Xtra/Brightmail does have very good filtering compared
    >> to Orcon. With Xtra I typically had 3 or 4 false negatives per day
    >> (out of approx 200 spams per day) and no observed false positives.
    >> With Orcon I had no better than 50% filtered as SPAM, and I had a
    >> false postive at least once per day. Orcon's was worse than useless,
    >> and I have fallen back to Mozilla's filtering and am looking at
    >> setting up SpamAssasin.
    >>

    >
    > With misfits account of 1 in a million being spread of the whole
    > customer base of xtra, would mean is xtra has 100,000 subscribers
    > then 1 in ten emails would be a false positive.


    Jus to clarify, it's not *my* 1 in a million, it's Xtra's:

    http://www.xtra.co.nz/products/0,,11195,00.html#cost3
    --
    ~misfit~
     
    ~misfit~, Jul 3, 2004
    #5
  6. Ralph Fox

    Brett Cooper Guest

    On Sat, 3 Jul 2004 19:10:50 +1200, "~misfit~" <>
    wrote:

    >Brett Cooper wrote:
    >> On Sat, 3 Jul 2004 15:02:33 +1200, "Howard" <>
    >> wrote:
    >>
    >>> ~misfit~ wrote:
    >>>> Ralph Fox wrote:
    >>>>> After 195 days and an estimated 9000 spams, I have just
    >>>>> had my first false positive from Xtra's spam filter.
    >>>>>
    >>>>> Not bad going, although it does suggest that the rate of
    >>>>> false positives might possibly be higher than the
    >>>>> 1 in 1,000,000 quoted on Xtra's website:
    >>>>> http://www.xtra.co.nz/products/0,,11195,00.html#cost3
    >>>>
    >>>> Statistically it doesn't suggest that at all. Your representative
    >>>> sample isn't big enough. You may have just got your 'one in a
    >>>> million', the next 991,000 could all be fine.
    >>>
    >>> I'm not a good enough statistician to be able to to supply the
    >>> confidence level at which one can say "the false positives in Xtra's
    >>> emails filtering is worse than 1 in 1 milllion" but a finding of 1
    >>> false postive in 9000 certainly does suggest the statement is more
    >>> likely true than false. I think I'll ask my statistician friend
    >>> precisely what the confidence level is, and report back.
    >>>
    >>> But anyway, Xtra/Brightmail does have very good filtering compared
    >>> to Orcon. With Xtra I typically had 3 or 4 false negatives per day
    >>> (out of approx 200 spams per day) and no observed false positives.
    >>> With Orcon I had no better than 50% filtered as SPAM, and I had a
    >>> false postive at least once per day. Orcon's was worse than useless,
    >>> and I have fallen back to Mozilla's filtering and am looking at
    >>> setting up SpamAssasin.
    >>>

    >>
    >> With misfits account of 1 in a million being spread of the whole
    >> customer base of xtra, would mean is xtra has 100,000 subscribers
    >> then 1 in ten emails would be a false positive.

    >
    >Jus to clarify, it's not *my* 1 in a million, it's Xtra's:
    >
    >http://www.xtra.co.nz/products/0,,11195,00.html#cost3


    What is a false positive?
    A false positive is when we mark up an email as being "[SPAM]" when it in
    fact is not. BrightMail claim that this will only occur with 1 in 1,000,000
    email messages.

    So it was brightmail making the claim. Maybe its out of date as spam
    grows.
     
    Brett Cooper, Jul 3, 2004
    #6
  7. Ralph Fox

    Tim Guest

    Try emailing an exe - it will reject 100% of programs that are not spam =
    100% failure rate in my books & none of these are false postives I suppose,
    they would be deliberate postitives.

    "Ralph Fox" <-echo.invalid> wrote in
    message news:...
    >
    > After 195 days and an estimated 9000 spams, I have just
    > had my first false positive from Xtra's spam filter.
    >
    > Not bad going, although it does suggest that the rate of
    > false positives might possibly be higher than the
    > 1 in 1,000,000 quoted on Xtra's website:
    > http://www.xtra.co.nz/products/0,,11195,00.html#cost3
    >
    >
    > --
    > Cheers,
    > Ralph
    >
    > Change is inevitable. Progress is optional.
    >
     
    Tim, Jul 3, 2004
    #7
  8. Ralph Fox

    Ralph Fox Guest

    On Sat, 3 Jul 2004 22:47:23 +1200, in message
    <cc62q7$74h$>, Tim wrote:

    > Try emailing an exe - it will reject 100% of programs that are not spam =
    > 100% failure rate in my books & none of these are false postives I suppose,
    > they would be deliberate postitives.


    Just tried an exe to my Xtra mailbox, and no rejection. This matches
    previous experience.

    The only blanket exe rejections I have seen are when sending to
    the work email of someone whose work runs MailMarshall.


    --
    Cheers,
    Ralph

    Change is inevitable. Progress is optional.
     
    Ralph Fox, Jul 3, 2004
    #8
  9. Ralph Fox wrote:

    > After 195 days and an estimated 9000 spams, I have just
    > had my first false positive from Xtra's spam filter.
    >
    > Not bad going, although it does suggest that the rate of
    > false positives might possibly be higher than the
    > 1 in 1,000,000 quoted on Xtra's website:
    > http://www.xtra.co.nz/products/0,,11195,00.html#cost3


    not necessarily... I mean if there are 1100 other users who have had the
    same number of spam as you, and no false positives, then it all works out.

    also, there is nothing to say that you wont get 2 false positives in a
    row... throw a dice and watch as the stats queue until you get a large
    enough sample.


    gah, my stats teacher was right, I have used it since high school.

    This is the same stats teacher who marked me down for including in a
    coin flipping test, the one time it landed on it's edge, he asked me to
    remove it as it doesnt happen, I pointed out that it had... eventually I
    and two friends who were in my group got sent to the rector... anal
    stats teacher he was.
     
    Dave - Dave.net.nz, Jul 4, 2004
    #9
  10. Ralph Fox wrote:
    > The only blanket exe rejections I have seen are when sending to
    > the work email of someone whose work runs MailMarshall.


    heh, add passworded zip files to that aswell.
    It says that they are virus's.
     
    Dave - Dave.net.nz, Jul 4, 2004
    #10
  11. Ralph Fox

    Ralph Fox Guest

    On Mon, 05 Jul 2004 10:45:51 +1200, in message
    <>, Dave - Dave.net.nz wrote:

    > Ralph Fox wrote:
    > > The only blanket exe rejections I have seen are when sending to
    > > the work email of someone whose work runs MailMarshall.

    >
    > heh, add passworded zip files to that aswell.
    > It says that they are virus's.



    I once emailed someone a corrupted zip file which
    they had asked for (in the hope of recovering it).

    The corrupted zip file didn't make it past MailMarshall.


    --
    Cheers,
    Ralph

    Change is inevitable. Progress is optional.
     
    Ralph Fox, Jul 5, 2004
    #11
  12. Ralph Fox

    Ralph Fox Guest

    On Mon, 05 Jul 2004 09:59:09 +1200, in message
    <>, Dave - Dave.net.nz wrote:

    > not necessarily... I mean if there are 1100 other users who have had the
    > same number of spam as you, and no false positives, then it all works out.



    Did you mean 1,000,000 ÷ 9000 = 111 other users who have had the same
    number of spam as me?


    For the stats to work out (to the claimed 1 false positive in 1,000,000
    genuine emails), what we would need is considerably more other users who
    have had the same number of _genuine_ _emails_ as me, and no false
    positives.


    --
    Cheers,
    Ralph

    Change is inevitable. Progress is optional.
     
    Ralph Fox, Jul 5, 2004
    #12
  13. Ralph Fox

    Brett Cooper Guest

    On Mon, 05 Jul 2004 10:35:34 +0000, Ralph Fox
    <-echo.invalid> wrote:

    >On Mon, 05 Jul 2004 10:45:51 +1200, in message
    ><>, Dave - Dave.net.nz wrote:
    >
    >> Ralph Fox wrote:
    >> > The only blanket exe rejections I have seen are when sending to
    >> > the work email of someone whose work runs MailMarshall.

    >>
    >> heh, add passworded zip files to that aswell.
    >> It says that they are virus's.

    >
    >
    >I once emailed someone a corrupted zip file which
    >they had asked for (in the hope of recovering it).
    >
    >The corrupted zip file didn't make it past MailMarshall.


    Sounds as bad as gmail, if MailMarshall can't read the contence of the zip
    than it will not deal with it.

    Brett
     
    Brett Cooper, Jul 5, 2004
    #13
  14. Ralph Fox wrote:
    >>heh, add passworded zip files to that aswell.
    >>It says that they are virus's.


    > I once emailed someone a corrupted zip file which
    > they had asked for (in the hope of recovering it).
    > The corrupted zip file didn't make it past MailMarshall.


    i assume that it is because MM can't open it, so marks it as "unknown"
    or something.
     
    Dave - Dave.net.nz, Jul 5, 2004
    #14
  15. Ralph Fox wrote:
    >>not necessarily... I mean if there are 1100 other users who have had the
    >>same number of spam as you, and no false positives, then it all works out.


    > Did you mean 1,000,000 ÷ 9000 = 111 other users who have had the same
    > number of spam as me?


    errr, yeah, Im home sick at the moment, so brain isn't quite right.

    > For the stats to work out (to the claimed 1 false positive in 1,000,000
    > genuine emails), what we would need is considerably more other users who
    > have had the same number of _genuine_ _emails_ as me, and no false
    > positives.


    yeah, see above. :)
     
    Dave - Dave.net.nz, Jul 5, 2004
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ellis_jay

    Re: avg false positives

    ellis_jay, Aug 11, 2005, in forum: Computer Support
    Replies:
    0
    Views:
    480
    ellis_jay
    Aug 11, 2005
  2. null

    Spy Sweeper 4.5 - False Positives

    null, Nov 8, 2005, in forum: Computer Security
    Replies:
    1
    Views:
    657
    Moe Trin
    Nov 8, 2005
  3. Nick

    False positive, false intrusion, false alarm

    Nick, Apr 23, 2006, in forum: Computer Security
    Replies:
    3
    Views:
    3,989
    Moe Trin
    Apr 26, 2006
  4. Heidi Manway

    Potential Software Conflicts and False Positives?

    Heidi Manway, Jan 19, 2007, in forum: Computer Support
    Replies:
    3
    Views:
    425
    Edwardo
    Jan 19, 2007
  5. Peter S

    Re: Spyware False Positives and idiots

    Peter S, Jan 9, 2005, in forum: NZ Computing
    Replies:
    3
    Views:
    341
    Peter Huebner
    Jan 29, 2005
Loading...

Share This Page