dodgy deals between IT mags and anti-virus commercial companies?

Discussion in 'NZ Computing' started by thingy, Jan 30, 2008.

  1. thingy

    thingy Guest

    I have seen some claims in here over which piece of anti-virus softwrae
    was the best...and several poo poos over open source ones....ClamAV was
    one if I recall correctly....

    Anyway while digging into the Barracuda v trend micro law suit, it
    revealed that Barracuda (which is a well regarded "black-box"
    anti-virus/spam gateway) uses clam-av (cant be that bad then can
    it...).....reading further in and I came across this,

    http://www.clamav.org/2007/08/09/untangle-tests-antivirus-tools-in-linuxworld-fight-club/

    "The “winners” in last night’s contest were ClamAV, Kaspersky and
    Symantec. All three tools caught 100 percent of the viruses they
    encountered."

    So good for ClamAV....next is the query on why ClamAV gets so little
    press....I wonder if it is to do with lack of potential advertising
    income or money that has mags/labs dis-ing or ignoring Clam-AV?

    "...They [The test labs] were eager to work with us for firewall and VPN
    testing, yet refused to test any antivirus functionality because we were
    using “the open source solution.” They would not explain why they
    refused to test ClamAV, although they did offer that they had tested it
    and it had done poorly relative to the commercial solutions. Given our
    testing and customer data had shown the exact opposite, we asked for
    their test results – to which they repeatedly refused. They also
    repeatedly refused to provide the test data set so we attempt to verify
    any results they had shown in their labs. I’m left to assume that the
    testing labs are biased in their testing, probably because they get
    their funding from the commercial vendors that pay them for testing."

    Now this is by "Untangle" who have open source "products" so maybe they
    are a little biased....but their tests are claimed to be open and
    verifiable....

    So who do you trust? I know I have used ClamAV on my sendmail gateway
    for 2+ years and I have yet to see a virus get through....and Barracuda
    use it....so I have to wonder...if it was no good, it would not get used...

    Intersting (also) is that Kaspersky seems to do consistently well....so
    if I was paying I think this would be where I'd be looking.

    Instead of course I use ClamAV backed up by Avast...at zero cost.

    regards

    Thing
     
    thingy, Jan 30, 2008
    #1
    1. Advertisements

  2. thingy

    thingy Guest


    "As always, we are surprised by how poor many of these solutions are
    performing. Contrary to many statements, Clam is a top performer, and
    also ran 10 times faster than many solutions. Kaspersky is clearly an
    excellent engine, and Norton also performed well although it consumed
    lots of resources on the test machine. The rest of the solutions, some
    of which are quite expensive, were mediocre to terrible."

    http://blog.untangle.com/?p=96

    regards

    Thing
     
    thingy, Jan 30, 2008
    #2
    1. Advertisements

  3. thingy

    ~misfit~ Guest

    Somewhere on teh intarweb "thingy" typed:
    Interesting stuff, thanks Thing.

    I've known for a long time that a lot of "tests" and "reviews" available on
    teh intarweb are seriously biased. However, my experience with the phenomena
    was mainly with hardware reviews. Interesting to see that it has pervaded
    all "independant" tests.

    It was going to happen. I know that there are millions of consumers who rely
    on web-based 'reviews' to make their purchasing decisions. What better place
    for big corporations to direct their advertising budget? I only use
    'reviews' as rough guides these days and, if I need an opinion as to
    efficacy of a product, I rely on good old usenet and peer reviews.

    Cheers,
     
    ~misfit~, Jan 30, 2008
    #3
  4. thingy

    impossible Guest

    I'm suprised you're giving this review any more credence than others that
    are conducted by interested parties. Untangle is "maybe a little biased"?
    I'd say so. They sell proprietary security products derived from open-source
    code, so it's definitely in their interest to show that open-source
    development is up to the task. Maybe ClamAV is an Untangle customer -- who
    knows. Ok, so I'm deeply cynical -- sue me. If you (or ClamAV or UnTangle)
    had cited actual test specifications and results I'd be more likely to take
    this story seriously. But since I've read similar undocumented testimonials
    for at least 25 different av programs over the years, I can't say as I'm
    particularly impressed with this one.

    ClamAV may be as good as Kasperksy or Symantec or MacAfee. And it certainly
    has the free thing going for it. But in the absence of actual data, that's
    about all anyone can claim.
    Most of these testimonials are aimed as much at making current users feel
    good about their choice as they are at attracting new users. Guess it
    worked for you.
     
    impossible, Jan 30, 2008
    #4
  5. thingy

    J Brockley Guest

    Easy enough to find tests and reasons why it doesn't rate as high as others.
    http://www.av-comparatives.org/seiten/ergebnisse/2ndgrouptest.pdf
     
    J Brockley, Jan 30, 2008
    #5
  6. thingy

    impossible Guest

    Yes, interesting. I was going to mention the av-comparatives site, but I
    noticed that they never seemed to run tests that included ClamAV. The report
    you cited clears that up. Thanks.
     
    impossible, Jan 30, 2008
    #6
  7. thingy

    thingy Guest

    They did...they offered the test suite/regime....
    Logic is not good for u is it....

    untangled wont get any money from clamav...so what would be the point?

    What worked? I found it interesting that clamav rarely if ever comes up
    in so called mag/lac tests....in fact I dont recall seeing one....and I
    wondered why....if it was that bad you would think the commercial guys
    would be falling over themselves to get it included as it would make
    them look good, instead it gets ignored....ghandi comes to mind....

    regards

    Thing
     
    thingy, Jan 30, 2008
    #7
  8. thingy

    thingy Guest

    or maybe you should look at the full tests....eg clamav 55% which on the
    face of it is not too good....

    On the other hand,

    kaspersky 55%
    McAfee 55%
    MS onecare 28% (can I say ouch here?)
    NOD 71% (interesting as ppl have said its "rated" but sucks for an
    interface)
    symantec 35%

    So clamav rates joint second....

    Macros...clamav 86%
    McAfee 100%
    the rest zero.....

    scripted malware...
    Clamav 46%
    the rest had issues trying to break 10%....

    worms
    clamav 65%
    Symantec and NOD did well here90% or better....


    While not top....overall clamav beat most of the competition....

    regards

    Thing
     
    thingy, Jan 30, 2008
    #8
  9. thingy

    impossible Guest

    Got a link?
    Are you that naive?! This "Untangle" outfit -- Ever heard of it before
    ClamAV posted a write-up of their LinuxWorld demo? Didn't think so. Now
    wrack tour brain and see if you can figure out the relationship between
    glowing testimonials and free advertising.
    You must have missed this:

    http://www.av-comparatives.org/seiten/ergebnisse/2ndgrouptest.pdf

    Sometimes products are ignored simply because they're not worth bothering
    with.
     
    impossible, Jan 30, 2008
    #9
  10. thingy

    impossible Guest

    Wrong, wrong, wrong. You need to be specific when you reference test results
    (and please always post links). In this case, there are different classes of
    tests run by av-comparativces that you have to consider. The test results
    referenced above
    (http://www.av-comparatives.org/seiten/ergebnisse/2ndgrouptest.pdf) by J
    Brockley are for what they call "on-demand detection rates", which is the
    familiar way of summarizing av performance. On those tests, yes, ClamAV was
    able to identify about 58% (not 55%) of the windows viruses thrown at. But
    if you want to compare those results with the results for other mainstream
    av programs **on the samebattery of tests**, then this is the report to
    look at:

    http://www.av-comparatives.org/seiten/ergebnisse/report15.pdf

    Here we see that that all the programs tested did **much better** than
    ClamAV.

    Windows viruses: 91-99%, compared to ClamAV's 58%
    Macro viruses: 99%, compared to ClamAV's 86%
    Script malware: 70-97%, cvompared to ClamAV's 46%
    Worms: 95-99%, compared to ClamAV's 65%

    ....and so on.

    The numbers you refer to below for the mainstream products all come from
    another class of tests run by av-comparatiuves, their
    "proactive/retrospective" tests, which are much more rigorous.

    See: http://www.av-comparatives.org/seiten/ergebnisse/report16.pdf

    None of these more rigorus tersts were run on ClamAV for the simple reason
    that it had failed so miserably at the basic tests.
    Apples and oranges. See explanation above.
    Apples and oranges. See explanation above.
    Apples and oranges. See explanation above.
    Sorry, but that's absolutely wronjg. You need to take more care with how you
    read and interpret these kinds of reports.
     
    impossible, Jan 30, 2008
    #10
  11. Scientifically speaking, results which cannot be reproduced are worthless.
     
    Lawrence D'Oliveiro, Jan 31, 2008
    #11
  12. Not really surprising. The market-for-lemons effect
    <http://www.schneier.com/blog/archives/2007/04/a_security_mark.html>
    strikes again.
     
    Lawrence D'Oliveiro, Jan 31, 2008
    #12
  13. thingy

    Murray Symon Guest

    Typically IT magazines are not science - more like infomercials (IMHO).
     
    Murray Symon, Jan 31, 2008
    #13
  14. Some of them are quite transparent about their testing methodology, laying
    out exactly what tools they use and where you can get them to try for
    yourself.
     
    Lawrence D'Oliveiro, Jan 31, 2008
    #14
  15. thingy

    Craig Sutton Guest

    Which version Nod32 the new version 3 interface is way better.
     
    Craig Sutton, Jan 31, 2008
    #15
  16. thingy

    thingy Guest

    I fail to see why....pointless trying to sell a product with a dodgy
    component you can easily swap out...

    second attempt...somehow our thread is not being re-produced....anyway I
    see you point re: apples and oranges (post elsewhere).

    yes, they need to be comparable, but they also need to be
    realistic....and true to real world...but consistent and repeatable is
    essential also.

    regards

    Thing
     
    thingy, Jan 31, 2008
    #16
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.