Video Screencast Help
Security Community Blog

Tolly Group Performance Testing of Endpoint Protection SBE (V12.0) vs. the Competition

Created: 21 Apr 2009 • Updated: 21 Apr 2009 • 2 comments
Carsten Hoffmann's picture
+1 1 Vote
Login to vote

The Tolly Group  tested the performance of Symantec Endpoint Protection Small Business Edition 12.0(SEP SBE) and the comparable products from McAfee, Trend Micro, Sophos, Kaspersky, BitDefender and AVG.

I worked with the Tolly Group on defining the scenarios and would like to give you some insight into the test. Let me be clear, this is not about the actual results.
There are quite a few tests (Virus Bulletin, AV-Comparatives, etc) that look into the pure technical numbers. For example, "through-put" by file type. That is important and interesting, but I was looking at it from a general user perspective. How long does take to boot the computer? How long does it take to open a presentation, etc.? That is what really interested me (and I hope you too!)

Tolly performed a similar test in November with  SEP 11.  I will try and incorporate the feedback we got on that test.
Some question the validity of any commissioned report and I can understand why. There are tests conducted where vendors identify how long it takes to scan a million viruses, which doesn't seem relevant in real-world applications. That simply doesn’t make sense -  even the most infected machine will have 1% or less malware files.

  • The test scenarios we came up with should all be relevant for normal PC users. Furthermore, all test results have been published. No test result has fallen under the table because we didn’t like it. You will see some results where Symantec is not the ‘fastest."
  • Each test was conducted locally and over the network,  as opposed to saving the files on File Share.
  • Some found the size of the documents used in the last test unrealistically large (or small). So I asked Tolly to run the tests with documents of different sizes (100kb, 1MB and 10MB).
  • A lot of people asked “what about product ABC”. While it will be impossible to run a test with all regional vendors, we tried to broaden the picture and double the number of competitive products to be included in the test.
  • I also added one class of scenarios to get a misperception about “Scan Speed” out of this world.  As you may know, it can take a long time to complete a “Full Scan” with SEP. Well, you basically have a choice – suck up all resources and get done with the scan as fast as possible, or let it run longer but leave the resources to the user. By default, Symantec takes the latter approach (administrator can change this).

To show the price for a fast scan I asked Tolly to perform the same set of tests while running a full scan of the computer.

So, the end result..... I just have to boast with the results SEP SBE.  It was faster in several tests while performing a full scan than McAfee and Trend.  And, the latter had no scan taking place.... send me a PM if you'd like more information or data!
 
Carsten

Comments 2 CommentsJump to latest comment

Tejas Shah's picture

We would need one of the actual Tolly performace testing report, especially a comparision report for SEP 11.x viz a viz McAfee and Trend. Can you help?

0
Login to vote
Bijay.Swain's picture

In my workplace I have seen Kaspersky is much faster than SEP11MR4 in terms of scan. and in detection rate again kaspersky beats SEP11MR4 .

I don't want to see the reports of a commisioned test. I want to see a test result where no antivirusa company has commisioned for the test.

0
Login to vote