Hi,
in this post I don't want to influence the votes.
For a constructive discussion let me post these FAQ from a 3rd part organization.
AVIEN Malware Defense Guide for the Enterprise, 2007 by Elsevier, chapter 10:
Frequently Asked Questions
Q: Surely all antimalware products detect much the same range of malicious programs?
A: This is less true than it was a few years ago, for two main reasons.
■ The range of malicious programs commonly encountered now is much wider than
it used to be, and non-replicative programs in particular pose more problems in
terms of heuristic detection. Also, while what we used to call AV programs detect a
good many non-viral malicious programs, they aren’t all equally focused on the
whole range of malware. So while the major players all tend to perform comparably
on replicative malware, they may vary dramatically on other malware types.
■ Modern malware (replicative or not) presents quite different problems in detection.
Despite all the advances made in scanning technology (such as advanced
heuristics), it’s easier for the bad guys to evade detection with short spamming
runs, multiple packing, and so on.
Q: Who has the resources to do in-house testing?
A: Good, safe detection testing takes appreciable time as well as expertise, and an isolated
network. However, testing for compatibility, configuration, deployment, network impact,
update and upgrade, and so on can be carried out on spare machines on a production
network, and is fairly painless.
Q: I understand the need to get the best possible performance, but my budget is very
restricted.
A: Been there, done that, set fire to the tee shirt out of sheer frustration. In fact, once, after
negotiating a particularly good deal one year, I discovered the next year that my budget
had disappeared altogether, in the expectation that I’d be able to repeat the coup in
perpetuity. It’s true that most of us have to fight the bean counters tooth and nail every
time, and mostly they are infinitely more impressed by low unit cost than by
performance metrics.
Q: Can you recommend a resource for malware-related metrics?
A: There’ve been many attempts to provide easy plug-in spreadsheets and other forms of
modeling over the years. Unfortunately, the malware management field has proved
particularly resistant to standards of measurement. Andrew Jaquith’s book “Security
Metrics: Replacing Fear, Uncertainty and Doubt” (Addison-Wesley, 2007) won’t give
you all the answers, but it’s well worth reading as an introduction to metric techniques
in general.
Q: Why do you say that detection isn’t important?
A: That’s not quite what I said. What I’m trying to say is that it doesn’t matter how good
detection is if the product is unusable, or beats your business processes to a pulp.
Detection is very important, of course; however, the days of near-100 percent detection
of all the threats you need to worry about are long gone.
Q: What’s the difference between the latent malware problem and heterogeneous malware
transmission.
A: HMT is concerned with the spread of malware, especially replicative malware, from an
infective or infected system to a vulnerable system via systems that aren’t themselves
vulnerable. (The expression was probably coined by Peter Radatti: at any rate, I first
encountered it in the 1992 paper “Heterogeneous computer viruses in a networked
UNIX environment” in the Proceedings of the First International Virus Prevention
Conference and Exhibition (NCSA),Washington, DC.) A latent virus is one that hasn’t
been executed in its present environment, but that doesn’t necessarily mean it can’t be.
Q: How do latent viruses relate to latency in other virus issues.
A: In “Viruses Revealed” I suggested that “dormancy” might be a better term in this
respect, since the term latency is used to denote impact on performance (e.g., the impact
of firewall processing on network throughput).
Q: Isn’t it rather convenient for the AV industry to protest that virus testing methodology
and creation of viruses for test purposes is unethical?
A: You could look at it like that, but there are sound reasons for this viewpoint.
■ Many in the industry are adamant that it’s inappropriate for those within the
industry to create new malware, even for research purposes (that probably has a lot
to do with the persistent allegations that the AV industry creates viruses in order to
keep itself going).
■ There is a safety issue.You may not think it’s that hard to keep a test network
isolated, but there’s a feeling that those who don’t understand the other points may
not understand the importance of safe practice, either.
■ The use of invalidated samples, poorly conceived modifications to malware, and
newly created malware can seriously bias the results in obvious and less obvious
ways.The most obvious problem is that where there is an invalid sample which
is incorrectly identified as viral or malicious by product A and not by product B,
product B is unfairly and inappropriately disadvantaged. Even if there is no
intention to skew the results, there is a question of ethical responsibility.
Q: Aren’t macro viruses specific to multiple operating systems?
A: In a sense. Macro viruses and some forms of script virus are actually specific to an
application, not to an operating system. However, their ability to replicate and the effectiveness
of any payload may vary according to the operating system, or between versions
of a given operating system.
Q: Isn’t the fact that the WildList is published monthly at most a drawback in terms of
WildList testing?
A: It does lessen its usefulness as the main component of a detection test. But it still offers a
useful way of assessing a scanner’s ability to detect a baseline set of properly validated
samples that a front-running product shouldn’t normally miss.
Q: Why is there no WildList for Trojans?
A: The idea has been discussed.The difficulties include:
■ Sheer volume of samples, arising from present-day patterns of distribution, raising
resource difficulties in terms of validating a core test set.
■ The additional technical difficulties of defining and automating the detection of
Trojans
Regards,