In an earlier blog, Symantec Labs posted the results of some internal tests that compared the efficacy of Symantec Endpoint Protection (SEP) to CylanceProtect (http://www.symantec.com/connect/blogs/cylanceprotect-symantec-labs-analysis).
Our Symantec Labs team believed that we had identified some glaring, objective weaknesses in the Cylance product when addressing real-world threats, and we suggested that third-party validation was a reasonable next step. Given Cylance’s lack of participation in any independent tests at the time, the industry lacked independent validation.
We did not have to wait long for independent researchers to step up to fill this gap. AV-Comparatives (www.av-comparatives.org) and MRG-Effitas (www.mrg-effitas.com) tested the efficacy of SEP versus CylanceProtect and have published their results.
The Test Results
“In this independent assessment Cylance clearly delivered inferior protection against In-the-Wild threats and exploits compared to Symantec.”
AV-Comparatives and MRG-Effitas recently completed an in-depth real-world protection comparison between Symantec Endpoint Protection and CylanceProtect. Results demonstrated that SEP protected against 100% of in-the-wild malware while Cylance trailed behind at 92% efficacy. Even more significantly, SEP protected against almost 45% more exploits than CylanceProtect, with SEP protecting against 90% of exploits versus CylanceProtect guarding against only 63% (http://www.av-comparatives.org/wp-content/uploads/2016/02/avc_mrg_prot_2016_02_24_cyl_sym_en.pdf).
Symantec’s superior preemptive detection and prevention capabilities were also evident during the AV-Comparatives and MRG-Effitas tests. In addition to achieving significantly higher overall efficacy, SEP prevented more than three times as many exploits from executing than CylanceProtect.
The Deeper Issue
Despite detecting some of the malware installed by the shellcode tests, these failures on Cylance’s part are indicative of a deeper product strategy issue. Hackers are never idle. Their toolkits and approaches are continuously mutating in an effort to find weaknesses to exploit the moment that cyber security products deploy new defenses. As a result, a multi-dimensional approach is critical to ensuring long-term confidence in your cyber security solution.
Consider the hacker as the player and completion of a marble maze as a successful exploit and exfiltration of data. A single-dimensional solution like CylanceProtect that relies on only one strategy (i.e., algorithmic detection) offers perhaps a single wall and hole for the hacker to circumvent. Once the hacker is successful, it will become easier over time to sidestep any single-approach product’s protection technologies. Hackers have yet to see enough of the Cylance marble maze in the wild to exploit all its weaknesses.
Machine learning (ML) is a great tool but shouldn’t be used in isolation. While the machine is learning, so are the attackers… learning how to circumvent the detection algorithms.
Some specific concerns come to mind from our extensive experience with machine learning:
- Many security firms use ML “classifiers” to detect new artifacts like malicious files or URLs. The problem with these systems is that their decision-making is based on behaviors that are entirely under the attacker’s control. For example, an attacker can simply change their threat to use a different sequence of behaviors and an existing ML classifier is vulnerable to fail to detect it,
- The other issue for security firms that rely entirely on endpoint based ML with no cloud component is that the entire software stack is available to the attacker for potential manipulation – on the endpoint. Symantec uses ML where it matters – on the endpoint and in the cloud where attackers cannot compromise the intelligence, while also optimizing for scale and speed, making it effective across a variety of enterprise conditions.
Unlike CylanceProtect, Symantec Endpoint Protection is a multi-dimensional Intelligent Endpoint Protection Platform with machine learning that leverages multiple unique classifiers, combined with analysis of real-time software adoption patterns across Symantec’s hundreds of millions of active customers. This multi-dimensional approach, deployed on the endpoint and the cloud, makes it extremely effective in proactive prevention while reducing false positives, and far more resilient to attack. Symantec endpoints have withstood attacks from an army of hackers who have played its complex marble maze many times.
Data and algorithms are key to “tuning in” protection, and having better data is the first hurdle to cross. Without the right data, you may be missing visibility that cannot be extrapolated. Without the right algorithms, you can’t focus on the relevant data. And lastly, without the right experts, you can’t make sense of it all. Fortunately, Symantec combines all these attributes and capabilities. SEP leverages multiple, complementary strategies and technologies to counter attacks: Machine learning-based, network-based, hardening-based, and policy-based protection all work together to provide customers with the best endpoint protection. These technologies are described in detail on our next Next-Generation Threat Protection site: https://www.symantec.com/solutions/next-generation-threat-protection.
Our proven technology also recently received the top industry award for the 2015 best enterprise protection solution from AV-TEST: http://www.symantec.com/connect/blogs/and-best-protection-award-2015-goes-symantec-endpoint-protection-av-test.
Now consider the full portfolio of Symantec’s threat and information protection products like Symantec Advanced Threat Protection and Data Loss Prevention to realize that the hacker’s marble maze becomes exponentially more daunting to navigate. For a complete look at some of our new offerings, check out our announcements at the RSA Conference (http://www.symantec.com/rsa/).
These dimensional marble maze differences between CylanceProtect and Symantec’s security portfolio are not purely academic. In the AV-Comparatives and MRG-Effitas tests, CylanceProtect failed to catch Dridex, a financial Trojan:
“Among the missed samples are Metasploit exploits with in-memory Meterpreter, Dridex financial malware, in-the-wild exploit (malvertisement) and Sandworm Office exploit.”
In 2015 we leveraged our email security footprint, detection of spam, and cloud-based machine learning to deliver meaningful data to SEP through our Global Intelligence Network that permitted us to quickly lock in protection against Dridex. For more details on this threat and our multi-dimensional capabilities, visit our Connect article: http://www.symantec.com/connect/blogs/dridex-financial-trojan-aggressively-spread-millions-spam-emails-each-day.
The Curtain
AV-Comparatives and MRG-Effitas made an observation about niche players like Cylance, who are delivering only single-dimensional protection strategies and who are reluctant to participate in independent tests:
“This behaviour is seen by many of the newer products that claim to be next generation. It looks like they try to avoid getting tested in order to continue to attract users simple [sic] by unproven marketing claims.”
A multi-dimensional approach is table stakes for effectively stopping today’s malware. If something looks too simple then perhaps it is time to check the math.
Data and algorithms are key to “tuning in” protection; and having better data is the first hurdle to cross. Without the right data, you may be missing visibility that cannot be extrapolated. Without the right algorithms, you can’t focus on the relevant data. And lastly, without the right experts, you can’t make sense of it all. Fortunately, Symantec combines all these attributes and capabilities.
For a more extensive listing of the recent third-party tests, please visit our new Performance Center (http://www.symantec.com/connect/performance-center/).
A special thanks to AV-Comparatives and MRG-Effitas for providing these test results.