Back in June of 1992, I joined Symantec’s nascent antivirus team as a scruffy intern after a brief stint with the Norton Commander and Norton Desktop teams. At the time, Norton AntiVirus was a third-tier product with virtually no market-share. But that was about to change. That summer, Symantec hired over a dozen contractors to drastically improve Symantec’s detection rate and make us a world-class product. To give you an idea, back in 1993, top-notch products detected about 1,400 virus strains.
Over the course of that summer, and during my follow-up internships over the next few years, my teammates and I quickly realized that viruses were evolving at an extremely rapid pace, and would soon prove impossible for NAV’s core detection engines to detect. A detection engine is the heart and brains of the antivirus product; it performs all of the actual virus fingerprint scanning, and ours was quickly becoming obsolete.
Clearly the word was getting up to our executives, because in 1993 Symantec acquired Certus corporation, a small startup known for its blazing-fast antivirus scanner, to provide the foundation for our next generation product. In retrospect, the Certus engine boasted excellent performance, but lacked many of the more advanced features for which our antivirus fingerprinting team had been pining. While posing a challenge, this was also an enticing opportunity for our fledgling team.
Challenged with the opportunity of building Symantec’s next-generation AV product, my manager, Jimmy Kuo, empowered us to rapidly innovate and evolve our new Certus engine. Everyone on the team, including myself (at the time, a Senior at UCLA) was encouraged to integrate their inventions into the new engine. While such a wild-west approach could have been a recipe for disaster under other circumstances, our small, tightly-knit team successfully incorporated dozens of innovations into the new engine. We launched NAV 3.0 in January of ’94, firmly under the impression that we had built the most advanced antivirus product in the world.
Unfortunately, as anyone familiar with the security industry knows, given the rate at which new attacks and threats are released, security software begins to lose effectiveness the moment it’s shipped. NAV 3.0 didn’t buck this trend, even with its many innovations. A small, but growing number of virus writers were producing a new class of viruses that mutated themselves as they spread. These so-called “polymorphic” viruses were designed such that no two copies shared the same set of programming instructions, making detection by the classic fingerprint-based antivirus engines of the time largely impossible.
At first, members of the team thought that they could write a specialized detection program for each new polymorphic strain that was discovered. In other words, an analyst would reverse-engineer the threat, figure out how it mutated, write a C or assembly language module to detect all possible mutations, and then ship an entirely new version of the antivirus software containing this new module – not a virus fingerprint update, but an entirely new version of the product (e.g. NAV 3.1 vs. NAV 3.0). I have vivid memories of one of my teammates strutting proudly down our aisle of cubicles toting a fluttering ten-page ribbon of dot-matrix pages containing his assembly language source code to detect the MtE (Mutation Engine) family of viruses. He spent weeks of time writing ten pages of assembly code to detect one family of viruses! It was at this point we realized that such an approach couldn’t be scalable. By the time we were seeing dozens of polymorphic viruses every month, our fears were confirmed that we needed a more efficient approach. At this point, it was mid-1994.
Over the few years preceding this spurt in polymorphic viruses, we had investigated using CPU emulators to repair certain classes of viruses. A CPU emulator is a software program that simulates a PC’s hardware; one can then run software in the emulator and safely watch how it behaves, without affecting the actual computer – like the movie The Matrix. We found that if we inserted a virus-infected file into the emulator and simulated its execution, we could use the information gathered during the simulation to remove the virus from its host program. (Viruses attach onto legitimate files much like the alien in the movie Aliens infected its human hosts.)
This gave us an idea. If we could use an emulator to remove traditional viruses, might it also be possible to use the emulator to scan for polymorphic virus infections? If we threw an unknown file into the emulator that contained a polymorphic virus, would the virus’s behavior manifest itself and reveal the infection?
In fact, we found that the CPU emulator was extremely effective at detecting polymorphic viruses. The only problem was ensuring acceptable performance. How long should we let an unknown program – one that may or may not be infected with a polymorphic virus – run within our emulator before we can, with certainty, categorize it as “clean” or “infected”? If we let a malicious program run for only a fraction of a millisecond, it may not have enough time to expose itself. On the other hand, if we run each program for 10-20 seconds in the emulator, the computer would slow to a crawl.
This was an extremely difficult challenge. Our antivirus program had to scan each file in no more than ten to fifteen milliseconds or the computer would be unusable. So in the fraction of a second between when the user clicked on an icon and the associated program loaded, we had to somehow simulate the program within our CPU emulator long enough for any viruses in the file to reveal themselves, but not so long that the computer would grind to a halt. After another year or research and an additional year of testing, Symantec solved the remaining challenges and integrated a CPU emulator-based scanning engine, called “Striker,” into Norton Antivirus alongside our enhanced Certus scanning engine. This engine enabled us to detect the vast majority of polymorphic viruses in just milliseconds of scanning time, with just minutes or hours of required analysis rather than the weeks required before. Enhanced versions of the Striker engine still power all of our antivirus products to this day.
But as I mentioned earlier, the security world doesn’t stand still. During the next few years, we had even more challenges and polymorphic viruses turned out to be the least of our problems. On the malicious code front, we had the emergence of Macro viruses (viruses that infect Word for Windows, PowerPoint and Excel), script viruses, and increasing numbers of 32-bit Windows viruses. In addition, Symantec’s antivirus business was booming, and Symantec started shipping antivirus software for multiple platforms like Netware, Windows 95, Windows 3.1 as well as the ever-present DOS. This unique combination of threats and increased platform requirements led us to realize that the industry’s existing antivirus model was broken.
Up until this point, every time a security company wanted to release a new antivirus engine to protect against a new class of threat (e.g. Macro viruses) we had to ship a whole new version of the product. This meant that our customers would have to install a new version of the software to upgrade their protection, and retailers would have to replace their entire inventory with new yellow boxes each time our researchers wanted to address a new class of threat. In addition, because security companies had products for multiple platforms: Windows, Netware, DOS, etc., they’d usually ship one updated product per month, meaning that the DOS update might ship in January, the Windows update in March, and the Netware update in July. The same product would have different levels of protection on each platform, making administration a nightmare! This was extremely important because Symantec’s corporate customers were adamant that they wanted protection against the latest threats on all platforms at the same time, but equally adamant that they didn’t want to re-install new versions of their antivirus software once per month.
We started considering architectural alternatives to our antivirus engines in late 1995 and early 1996. Our challenge: How could we produce a platform-agnostic engine technology which could be updated as part of our standard fingerprint updates? How could we ship fundamental new enhancements to our antivirus scanning engines on a moment’s notice, rather than require multi-month, full product updates and retail inventory shuffles, not to mention angry customers? The result of our research was the NAVEX engine, or Norton AntiVirus EXtension technology.
NAVEX technology, shipped in 1996, enabled Symantec to rapidly ship entirely new scanning engines, across all supported platforms, to address entirely new threats. All customers needed to do to have the latest protection was click the LiveUpdate button in Norton AntiVirus, or use our corporate LiveUpdate Administrator technology to update thousands of enterprise desktops. To my knowledge, Symantec was the first security company to deploy such a modular engine architecture, and it took several years for our competitors to follow suit. Today, these modular engine designs are considered standard, and virtually every antivirus and antispyware product employs this architecture.
Well, hopefully this gives you some insight into the early days of antivirus engine development. As you can see, the 1990s posed some huge challenges to security companies and customers alike. On the other hand, these same challenges drove us to invent some of the most far-reaching and important innovations in the security field. Today, of course, our challenges are entirely different an in many ways much worse, but we’ll leave those for a separate blog entry.
For more on Symantec's 25th anniversary, click here