Measuring Up (cont.)

Finding the Right Security Metrics

Demonstrate success

Terry Stern, information risk and security manager for the U.K.-based Xansa Ltd., also grapples with the challenge of accurately measuring security strategies intended to protect the technology outsourcing company he works for headquartered in Reading. "The biggest challenge of all is being able to demonstrate compliance with our policies through metrics," says Stern, who uses Symantec AntiVirus and Norton Ghost. "The whole business of compliance and governance is an essential part of our work."

When Stern evaluates new security software he bases his buying decisions on the package's ability to gather security metrics and compile them into reports. "We want easy, high-quality reporting from our security tools so we know what's going on from a security standpoint. Reporting becomes a critical consideration whenever we're assessing any security tool. The manageability of the management reporting and how easy it is to paint and present those reports is crucial."

In addition to measuring effectiveness, the reports help security managers demonstrate success to management. "If we can feed the data into a dashboard, that helps us. Management wants to see all the statistics on one slide."

A third-party security service tracks inbound and outbound emails for Xansa, and provides monthly reports on how many and what kinds of viruses it thwarted.

When Xansa first established the outsourcing arrangement almost four years ago, it found itself awash in security statistics-data points that would have previously been available only by "mucking about with the mail gateways," Stern says. Those investigations didn't typically occur because "it would have taken people away from their day-to-day jobs."

Today, Stern can scan online report details for key security metrics such as an unusual spike in incoming and outgoing emails, which might signal a potential virus or spamming attempt.

Stern says that stealth viruses are potentially more destructive than early incarnations that did their damage blatantly when they wiped data from a hard drive. "Now they have moved into the criminal arena as they endeavor to fool people into going to unsafe places and revealing information or installing Trojans and key loggers. That's a lot nastier," he says.

Other data points include the number of blocked attachments that carried viruses or the latest list of Web sites Xansa cordons off from its employees because of security breaches. Statistics like these keep Stern abreast of new outbreaks, and provide summaries for senior managers to substantiate Xansa's security software and services investments.

Stern can also use the data for historical analyses. He compares the number of viruses encountered year to year, and the annual rise in the amount of spam Xansa receives. "We are able to monitor trends and produce reports that show what happened over the last month or over the last year," he says. "We are able to see whether things have gotten better or worse."

In addition to statistics like these, Stern would like to see better reporting tools that calculate the possible consequences if antivirus or intrusion-detection technologies weren't in place to foil specific attacks. "I want reporting to be able to anticipate the type of reports that we want," he says. "For example, if I want to know the top 100 sites that some of our people tried to go to but were blocked, and the users who tried to access those sites. I don't want to have to be writing too many scripts in order to produce those reports."

Stern also counts on two additional security metrics that don't appear in the regular monthly reports. While perhaps unscientific, these indicators are nevertheless informative. "One of the most underrated security metrics is your own people," he says. "If you're doing your job well, then people will be talking about security; they're talking about what's going on in the news-if there is a high-profile phishing or key-logging incident, for example. Hearing people discuss things like that, I would argue, is a metric. It's a good indicator that your awareness training is working."

And the other informal security indicator? "I always judge how we're doing by how well I sleep at night," he says, adding that he usually rests easy because Xansa's strategies have proven successful. But he remains vigilant. "I always consider this caveat: Somebody, somewhere will be trying to find a way of doing something, either maliciously or not, to circumvent the security controls Xansa has in place."

Alan Joch is a business and technology writer whose work has appeared in Federal Computer Week and FedTech.