Video Screencast Help

Enterprise Intrusion Analysis, Part One

Created: 01 Apr 2009 • Updated: 02 Nov 2010
Language Translations
Admin's picture
0 0 Votes
Login to vote

by Stephen Barish

We all remember the early days of intrusion-detection systems — IDS was supposed to be the silver bullet that ensured the security of our enterprises against every conceivable attack. It was the same premise that the firewall industry and the giant antivirus conglomerates were built around: Buy our product and your worries are over.

Obviously this hasn’t proven to be the case. Even though intrusion-detection systems are readily available, many organizations still don’t use them effectively. Too often the IDS sits without maintenance or updates, poorly monitored, generating alerts that are completely irrelevant to the daily work of the security and staff.

The key to realizing the benefits an IDS offers is to focus less on the technology, and more on how it will be used by a security analyst. This article explores the discipline of intrusion analysis, focusing primarily on techniques to extend IDS capabilities beyond simple alert data into a tool for attack indications and warning, policy enforcement, and network defense.

Emphasize the Analyst, not the IDS

By now pretty much everyone in the security industry understands the basic ideas behind an IDS: monitor observable behavior, conduct some kind of automated test to determine if it is potentially malicious, and alert the analyst as required. While the security industry and professionals continue to seek the "Best in Class" solution to all IDS needs, the reality is there is no solution that eliminates reliance on human decision-making as part of the analysis process. Too many companies forget this, investing heavily in the infrastructure without making a comparable investment in their analytical personnel. Even large companies make the mistake of relying on the machine rather than the analyst.

From a balance-sheet perspective, it makes a weird sort of sense. On one hand, you have a capital asset you can depreciate annually, while on the other, you have a recurring expense in training. Many companies simply stop right there. The reality, however, is that a sufficiently skilled analyst can analyze network or host data with even the most rudimentary of tools. In fact, that’s how the industry got started with tools as simple as tcpdump, shadow, and the early versions of snort.

This is one of the classic mistakes in deploying IDS: over-reliance on the technology. Whether the solution is network or host-based, whether it works on a traffic analysis, signature matching, anomaly detection, or a hybrid model, ultimately all IDS does is present data of potential interest to a human analyst.

One good model for training IDS analysts can be found in the Department of Defense’s Information Assurance Workforce Improvement Program, documented in DoD Instruction 8570.01-M. This program establishes a training plan for Computer Network Defense (CND) analysts that yields an extremely effective skill set for IDS analysis, regardless of the technology used in the enterprise. Using this standard, a workforce model might look like the table below.

IDS Analyst Training Program
  Entry Level Journeyman Senior Specialist
Education & Experience 0-2 yrs in security
background in IT
no degree required
2-4 yrs in security
6-10 yrs in IT
bachelors in CS or equivalent
4-8 yrs in security

bachelors in CS or equivalent

8-10 yrs in securitybroad experience in multiple roles
Certifications (in first year of hire) Network+, Security+ SSCP, GISF GCIA, CISSP Multiple certifications from GIAC and vendors

Table 1 - A model for a security-analyst workforce

Pre-Deployment Planning

Many times companies deploy IDS too early. Before investing heavily in an IDS solution, it is a good idea to have a security program and architecture for it to operate within.

Unfortunately, there is a tendency to see IDS as one more check on a laundry list of controls that should be procured and implemented to have a "best practices" security solution. This is especially common where IT auditors identify the need for additional security controls in the enterprise. Companies simply select a product and deploy it, failing to determine what kind of product they need, and, more importantly, what they intend to do with the system once it is deployed.

This kind of IDS solution is ineffective, rarely used properly, and easily circumvented by attackers. Before heading down that path, companies should prepare using the following four steps.

1. Identify Assets

Of course, we’re really talking about information assets and the critical infrastructure they reside on here. This might include systems performing critical corporate processes or holding important data — such as billing, financials, marketing data, and intellectual property — or generating revenue for the enterprise. Ideally, security analysts should be able to identify the information that makes the company run, the processes that use that data, and tie it back to a physical portion of the infrastructure.

2. Assess Your Risk

Virtually everyone with an IP address is at risk from a certain perspective. You can see this by looking at the Honeynet Project, where systems are exposed on the Internet and compromised by automated attack tools literally within seconds of provisioning. We are all under attack at virtually every moment our systems are powered up and connected, and it is pretty much accepted wisdom that eventually an attack will be pressed home and your network compromised.

But the impact of a successful compromise varies widely by industry, and according to the data compromised by attackers. For instance, it would hurt significantly if their website were compromised, even if the back-end financial systems weren’t affected, simply because consumer confidence would suffer. But the impact of a website compromise on other companies might have very little impact if there were no tie to revenue generation.

3. Develop Security Policy

Everyone eventually suffers a compromise of some kind, and it is far less expensive, time-consuming, and more efficient to think through how you will respond in advance. Security policy sets the foundation for these decisions, establishes controls on the use of company infrastructure, information assets, and limits user behavior. It establishes the right to monitor, what kind of material you can examine and how you can use it. It doesn’t have to be complex or be thousands of pages long either.

Too many companies spend money on consultants to provide best-of-breed policies that comply with ISO 17799 or other standards, and don’t think about how they will implement them. Policy that doesn’t get implemented is worthless. On the other hand, without some idea of how you will operate your security program, it is difficult at best to effectively operate an IDS and get any real value out of it.

4. Know Your Enterprise

One of the biggest problems with deploying IDS in an enterprise context is the potential for data overload. Network IDS in particular can generate a large volume of event-data that analysts have to process. In fact, the most common complaint about IDS is the sheer volume of false-positives — alerts that have entirely benign explanations. While IDS technology gets a bit better every year, the reality is this problem will never go completely away.

That’s why sensor tuning is such an important part of the deployment plan. Unfortunately, it takes in-depth knowledge about an enterprise to really tune the IDS without completely desensitizing it. Prior to deploying and tuning the sensors, security architects should identify the major ports as well as services the network needs to support, protocols in use and those definitely prohibited, and document the known access paths to the enterprise.

Post-Deployment Sensor Tuning

Once the IDS is deployed, the work has just begun.

IDS implementation is rarely a turn-key operation, especially in mid-sized to large enterprises. Most of the time the deployment team will need to tune the sensors to reduce false-positives, as IDS solutions out of the box tend to employ so many alert criteria that analysts are easily overwhelmed. This is where good pre-deployment planning really helps, as a solid understanding of your enterprise and what you are trying to achieve with your IDS solution will really help reduce analyst workload.

1. Accurate Network Information

Every IDS requires some information about the network it is monitoring. In snort for instance, setting the HOME_NET variable incorrectly will render it extremely ineffective. In some cases, the directionality of TCP traffic is a major indicator of a potentially malicious event. When HOME_NET is set incorrectly, or not at all, this directionality is lost in the signatures, forcing an analyst to determine what exactly occurred in any given network trace. The same philosophy applies to applications and protocols in use in the enterprise. Understanding what "normal" is helps IDS analysts tune sensors to ignore benign behaviors.

2. Baselining the Solution

Benign network traffic tends to follow patterns. For example, applications usually run on scheduled intervals, and users interact with the network during business hours.

In many instances, this is also true of attacker behavior. The most common attack enterprises experience today are automated scanning and hacking tools that are largely untargeted, meaning they are launched against large network blocs in the hopes of gaining some degree of access or success. By watching the traffic at the perimeter for a few days or weeks and correlating with firewall logs, analysts can tune IDS signatures to eliminate alerts that are simply not of interest, such as eliminating signatures for UNIX based attacks in Windows-only shops.

3. Managing Thresholds

The whole point of employing an IDS is to enable a human to act. This is one reason that policy and incident response planning during the pre-deployment phase are so critical. By understanding the points at which the security staff or management needs to take actions, it is possible to tune out IDS data that will simply never be looked at.

Many organizations treat denial-of-service and port scan alerts this way. There is a potential danger in this however, as this type of tuning deliberately ignores data that is potentially harmful to the enterprise and is likely occurring on a regular basis. During the analysis process it is important to remember these kind of decisions and cross-check your analysis with the underlying assumptions that affected sensor tuning.

4. Plan for Correlation

Realistically, most enterprises are rich sources of security-related data. Firewall and router logs, vulnerability data, and even system logs can provide useful data to correlate IDS alerts. Although many IDS now provide some ability to perform automated correlation, this is often an effort undertaken by the analyst. There are emerging solutions to perform this function outside the IDS environment: security information management systems (SIMS) such as ArcSight, CiscoWorks, SecureWorks, and IntelliTactics. Even if no technology is used to augment the analyst’s know-how, you need to plan for where you are going to get data when you need it, and understand how that data either ratifies or undermines what your IDS is telling you.

Showing Return on Investment

The traditional downfall of IDS solutions is that they never show return on investment.

Of course, that’s the business answer, not the technical or operations answer. Those of us who actually secure networks and monitor IDS see their value every day. The challenge is demonstrating the value to management in a way that meets their needs, and that provides accurate, truthful information about the value of IDS in the enterprise. This is more than just providing the stock reports most IDS solutions allow you to create. IDS analysts need to show what was achieved as a result. Reporting how many attacks were blocked and by what security device is very useful data, for example, as is an analysis of which areas of the enterprise are most at risk. This level of reporting requires a high degree of understanding of the enterprise and a correlated analysis approach. It also provides real, actionable intelligence about what is happening in the network.

At the end of the day, that’s what an IDS is supposed to provide: Information that allows us to know what is happening in our network, and who is targeting us. Then it’s our job to decide how to react.

This article originally appeared on -- reproduction in whole or in part is not allowed without expressed written consent.

Article Filed Under: