Video Screencast Help

Demonstrating ROI for Penetration Testing (Part Four)

Created: 06 Oct 2003 • Updated: 02 Nov 2010
Language Translations
Anonymous's picture
0 0 Votes
Login to vote

by Marcia Wilson

From the outside in

Bringing business to the Web is in and of itself risky business, just through the act of taking data from the inside network to the outside network. Data that was once protected by routers and firewalls is brought through the layers of security with remote procedure calls and database queries and made available to the public network. Part one of this series provided a general discussion of ROSI (Return on Security Investment) and likened performing penetration testing to having a health physical. Part two focused on defining penetration testing as a subset of a security assessment, by introducing information asset valuation and risk management concepts. Part three discussed asset valuation, risk analysis, the need for layered security, the reality of today's complex information systems, and argued the value of Pen Testing. In this, the last article of the series, we will discuss a Pen Test process and make final assertions about how ROSI can be shown.

Jody Melbourne and David Jorm have written an excellent three-part article series on Web penetration testing that discusses known security exploits and flaws in Web applications. The articles include useful information regarding input validation, authentication methods, code and SQL injection, cross-sight scripting, cookies, etc. The key point to understand is that through the Web application itself, a Pen Tester can exploit the backend database regardless of what types of firewalls, routers, load balancers, etc. are in place.

The process of penetration testing

There are two approaches to penetration testing: "black box" and "white box". Black box implies that the pen tester has no previous knowledge of the environment to be tested other than the name of the company. White box implies significant background knowledge of the environment including the type of network (i.e., Cisco gear, TCP/IP), Web platform (i.e., Solaris, Apache), application server platform (i.e., Windows NT, Peoplesoft), database platform (i.e., HP-UX, Oracle), load balancers (i.e., Alteon), firewalls (i.e., Cisco PIX), etc. Black box testing is supposed to imitate the behavior of a hacker who must gather that is information publicly available, such as domain and IP information. The WHOIS utility is often used (i.e., ARIN, InterNIC, NSI, RIPE). WHOIS provides the name and address, phone number and email address of the person who registered and manages the company's domain registration. You can find autonomous system numbers, public IP ranges, and even find out if the company is associated with a parent company. For most organizations a tremendous amount of information is gleaned through a WHOIS lookup. For smaller companies, sometimes the registrant is an important person in the company, such as senior network administrator. Sometimes it's the name of a founder, CIO, CEO, CTO or board member -- people who are notorious for not changing their passwords.

Google can easily be used to find just about anything and everything about a person or company. In five minutes the black hat approach can provide the public IP range in use by the company and the name and email address of a key employee. All that may be left to discover is the password of that person, assuming that the email address gives a clue to the format of a login name for the Web front-end to their email, or their terminal services account, or VPN access, or others. Social engineering comes easily into play to verify the login name format, and a clever hacker will just call the operator, get transferred to helpdesk and ask for a password reset with some ruse about being remote and traveling and having a critical customer meeting and not being able to logon. Make the story as colorful as you like and apply pressure to the poor bloke on helpdesk. As you can see, reconnaissance is the first step in the process as the WWW simplifies information-gathering efforts.

Even from a white box perspective, the gathering of information other than that given to the Pen Tester is a relevant activity. Some companies are not aware, believe it or not, where their networks begin and end. They may be connecting to a subsidiary or partner or parent company with poor security and they do not restrict or monitor traffic between entities. Companies give away a tremendous amount of information on their own Websites, but that's another topic for another day. As far as a procedure for the Pen Test, here is a short version of the process. It's important to review as it relates to proving out ROSI.

  1. Plan
    Determine if it's a black box or white box approach, realizing that the black box approach is just a matter of minutes away from the white box approach. Decide what the success criteria are. Plot out whether exploits will be performed and to what extent. The plan should include working with "someone on the inside" to monitor the work either via an IDS or manual monitoring so as not to adversely affect operations. Obtain signoff from appropriate management.

  2. Gather information
    Gathering information is sometimes called reconnaissance, which adds a bit of spy movie mystique to the process. After obtaining IP address information, the next step typically involves scanning the range for listening ports and services, OS fingerprinting, grabbing banner information that might include contact information, running an SNMP sweep looking for public community strings, and gathering any other information that might come in handy.

  3. Verify vulnerabilities
    This is typically called vulnerability analysis and can be accomplished in a number of ways, but all have to do with scanning the target IP range and comparing that information to a vulnerability database for matches. Many commercial and open source tools are available to achieve the goal and the outcome will depend upon how up to date the vulnerability database employed is.

  4. Exploit
    The exploit(s) are framed around the vulnerabilities found. A number of tools have exploit functionality built-in and provide automated results. The good news for companies is that Pen Testing doesn't have to take months. It can be done in a matter of hours. The bad news is that the same tools available and developed by hackers are also used by security professionals. If social engineering methods are used, the tester may not have to even exploit, for example, a buffer overflow vulnerability. He/she may have found the login name of a senior network administrator from the WHOIS lookup and may have discovered that, with the appropriate password, the Web application allows that particular login key access. Hopefully, this is not the case.

  5. Cleanup
    This is about cleaning up log files and making sure whatever settings or parameters were changed during the Pen Test are set back to their original condition. This requires excellent documentation during the Pen Test and work with the system and network administrators afterwards to make sure that the situation is satisfactory upon completion. Always obtain signoff at this juncture.

  6. Report out
    This is the most important part of the Pen Test. The final report must map the findings (vulnerabilities found, exploits performed) to the risk the company might realize if the threats were realized. The security professional should also provide a remediation roadmap based on "best practices" and be available to discuss the various technology options and costs.

Results and risks

The goal of a Pen-Test is to "connect the dots" between an asset and its value to the organization, the associated threats or vulnerabilities, and how that translates into a quantifiable risk in hard dollars or intangibles, such as the company's reputation. This is where, depending upon the level of vulnerabilities found and the potential impact to the environment, a level of risk is assigned to the results. For our Web-based ERP project example, let us assume that the Web server is a Microsoft Windows 2003 Server and the results of the Pen Test highlighted several vulnerable areas -- one of which is the recently reported "Buffer Overrun In RPCSS Service Could Allow Code Execution (824146)". If exploited, an attacker could run code with Local System privileges on the Web server, or could cause the RPCSS Service to fail. The attacker could then take any action on the system, including installing programs, viewing, changing or deleting data, or creating new accounts with full privileges.

Owning the Web server, which the above exploit could provide, is the first conquest. However, hackers don't usually stop there. The goal is to own the network. In making our prior ROSI assumptions, it was clear that the Web server itself has not been deemed to be the most valuable asset in the chain. The database is the primary target, however "owning the network" is much more interesting. Owning a system, exploiting trust relationships, compromising the database and finally owning the network is the real goal of a hacker. The Pen Tester's goal is to see if it can be done and in how many ways. Remember we are talking about calculating a Return on Security Investment, so management needs to understand why they may need more than one or two routers, one or two firewalls, switches with specific functionality, intrusion detection systems, tighter systems administration procedures, specific policies, and change control processes. They want to know why the security layers are necessary and how much it's going to cost. The results of the Pen Test provide this information to validate/invalidate the existing security architecture.

Mitigation efforts and risk averted

Risk analysis results provide insight into the cost/benefit of protection. In order to obtain the benefits, an organization needs to understand what it is they are trying to protect and what the risk is in not protecting the asset(s), but also and more importantly how that protection needs to be applied. In our example, the project plan included a network design that placed the Web and application server in the external DMZ, and the SAP database on the internal network. The firewall was given the task of filtering traffic between the Web/app servers and the SAP database. All good and well -- standard ports, standard traffic, right? The existing security design allowed inbound traffic limited to relevant services on the DMZ, SSH is allowed for encrypted remote administration, full outbound traffic allowed, and the internal network is "open". Compromising the Web server, in an effort to exploit trust relationships, includes taking a look at log files (syslog, lastlog, rhosts, shell history, etc), running processes, config files, password cracking, and sniffing. The Pen Tester can discover user/password pairs for the Web server and see how the Web server communicates with the application server or database server. One method to penetrate through to the back-end database server could be to setup port redirection (netcat) to redirect traffic coming into the Web sever on port 25, out of the Web server to the back-end database server on port 22 (SSH) and launch an SSH from the Pen Test box on port 25 to the Web server. The Web server will redirect the traffic to the backend database on port 22 and have an interactive session with that. Depending upon the trust relationships that have been setup, the Pen Tester, now owning the Web server, could gain ownership of the database server. At that point, mission accomplished. However, the greater goal is still to own the internal network. Since the back-end database server is on the internal network, and most internal networks are often very insecure, the Pen Tester can do more reconnaissance, run ping/snmp sweeps, port scans, sniff traffic, run vulnerability scans and exploit whatever is available for exploitation.

In our example we have shown how an entire network can be compromised starting with a compromised Web server. Since the traffic was considered valid traffic, the routers and firewalls didn't have a job to do. Intrusion detection would have been useful, though it would depend upon how well the IDS are configured. And of course you'd need a real person to spot the unusual traffic on the network and respond. Systems Administration policy and procedures become a very important part of designing and implementing any information system. There are many considerations when designing secure ecommerce environments, and some may now be considered elementary but are worth repeating anyway: a proper trust model between servers in the DMZ and internal network; configuring for CAR (committed access rate); Filtering (RFC 1918 and RFC 2927); ingress/egress filtering; VLANs, and host-based/network-based IDS.

Host-based IDS can stop attacks at the system level, while network-based IDS is effective in stopping attacks at the network layer such as DoS attacks. Network IDS can also monitor layer 7 information. False positives remain a problem with IDS deployments as well as placement of NIDS, although IDS/IPS technology is improving.

Even with all the right security layers in place, and best practices followed, new vulnerabilities are discovered every day. There is no such thing as a completely secured, invulnerable environment. What the Pen Test does is validate/invalidate the security of the design and makes it possible for the implementation to be as secure as possible before being put into final production.

Correlating Results with ROSI

We are left with justifying in advance the value of a Pen Test, when it would be so much easier to do the test and then justify the expense after-the-fact based on the relevancy of the results. The value is where a Pen Test can uncover weaknesses in the security of the system/implementation, and can either prove or disprove a perceived level of security. The results will determine what level of changes need to be made. If severe security vulnerabilities are discovered, everyone applauds the Pen Tester and the foresight involved in requesting a test be done. If no serious security vulnerabilities are found, everyone wonders one of two things: was it not necessary, or was it not thorough enough? In regards to doing specific ROSI calculations, the Pen Test results will show what the true numbers could be, but in advance of the test the only process that will give insight is:

  1. Valuation of Assets
  2. Risk Analysis
  3. Assessment of Security Posture
  4. Pen Test Internet-facing applications

You can hire the best network and systems engineers and follow the best practices and someone can still make a mistake. This is why strict change management is necessary and why an iterative process of validating the security posture is critical.


In our Web-based ERP implementation example for Widget Manufacturing, first described in part one of this series, the Pen Test was part of the project plan. Moving the application into full production was contingent upon the results of the test. The cost of the Pen Test was folded into the TCO numbers and the overall project ROI had been calculated, therefore no specific ROSI numbers for the individual components of the security architecture were understood even though each component had been given an "asset value" and was considered in the overall cost of the project. What happens to the numbers if the Pen Test results show that a revised security architecture may be required? What happens to the pay back period, the internal rate of return, and the net present value of the project?

With the above disturbing questions in mind, can the Pen Test provide cost savings and productivity gain by avoiding further costs should the security architecture have to be redesigned? If the delay of a project is considered to be detrimental to revenue generation, then the significant delay of the project could threaten the sustainability of an organization. Having hard numbers on ROSI in advance is a difficult thing to do. Proving out ROSI reminds me of the days when "Quality Assurance" was an afterthought. Now QA is part of how we do business because experience taught us that if we didn't do it as close to right as possible the first time, we'd have to repeat ourselves and do the numbers again and again and again. In much the same way, ultimately penetration testing should become part of how we do business, instead of having it regarded as an afterthought.

Author Credit

View links to all four parts of this series by Marcia J. Wilson on SecurityFocus.

This article originally appeared on -- reproduction in whole or in part is not allowed without expressed written consent.

Article Filed Under: