PortSentry for Attack Detection, Part One
by Ido Dubrawsky
|PortSentry for Attack Detection, Part One
last updated May 15, 2002
Portsentry by Psionic Technologies (now a part of Cisco) is a component of their TriSentry suite of attack detection tools: portsentry, hostsentry, and logsentry. This article is the first of a two-part series that will describe in detail how Portsentry works from both a theoretical and a technical point of view.The second article will discuss installing, configuring, and tailoring PortSentry for individual systems.
Port Sentry - The Basics Theory
The basic theory behind PortSentry operation revolves around the detection of scans on a host and the response that should be implemented in response to those scans. This is one reason why PortSentry should be classified as an "attack detection" tool. A network or system scan serves as a precursor to an attack and possible intrusion. Unless the attacker knows beforehand which ports are open or available on a system, they will initiate a scan to determine what services the host is running. This is where PortSentry comes into play. This tool monitors the TCP and UDP ports on a system and, depending on how the system is configured, will respond appropriately to an identified scan.
PortSentry monitors for both TCP as well as UDP scans and, as of version 2.0, can detect stealth scans such as those produced by Nmap. Some of the scans which it can detect include:
If PortSentry cannot identify a scan as one of the above, it has a default case in which it will still trigger on the scan. Many other tools may actually ignore "unidentifiable" scans and let them through: not so with PortSentry, which turns out to be a very nice feature.
One obstacle that PortSentry faces is how to determine whether a packet is part of a scan or whether it is part of normal traffic for a given port. It does this using two methods. First, PortSentry ignores ports that are in use (that is ports with a service bound to them). Second, the user specifies a list of ports for PortSentry to monitor in the configuration file. If a port in the configuration file list also happens to have a service bound, then PortSentry ignores that particular port.
One nice feature in PortSentry is that it is intelligent about how it monitors ports. For protocols like FTP, for which the client opens up ports in the ephemeral range (TCP ports greater than 1024) and the server then connects back to the host, PortSentry is able to examine the incoming connection and, if it is destined for one of the ephemeral ports, ignores it. As soon as the connection is terminated, PortSentry monitors that port as before.
Checking for TCP or UDP scans are not the only methods PortSentry uses to detect a scan. If the IP header length is less than 5 (the minimum length specified in RFC 791), PortSentry will log a message noting this fact. Similarly, if any IP options are set, PortSentry notes that as well. While at first this may seem problematic, the use of options such as source-routing, record-route, or timestamp may be used by an attacker for reconnaissance information.
In a very basic way, PortSentry operates by monitoring ports for potentially suspicious packets and then responding if it has determined that a scan has been detected. To do this, PortSentry must maintain state with all of the IP addresses connecting into a host.
PortSentry utilizes an internal state engine in the form of an array of IP addresses that determines whether a host has previously connected to the system. The array is a two-dimensional table consisting of an IP address and a counter. As an attacker scans a target (whether the scan is a sequential port scan or a random port scan) PortSentry checks the array to determine if the attacker's IP has been seen before and, if so, increments the counter. It does this for every port it monitors and detects packets coming from the attacker's IP address. Once the trigger value has been reached, PortSentry reacts to the scan.
The method by which PortSentry reacts to a scan varies depending on what the user selects in the configuration file. PortSentry provides for three ways of preventing an attacker from completing their scan, which shall be discussed in greater detail below. All three are equally valid methods of blocking a host and each method has its risks and its benefits.
First, it can insert a route into the host's routing table so that all traffic from the scanning IP is sent to some "bit-bucket". By using the host route tables to re-route traffic from a scanning host to a non-existent IP address or network PortSentry makes the system look like a black hole...traffic goes in, but nothing comes out. One problem with this method is that it increases the routing table size on a host. Normally, these days, hosts are not actively participating in routing network traffic (unless, of course, it has more than one interface and is being used as a router). As the routing table grows more memory is needed to remember the routes. Theoretically an attacker can forge the source address of packets used in a scan which would cause PortSentry to indiscriminantly add route table entries in response. This would then fill up the system memory and may create performance issues.
Second, PortSentry can be tied into various firewall software packages (as of version 2.0 the firewalls supported by PortSentry are ipfw, ipfilter, ipfwadm, ipchains, and iptables) so that a firewall rule can be use to block traffic from the scanning IP. With firewalls PortSentry would detect a scan and then add an appropriate rule to the firewall configuration. This would then block the IP address of the scanning host. Again, as with using the route tables to block traffic, this can be abused. A skillful attacker using forged packets can cause PortSentry to firewall off hosts indiscriminately. This could prevent legitimate hosts from reaching available services on the scanned host.
Finally, a TCP wrapper rule for the attacking IP can be added to the system
While it may appear that manipulating the route tables or using the firewall rules to block scans can easily lead to a host blocking all traffic from reaching it, it is not that simple. Yes, there are dangers in using these techniques; however, by properly configuring PortSentry and monitoring its log output, that risk is significantly reduced. As the authors of PortSentry state in their documentation:
Once a scan is detected and a response initiated PortSentry keeps state on previously blocked hosts by writing the IP address of the blocked host to a Portsentry-blocked.<:protocol>: file (where
This article has provided a theoretical and technical overview of the operation of PortSentry. This useful tool can be used to complement other security measures to prevent an attacker from scanning a host and possibly using such information to launch an attack against the target. The second article in this two-part series will cover the installation, configuration, and operation of PortSentry on a host.
This article originally appeared on SecurityFocus.com -- reproduction in whole or in part is not allowed without expressed written consent.