Focus On Linux: Intrusion Detection on Linux
by David Del Elson
This article focuses on several host-based intrusion detection systems that are available on Linux. In particular, I will cover some of the basics of installing setting up these packages, how they are useful, and in what circumstances they can be used.
Systems Security 101
This article assumes a basic knowledge of systems security. In particular, I will assume that the most basic security measures have already been taken to secure a host against intrusion from the internet. These measures could include:
Lines of Defence
Each layer in the diagram provides additional data protection to the layers above it. For example, the first layer is the firewall. Should an intrusion attempt not be defeated by the firewall, a second layer, the Port Sentry program, can provide additional protection.
Further inside the security system are the LIDS and LogCheck programs, that provide additional protection should an intrusion attempt not be intercepted by the Port Sentry program.
Monitoring Incoming Connections
The first layer of protection behind the firewall is a software package that will monitor incoming attempts to connect to the machine. The PortSentry package (http://www.psionic.com/abacus/portsentry/) provides a simple and effective method of doing this.
What does PortSentry do?
PortSentry is a program that monitors activity on specific TCP/IP ports. Activity on the ports that are monitored by PortSentry is reported, and one of several options can be taken, including denying further attempts to access to your system from the source of the activity. This is an important defence mechanism, because a hacker will typically probe your system for weaknesses ("port scanning") before attempting an intrusion. Detecting the probe or port scan, and completely denying further access to your system by a potential hacker, robs that hacker of the ability to follow up on any port scans with a real intrusion attempt.
For users of Red Hat Linux, PortSentry is available in RPM format on the Red Hat contrib FTP site. This site is mirrored in various locations around the world, check at www.redhat.com for the location of your nearest mirror. I haven't yet determined the availability of a .deb format package for PortSentry but I am sure there is one out there.
For other Linux systems, installing PortSentry from the source code is relatively simple.
PortSentry runs in a number of modes, including various TCP and UDP stealth modes. The mechanism that I prefer to use for running PortSentry is to bind it to a TCP port that (a) is not in use, and (b) is known in some systems to have potential for intrusion attempts. For example, port 143 (imap2), port 111 (portmap) and port 23 (telnet) are TCP ports that I do not use on my internet systems, and my web server was scanned on both of those ports in the last 24 hours.
To start PortSentry in basic TCP mode, ensure that your system start-up scripts run this command somewhere:
Also, ensure that the PortSentry config file (portsentry.conf) contains a TCP_PORTS line enabling scanning on the ports that you require.
The "Response Options" section of the portsentry.conf file allows you to specify what response that PortSentry will take on detecting unwanted activity. The mechanism that I normally choose is to use ipchains to block further access from the source of the activity. This is done by uncommenting the following line in the portsentry.conf file:
On systems that receive a high level of port scanning activity, removing the "-l" at the end of the above line will prevent logging of further incoming connections, which might be useful to save space in the log files.
Monitoring System Logs
Firewalling systems, and software like PortSentry perform one useful function, in that they monitor and prevent connections coming in to unwanted ports on the system. This can prevent access to a system via a standard scan-and-intrude method.
Where a system is required to run a particular service (eg: Apache on a web server, or BIND on a DNS server), and a hacker has uncovered a particular loophole in the service, these programs will unfortunately not achieve the result of keeping all intruders out of the system. A system acting as a DNS server that has a vulnerable copy of BIND running on it will eventually be discovered by a hacker that scans a wide range of machines for a single port (the DNS port) on each machine, and attempts intrusion against that port only. The firewall and PortSentry will unfortunately see this intrusion attempt as a legitimate access to the system.
LogCheck (http://www.psionic.com/abacus/logcheck/) is a useful program for scanning system logs for unusual activity. LogCheck works by scanning the various system log files (under Linux these are located in /var/log), and notifying the system administrator by e-mail if there is any unusual activity. Unusual messages in the log files can often be generated by intrusion attempts, or actual intrusions against your system.
LogCheck is available in RPM format from the Red Hat contrib archives, and from the same sources as PortSentry. Installing LogCheck from the RPM file or from the source code (read the INSTALL file provided with the source code) is relatively simple.
LogCheck has four main configuration files. In the RPM version, these are stored in the /etc/logcheck directory. Normally, only the logcheck.ignore and the logcheck.violations.ignore files need modification. The normal process that I go through after installing LogCheck is as follows:
Note that the RPM file specifies that LogCheck is to be run hourly, but normally I only run it daily except on critical systems that need regular monitoring. This is done by moving the /etc/cron.hourly/logcheck file into /etc/cron.daily.
Kernel Based Intrusion Detection
Kernel based intrusion detection is a relatively new art form for Linux. The main kernel based intrusion detection system currently available is called LIDS, and is available from http://www.lids.org/.
What is LIDS?
LIDS is an intrusion detection and prevention system that resides within the Linux kernel.
LIDS' protection is aimed at preventing the root user (who would normally have access to the entire system) from tampering with important parts of the system. LIDS' most important features include increased file system protection, protection against direct port access or direct memory access, protection against raw disk access, and protection of log files. LIDS also prevents certain system actions, such as installing a packet sniffer or changing firewall rules.
The LIDS system is somewhat more complex to install than either PortSentry or LogCheck. Fortunately, the LIDS web site contains quite good documentation on the LIDS project, including installation and configuration instructions.
First, before installing LIDS, make sure that you have the most up to date LIDS patch (I am using 0.9), and the correct kernel version. I am using the updated kernel (2.2.14-12) from the Red Hat Updates FTP site, because this contains some security fixes. You also need the source code for the kernel that you are using.
LIDS is currently targeted towards the 2.2.14 kernels. I installed LIDS on a Red Hat 6.2 system, this includes the 2.2.14 kernel. Before I installed LIDS, I obtained the updated kernel (from ftp.redhat.com/updates/ or one of its mirrors) and installed it according to the instructions at http://www.redhat.com/support/docs/howto/kernel-upgrade/kernel-upgrade.html.
The next thing I obtained was the updated kernel source, which also came from ftp.redhat.com/updates/ This I installed using:
Next, compile and install the lidsadm program:
Generate a RipeMD-160 password that will later be installed into the kernel:
I entered the password "anypass" and got back the key "d502d92bfead11d1ef17887c9db07a78108859e8".
Next, I copied the standard Red Hat configuration file for my architecture into the /usr/src/linux directory:
Next, I installed the LIDS patch using the following commands:
Note that the Red Hat supplied kernel is slightly different from the standard 2.2.14 kernel distributed by Linus, as it contains some updated drivers. The lids-0.9-2.2.14-redhat.patch file that is available is slightly different to the standard lids-0.9-2.2.14.patch file distributed with LIDS, as the latter will not apply cleanly to Red Hat's kernel.
Finally, I configured, compiled, and installed the kernel:
The following script shows the LIDS configuration options that I chose during the kernel configuration:
Note that since I don't have a UPS, am running a headless server (no X installed), and need to access this system remotely, I chose the configuration options above. The options that you choose for your environment may vary.
One important note: After compiling the kernel you must configure LIDS before you next reboot!
LIDS stores its configuration in the /etc/lids.conf file. This file should never be edited by hand, instead, you should configure LIDS by using the lidsadm program.
Running "lidsadm -h" gives a page or so of help as to how to use the lidsadm program. The LIDS documentation (on the LIDS web site) gives some examples of using LIDS to protect files, for example:
... which protects (marks read-only) the entire /sbin directory.
My preferred LIDS configuration script looks like this:
Once the LIDS system has been configured, you need to update your boot scripts to ensure that the "lidsadm -I" command is run during the boot process. This effectively "starts" the LIDS functions in the kernel. I normally place lidsadm at the end of the /etc/rc.d/rc.local script, as this ensures that the LIDS functionality doesn't prevent the rest of the system scripts from operating correctly.
This is the command line that I use at the end of /etc/rc.d/rc.local to start LIDS:
/sbin/lidsadm -I -- -CAP_SYS_MODULE -CAP_SYS_RAWIO -CAP_SYS_ADMIN -CAP_SYS_PTRACE -CAP_NET_ADMIN -CAP_LINUX_IMMUTABLE +INIT_CHILDREN_LOCK
Note that since the Kernel was updated using Red Hat's RPMs, you will need to follow the instructions in the Red Hat kernel upgrading web page mentioned above to modify the /etc/lilo.conf file. This will ensure that the new kernel that has been compiled with LIDS functionality will be the one booted when your system reboots.
After the next reboot, LIDS will be running on your system. If you need to stop LIDS to perform system administration tasks, then you should use one of the following commands:
You will need to provide the LIDS password, which was inserted into the kernel in RipeMD-160 format during the kernel compile.
You will also note that on shutdown, most of the shutdown scripts will fail. This is normal. The final shutdown script (/etc/rc.d/init.d/halt) will kill all of the processes and unmount the file systems. No other process will be allowed to kill any of the children of the init() process, due to the "+INIT_CHILDREN_LOCK" protection made in the rc.local file (above).
Also, every 10 minutes, you will get an error message about "rmmod \as" being unable to remove a module. This is because the "-CAP_SYS_MODULE" protection stops insertion or removal of modules once LIDS has started. To stop the error message happening, delete the /etc/cron.d/kmod file.
What Can LIDS Protect?
A quick read through the LIDS documentation will reveal the full set of features in LIDS. The most important features, in my opinion, include the following:
All of the above features can be turned on at any point using "lidsadm -I". The features can also be disabled at any point (to allow the real system administrator access to the system configuration) by using "lidsadm -S", and providing the LIDS password which was installed into the kernel (and encrypted with RipeMD-160).
Anatomy of a Break In
I was recently asked to examine a system that had been hacked, to determine the cause of the break-in, and to determine what damage the hacker had done to the system. Fortunately, the system was hacked by someone who was not particularly clever, and didn't manage to conceal their tracks entirely.
The break-in occurred when the hacker overflowed the buffer of a system daemon running as root (in fact one that should not have been running on the system at all, but the person who installed Linux was careless and left it running, and also failed to install Red Hat's released updates which would have fixed the buffer overflow problem). The hacker, however, was also careless in that when they managed to open a shell (BASH) on the hacked system following the break-in, they forgot that the BASH shell logs all activity to a .bash_history file for use by the command line recall functions. A simple read through /.bash_history revealed exactly what the hacker had done while logged on to the system.
The file read as follows (edited slightly for brevity):
Reading through this file, we can note the following activity:
What Can We Learn From This?
Firstly, note that LIDS would not have prevented the actual break-in. The hacker obtained root access to the machine by connecting to and overflowing a buffer in a process that was running as root.
Once the hacker had broken in, we can note how LIDS would have minimised the damage:
Note that LIDS would not have prevented the break-in, but would have prevented the hacker from causing any significant system damage after the break-in. A backdoor process could have been installed (eg: the pt07 backdoor could have been placed in /tmp, or any other non-immutable directory), but the non-trojan versions of ps, netstat, and pstree would have detected this process fairly easily and we could have come back and killed it off.
Without LIDS being installed we have no other real clues as to what the hacker might have done via this backdoor, and so our only available method to clean up the hacker's damage is to re-install the system completely.
OpenWall and LIDS: An Extra Layer
Another similar system to LIDS is the OpenWall project (http://www.openwall.com/linux/). The OpenWall project contains some different security features to LIDS, and one of the OpenWall patches in particular makes the stack area non-executable. An excerpt from the OpenWall README file states:
Recently, the LIDS web site has contained some integrated LIDS + OpenWall kernel patches that apply the security features of both LIDS and OpenWall to the kernel in a single integrated patch set.
Using a set of layered security tools on the Linux system, it is possible to prevent a wide range of system attacks, and to protect your system against intrusion or tampering. A hacker's point of entry into your system will be the network interfaces, and protecting these, and under the network interfaces, the system kernel, can discourage many attacks and prevent others.
Be aware of any potential security holes in your system. Any daemon or service running on your system, either as root or as a non-root user, can be a potential security threat. Be prepared to face attacks against these threats.
Articles and General Resources in this Section
Subscribe to the FOCUS-Linux Mail List
This article originally appeared on SecurityFocus.com -- reproduction in whole or in part is not allowed without expressed written consent.