Chrooted Snort on Solaris
by Andre Lue-Fook-Sang
Do most people ever worry about the security of their Intrusion Detection System (IDS)? They should. With high speed Internet access being so common around the world, many personal PCs are being hijacked or shared by hackers these days. These hijacked PCs serve as a launch point for attacks, making it easier and more tempting for anyone to try their latest exploit because a reverse trace will most likely lead back simply to a compromised PC. Having a secure IDS is a necessary tool to see what kind of attacks or exploits are being tested against your network and systems. It can also reveal traffic from a prior hack if the intruder is still occasionally logging in. Hopefully, the latter never occurs.
With the addition of an IDS to your network, you do not want to neglect securing it and you do not want to also add the possibility of an attack or entry point. This would be all around embarrassing, especially if the IDS was a hard sell to management to begin with.
This paper will focus on installing a Snort IDS on the Solaris operating system, and securing it in a chrooted environment. Snort is an easy to use, cost effective, lightweight IDS that compiles and runs on every major operating system. Snort has had one major security flaw in it's time, due to a RPC protocol buffer overflow, [ref 1] which is why we will chroot its installation.
The effect of chrooting a process minimizes the impact of future buffer overflows like this one. The IDS process would still likely crash, taking the IDS offline, but the hacker would not gain access due to the overflow because the chrooted environment would not contain a shell.
Chrooting is the process of executing commands relative to the directory provided, which to the application becomes the new system root. All future system calls by the chrooted process will use the new system root directory as its filesystem root. As an example, if /export/home/ids is to be the new system root and Snort is normally located in /usr/local/bin/snort, it would now be located in /export/home/ids/usr/local/bin/snort. Nothing outside of /export/home/ids would be accessible.
To further reduce the chances of an IDS exploit we will also:
It's a good idea to also read the following two articles on Snort before continuing with this article:
2.0 Tested Platforms
The following instructions have been tested with all of the following environments:
The same techniques should also work on Solaris 10, however a native, more ideal solution for chrooted environments called "zones" has been implemented in Solaris 10.
3.0 Placement of the Snort IDS
A Snort IDS is best placed at a point on the network that is a shared entry or exit point. Sometimes, this translates to multiple Snort installations running on multiple machines. An alternative to this could be one machine running multiple Snort processes connected via multiple interfaces to different networks. This is the model we will explore because it is a little more cost effective for our setup, illustrated below in Figure 1. Keep in mind that the total cost of an IDS can become quite expensive depending on the tapped link speed (100Mb or 1000Mb), the amount of tapped points and the number of Snort machines needed.
In our example production environment, interface bge0 with IP address 172.30.8.102 is used as the service interface. No Snort process is run on this interface. We wish to watch network 172.30.3.0/24, which is connected to interface bge1, and interface with no IP address. A Snort process is therefore attached to bge1. This is repeated for network 172.30.1.0/24 on interface bge2 and network 184.108.40.206/24 on interface bge3. This requires a change to each respective snort.conf configuration file, as follows:
An second example, more suited to a small office environment, uses a cable modem setup and can be seen below in Figure 2.
In this setup the machine functions as a firewall and gateway to the Internet for various Windows PCs. It also runs Snort in a chrooted environment. Here, Snort will log anything it can match against interface elxl1 (which is the Solaris intel labeling for a 3com Ether Link XL card, iprb for intel pro B), connected to the Internet via DHCP. The snort.conf entry is as follows:
Using this configuration one can avoid ever having to specify an IP address, regardless of whether Snort is listening on an interface that has an IP address or not. This configuration also saves the user from having to change the setup when the cable modem's DHCP address inevitably changes.
4.0 Getting the required software
The following dependencies are needed to build our Snort binary, and are assumed to be installed and functional: the gcc compiler 2.95+, flex 2.5.4+, and bison 1.75+). If these dependencies are not available, they can be downloaded in pkgadd format from sunfreeware.com. Then you will need to download, compile and install the following software:
Optional hardware info:
5.0 Compiling the packages
The machine you compile packages on and the Snort IDS machine might be different machines. The following steps are done on the compiler machine. Download all packages to the /snort directory.
Building libpcap, libpcre and Snort is very straightforward.
# cd /snort # gzip -dc libpcap-0.8.3.tar.gz | tar -xf - # cd libpcap-0.8.3 # ./configure ; make # make install
Step 2: Building libpcre
# cd /snort # gzip -dc pcre-5.0.tar.gz | tar -xf - # cd pcre-5.0 # ./configure ; make # make install
Step 3: Building Snort
# cd /snort # gzip -dc snort-2.3.2.tar.gz | tar -xf - # cd snort-2.3.2 # ./configure ; make
Now that the packages are built, let's create a snort.conf for testing inside the chroot. For simplicity, I've included a script that will make a generic snort.conf from the entire ruleset that came with Snort. While this is not optimal for a production environment, it works fine for testing purposes on a cable modem and an old Pentium II running Solaris on Intel. We just need a snort.conf to test Snort in the chroot. Save the script as /snort/fxsnort, chmod 755 fxsnort, and then run it as follows:
# ./fxsnort snort-2.3.2/rules out.conf
It should have created a generic /snort/snort.conf. Edit the generic snort.conf as follows:
6.0 Building the chrooted environment
The next steps must be done on the Snort machine. Building a chrooted environment means re-creating a minimal system directory structure and copying every needed file and device inside the chrooted directory. For example, let's look at what's required to re-create /dev/tcp inside our chroot path /export/home/ids. This would involve:
# cd /export/home # mkdir ids # cd ids # mkdir dev # mkdir -p devices/psuedo # cd devices/psuedo sparc specific # mknod tcp@0:tcp c 11 42 intel specific # mknod tcp@0:tcp c 42 0 # cd ../../dev # ln -s ../devices/psuedo/tcp@0:tcp tcp
This would take up alot of time each time you had to build a chroot. Luckily, there are scripts to do this for us, downloadable here [ref 2]. I have made some modifications and added a perm_fix script that copies the current permissions of everything it re-creates. The modified package should be used and can be downloaded here at SecurityFocus: cell.tar.gz.
Adding your specific solaris interface to the /cell/snort/devlist file will be required unless you have a device matching bge, hme, qfe, eri, ce or elxl. To find your interface do the following:
# ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 elxl0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 192.168.43.7 netmask ffffff00 broadcast 192.168.43.255 elxl1: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu 1500 index 3 inet xx.xxx.xxx.x netmask fffff800 broadcast xx.xxx.xxx.255
Let's build the cell inside /export/home/ids. Download the cell.tar.gz package to root, then do the following:
# gzip -dc cell.tgz | tar -xf - # cd cell # ./make_cell snort /export/home/ids
7.0 Installing the Snort binaries into the chroot
We now have a minimal environment for Snort in /export/home/ids. All we need to do now is transfer the Snort binary to /export/home/ids/usr/local/bin. Let's give it a test run using:
# chroot /export/home/ids /usr/local/bin/snort -D -b -i bge1 \ -c /usr/local/etc/snort.conf -u nobody -g nobody -l /tmp
Note the Snort switches used:
-D Run Snort in background (daemon) mode -b Log packets in tcpdump format (much faster!) -c <rules> Use Rules File <rules> -i <if> Listen on interface <if> -t <dir> Chroots process to <dir> after initialization -u <uname> Run Snort uid as <uname> user (or uid) after initialization -l <ld> Log to directory <ld> -g <gname> Run Snort gid as <gname> group (or gid) after initialization
The prompt returned and thus no Snort process is running. What went wrong? Well, three files were purposely left out from /export/home/ids/usr/local/etc. I left out unicode.map, snort.conf and virus.rules. The file virus.rules can be left out by commenting it out in snort.conf. But let's see if we can use truss to see what really happened, because when dealing with chroots you will need a handy tool like truss to see what's really going on. Truss is a utility that executes a command and produces a trace of the system calls, signals and machine faults.
Note the truss switches used:
-f Follows all children created by fork() or vfork() -a Shows the argument strings that are passed in each exec() system call -e Shows the environment strings that are passed in each exec() system call -vall Verbose syscall. In this case all syscalls.
# truss -fae -vall chroot /export/home/ids /usr/local/bin/snort -D -b -i bge1 \ -c /usr/local/etc/snort.conf -u nobody -g nobody -l /tmp 12099: execve("/usr/sbin/chroot", 0xFFBEFB74, 0xFFBEFBB4) argc = 15 12099: argv: chroot /export/home/ids /usr/local/bin/snort -D -b -i 12099: bge1 -c /usr/local/etc/snort.conf -u nobody -g nobody -l 12099: /tmp 12099: envp: HOME=/ HZ=100 LC_COLLATE=en_US.ISO8859-15 12099: LC_CTYPE=en_US.ISO8859-15 LC_MESSAGES=C 12099: LC_MONETARY=en_US.ISO8859-15 LC_NUMERIC=en_US.ISO8859-15 12099: LC_TIME=en_US.ISO8859-15 LOGNAME=root MAIL=/var/mail/root 12099: PATH=/usr/sbin:/usr/bin SHELL=/sbin/sh TERM=xterm 12099: TZ=US/Eastern _INIT_NET_STRATEGY=none _INIT_PREV_LEVEL=S 12099: _INIT_RUN_LEVEL=3 _INIT_RUN_NPREV=0 _INIT_UTS_ISA=sparc 12099: _INIT_UTS_MACHINE=sun4u _INIT_UTS_NODENAME=webtkr5p 12099: _INIT_UTS_PLATFORM=SUNW,Sun-Fire-V210 _INIT_UTS_RELEASE=5.8 12099: _INIT_UTS_SYSNAME=SunOS _INIT_UTS_VERSION=Generic_108528-18 12099: HOSTTYPE=sun4 VENDOR=sun OSTYPE=solaris MACHTYPE=sparc 12099: SHLVL=1 PWD=/export/spare USER=root GROUP=other HOST=webtkr5p 12099: REMOTEHOST=172.27.100.74 12099: mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, \ MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFF3A0000 12099: resolvepath("/usr/lib/ld.so.1", "/usr/lib/ld.so.1", 1023) = 16 12099: open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT 12099: open("/usr/lib/libc.so.1", O_RDONLY) = 3 12099: fstat(3, 0xFFBEF29C) = 0 12099: d=0x00800005 i=484614 m=0100755 l=1 u=0 g=2 sz=1146168 12099: at = Apr 20 12:52:09 EDT 2005 [ 1114015929 ] 12099: mt = Aug 8 11:01:36 EDT 2002 [ 1028818896 ] 12099: ct = Dec 17 12:39:34 EST 2002 [ 1040146774 ] 12099: bsz=8192 blks=2256 fs=ufs 12099: mmap(0x00000000, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) \ = 0xFF390000 12099: mmap(0x00000000, 794624, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) \ = 0xFF280000 12099: mmap(0xFF33A000, 24652, PROT_READ|PROT_WRITE|PROT_EXEC, \ MAP_PRIVATE|MAP_FIXED, 3, 696320) = 0xFF33A000 12099: munmap(0xFF32A000, 65536) = 0 12099: memcntl(0xFF280000, 113332, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 12099: close(3) = 0 12099: open("/usr/lib/libdl.so.1", O_RDONLY) = 3 12099: fstat(3, 0xFFBEF29C) = 0 12099: d=0x00800005 i=484626 m=0100755 l=1 u=0 g=2 sz=5296 12099: at = Apr 20 12:52:09 EDT 2005 [ 1114015929 ] 12099: mt = Sep 6 17:41:11 EDT 2002 [ 1031348471 ] 12099: ct = Dec 17 12:39:34 EST 2002 [ 1040146774 ] 12099: bsz=8192 blks=12 fs=ufs 12099: mmap(0xFF390000, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, \ 0) = 0xFF390000 12099: close(3) = 0 12099: open("/usr/platform/SUNW,Sun-Fire-V210/lib/libc_psr.so.1", O_RDONLY) = 3 12099: fstat(3, 0xFFBEF12C) = 0 12099: d=0x00800005 i=377643 m=0100755 l=1 u=0 g=2 sz=4852 12099: at = Apr 20 12:52:09 EDT 2005 [ 1114015929 ] 12099: mt = Nov 12 18:51:58 EST 2002 [ 1037145118 ] 12099: ct = Dec 17 12:36:16 EST 2002 [ 1040146576 ] 12099: bsz=8192 blks=10 fs=ufs 12099: mmap(0x00000000, 8192, PROT_READ|PROT_EXEC, \ MAP_PRIVATE, 3, 0) = 0xFF380000 12099: close(3) = 0 12099: getuid() = 0  12099: chroot("/export/home/ids") = 0 12099: chdir("/") = 0 12099: execve("/usr/local/bin/snort", 0xFFBEFB7C, 0xFFBEFBB4) argc = 13 12099: argv: /usr/local/bin/snort -D -b -i bge1 -c 12099: /usr/local/etc/snort.conf -u nobody -g nobody -l /tmp 12099: envp: HOME=/ HZ=100 LC_COLLATE=en_US.ISO8859-15 12099: LC_CTYPE=en_US.ISO8859-15 LC_MESSAGES=C 12099: LC_MONETARY=en_US.ISO8859-15 LC_NUMERIC=en_US.ISO8859-15 12099: LC_TIME=en_US.ISO8859-15 LOGNAME=root MAIL=/var/mail/root 12099: PATH=/usr/sbin:/usr/bin SHELL=/sbin/sh TERM=xterm 12099: TZ=US/Eastern _INIT_NET_STRATEGY=none _INIT_PREV_LEVEL=S 12099: _INIT_RUN_LEVEL=3 _INIT_RUN_NPREV=0 _INIT_UTS_ISA=sparc 12099: _INIT_UTS_MACHINE=sun4u _INIT_UTS_NODENAME=webtkr5p 12099: _INIT_UTS_PLATFORM=SUNW,Sun-Fire-V210 _INIT_UTS_RELEASE=5.8 12099: _INIT_UTS_SYSNAME=SunOS _INIT_UTS_VERSION=Generic_108528-18 12099: HOSTTYPE=sun4 VENDOR=sun OSTYPE=solaris MACHTYPE=sparc 12099: SHLVL=1 PWD=/export/spare USER=root GROUP=other HOST=webtkr5p 12099: REMOTEHOST=172.27.100.74 12099: mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, \ MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFF3A0000 12099: resolvepath("/usr/lib/ld.so.1", "/usr/lib/ld.so.1", 1023) = 16 12099: open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT 12099: open("/usr/local/lib/libpcre.so.0", O_RDONLY) Err#2 ENOENT 12099: open("/usr/lib/libpcre.so.0", O_RDONLY) Err#2 ENOENT ld.so.1: /usr/local/bin/snort: fatal: libpcre.so.0: open failed: No such \ file or directory 12099: write(2, " l d . s o . 1 : / u s".., 91) = 91 12099: getpid() = 12099  12099: *** process killed ***
Above in red, it looks like the attempt to open libpcre.so.0 ending in a Err#2 ENOENT, was critical. Note that not all Err# after a system call are critical, however. If you notice right above it an open of /var/ld/ld.config also failed with Err#2 ENOENT, but was not critical. Learning which is critical and which isn't, comes with using truss more. As a guideline, just check the error with grep, as below, and assess whether that sounds like the problem or not. It seems I also forgot a file other than the 3 mentioned above. We will copy libpcre.so.0 from our compiler system to /export/home/ids/usr/lib. Notice it checks for /usr/local/lib/libpcre.so.0 and /usr/lib/libpcre.so.0, so placing it in either would be ok. Let's give it another try:
# grep ENOENT /usr/include/sys/errno.h #define ENOENT 2 /* No such file or directory */ # truss -fae -vall chroot /export/home/ids /usr/local/bin/snort -D -b -i bge1 \ -c /usr/local/etc/snort.conf -u nobody -g nobody -l /tmp begin snipped ... 12330: dat: maxlen=1280 len=108 buf=0xFFBEC728: " A p r 2 0 1".. 12330: open("/var/run/syslog_door", O_RDONLY) Err#2 ENOENT 12330: brk(0x001ED350) = 0 12330: brk(0x001EF350) = 0 12330: brk(0x001EF350) = 0 12330: brk(0x001FF350) = 0 12330: brk(0x001FF350) = 0 12330: brk(0x0020F350) = 0 12330: open("/usr/local/etc/unicode.map", O_RDONLY) Err#2 ENOENT 12330: fstat(4, 0xFFBEC8D0) = 0 12330: d=0x00800000 i=21702 m=0020666 l=1 u=0 g=3 rdev=0x00540000 12330: at = Apr 20 12:50:38 EDT 2005 [ 1114015838 ] 12330: mt = Apr 20 12:50:38 EDT 2005 [ 1114015838 ] 12330: ct = Apr 20 12:50:38 EDT 2005 [ 1114015838 ] 12330: bsz=8192 blks=0 fs=ufs 12330: time() = 1114032987 12330: putmsg(4, 0xFFBEBF88, 0xFFBEBF7C, 0) = 0 12330: ctl: maxlen=24 len=24 buf=0xFFBEC8B8: "\0\0\0\0\0\0\010".. 12330: dat: maxlen=1280 len=176 buf=0xFFBEC3B8: " A p r 2 0 1".. 12330: open("/var/run/syslog_door", O_RDONLY) Err#2 ENOENT 12330: llseek(0, 0, SEEK_CUR) = 0 12330: llseek(5, 0xFFFFFFFFFFFFE267, SEEK_CUR) = 615 12330: _exit(1)
Above in red, we copied libpcre.so.0 but we still don't have unicode.map. Let's do that and try again.
# truss -fae -vall chroot /export/home/ids /usr/local/bin/snort -D -b \ -i bge1 -c /usr/local/etc/snort.conf -u nobody -g nobody -l /tmp begin snipped ... 12236: sysinfo(SI_SRPC_DOMAIN, "", 256) = 1 12236: setgroups(1, 0x001E6A60) = 0 12236: 60001 12236: setuid(60001) = 0 12236: open("/tmp/alert", O_WRONLY|O_APPEND|O_CREAT, 0666) Err#13 EACCES 12236: fstat(4, 0xFFBEED18) = 0 12236: d=0x00800000 i=21702 m=0020666 l=1 u=0 g=3 rdev=0x00540000 12236: at = Apr 20 12:50:38 EDT 2005 [ 1114015838 ] 12236: mt = Apr 20 12:50:38 EDT 2005 [ 1114015838 ] 12236: ct = Apr 20 12:50:38 EDT 2005 [ 1114015838 ] 12236: bsz=8192 blks=0 fs=ufs 12236: time() = 1114030575 12236: putmsg(4, 0xFFBEE3D0, 0xFFBEE3C4, 0) = 0 12236: ctl: maxlen=24 len=24 buf=0xFFBEED00: "\0\0\0\0\0\0\010".. 12236: dat: maxlen=1280 len=141 buf=0xFFBEE800: " A p r 2 0 1".. 12236: open("/var/run/syslog_door", O_RDONLY) Err#2 ENOENT 12236: llseek(0, 0, SEEK_CUR) = 0 12236: llseek(6, 0xFFFFFFFFFFFFE3C7, SEEK_CUR) = 9159 12236: _exit(1)
We have all the files now, so now what's wrong? Using truss and a little patience it is clear we can see what is missing from our chrooted environment. Without truss or strace [ref 3] this would be a difficult task. The line in red above looks like our problem, but once again, what does it mean? It tries to open /tmp/alert and receives an Err #13 EACCES response. Let's see what that means, using grep below. You will quickly discover that it means permission denied. A listing on the directory shows that only root can write in /export/home/ids/tmp (permission: drwxr-xr-x = 755 for user root and group other), but our Snort process is running as the user, nobody and the (-l /tmp) on our command line tries to create log files in /tmp of the chroot. To fix this, let's set the chrooted /tmp directory to the same permission as the system directory /tmp using chmod. That should take care of this error and our Snort process should finally be running in the chroot (full listing).
# grep EACCES /usr/include/sys/errno.h #define EACCES 13 /* Permission denied */ ls -ld /export/home/ids/tmp drwxr-xr-x 2 root other 512 Apr 21 17:26 /export/home/ids/tmp # chmod 1777 tmp
The following scripts automate the startup of Snort and the plumbing of the non-IP'ed interfaces. The /etc/rc3.d/S99sniff script is needed if using and interface without an IP address. Only /etc/rc3.d/S99snort is needed if automating Snort for an interface with an IP address.
7.1 A look at utilizing the chroot method integrated into snort
Snort also comes with its own chrooting method after intialization. Using this technique does not require a chrooted environment, it only requires a chroot path. I'm sure you are then asking why one would go through all the work needed to build a chrooted environment? Well, let's take a look at Snort's chrooting method. This is the command line we'll use to start it:
# /export/home/ids/usr/local/bin/snort -D -b -i bge1 \ -c /export/home/ids/usr/local/etc/snort.conf -u nobody -g nobody \ -l /export/home/ids/tmp -t /export/home/ids
This will fail to start Snort because the Snort process now needs to initialize before it can chroot. In other words, it needs the libraries outside the chroot. Therefore this works after copying the library libpcre.so.0 to the system's /usr/lib or /usr/local/lib directory.
After fixing this, Snort starts. However, the portscan file is being written to /tmp/portscan instead of /export/home/ids/tmp/portscan, the chroot. This can be fixed by changing the snort.conf entry (preprocessor portscan: $INTERNAL 5 4 /tmp/portscan) to /export/home/ids/tmp/portscan. Please note that all these inconsistencies with reading some things inside the chroot and some outside the chroot lead this author to believe that the chroot method built into Snort has allowed the running daemon to see both inside and outside the chroot environment. So the question is, can an attack on the process do the same? This method still seems vulnerable to buffer overflows because Snort is still really running in the real file system. This means a hacker could still possibly work their way back to executing a shell or other malicious code. The chrooted environment as presented in this article requires more work to setup, but pays off by being more secure because Snort does not know its running inside a bubble. This way, one can be sure that Snort is definitely limited to the contents of the chroot which does not contain a shell and thus results in the daemon being more trustworthy.
The concept of chrooting is not new and everything done in this article using Solaris works very similarly on other flavors of Unix (Linux, BSD, AIX, and so on). The library and device requirements will be different but everything else is pretty much the same. Let's briefly look at the steps we've taken to secure our Snort IDS. We created a minimal clone of the file system and the devices that would sustain the Snort daemon. Next, we ran the Snort daemon as "nobody," a minimal privilege user. Finally, when possible we ran it on an interface without an IP address. What we built, in the end, is a secure IDS that adds minimal risk to the network perimeter. It also serves as a stealth syslog server for our Cisco border devices. The Cisco units require a syslog server entry pointing to a fake IP on the network and a static ARP entry binding the fake IP with the MAC address of the Snort interface. On the Snort machine, you add a simple Snort config, logging for traffic destined for the fake IP to a specified file. This file will contain your Cisco syslog messages. This IDS is also used as a sniffer if needed for troubleshooting on any of the connected networks.
This article originally appeared on SecurityFocus.com -- reproduction in whole or in part is not allowed without expressed written consent.