| Scenario: it's Friday afternoon. You've just finished a three-month implementation of the industry leading (and expensive) firewall architecture designed to secure your global perimeter networks and business partner connections. You spent even more time formulating the security policies that will run on those firewalls. You've incorporated all of the best practices for firewall management into your standard operating procedures and are about to leave early for a well-deserved weekend getaway when your phone rings. It's the operations desk. A virus is in the early stages of total network bandwidth domination and the mail system is on overload. You have the administrators follow your procedures and block outbound mail then follow the usual cleanup and recovery procedures. When the crisis is over and the post mortem is complete you discover that your weekend was ruined and your bulletproof firewall security subverted when Steve in accounting simply opened up a single mail message.
Chances are good that you've faced a similar situation if you've been a security professional within the past few years. The intense rise of spam along with viruses, worms and other malicious code (i.e. malware) has made operating a secure, efficient IT environment a real challenge. This article provides you with the tools and methodology for creating and sustaining a secure and robust mail environment, with a focus on defending an Exchange/Outlook based e-mail system.
Defining the battlefield
Large portions of many mid- to large-size corporations have based their core IT infrastructure on Microsoft technologies. This includes the Microsoft Exchange Server and Outlook client. The combination of these two products enables (among other things) centralized address books, integration with Windows accounts and passwords, and efficient calendaring and scheduling. For smaller environments, Microsoft makes it very easy to obtain, install and configure a basic mail system with it's Small Business Server product line. For larger environments, there are significant benefits from having all the aforementioned features integrated on a global scale. However, such widespread deployment of common components helps to create, in the case of malware, the avenue of attack. It's much simpler to create malicious code if you only have to worry about one flavor of operating system or mail client or office suite, especially one that facilitates programmatic access either through built-in mechanisms (e.g. macros) or inherent security flaws (e.g. buffer overflows). It is this very environment, which was designed to facilitate easy administration, global collaboration and information exchange, that requires the most protection.
Even if you're not operating in an Exchange/Outlook environment there is another enemy in the e-mail battle: spam, or unwanted, unsolicited e-mail. According to InternetNews.com [ref 1], 50-60% of all e-mail in your inbox is comprised of the stuff (they even break it down into categories) with no end in sight to the growth. All of the anti-spam efforts to date have had little-to-no effect and company mail systems are barely keeping up.
So, no matter what environment you choose, you have a battle to fight on two fronts.
A plan of attack
As is the case with all IT security initiatives, it is absolutely necessary to define policy before infrastructure pieces are deployed. Without a well-defined policy, engineers, administrators and operations personnel will each have different ideas of what they are supposed to do and users will have no idea what to expect. Furthermore, a good policy will be technology-neutral, enabling you to swap out components as time goes by (e.g. replace sendmail+MIMEDefang with qmail+qmail-scanner+ClamAV) without negatively impacting the process.
For a spam/malware gateway, you'll need to define what you want to filter, how you want to filter and be able to explain (at a high level) the process so end users will understand what to expect from the environment. To help define your policy, you'll need to ask (and answer!) questions like:
- Do we want to do anti-virus checks on incoming mail?
- Do we want to do anti-virus checks on outgoing mail?
- Do we want to filter/remove all incoming spam or tag it as spam and let the e-mail clients or Exchange post-process the messages?
- Do we want to filter/remove all attachments within certain categories (e.g. remove all executables)?
- Do we want to do content-filtering (e.g. non-spam messages that meet certain criteria)?
- Do we want to maintain a list of malware and filter/remove messages based on that list or rely on the anti-virus and spam-detection software alone?
- Do we want to generate, review and archive reports for the filtering gateway?
The above list is just a start. Rules that govern spam/malware e-mail handling may be spread across multiple security policies of an organization. A good place to start for policy development is the SANS Security Policy Project [ref 2]. They have templates and ideas you can reuse plus links to other resources for policy development.
There are other, more technical items to consider as well, such as:
- Should all spam/malware flagged messages be removed in-stream or be tagged and delivered to the end-users?
- Should users be allowed to tag items as spam and be able to add messages to the spam pool?
- Should users be able to maintain their own black-/white-lists?
The answers to questions like these lie both in the capabilities of the solution as well as the ability/preferences of the IT staff and users in the organization.
Choosing your pedigree
Dogs have been used in warfare for almost as far back as people have been fighting [ref 3]. They have been guards, scouts, sentries, couriers, messengers and fighters on the front lines, much like the tools you would use in fighting the spam/malware war. There are many different ways to protect your mail environment from attack, each having different costs and implementation complexities. The tools presented here are open source and run on almost every flavor of *nix including Linux, Solaris, OS X and FreeBSD/OpenBSD. Choosing open source tools provides a great deal of flexibility and makes it much easier to mix-and-match solutions as environments change. It also generally costs less (usually just the hardware) to implement.
Part one of this article will use the following tools for your spam/malware fighting toolbox:
ref 4
This oft-maligned MTA (mail transfer agent) has been a staple of Internet mail for two decades. It is powerful and complex, but is well suited as a spam/malware mail filtering gateway. A big benefit to using it is that you can easily obtain commercial support from Sendmail Inc. [ref 5] if your management requires a company to fall back on for support.
An especially useful feature of sendmail, which makes it even more ideal as a filtering agent, is the milter [ref 6] API built into the software. This (m)ail f(ilter) protocol/library lets you incorporate other components into your e-mail server which can monitor the mail exchange and perform actions or modify/redirect the output.
MIMEDefang [ref 7]
This tool is a sendmail milter that provides a facility to customize rules to perform many tasks such as virus blocking, spam blocking/removal, attachment processing (including blocking/deleting attachments) and much more. It requires Perl and a few additional Perl modules to install and run and supports filter writing and extension development in Perl code, which provides a great deal of power and flexibility when you start implementing your e-mail policies.
SpamAssassin [ref 8]
SpamAssassin is one of the most well known spam fighting tools in the open source world. It is regularly maintained and updated, works for individual user (via procmail) as well as in site-wide configurations, is highly customizable and integrates with many different mail servers. MIMEDefang uses the SpamAssassin Perl library to do the dirty work of spam detection.
Next time, in part two of this article we'll look at an alternative approach, using Qmail, Qmail-Scanner, Clam Anti-Virus and SpamAssassin to accomplish the same thing.
I highly recommend working in a FreeBSD/OpenBSD [ref 9] environment when trying to setup the gateway. All of the tools being used in this article are in the ports tree and installing them (and automatically installing their various dependencies) is as simple as going to their ports directory and typing "make install clean". Everything runs equally well under Linux, but you'll end up doing a bit more work to download the software and get it compiled/installed (using a Debian-based distribution might ease things a bit, though). Furthermore, OpenBSD's main priority is security, which isn't a bad feature to have in an e-mail gateway.
The pieces are set
Now that the tools have been chosen, it's time to put them together. Since we're dealing with sendmail, one assumption being made is a certain bit of proficiency with the tool. You should have experience compiling and performing basic configuration with sendmail before proceeding. It is also highly recommended that you do initial testing with the spam/malware gateway in a lab environment before opening up your creation to folks on the Internet.
The first thing to do is ensure that you've compiled sendmail with milter support. You can check the output of "/usr/lib/sendmail -bv -d0.1" and verify that it contains the word "MILTER". If it does not already have milter support, then add the following lines to "devtools/Site/site.config.m4" (under the main sendmail directory tree):
dnl Milter
APPENDDEF('conf_sendmail_ENVDEF', '-DMILTER')
And re-compile sendmail.
If you've never used CPAN [ref 10] before, now would be a good time to get to know it, since it will save you a great deal of time if you ever need to install Perl modules on any system. For an initial configuration, do the following:
(this assumes you have Perl installed). It will lead you through configuration options for setting up access to the CPAN library. A big time-saver is to choose "follow" when it asks about how to install dependent modules. Once CPAN access is configured, you can now install any Perl module (like, say, SpamAssassin) with one line from a shell prompt:
perl -MCPAN -e "install Mail::SpamAssassin"
and let it do all the work of downloading, compiling (if necessary) and installing.
With sendmail prepped, the MIMEDefang installation is the next course of action. If you're not doing the automatic build via the *BSD ports tree, then I highly recommend starting with The MIMEDEfang HOWTO [ref 11]. It covers all the detail on gathering the necessary components (required and optional), basic sendmail configuration. The HOWTO lists the pre-requisite and optional Perl modules necessary for the install. I suggest using the automated CPAN installation method outlined above to install them.
To make sendmail act as a gateway for your Exchange environment, you'll need to (at a minimum) add mailertable support and configure it to handle relaying for your domain(s). The sendmail mailertable feature enables you - in this case - to forward mail destined for specified domains to alternate servers. First, edit the m4 file and add:
FEATURE('mailertable')dnl
to the configuration if it is not already there and re-generate your sendmail configuration (cf) file.
For relay control, it's also a good idea to enable the sendmail access_db:
The mailertable file is usually installed as /etc/mail/mailertable and is a plaintext file with the format of:
example.org esmtp:[exchangeserver.internal.example.org]
Each line in the file says "mail for this domain is to be sent to the following host for processing". It's a good idea to make sure your split DNS configuration is working properly to ensure seamless mail delivery. Next, generate the mailertable database with the makemap command:
makemap hash /etc/mail/mailertable.db < /etc/mail/mailertable
(assuming you're using the hash db format).
You'll now need to let sendmail know you can take and relay mail for the domain by editing and generating the access_db database. First, edit /usr/mail/access and add your domains:
and generate the /etc/mail/access.db file via the makemap command.
Depending on your particular sendmail setup, you may have to modify /etc/mail/local-host-names, /etc/mail/relay-domains and other configuration files. Again, sendmail's biggest strength and biggest weakness lies in it's flexibility and complexity.
The final step is to configure DNS so that the new spam/malware filtering gateway handles all the mail by adding something like the following to the zone file:
example.org IN MX 5 defender.domain.org
defender.example.org IN A 192.168.10.10
Once you configure your Exchange server to allow the receipt of mail from your sendmail gateway you will have all the components in place and can begin implementing your e-mail security policy.
The pieces are moving
The bulk of our mail filtering will focus on the MIMEDefang milter since it sits in-stream and handles all of the processing. As each message enters the environment, MIMEDefang breaks the message into parts in a temporary directory and runs its filtering routines (/etc/mail/mimedefang.pl -> /etc/mail/mimedefang-filter.pl). The bulk of your work will be done in customizing the processing performed in the filter functions. All customization is done within perl subroutines.
MIMEDefang's filters operate at different points during the e-mail conversation. The sendmail milter API exposes different portions of the conversation to MIMEDefang and your filter customizations (additions to the subroutines) can operate during any phase. You can perform custom relay checks with filter_relay, do additional sender checks with filter_sender, handle custom recipient verification with filter_recipient and do full message processing (during the milter scan phase) with filter_begin, filter, filter_multipart and filter_end. The rest of the article is primarily concerned with the scan phase of filter operations. The other filters are covered in the MIMEDefang FAQ and documentation.
As stated previously, MIMEDefang is called when a message is received. At that time, it creates a temporary directory and breaks the message into parts:
- INPUTMSG - the complete message with headers
- HEADERS - just the message headers (each on a separate line)
- COMMANDS - a list of single-letter commands
If there are attachments, other files are also created for them.
When the scan phase initiates, MIMEDefang parses the mail message and calls filter_begin. As each MIME part is processed, filter or filter_multipart is called. Finally, at the end of processing, filter_end is executed.
When filter_begin is called, you can do any type of processing on the files MIMEDefang created, as well as setup global variables for flags. The real run starts when filter is called. Here's an example filter:
sub filter { my ($entity, $filename, $extension, $type) = @_ ; return action_accept if ($ext eq '.txt') ; return action_drop_with_warning('No compressed files!') if ($ext eq '.zip'); if ($Subject =~ /viagra/i) { action_discard() ; } # if # } # filter() #
MIMEDefang passes the MIME Entity, the filename, the extension and the MIME type to the filter subroutine. It also defines some global variables that it creates from the information from the message such as the subject line ($Subject), recipients array (@Recipients), relay address ($RelayAddr) and others. In the example filter (NOTE: it's not very useful in production) above, it first checks to see if the extension is a text file, and if it is, the subroutine lets it through via the action_accept() event. If that test failed, it goes on to the next one and checks to see if the extension is a ZIP file. If it is, the user is notified (within the message) and the attachment is removed. Lastly, if the subject line contains the word "viagra" (in any case), the message is accepted (the remote server receives a 2## SMTP success code), but the user never sees it.
MIMEDefang has many routines which enable you to quarantine messages or parts of messages, reject the connection (send a 5## SMTP failure code to the remote server), separate attachments into a web-accessible directory, call external routines and more. There are also built-in routines which utilize additional Perl modules for virus checking. Part of the process can also involve spam checking and that's where SpamAssassin joins in the fun.
SpamAssassin performs tests on message headers, the message body, RBL checks and statistical analysis based on reference spam (Bayesian analysis). Each test generates a value which is added to an overall score (or weight) that is assigned to the message. When the score reaches a certain threshold (site or user-definable), the message is tagged as spam. SpamAssassin checks are generally run in the filter_end stage and MIMEDefang has built-in access to the SpamAssassin module via the spam_assassin_check() routine. A simple filter would be:
sub filter_end { my ($entity) = @_ ; my ($score, $thresh, $names, $report) = spam_assassin_check() if ($score >= 5*$thresh) { action_bounce("SPAM Rejected"); return; } # if # if ($score > $thresh) { action_change_header("Subject", "[SPAM] $Subject") ; } # if # } # filter_end() #
The "spam filter" runs SpamAssassin and then looks to see if the score for the current message is greater than or equal to five times the threshold (again, for example purposes only). If it is, it bounces the message. If the score is just greater than the threshold, the Subject line is altered to indicate that the message has been tagged as spam. This would allow for further downstream processing. You can define the site-wide SpamAssassin preferences in /etc/mail/spamassassin/local.cf. For example, the following local.cf:
required_hits 10
blacklist_from *.msn.com
score NIGERIAN_BODY1 10
score NIGERIAN_BODY2 10
score HTML_IMAGE_ONLY 5
states that it takes ten points to be considered spam, no mail from msn.com is accepted and that ten points should be added if the common Nigerian scam body texts are found. Additionally five points should be added if the message is just an HTML file with an IMG tag. Running:
perldoc Mail::SpamAssassin::Conf
will provide you with a full list of configurable options. You will need to spend a few days (weeks) fine tuning your spam settings to get it to the point where spam is stopped pretty well. One major caveat to this setup is that SpamAssassin/MIMEDefang will be running as the "defang" user. All additions to the Bayesian database (via sa-learn) must be done as this user. To get a good feel for how the Bayesian setup works, I strongly recommend running SpamAssassin in a test bed via procmail and a local Unix mail account (you can just feed spam to a test machine directed at a test user account). You'll see how SpamAssassin tags mail. When it finds spam, it adds it to the Bayesian database automatically, but you can also "feed" it spam by setting aside a mail folder for spam, tossing messages SpamAssassin missed into that folder and then running the sa-learn program on that mail folder.
If this all sounds like a daunting task (especially if your Perl skills are a tad rusty) don't worry. The MIMEDefang HOWTO provides a sample filter which performs virus checks, filename extension checks, and spam checks that you can do basic customization to. You should also check out David Skoll's LISA slides [ref 12] from his presentation on MIMEDefang and SpamAssassin.
Concluding part one
In this article we've look at the combination of Sendmail, MIMEDefang and SpamAssassin as tools to build our MTA. However, some people prefer an alternate approach. In part two we look at the same problem from another angle, through use of Qmail, Qmail-Scanner, Clam Anti-Virus and once again, SpamAssassin to accomplish the same job using different tools.
|