qmail Anti-Spam HOWTO
Version 0.32 by Chris Hardie, part of his qmail stuff ~ chris@summersault.com


Table of Contents


I. Introduction
Click to receive email
when this page changes
Powered by NetMind
This document discusses anti-spam philosophies from a variety of perspectives and provides information about available options for dealing with spam.

II. Who Should Be Reading this Document

This document could be useful for

  • UNIX system administrators
  • qmail administrators
  • System Use/Abuse Policy Makers
  • Casual users of e-mail concerned about SPAM

III. General Issues

Spam is defined here as unsolicited commercial e-mail, usually sent in bulk. In other words, spam is simply electronic junk mail. Dealing with spam is, at best, a very difficult task. This is mostly true because spammers have a wide array of tools and circumstances available to them that make it easy for them to send you mail but difficult for you to communicate back with them or any authority over them. Spam is also difficult to deal with because it almost always comes in under the guise of being a normal e-mail message. No amount of technology can automatically decide what content is undesirable to you, but there are many ways to use technology to reduce the amount of unwanted e-mail you or your users receive.

For a more thorough explanation of the issues here, I recommend reading the IETF Anti Spam Recomendations. David E. Sorkin has produced an excellent document called Technical and Legal Approaches to Unsolicited Electronic Mail.

IV. Specific Issues of Policy

Anyone dealing with spam prevention will have to make definite and impacting decisions about the following issues:
  • Is the prevention of spam worth the time and resources required to reach a given level of spam reduction?
  • Is the prevention of spam the responsibility of a system administrator or the responsibility of the end user?
  • Should e-mail identified as potential spam be flatly rejected, or just tagged as spam and routed accordingly?
  • Should system administrators (yours or anyone else's) who have misconfigured their systems be held responsible for any problems that result?
  • Should you reject e-mail messages that are legitimate in content but that do not conform to known and accepted standards?
  • Should you accept for delivery mail that does not have valid reply information (either in the envelope or From address)?
  • What criteria should be met before an individual or ISP is justifiably classified as "spam-friendly"?

V. Basic Things You Can do to Prevent/Reduce Spam

  • Don't use a dot-qmail-default file in a lazy way
    The dot-qmail-default file (e.g. '.qmail-default') makes any mailbox name on your system valid, whether or not it actually exists. While it can be useful as a catch-all for setups where there is a possibility that the sender would misspell the address, it often leads to an increased volume of spam from spammers who use made up aliases. Typical examples of this are "sales@domain.com", "webmaster@domain.com", and so on. Note that the dot-qmail-default file can actually be useful in fighting spam, if you set it up to process mail sent to invalid addresses and then reject the mail with useful error messages.

  • Report any spam that you do get.
    Services like SpamCOP make this straightforward and efficient. See the resources section below for links.

  • Educate
    Educate your users, friends, and family about spam, why it is (or is not) worth fighting, and what they can do. Spammers are successful in getting mail through because the typical end user doesn't understand the technical issues enough to make decisions about how to fight back.

  • Make sure your system is properly configured
    All of the hosts on your system should have proper hostnames and corresponding IP addresses. Other systems should be able to properly resolve your hostname and IP address.
    Your mailer should issue standard error and warning messages when appropriate, and you should be aware of its configuration regarding rejection of messages. Adhering to standards is a good thing.

VI. Commonly Held Views About Spam

The following are some commonly held views about spam. They are presented here to provide you with some perspective about the issues involved in preventing spam. As you will note, the disparity between the different views makes it even more difficult to develop a commonly acceptable solution to preventing and reducing spam.

Commonly Held View: "Spam Can't Be Stopped"
This view holds that, because of the seemingly impossible task of accurately identifying e-mail messages as spam, and the difficulty in holding spammers responsible for their actions once they have been identified, efforts at stopping spam are a waste of time and resources, and can even result in losing legitimate mail.

People holding this view usually think that the tolerable ratio of lost legitimate e-mail to rejected spam is zero.

Commonly Held View: "Spam Prevention is the Responsibility of the End-User"
This view holds that, because the identification of spam is so difficult, the reduction of spam is the responsibility of the end user. This view is usually held by system administrators because of the liability issues in rejecting any e-mail on a system-wide basis. Many also think that user-level spam prevention is more effective because it allows the user to have more control over the kinds of messages that should be automatically blocked or should automatically get through. People holding this view sometimes acknowledge that, believe it or not, some people actually like receiving commercial e-mail, and tend to believe that rejecting any e-mail without obtaining the permission of the intended recipient is an invasion of privacy.

People holding this view usually think that the tolerable ratio of lost legitimate e-mail to rejected spam is a matter of personal opinion to be determined by the user affected.

Commonly Held View: "Spam Prevention is the Responsibility of the System Administrator"
This view holds that, because there are some tools available for rejecting spam at the system level, and because the prevention of spam is a worthwhile effort in the name of conserving system resources, spam prevention is the responsibility of the person or entity maintaining the mail server. This view is often held by administrative bodies in the context of reducing the amount of time their staff (both sysadmins and end-users) spend dealing with unwanted spam. It is also often held by system administrators who have a particular dislike for spammers who are, as they perceive it, misusing or abusing their systems for unwanted commercial e-mail.

This view also affects system administrators who maintain systems that are configured in such a manner that spammers can more easily use that system for spamming. The most common example of this is when a system is classified as an "open relay", allowing mail to pass through it that is neither to or from a user on the system. Systems like this are sometimes seen as facilitating spam in a significant manner, and that it is the responsibility of the system administrators to alter that configuration in the name of fighting spam.

People holding this view usually think that the tolerable ratio of lost legitimate e-mail to rejected spam is acceptable at low levels. They are usually willing to accept that some legitimate e-mails will be rejected because of a misconfiguration or error on the part of another system administrator.

Variations
Within each of the above views, there are many variations. Often the variations involve decisions about the issues stated above in "Specific Issues". Some common variations:

  • Some people believe that any messages from senders who have been listed in one of the various "black hole lists" (see resources section for links) should be rejected without exception. Others sometime believe that these black hole lists are not always fair or just in their criteria for inclusion, and that depending on these lists would result in too many legitimate e-mails being rejected. Even others disagree with the use of the black hole lists because of grievances with the methodology the owners use to develop and maintain the lists.
  • Some people believe that messages not conforming to known standards for mail delivery should be rejected or identified as potential spam. The most common example of this involves "From" headers or "envelope" addresses. These addresses are necessary to handle the "bouncing" of messages and for any sort of reply. For a variety of reasons, many spam messages do not have valid From or envelope headers. So, while some believe that any such message should be rejected, others hold that there are too many exceptions where these headers might be invalid but the content or intent of the message is legitimate.

VII. Options for Individual Users

This section discusses spam prevention options for individual users. It assumes that you have the access necessary to modify your personal mail setup and that you have basic knowledge of software configuration procedures. If either of the above is not true, you should ask your system administrator for assistance.

The spam-prevention options for individual users at first seem limited, given that if an e-mail has made its way to you, it has been successfully accepted for delivery by your system, and is now your responsibility for filtering. However, filtering mail (also sometimes called "rule-based delivery") can be a powerful tool (in spam-prevention and other arenas), and is probably your best option.

The most commonly used program for filtering mail under UN*X is procmail, a recipe-based set of scripts that will route, reject, forward, and modify your mail based on criteria you specify. Tools for filtering mail under other platforms (Windows, Mac, etc) are numerous, and are usually tied in to whatever mail client you use. Modern versions of Eudora, Outlook, Netscape Mail, etc. all have filtering capabilities.

Using Realtime Third-Party Black-hole Lists

With the basic procmail filter below (to be used in your .procmailrc file) and two simple programs rblcheck and origip.pl, you can block/route mail from senders who have been listed in various realtime black-hole lists.

MAILDIR=$HOME/mail
SPAMFOLDER=$MAILDIR/junkmail
LOGFILE=$MAILDIR/log


:0h
TCPREMOTEIP=| origip.pl
LOG="Remote IP: \"$TCPREMOTEIP\""

:0
* !^From.*myfriend@domain.com
* ! ? if [ -n "$TCPREMOTEIP" ]; \
      then /usr/local/bin/rblcheck -q "$TCPREMOTEIP"; fi
{
        LOGABSTRACT=all
        LOG="Filter: RBL-filtered address: \"$TCPREMOTEIP\""
        :0:
        $SPAMFOLDER
}

This recipe will filter any mail with IP addresses on the ORBS (Open Relay Blocking System), RBL (Realtime Blackhole list), OR DUL (Dial-up User List). (See Other Resources below for links).

Does anyone know of RBL-checking tools for Windows, Mac, etc that one could use in conjunction with client software filters?

Using Your Own Black-hole Lists

There are several different ways to set up procmail to use a locally maintained list of potential spammers to automatically filter spam, and a list of known good senders to automatically safeguard legitimate mail.

Probably the best place to start here is to use SpamBouncer, an extensive set of recipes for procmail designed for the novice procmail user. To use SpamBouncer, just follow the instructions provided with the software. After SpamBouncer is installed, you can modify the lists of good and bad senders to meet your needs.

Using Tagged Messages and Special/Dedicated E-mail Addresses.
Tagged messages can mean several different things.

  • Requiring people sending you e-mail to verify the address they send from before the mail will go through to you. While it usually requires they do this only once in a lifetime of their e-mail address, it can be a hassle for "new contacts."
  • Allowing persons sending mail from known spam domains to bypass spam filters by including a secret passphrase.
  • Using e-mail addresses that "expire" or are dated when you send mail, so that responses or new mail to that address (especially those posted on newsgroups, websites, etc) are protected from spam.

Here are some tagged message resources:

  • SpamBouncer has a mechanism for tagged messenging
  • Thomas Erskine wrote tms (Tagged Message Sender), which also does this.
  • Jason R. Mastaler has taken tms and extended the project into Tagged Message Delivery Agent.
  • Brett Randall has developed an avoidance technique especially for users participating in usenet and mailing list discussions.
VIII. Options for qmail Administrators

This section discusses options for system administrators who want to implement anti-spam mechanisms at the system-wide level. Please note that you should resolve the specific issues listed above before implementing any of these solutions, and that you should always notify your users of any changes to the system that affect the mail they do or don't receive.

Rejecting SMTP connections at the network level from hosts with bad DNS
It is becoming common for the default installation of many Unix operating systems like FreeBSD and Linux to include a mechanism to block network traffic based on certain criteria, commonly referred to "host-based access control" and commonly implemented using the tcp_wrappers package. In some of these installations, network traffic from hostnames that do not map to valid IP addresses is blocked. While not an e-mail specific measure, this is one way to cut down on e-mail from hosts that have misconfigured their DNS, and therefore are thought by some to be more likely to be spam-friendly.

If you're using inetd, an example line in a FreeBSD /etc/hosts.allow is here:
ALL : PARANOID : RFC931 20 : deny

One can also achieve this using the ucspi-tcp package's tcpserver (now the recommended alternative to inetd), by enabling the "-p" option, for paranoid, e.g.

tcpserver -p -v -x/etc/tcp.smtp.cdb -u1007 -g1007 0 25 \
qmail-smtpd 2>&1

Using your SMTP daemon to reject "known" spammers
In the ucspi-tcp package there is the rblsmtpd package, an alternative to the usual qmail-smtpd, and works with any SMTP server that runs under tcpserver.

So, if you follow the Life With qmail Installation guide, and then update your supervise scripts accordingly, your /var/qmail/supervise/qmail-smtpd/run script looks like this:

#!/bin/sh
QMAILDUID=`id -u qmaild`
NOFILESGID=`id -g qmaild`
MAXSMTPD=`cat /var/qmail/control/concurrencyincoming`

exec /usr/local/bin/softlimit -m 2000000 \
/usr/local/bin/tcpserver -v -p -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \
-u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \
/usr/local/bin/rblsmtpd /var/qmail/bin/qmail-smtpd 2>&1

Note that the above is an updated version of the call to rblsmtpd; the previous version was only correct for older versions of daemontools, and is now deprecated. The upgrade is worth it.

If you're still using inetd, you can patch qmail to do about the same thing, or you can patch tcp_wrappers to use the RBL with software from SmallWorks.

If you want to use the ORBS database (or any other database with DNS access) in addition to the RBL, you can modify your rblsmtpd configuration with the "-r" option to do so.

rblsmtpd -rrelays.orbs.org -rrbl.maps.vix.com

Mike Silbersack says "If you wish to use the *.mail-abuse.org black-hole lists, you'll have to apply the patch to make rblsmptd work with A records"

Using qmail-smtpd to reject mail with invalid envelope or From headers
This area is a little blurry right now; it is the author's hope that readers will contribute their experiences here to improve the recommended options.

There are several patches out there that claim to make qmail reject messages with bad envelopes or From headers (i.e. if the envelope is blank or if the hostname in it doesn't have a valid DNS entry) or otherwise deal with suspected SPAM mail. Note that most of these patches are not featured on the qmail site and are therefore assumed to be "nonstandard".

  • Nagy Balazs wrote a patch to ensure that the domain name on the envelope sender is a valid DNS name. This ensures that you do not receive email which you cannot bounce, should that prove necessary.
  • Erwin Hoffman has written a patch for qmail-smtpd called SPAMCONTROL which improves qmail's filtering abilities and makes it RFC 2505 compliant. He and Noel Mistula also produced some scripts for filtering attachments and subject lines (something you can also do with procmail).
  • The folks at flame.org wrote another patch that performs various header checks and bounce/flagging functionality
  • Will Harris wrote a patch that allows you to use a new control file to specify Perl regular expressions to be used when checking the validity of the envelope sender.
  • Mark Delany wrote a patch for qmail 1.01 which lets you match the envelope sender against a regex and accept or reject the mail accordingly.
  • Jonathan McDowell has written an X-Spam-Warning header patch that adds warning headers for messages from senders in ORBS, RSS, RBL and DUL without the use of any external programs. This is useful if you want to allow your users to decide how to handle SPAM while tagging it as such at the system level.
  • Jay Soffian has a script that also adds warning headers, but uses the existing QMAILQUEUE patch instead of patching qmail source itself.
  • Chris Johnson wrote a patch to log attempted relay attempts

It should also be noted here that messages with recipient addresses in the form "user%hosta@hostb" are not going to be relayed through your system unless you have misconfigured something. See the qmail.faqts page on this issue for futher details.

Make it hard to spam from your system to the outside world

There are a variety of ways to make it difficult for your users to create spam. This is an important effort; while most of this document focuses on avoiding incoming spam, don't forget that a lot of incoming spam is generated because of overly lax mail sending policies.

  • Chris Johnson has written a patch for qmail called tarpit. Tarpitting is "the practice of inserting a small sleep in an SMTP session for each RCPT TO after some set number of RCPT TOs." This discourages a user from using a given system as a relay.

IX. Other Resources

Real-time Third-Party Blocking Solutions

Third Party Spam Reporting Services

General Related Mail and System Tools

Writings on Spam

Anti-Spam Manifestos and Organizations

X. History

  • April 28, 2001 - v0.31, added some new misc links
  • April 21, 2001 - version 0.30; added a new section of resources for administrators, updated some links, updated daemontools syntax to be current, added misc links
  • January 31, 2001 - minor update regarding -r syntax for rblsmtpd
  • December 16, 2000 - added section numbering; removed links to contributor's e-mail addresses (too much potential irony); added some new patches and software; expanded user section to include non-Unix-centric perspectives; other minor fixes and updates
  • October 7, 2000 - added IETF recommendations, submitted by Robert Dalton
  • October 5, 2000 - updated some links, reviewed content
  • July 14, 2000 - added rblsmtpd syntax that includes logging; added more spam-related resources.
  • July 3, 2000 - freshened up some links
  • May 26, 2000 - updated rblsmtpd ORBS syntax with corrections provided by Konstantin Riabitsev
  • April 30, 2000 - added new links
  • April 4, 2000 - version 0.2, implemented some suggested changes from Dave Sill
  • April 3, 2000 - Implemented suggested changes from Mikko Hanninen, Peter van Dijk
  • April 3, 2000 - version 0.1 created by Chris Hardie

XI. Comments/Feedback

Click to receive email
when this page changes
Powered by NetMind
If you have specific suggestions for changes, corrections, and additions to this HOWTO, please send them to Chris Hardie and they will be integrated into the document as appropriate. If you have general comments, reactions, or points of discussion, please use the comment form below. Please do NOT use this section to ask technical questions about your specific system setup; use the qmail users list for this.


7 most recent User Comments on this Page      [view all comments] [all comments on this site] [posting policy]
(powered by SSIComment)


blocking from and tos posted by Stefan Sels on 07/11/2001 at 8:56:21 AM

there are some patches to use:

badrcpto solution :
with this patch, you can block sender and domains. this is done after the RCPT TO: handshake in SMTP. This is much better than bouncing. Spammer won´t use valid adresses.

you can get the patch from my hompage. this is a mirror, I am not the author. badrcptto patch

There are serveral solutions to stop stupid spammers who use always the same sender (badmailfrom) one is the patch (linked on qmail.org) called wildmat (= wild match).
it allows to mach sender against regular expressions.

you can get it

here


Another orbs replacement posted by Andres Salomon on 07/10/2001 at 10:27:04 AM

http://www.orbz.org/. This one is doing active scanning of relays, using lists imported from various other relay databases. Net-wide blocks aren't done, no politics involved; if an email finds it's way back to the scanner, it's blackholed. Plain and simple.


RBL style lists... an update posted by Larry M. Smith on 07/03/2001 at 7:37:48 AM

ORBS has died last month (JUN 01)

However there are several replacements popping up;
http://www.orbl.org
http://www.orbs.org
http://www.orbd.org
http://www.dorkslayers.org

They may be others...

Also another site has come up in Feb 01 to deal with sites that do not want to follow the RFCs. However, as yet (Jul 01) no one has patched qmail to use this list.

http://www.rfc-ignorant.org


rblsmtpd patch posted by Ian Scott on 07/02/2001 at 11:33:41 AM

The patch to rblsmtpd to allow it to use A records instead of TXT records is available here:

http://www.qmail.org/ucspi-rss.diff


rblsmtpd posted by Stefan Sels on 06/09/2001 at 4:48:52 AM

As far as I know, rblsmtpd does not support MAPS because they changed their zone files. as you quote (Mike Silbersack says "If you wish to use the *.mail-abuse.org black-hole lists, you'll have to apply the patch to make rblsmptd work with A records")
What is "the patch" ? does ist exist ? have I missed a link ?
there seems to be everyone knows the problem but nobody wrote a patch/solution ...?

Any comment (preferable to my emailaccount) will be highly apreceated.


add this qmail anti-spam links posted by Noel on 05/14/2001 at 8:30:53 AM

http://www.fehcom.de/qmail/filter.html

http://www.ornl.gov/its/archives/mailing-lists/qmail/2001/03/msg01041.html

for filtering attachments, etc

cheers
Noel


rblsmtpd posted by John Nastasi on 04/22/2001 at 11:46:28 AM

FYI--

Rblsmtpd does in fact work with multiple r's. One thing I ran into, however, was the fact that RBL and DUL work with the following syntax:

-rblackholes.mail-abuse.org
-rdialups.mail-abuse.org

...but RSS "only" works with the following syntax:

-r 'relays.mail-abuse.org:Open Relay - see '

...that is to say, it will fail if you use:

-rrelays.mail-abuse.org

...haven't tried ORBS yet, but now have RBL, RSS, and DUL (in that order) working on the same rblsmtpd instance.

Thanks for the great resource.


Post your comment (all fields but email are required):   

E-mail
Name
Subject
Comments
You may use the following HTML tags in your comments:
<B><I><P .*><P><A><LI><OL><UL><EM><BR><STRONG><BLOCKQUOTE><HR><DIV .*><DIV><TT>
  Notify me of new comments/changes on this page
 

 


Last updated: Monday, 14-May-2001 19:43:46 EST
Copyright © 1995-2001 by Chris Hardie ~ chris@summersault.com ~ [ sitemap | what's new ]