Configuring procmail mail processing

Background.

I was attracted to pair.com because every pair.com account offers mail delivery for the virtual domains hosted under that account. As the email delivery is implemented using qmail, it is fairly easy to configure email redirection using .qmail (dot-qmail) files. However, what few people know is that you can configure a pair.com account to use the program procmail to do all kinds of useful mail processing. The particular processing of interest to me was the ability to pass selected mail on to another smtp server (my Mac OS X box at home, also running procmail) for processing, with the smtp envelope intact. So over many months I have developed and refined a .procmailrc file to implement this, along with some basic spam filtering and forwarding, and I'm passing on a little of what I have learned well.

Invoking procmail

If you are using qmail (as pair.com does), invocation of procmail instead of the usual qmail delivery is done from your .qmail file(s). It's is as simple as inserting the following into the appropriate .qmail files:

For most users, the appropriate .qmail files would be .qmail and .qmail-default in your home directory. If you are using sendmail rather than qmail, the procmail man page tells you what to put into your sendmail .forward file (look for the section entitled notes.)

What the above command does is cause qmail to execute the program preline (using the | token). Preline, which is part of the qmail install, , prepends the smtp envelope sender and recipient to the mail message so that procmail can get at them, and then runs procmail. The -t flag, while not essential, is a good idea, it directs procmail to requeue and attempted redelivery later if it cannot deliver the mail due to some temporary error.

A summary of procmail operation.

Conceptually procmail is quite straightforward in its operation. It looks for a file called .procmailrc in your home directory and then reads so-called delivery "recipes" from this file. The delivery recipes allow you to examine the headers (and body if you wish) of the email message being processed, to write stuff to logfiles, and to carry out various delivery actions based on the headers.

The allowed delivery actions are:

As soon as at least one of these actions has been carried out, procmail stops reading recipes and exits, telling qmail that the message was delivered. Otherwise it keeps reading recipes until it gets to the end of your .procmailrc file (or immediately you don't even have a .procmailrc file) in which case it performs the default delivery action of filing the message in your default mailbox itself (and of course telling qmail that it has been delivered.) There is one more way procmail can exit, that is if it hits a token HOST without anything after it. This last method is useful for getting qmail to bounce the mail message (i.e. return it to the sender with an error message) but more about that below.

Getting at the SMTP envelope sender and recipient from within .procmailrc

Details about what to put in your .procmailrc file can be found in the procmailrc man page, or from various FAQs on the web. You will most likely have to learn a little bit about grep patterns, but the procmailrc man page explains most of it, so don't stress. I won't repeat the detail here, other than to make it very clear that you can get at the smtp envelope sender and recipient, since these important pieces of information are added by preline. This means that you can happily ignore one of the main points of the the procmail FAQ, the one about not being able to figure out who the message is to.

The headers added by preline are a line beginning with 'From ' indicating the smtp envelope sender (that's right, that's a space, not a colon after the From) and a line beginning 'Delivered-To:' indicating the smtp envlope recipient. If you want to determine who the message is for, it's the Delivered-To: header that you should examine, not the To: or Cc: headers (since it is entirely possible that your address will not be in the To: or Cc: headers, e.g. it was Bcc:'ed to you. And no, there is no Bcc: header on a delivered message.)

Here are examples of .procmailrc recipes using the 'From ' and 'Delivered-To:' headers. This first one looks for the 'From ' header, gets its contents into a variable ENV_SENDER, and then writes the contents of this variable(accessed using the token $ENV_SENDER) to the logfile (specified at the start.)

LOGFILE=$HOME/procmail.log
:0
* ^From[ ]\/[^ ]*
{
    ENV_SENDER=$MATCH
    LOG=$ENV_SENDER
}

This second recipe looks for mail addressed to [email protected] and then forwards the mail (using the ! procmailrc token) to [email protected]. Note the use of the backslash character to match against the dots in the email address, this is necessary to prevent the dot being interpreted as the egrep 'any character' token.

:0
* ^Delivered-To:.*-.*-mypairusername@mypairservername\.pair\.com
! [email protected]

Now we can put together a more complicated example which will pass on all mail addressed to any user, for example [email protected], or [email protected] to the the same user at the smtp server, e.g. [email protected], [email protected]. (Lines beginning with # in .procmailrc are comments.)

# Get username portion of recipient from Delivered-To: header,
# and concatenate it with desired downstream smtp server.
:0
* ^Delivered-To:.*-.*-\/[^@]*
{
    ENV_REC_USER=$MATCH
    [email protected]
}

# Now deliver the mail
:0
! $ENV_REC

Preventing mail loops

If you're using procmail to do forwarding, it's good practice to add a header to outgoing mail sent by procmail so that if it is somehow bounced back to your account, procmail doesn't try to send it again and thus enter an infinite mail loop. This can be accomplished quite simply by piping outgoing mail through formail first to add a header like X-Loop: [email protected]. Incoming mail is then checked for this header, and if it is found, is not forwarded but is stashed in your popmail box for you to find sometime. The following code inserted before your other recipes will do this:

# Trap looping mail and stuff in home mailbox,
# otherwise add an X-Loop: header (using formail) to all mail.
# E acts an else for previous recipe, f treat action as filter.
LOGFILE=$HOME/procmail.log
CRLF="
"
:0
* ^X-Loop: mypairusername@mypairservername\.pair\.com
{
    LOG="Loop detected"
    LOG=CRLF
    :0
    $HOME/mail
}
:0Ef
| formail -A "X-Loop: [email protected]"

The right way to bounce messages

It is possible, but a little tricky, to get incoming email to be bounced by qmail from a procmail recipe. There are three important things to note: Firstly, the mail must be worthy of bouncing, which means that it must have a valid return address. So most spam, which does not have a valid return address should not be bounced, it should be delivered to /dev/null and thereby prevented from entering into your mailbox. This is the equivalent of throwing junk snail mail in the bin, you wouldn't waste your time returning every little bit of junk snail mail to the sender would you? Well you should treat your email the same way. The following recipe looks at the 'From:' header (not the 'From ' header) for the address [email protected] and discards the email (without generating an error) if it is found.

# This recipe traps mail to just throw away.
# (Eg. forged return address that makes bounce impossible.)
LOGFILE=$HOME/procmail.log
CRLF="
"
:0
* ^From:.*someone@spammer\.com
{
    LOG="Dumped [email protected]"
    LOG=$CRLF
    :0
    /dev/null
}

Ok, well if you are sure that someone who is sending you unwanted mail has a valid return email address, or you are receiving email for a particular address that is no longer valid (say an employee who is long gone from your firm or something) the correct thing to do is bounce the mail. That way the sender will hopefully realise and not send any more emails. This is especially useful if the sender is automated, since automated systems are likely to deal properly with bounces. To bounce mail from procmail, you set the variable EXITCODE to a special value, and then get procmail to terminate. When it terminates, procmail reads the value of EXITCODE if it is set and passes this as its result code back to the mailing program (qmail or sendmail) which can decide what to do. We want to do a hard bounce, so that the mail never comes back. qmail will do this if it sees an exitcode of 100. If sendmail is being used, the two most useful codes are 67 (unknown user) and 77 (operation not permitted). To get procmail to exit we use the HOST token in our recipe. The recipes below show examples of bounces in action, bouncing emails From: *@unwantedcompany.com and To: [email protected], respectively.

LOGFILE=$HOME/procmail.log
CRLF="
"
# This recipe traps mail to bounce by sender
:0
* ^From:.*@unwantedcompany\.com
{
    LOG="Bounced unwantedcompany.com"
    LOG=$CRLF
    #EXITCODE=77 # for sendmail
    EXITCODE=100 # for qmail
    HOST
}
# This recipe traps mail to bounce by recipient
:0
* ^Delivered-To:.*someonegone@mydomain\.com
{
    LOG="Me: Bounced someonegone"
    LOG=$CRLF
    #EXITCODE=67 # for sendmail
    EXITCODE=100 # for qmail
    HOST
}

Now, you may also recall that at the top of this page I mentioned that invoking procmail with the -t flag causes it to attempt to redeliver mail that is not delivered, and you might be thinking "Won't it just try to redeliver this message that we want to bounce, since it exited undelivered with an error?" Well, nice try, but if you had read carefully, you would have seen that I said temporary error. A hard bounce is not considered a temporary error, and procmail will not requeue it and it will bounce like a rubber ball. Try it and see.

Well, that's all for now! Happy recipe making!

Phil

1 November 2001