[ English / Japanese ]

- Anti bad-mail SMTP wrapper -

What's new?

What's this?

The SMTP wrapper `antibadmail' provides generic spam rejection on SMTP session. This program is a successor of my qmail patches.

Because `antibadamail' is a wrapper, it can work not only with qmail, but also with sendmail, postfix and any other RFC2821 compliances.

Unlike the `Message contents spam filter', antibadmail rejects bad mail without seeing contents of them. Imagine if you were a large scale bad mail sender. Do you send bad mails from valid SMTP server? Do you send bad mails with your correct email addresses?

No.

Antibadmail reject all bad mails seeing the bogus parameters in SMTP session. Doing so reduces the load of mail servers because antibadmail program check only three SMTP parameters(HELO, MAIL-FROM and RCPT-TO) and smtp-client's DNS-record settings.

How to get

To get latest version, use cvs.

cvs -d :pserver:anonymous@yatex.org:/qmail co antibadmail

Once you've checked out source, all you have to do when you want to get latest source is typing

cvs -z1 up -Pd

in antibadmail/ directory. If you want to improve this program by yourself, please feel free to email to abmusers ML. Then we'll create developer's account in CVS repository for you.

Snapshot here.
antibadmail-1.43.tar.gz

And join abmusers ML.

Features

The antibadmail program enables handy bad mail rejection for qmail/sendmail/Postfix as below;

($CONTROLDIR defaults to /var/qmail/control)

Note that this is not a virus scanner. Most virus-infected PC sends malicious email with infected person's email addresses. Introduction of antibadmail does not mean unnecessity of virus scanner. But `contents filter' including virus scanners always waste tremendous computing resources which are essentially unnecessary.

You'll find 60%-99% of undesirable emails are comfortably rejected by antibadmail. Save your mail server's resources!

Installation

Antibadmail should be invoked by modified version of tcpserver, which can set environment variables for both TCPREMOTEHOST match and TCPREMOTEIP match.

cvs -d :pserver:anonymous@yatex.org:/qmail \
  co -r paranoid+bsd ucspi-tcp
cd ucspi-tcp
vi conf-home
(edit conf-home to define installation prefix)
make && make setup check

NOTE: If you use FreeBSD, checkout

cvs -d :pserver:anonymous@yatex.org:/qmail \
  co -r paranoid+freebsd ucspi-tcp

instead.

You need unprivileged user for antibadmail.

groupadd abm
useradd -g abm abm

User name `abm' is arbitrary. Suppose uid and gid of `abm' user are 250 and 25 respectively. Replace uid=250 and gid=25 as described below with real values on your host.

Then, build antibadmail main programs.

gtar zxpf antibadmail-VERSION.tar.gz
cd antibadmail-VERSION
make all install

At the beginning of `make', you have a chance to determine installation prefix of antibadmail. Assume prefix is `/var/qmail` in latter of documetation.

The final step is to wrap running smtpd:

  1. Installaion step to wrap sendmail/postfix smtpd
  2. Installaion step to wrap qmail-smtpd
  1. Installaion step to wrap sendmail/postfix smtpd

    1. Change smtp daemon port other than 25. 10025 for example here.

    2. Start antibadmail as follows;

      HOSTANDIP=1 RELAYCHECK=1 tcpserver -u 250 -g 25 -x /etc/smtp.cdb 0 25 \
        antibadmail mconnect 127.0.0.1 10025
      

      Make sure antibadmail and mconnect command are located in $PATH. `mconnect' is a SMTP connection client, which comes with ucspi-tcp. /etc/smtp.cdb is the tcpserver's connection control rule database. If you are not familiar with tcpserver, see the tcpserver rule section below.

    3. Put list of domain names your server can accept.

      If your server can accept(or relay) foo.example.com and *.bar.example.net, create empty files below in /var/qmail/control/rcpthostsdir/

      foo.example.com
      .bar.example.net

      Note that no at-marks(@) are necessary for domain patterns, unlike patterns for bad*dir described below. RELAYCHECK=1 on a startup command line specifies enabling acceptable RCPT-TO domain check. Even if RELAYCHECK=1, when the client pass SMTP-AUTH, any RCPT-TO domain the client send will be accepted by antibadmail.

      If you alter acceptable domains by "POP before SMTP" control, please ask it at the abmusers ML.

  2. Installaion step to wrap qmail-smtpd

    First, genuine netqmail is strongly recommended. If you have old qmail-1.03 plus many spam-control patches such as qmail badrcpt to patch, qmpatch or so, replace it with netqmail. Now is the chance to shift. And if you are using ancient inetd+tcpwrapper for qmail-smtpd, replace it with tcpserver.

    The qmail-smtpd daemon program is well designed to be wrapped by others. All you have to do is to add antibadmail to starting script. For example, you may already have script like this;

    tcpserver -u 250 -g 25 -x /etc/smtp.cdb 0 25 qmail-smtpd
    

    Rewrite it as follows;

    HOSTANDIP=1 tcpserver -u 250 -g 25 -x /etc/smtp.cdb 0 25 \
        antibadmail qmail-smtpd
    

    That's all. Never fail to replace tcpserver with modified version described above.

tcpserver rule

If you are not familiar with tcpserver yet, try this simplest rule file /etc/smtp.

127.0.0.1:allow,RELAYCLIENT=""
10.0.0.:allow,RELAYCLIENT=""
all:allow

where 10.0.0. is IP-address prefix of your LAN. RELAYCLIENT="" means setting environment variable like that at invocation of antibadmail when smtp connection comes from corresponding address. Like qmail-smtpd, antibadmail assume that the client is located in LAN when the environment variable RELAYCLIENT is set, so that connection at that time must not be abusing. Antibadmail accepts all message when RELAYCLIENT set.

If you wrote rule database in /etc/smtp, you have to convert it to cdb-format as follows.

cd /etc
tcprules smtp.cdb tmp < smtp

After starting tcpserver+antibadmail, try `telnet localhost smtp' to confirm it is running. If the SMTP greeting message of original smtp-daemon shows up, almost all goes well.

To record the rejecting/accepting result, add the following line to /etc/syslog.conf.

local1.info                  /var/log/smtp-stat

It is more desirable to add a notation for log rotation to /etc/newsyslog.conf(BSD) or /etc/logrotate.conf(Linux).

Constructing badmail database

Antibadmail refers `datadir' database structure. Datadir is maildir-similar structure where an entity exists in a form of `file in a directory' instead of `line in a file'. By default, antibadmail referes three directories.

Again, you can change the prefix /var/qmail/control by build time setting or environment variable $CONTROLDIR at running time.

Blacklist

A filename should be the form of one of as follows;

Rejection by header

The policy of antibadmail is ``Don't inspect contents''.

However, as to badmail forwarded by (friendly) SMTP server, we can find rejection ground only in message header.

Antibadmail can reject mails by mail header pattern. Rejection by header is a `last resort', and can't be diverted once message header matches with a pattern even if the pattern is misspelled. Use with great care. Header inspection defaults to ON. To disable header check, set environment variable HEADERCHECK to 0 at invocation of antibadmail.

HEADERCHECK=0 tcpserver ..... antibadmail .....

When HEADERCHECK is enabled, antibadmail receives DATA, it reads header patterns from regular data-directory. The datadir structure for header rejection is little bit different from above. Datadir for one header pattern consists of as follows;

$CONTROLDIR/badhdrdir/FieldName/EntryName/p*
$CONTROLDIR/badhdrdir/FieldName/EntryName/v*:VAR
$CONTROLDIR/badhdrdir/FieldName/EntryName/errmsg

where FieldName is header-field name with all lower case, EntryName is arbitrary name for pattern set.

Suppose you have been receiving many spams via smtp server, which is in the office you previously belonged. Messages to your old address are forwarded to current address. In case as this, antibadmail can reject forwarded spam by seeing headers. If your old server's FQDN is `oldserver.you.used', messages with these header can be assumed to be spam.

Received: from hogehoge.com (HELO oldserver.you.used) ....(1)
  or
Received: from unknown .... by oldserver.you.used ....(2)

where `....' is any string. Both (1) and (2) are received header attached by oldserver. Let's think about (1) first. It is the typical received header of spam. So we define the first pattern as follows;

(HELO oldserer.you.used)

Why is this bad pattren? Because "HELO oldserver.you.used" was told by spamming server when it connects to oldserver.you.used. Sending opponent's name as HELO is typical spamming behavior.

Then, think about pattern for (2). It's as follows;

from unknown
by oldserver.you.used

Note that pattern(2) is written in two lines so that both string ("from unknwn" and "by oldserver.you.used") must match with received header. Now define this rule set name as `foo'

--- File: $CONTROLDIR/badhdrdir/received/foo/ptn-1 ---
(HELO oldserer.you.used)

--- File: $CONTROLDIR/badhdrdir/received/foo/ptn-2 ---
from unknwon
by oldserer.you.used

--- File: $CONTROLDIR/badhdrdir/received/foo/errmsg ---
We cannot receive suspicious messages.

refuses the messages which have received-header that matches with "(HELO oldserer.you.used)", or matches with both "from unknwon" and "by oldserer.you.used", returning the SMTP error message of "We cannot receive suspicious messages.".

Conditional header pattern

More precise pattern for headers

When you are aware of the power of header rejection, you might want to describe patterns more precisely. The first character of each line determines the matching method. There are five methods for matching.

STRING
Pattern lineMeaning
=STRING Whole line is exactly same as STRING
^STRING Line is beginning with STRING
$STRING Line is ending with STRING
/REGEX Line is matching with regular expression REGEX
:STRING Line has the part which match with STRING exactly
Same as above

Note that all header field contents will be joined into one line, converted all lower case. Therefore all matching will be done in case-insensitive.

Regular expression engine is supplied by libc of your driving operating system. So it is different from that of Perl, Emacs-Lisp, (GNU)egrep which you may be experienced with. If you want to examine how regexp pattern matches with, use debugging mode of header module. It is obtained by typing this;

make h

Using h command interactively as follows.

./h
Received: from hogehoge.blah.example.org (HELO oldserver.you.used) by oldserver.you.used

The `h' command acquires all patterns in $CONTROLDIR/badhdrdir/*/* and apply all patterns over given strings from stdin. To change badhdrdir, set that directory to environment variable BADHDRDIR.

Be aware that although regular expression is friendly, using regexp easily causes configuration errors. It is difficult enough to confuse system administrators. Absolutely NO errors for mail configurations!

Exclusion from blacklist

You might want to reject all false `*@hotmail.com' mails. But you might want to receive `*@hotmail.com' from real hotmail server. In this case, do as follows;

  1. Put reject pattern in badmailfromdir/
    touch /var/qmail/control/badmailfromdir/@hotmail.com
    
  2. Authorize client whose PTR-record matches with *.hotmai.com. Put the following line to /etc/smtp and convert it to smtp.cdb.
    =.hotmail.com:allow,GOODMAILFROM="@hotmail.com"
    

If you want to receive any message from certain server, Set environment variable RELIABLECLIENT for the server.

=smtp.server.you.wantto.rely:allow,RELIABLECLIENT=""

Antibadmail stops all rejection check except extremely insecure parameter when RELIABLECLIENT is set.

Soiled recipient address

If you or users of your SMTP server want to receive all email even if the sender's SMTP server has wrong DNS-record settings. Suppose when you apply web-shoping, auction, mail-magazine or so. Those sites as a whole are held in ill-mannered service provider. Many of them don't have correct settings of DNS(A and PTR record) nor SMTP-HELO. Althogh antibadmail reject emails from those incorrect servers by default, you can stop rejection upon certain receipient addresses.

You can make `soiled recipient address' as follows.

  1. Create datadir for soiled recipients.

    mkdir /var/qmail/control/soiledrcpttodir
    
  2. Make the entry of recipient address for no rejection.

    mkdir /var/qmail/control/soiledrcpttodir/local-foo@your.domain
    

Then antibadmail will pass all emails for `local-foo@your.domain'.

`Domain specific control' is also available. If you and/or unprivileged users on your server hold virtualdomain, @v.example.com for example, create directory with that name and touch any files whose names are local-part for which you want to omit spam control.

mkdir /var/qmail/control/soiledrcpttodir/@v.example.com
cd /var/qmail/control/soiledrcpttodir/@v.example.com
touch foo bar- ./-bar-baz ./'!bar-foo'
ls
!bar-foo  -bar-baz  bar-      foo

In this case above, each address are treated as follows.

local-partspam-inspectionreceive/reject
foono (omitted)receive
bar-bazyesdepends on other parameters
bar-foonoREJECT
bar-anyno (omitted)receive

Filenames are recognized as follows.

name

No spam inspection for name@domain.

name-

Wildcard. No spam inspection for any name-*@domain. It should work fine with qmail's address extension.

-name

DO spam inspection for name@domain. Minus sign stands for `remove(-) from soiled entry'. Note that, this doesn't mean all messages to name will be delivered. They all will undergo spam inspection. Note also that, this minus sign can be combined with wildcard of `-'.

!name

Reject name@domain immediately even if it might be clean message.

ALL

Through all if local-part doesn't match any name in this directory.

Note that rejection avoidance don't occur when smtp client sends parameter which matches strictly with entry of one of badhelodir, badmailfromdir and badrcpttodir.

Reject unknown RCPT

For qmail users and admininstrators of secondary SMTP server, it was long time worry to stop mails to non-existent recipient. Both qmail-smtpd and secondary SMTP server don't know the existence of recipient(RCPT TO) address. Now antibadmail can drop mails for non-existent recipient.

Environment Variables to Control Behaviour

When each variables in this table is set, antibadmail alter the action. These variables can be set via antibadmail start-up script or tcpserver according to client's IP-address/hostname.

VariableAction when setNegation
RELAYCLIENT
RELIABLECLIENT
Accept all message except when the client sends seriously bad parameter.no
CHECKHELODOMAIN
CHD
Expect HELO paramter has correct DNS recordyes
REJECTNODOTHELO Reject HELO without dots.yes
REJECTIPINHELO Reject HELO which contains client IP address that is presumably hostname with dynamically allocated IP address.yes
PERMIT_STATIC In the case of REJECTIPINHELO, permit if client's PTR-record has substring "static".yes
PASSKNOWNNODOTHELO By default, antibadmail rejects no-dot HELO from unknwon host. Set this variable when overlook it.yes
PASSUNKOWNIPHELO By default, antibadmail rejects IP-address formed HELO from unknown host. Settng this variable passes it.yes
NOMFDCHECK Stop MAIL-FROM domain check.yes
PERMIT_NXRCPT By default, antibadmail disconnect session when clients sends non-exixtent RCPT for protection from `random dictionary attack'. When a good user sends to multiple recipients with some of them misspelled, the SMTP session will end in rejection. To rescue this, set this variable. But it is subject to dictionary attack. yes
BADHOSTReject.yes
NOTERM Keep SMTP session after it is proved to be spam.yes
QUICKREJECT Reset TCP quickly before all SMTP parameters are accepted. yes

Negating effect of each variable can be accomplished by set its value to "0".

Public spamdb

The public spam rejection database is available via anoncvs.

cvs -d :pserver:anonymous@yatex.org:/qmail co spamdb

Or just type

make spamdb

In the top directory of antibadmail source.

This database is in plain text format, not datadir structure. You can convert plain text database to datadir struct by f2d command, which comes with antibadmail package.

You can convert, for example, badmailfrom file from spamdb to badmailfromdir/ structure by executing f2d as follows.

f2d -d ./badmailfromdir badmailfrom

Or just type `make' in spamdb/ directory.

Note that ./badmailfromdir/ and badmailfrom is accessible from working directory.

cd $CONTROLDIR
ln -s $SPAMDBDIR/bad*dir .

Finally, create a Makefile to merge your site's smtp-rule and spamdb's smtp-badhost referring an example Makefile in spamdb/Makefile.tcprule.

TR	= tcprules
OWNER	= qmaild
#OWNER	= abm

SRC	= smtp smtp-badhost

all: smtp.cdb

smtp.cdb:	${SRC}
	(cat ${SRC} ; \
	 echo all:allow ) | ${TR} $@ smtp.tmp
#       chown ${OWNER} smtp.cdb

Locate this as Makefile in smtp starting directory(eg. /service/smtpd).

cd /service/smtpd
ln -s $CONTROLDIR/smtp-badhost
make

The Datadir structure

Any database which has multiple records in a file always suffers from these difficulties;

  1. file locking, which is the goal of `Don Quixote'
  2. one data file corruption means whole data corruption
  3. Updating data file without atomic-operation causes another referer to confusion

With `Datadir' structure, there's no need for file locking because all the updation on an entity can be done without referring any other entities. All addition/deletion operation is automatically atomic because they are file creation or file deletion.

Antibadmail Users ML

There is Mailing List for discussing about development and trouble shooting related to antibadmail. Please join it to nourish antibadmail!

To join the antibadmail users ML(abmusers), send your self introduction(more than 5 lines) to abmusers@ml.gentei.org with subject="subscribe". Here is an example.

To: abmusers@ml.gentei.org
Subject: subscribe
--
(Self-introduction more than 5 lines)
I'm newbie administrator of FOO company.
I love email!
I don't like spam!
---
YourMail@foo.comp....

Do not mimic above. :)

No Warranty

This program is free software and comes with absolutely NO WARRANTY. The author is not responsible for any possible defects caused by this software. You can freely modify this program for your convenience. But if you want to publish modified program, please tell me before announcement. Take it easy to write me comments and bug-reports.

yuuji@gentei.org

Publications

Sorry, Japanese only below.


82223
yuuji@example.org
Fingerprint16 = FF F9 FF CC E0 FE 5C F7 19 97 28 24 EC 5D 39 BA
HIROSE Yuuji - ASTROLOGY / BIKE / EPO / GUEST BOOK / YaTeX [Tweet]