First read this.
+------------------------------------------------------------------+
Spam control with OpenBSD greylisting
Spam control has to invariably fall under one of the following
categories.
a) Bayesian filtering and contextual analysis
b) Heuristical filtering based on known keywords/bad words
c) CRM114 Markovian chain based filtering (related to a)
d) Vipul's razor approach of DCC
(Distributed checksum computation) with manual
interference - gmail uses this heavily
e) Greylisting to stop spam right at the MTA level
f) IP address blacklisting and
e-mail address whitelisting
g) TMDA - cure worse than the
disease (Only approved senders can send mail)
h) RBL lists ,
spamhaus (politically sensitive spam control techniques)
i) SPF from Microsoft (not a bad idea per se)
but does not work well
This is more or less it.
And most of these techniques are based on content scanning/filtering and
actually reading e-mails with a computer.
Since this is an activity that requires a high end CPU and memory, spam
control software and virus scanning software typically end up grinding
your machines to a halt or even slow down your legitimate e-mails.
Also there is the very scary possibility of losing e-mails due to false
positives.
My product 'anjal' uses the technique called greylisting. This is a very
smart way to combat spam since it is stopped right at the MTA level.
Since this never reads e-mail it is also very fast and highly efficient.
It is impossible to get a false positive here though the first mail from
a domain will experience a delay.
Basically greylisting forces mail servers to be RFC 2821 compliant and
retry mails until the receiving site is ready. This also has an added
advantage of hurting spammers sometimes and also stopping the spam that
is meant for some other sites.
The architecture of our solution is something like this.
+--------------------------+
| Internal company network |
| |
+--------------------------+
\ (Only legitimate senders
\ reach here)
\ --------------> SMTP port 25
\ / +---------------------+
\-> SMTP port 25 / | company mail server |
/ <------+ +---------------------+
+--------------------+ \
| | \
| anjal | \
| antispam appliance | \ +----------------+
| (filters senders) | \ | |
| | \ | |
+--------------------+ | |
SPAMD port 8025 | |
^ | The Internet |
| | |
| | |
SPAMMERS end up here | |
+----------------+
Here is a schematic to explain how anjal works.
The firewall that works in anjal redirects e-mail traffic depending on
three parameters -
a) sending IP address (From IP)
b) envelope sender (who sends you mail?)
c) envelope recipient (who is mail addressed to?)
If the above '3 tuple' are seen for the first time then the mail sender
is subjected to the torturous SPAMD filtering (running on port 8025
above). There is a phenomenon called 'initial stuttering' that happens
here. Instead of talking at full speed the MTA accepts mail one
character at a time. This will piss off spammers and many go away. But
legitimate senders have just one mail to send. Moreover they have to be
RFC compliant. So they survive the test.
Once this process is completed, any subsequent mails from this sending
IP address is assumed to be legitimate and they directly talk to the
company mail server.
This in a nutshell is what my antispam solution does.
And you don't waste your storage space and bandwidth receiving spam
first and then rejecting them.
Overall a very brilliant idea no doubt.
Anyway as a bonus this also stops all sorts of irritating malware like
virses, Trojans, worms and other annoyances. Such mails usually
propagate with reckless abandon and my firewall running in anjal can
rate limit them.
+--------------------------------------------------------------------+
That was not a document written for public consumption but then that
should give you the context for what I am going to say now.
And that is what counts.
Spam control is really not that hard if you understand how spammers do
their job.
There is a lot of difference between the way spammers send out mail and
the way legitimate mailers send.
Let me give you a real life analogy.
Assume that you run a hotel that serves Masal Dosai.
If you have only ten customers in a day you have all the time to make a
perfect Dosa for everyone. You spend the time required for preparing a
proper Dosa.
But instead if you have to deliver 200 Dosais to a party and you only
have two hours. What do you do? And the quality really does not count
since most people may order something else or not eat your Dosa at all.
In such a situation you will be aiming for speed and automation.
Not quality.
This is what happens with spam senders.
Spam sending botnets (which are programs) send out e-mails in bulk. And
their business is really dependent on volume. Even if one person in a
million clicks on a p0rn site or is stupid enough to opt for some
pleasure enhancement pill or some such thing they are fine.
And spam is not only about this or Nigerian widows. It is a lot more
than that. Viruses and worms do not propagate one mail at a time.
This is precisely what we use for controlling spam by using OpenBSD
greylisting.
Remember OpenBSD greylisting is not some other greylisting.
This is something you have to realize. I feel sorry for people who think
that OpenBSD is not very different from linux.
It is not true. As simple as that.
Anyway now coming back to the point, my product was deployed in a high
pressure corporate environment. I did an ifstat and found that it was
getting impounded with mails like bullets spraying from an AK 47 rifle.
And much to my delight my hardware could withstand it. So could OpenBSD
of course.
You should also understand that my low end hardware would not even
withstand a fraction of this load had I gone in for an alternate spam
control strategy that involves content filtering in one way or other. No
matter if it uses heuristics or not.
Here are some even more interesting features.
a) No false positives at all since this technique does not
differentiate between e-mails, only senders
b) Spammers actually get hurt by this. So if everyone used
OpenBSD greylisting then spammers would go out of business. This is one
of the purpose of this mail. ;)
c) Since greylisting pushes back mail to the spammer, we do not
accept spam even at an MTA level. This means that your bandwidth bills
and storage space is saved.
I can go on and on but you get the idea.
It is not a long time since my appliance was deployed but given their
incredibly high mail traffic and spam possibility, it is indeed quite
heart warming that not a single spam has slipped thro' yet.
So what are you waiting for?
You should wait for the next issue of BSDmag, a Polish magazine where I
talk about setting up OpenBSD greylisting.
Or simply read the man page after installing OpenBSD.
$ man 8 spamd
;)
-Girish
