[antispam-f] Re: Block list processing

  • From: Martin <freelists@xxxxxxxxxxxxxxxx>
  • To: antispam@xxxxxxxxxxxxx
  • Date: Sat, 17 Mar 2007 12:50:23 +0000 (GMT)

In article <4ec4eca1c2antispam@xxxxxxxxxx>,
   Frank de Bruijn <antispam@xxxxxxxxxx> wrote:
> In article <4ec2e7e98cfreelists@xxxxxxxxxxxxxxxx>,
>    Martin <freelists@xxxxxxxxxxxxxxxx> wrote:
> > I have been using AS 1.60.a5 for about 10 days now in order to
> > experiment with the BlockList processing - which seems to work well,
> > incidentally. I have not detected any false spams, as yet - although
> > I do have the Weight test near the bottom of my rules at the moment.
> > It is a really nice addition to AS.

> > However, I do have some questions, which Fred
> Who?
Oooops! Big big apologies there. Of couse I meant you, Frank ... but
recent correspondance with a Fred confused me. Sorry.

> > may be able to answer now, or when the documentation is updated in due
> > course.

> > 1.  What is the significance of the number in the Weight rule?
> It indicates 'how heavy' a message must be for the Weight rule to take
> effect, i.e. how many positive results the queries have returned, both
> for one IP address and all the addresses in all trace fields taken
> together.
Rightho. Much more obvious now I realise more about how it all works! 

> > Are numbers over 10 only found when the IP address is on more than one
> > blocklist?
> > Or more than one IP address in a blocklist?
> Both. Every time a DNSBL query returns a result, the 'weight' related to
> that result (see NSQ's Settings file in !AntiSpam.Resources.!NSQ) is
> added to the running total.

> As soon as the value set in the rule is exceeded, the rule matches.
Of course.

So presumably I could have two rules, say ...

 Delete Weight   > 19           at a highish priority to delete heavy ones
 Divert Weight   >  9           at a lower priority to divert lighter ones

> > How many blocklists are consulted for each IP address?

> The current default is three: zen.spamhaus.org, bl.spamcom.net and
> db.wpbl.info. 
Ahah. I had seen the Setting file, but was not sure if it listed DNSBLs
which could be used, or which were all used.

> You can always add more (or remove any of the defaults) if you feel
> like it. Just make very sure you understand the format of the Settings
> file before you do.
I will leave well alone!

> > Are only numeric IP addresses looked up, or named ones as well?
> Currently just the IP addresses found in the trace fields. Future
> versions may have more options here.

> > 2.  Because my Weight keyword is low down (ie low priority) in my
> > rules, I was expecting that the DNS lookups would only happen if the
> > message had not matched any higher-priority rules. This does not seem
> > to be true, as the check is done for ALL emails - and presumably for
> > all IP addresses found in Received headers?

> That's how AntiSpam has always worked. For every header it checks all
> rules with a lower line number (i.e. a higher priority) than the current
> highest priority rule. The trace fields are at the start of a message,
> so the program is likely to 'see' most of them before it gets to the
> headers that trigger rules with a higher priority.
I thought I understood how the rules work, but had not appreciated that a
high priority test (on Subject) would not stop the low priority DSNBL
lookups (because they are done when the Received header is ... err ...
received, which is before the Subject).

> > Thus it appears that AS is actually spending quite a lot of time doing
> > (or waiting for) the lookups, when (in my case) 80% of the messages
> > are deleted by a much higher priority rule. Thus AS seems much slower
> > than without the Weight keyword,
> That will depend on the speed of your connection to the Internet. I have
> a relatively fast connection, so the delay doesn't bother me.
Mine is around 4Mb, so I it does not normally bother me. But while
downloading, the Status now tends to show DSN lookup for a while, and
just flicks though the other states rapidly. Just before I implemented
using DSNBL here I had returned from holiday to find 8,000 emails waiting
:-(( They took some considerable time to process, and I just wonder if
that number of lookups *would* have been noticeable. Particularly as 90%
of them were deleted by my high-priority rule!

> > although I am not sure it is sufficient to be concerned about. I have
> > toyed with the idea of setting up a second set of rules for the same
> > mailbox, which just contained the one Delete rule which hits 80% of my
> > Spam, and leaving the others alone, and then running the normal rule
> > set to do further checks and download the ones I really want - but
> > this does not seem the right way to do it at all! 
I may consider this when in 'return from holiday mode', though.

> > I suspect that queuing the IPs found, and only doing the checks later
> > if really required, would be a major change to the program logic.

> Short answer: yes.

> Slightly longer answer: I thought about queuing and decided against it
> for various reasons.
I can understand that - it would add a whole set of extra bits to go

Thanks for your helpful reply, Frank. It has certainly expanded my
understanding of how DSNBL processing works.

I really do not know how I would manage without AntiSpam!


Martin Avison      using a British Iyonix running RISC OS 5 
                   and the Pluto mail and newsreader

Other related posts: