How To Tune The CleanMail Filter Pipeline

CleanMail feeds incoming mail to a series of mail filters, the so-called filter pipeline. Examples of mail filters are the built-in attachment blocker, third-party virus checkers, or SpamAssassin.

Each filter analyzes the message and returns a filter result telling CleanMail what to do with it. Depending on the result, the message is passed on to the next filter in the pipeline, discarded, or delivered to its intended recipients.

Obviously, the order of the filters in the pipeline matters. If the first filter is known to consume lots of resources overall server throughput will be reduced. On the other end of the spectrum, light-weight filters may be used as a "triage" stage: obvious spam is discarded, freeing precious server resources, possibly at the cost of a higher probability of classifying legitimate mails as spam (false positives). However, resource usage is only one aspect of a much more complex issue, there are other criteria to look at when configuring your filters. Here's a list:

CleanMail message filters can be classified according to the following table:

FilterAggressivityResource UsageSelectivityProtection
Anti-Viruslowhighmediumanti-virus
Attachmentlowlowmediumanti-virus
SMTP Delaylowlowmediumboth
DNSBLhighlowhighboth
Fingerprintmediumlowhighboth
SMTP Checkslowlowmediumboth
SpamAssassinlowhighmediumanti-spam

The built-in SMTP-level filtering (traffic limiting, anti-abuse), is not configurable in filter pipeline, and is only available in SMTP proxies.

CleanMail by default orders the filters to optimize throughput, using the following guidelines:

Choosing the Right Filters

Judging from the list above, the following filters are a must-have in every CleanMail configuration, in order of their execution:

The other filters are situational - your milage may vary:

When configuring CleanMail with the Admin application, every new filter will be automatically moved to the best position in the filter pipeline. Afterwards, you can still change the order of filters, but only within limits.

Example Filtering Results

The figure below shows typical filtering results for a CleanMail filter pipeline, using an attachment blocker, the fingerprint filter, the delay filter, SpamAssassin and Clam Anti-Virus, in that order, with the built-in SMTP checks as an added bonus filter getting rid of abusive SMTP traffic even before the filter pipeline is invoked.

CleanMail Filtering Results

The low-resource usage filters are able to discard 77.6% of all incoming mail traffic, before the rest (22.4%) is passed to the more elaborate filters such as SpamAssassin and Anti-Virus, finally leaving only 15.2% of all messages classified as legitimate and passed on to the recipients. The fingerprint filter in this example, being one of the first filters, is able to get rid of the lion's share of all unwanted messages, removing this filter would increase the slices for SpamAssassin and Anti-Virus proportionately. In summary, the light-weight filters are able to increase your CleanMail server's message throughput almost five-fold, in comparison to a solution only using SpamAssassin and Anti-Virus.

Looking at the chart and the numbers, you might be misled into thinking that SpamAssassin and Anti-Virus have a rather small effect on the filtering results and can be removed from the filter pipeline with only little impact on results. However, precisely these filters are needed to teach spam fingerprints to your fingerprint database, as a spam message has to be filtered at least once by some other filter, before all subsequent occurrences of similar messages can be discarded by the fingerprint filter. After removing SpamAssassin and Anti-Virus from the pipeline, the fingerprint filter would effectively stop to work.

Filtering Problems

Closing Remarks

Your feedback is welcome! Please submit hints and suggestions to .