Byteplant Forum

Home » CleanMail Support » CleanMail SpamAssassin Filter » Spam removal getting worse
Spam removal getting worse [message #581] Mon, 07 February 2005 09:26
Mario Konst
Messages: 12
Registered: November 2004
Junior Member
NST was working perfectly last year. After installing an update last year NST performance degraded. Maybe with the latest version I found where the problem lies.
On the maintenance tab it shows:
words : 166994
HAM : 12574
SPAM : 6153
After learning 153 SPAM mails it shows:
words: 167069
HAM : 12584
SPAM: 6153
It seems while learning SPAM, specific word are classified as HAM instead of SPAM.
Is this correct? I would imagine by learning SPAM , specific word are learned as SPAM tokens.
(batch)command for learning spam:
c:
cd "c:\program files\no spam today!\sa"
sa-learn -c ruleset --spam --mbox "C:/Documents and Settings/Administrator/Application Data/Thunderbird/Profiles/default/steoiizz.slt/Mail/post/spam
pause

Mario Konst
Re: Spam removal getting worse [message #582 is a reply to message #581] Mon, 07 February 2005 12:07 Go to previous message
support
Messages: 918
Registered: April 2004
Senior Member
The most likely explanation for the performance changes last year were the changes in scoring introduced by version 3.0 of SpamAssassin.
SpamAssassin 3.0 was designed to decrease the likelihood of false positives.

I would not put it past sa-learn, that after learning spams, the number of ham tokens increases, for reasons hard to guess at.

Unless you have lots of wrong results from the Bayes tests (high scores for ham mail, leading to false positives, and low scores for spam, leading to false negatives), I would not worry.



Customer Support
Byteplant GmbH
Re: Spam removal getting worse [message #583 is a reply to message #581] Wed, 09 February 2005 08:09 Go to previous message
Heidner
Messages: 121
Registered: February 2005
Senior Member
The scoring points for the bayesan filter were lowered significantly. I think the points were dropped from nearly 5 for the 99% spam category down to about 1.9 points. For me that made a very big difference. (I had widened the threshold for spam detection so lowering the bayesan score opened the flood gates...)

I really did want the new ruleset from the newer Spamassassin, it includes support for SPF, etc... so my solution was to copy the scores for the bayesan filter into a local.cf file... (saved a copy also for backup) and used the prior bayesan filter scores. I also boosted the score for domains that are listed on the spamhaus lists. Between the two the catch without false positives climbed back up to nearly 80%+ (I have a large whitelist) If you excluded the whitelisted items - I think NoSPAM TODAY is hitting nearly 95% accurate blocking.
Re: Spam removal getting worse [message #584 is a reply to message #583] Wed, 09 February 2005 12:24 Go to previous message
support
Messages: 918
Registered: April 2004
Senior Member
> The scoring points for the bayesan filter were lowered
> significantly. I think the points were dropped from nearly 5
> for the 99% spam category down to about 1.9 points. For me
> that made a very big difference. (I had widened the
> threshold for spam detection so lowering the bayesan score
> opened the flood gates...)

The Bayes scores are back now to the SpamAssassin distribution scores. We had them increased, because we felt that the SpamAssassin developers were a little to cautious here. But after many users complained about false positives caused by the Bayes filter, we went back.

If you have a well-maintained Bayes database, it is probably still advisable to increase the scores. Add something like this to your local.cf file, using a text editor:

score BAYES_40 0.1
score BAYES_50 0.2
score BAYES_60 0.9
score BAYES_80 3.0
score BAYES_95 4.5
score BAYES_99 5.4



Customer Support
Byteplant GmbH
Previous Topic: 5.4 BAYES_99 appearing very frequently
Next Topic: White list autolearn
Goto Forum:
  


Current Time: Thu Sep 29 00:11:24 CEST 2016