This program reads the logfile created by NO SPAM TODAY! and writes to STDOUT a summary of the e-mails processed by NO SPAM TODAY. The summary includes the SpamAssassin score, accept/reject and delete status. Also included is an abbreviated subject line. A more detailed comma separated variable file can be produced with the 'csv' option.
Release History:
1.0.0.0 - Originally thought up and writing started late January - as a means to make it easier to read the
logfiles made by No Spam Today (NST). Periodically looking at the logfiles is necessary to tune
the spam filters to improve the accuracy of the filters.
1.0.0.1 - First compiled version - added improved Getops::long
1.0.0.2 - Fixed some documenation, added the --days switch, fixed .csv output to include message size as well as the
mail storage file name (if there is one).
1.0.0.3 - Add support for scanning the 'log~' first, then the current logfile - this handles splits in the logfile
that often occurs with Nospamtoday. Slow, painful, it it does work.
1.0.0.4 - Set ERRORLEVEL to number of messages selected. Typically no SPAM = good and errorlevel =0
1.0.0.5 - Fixed date range checking that failed for May 1.
1.1.0.1 - Changed to support NST 3.x, added check for mail from alias, I.E. "MAIL FROM <> From"
added check for unicode in subject, added check for AUTH mail attacks, if client reset
before subject or from was available - spamlogs will now display the IP address.
Unicode characters in subject are now replaced with "--unicode character--" unless a new
-unicode switch is added to the command line. This surpresses the strange random characters
for those of us that use standard ascii character sequences.
Email authentification attacks will now have the blank subject line replaced with "AUTH ATTACK"
Sender is decoded from 1) the stand alone "From" log entries, or 2) the "MAIL FROM:" when
the initial connection with the sender is established. (see NST logfiles for examples)
Receipent is now decoded from 1) the stand alone "To" log entries, or the "RCPT TO:" when the
intial connection with the sender is established. (see NST logfiles for examples.)
spamlogs [options] [file | value ...]
Options:
-file the path and filename for the NO SPAM TODAY! logfile -csv the path and filename for the detailed csv file -nosum surpress printing the mail summary to STDOUT -help brief help message -man full documentation -rcpt select only log entries whose receiptent address matches -remark select only log entries with the matching remark -score select log entries that have spamassissin scores equal to or higher -days select log entries based on todays entries and n day(s) before today -unicode subject line displayed ASIS even if it contains unicode characters (non-ASCII) -version display version of this program
Print a brief help message and exits.
Prints the manual page and exits.
Specify the NO SPAM TODAY! Logfile that should be analyzed.
--file=nospamtoday.log
--file="C:\program files\no spam today!\nospamtoday.log"
NOTE if the path for the file contains "spaces" you should put the whole path & file
in quotes.
Select the log entries whose receipent address matches the optional string. For example:
--rcpt=amy would match all incoming mail sent to ANY amy -- all domains.
--rcpt=yazook.com would match all incoming mail sent to the yazook.com domain
--rcpt=amy@yazook.com will match only mail for amy at the domain yazook.com
If --rcpt is left off mail for any receipent and domain will match. The rcpt switch is case insensitive. That means "amy" or "Amy" or "AMY" are equal.
Select the log entries with matching filter actions. Remark is case-insensitive. Typical actions are accept, reject, delete, and deliver. In addition the terms above can be combined to in forms such as "reject/deliver" or "reject/delete".
--remark=delete would match and display only entries that had been deleted.
--remark=accept/delete would match entries that are learned by the spam trap.
Select items whose spam assassin score is equal to or higher than the value specified. If the score field is blank (mail deleted by an eariler filter) then it ALWAYS qualifies in this selection criteria. For example:
--score=5 Only e-mails that had a spamassassin score equal to or greater than 5 would
be included in the summary (or csv file)
--score=0 include all positive spamassissin scores
--score=-10 include e-mails that are on the automatic white list (AWL)
If --score is not used, all e-mails will have matched this pattern
Send detailed log information to a comma separated variable (csv) file. This file can be read by spreadsheet programs for additional analysis. The items written to the file are in the following order:
sender's IP address,
senders e-mail address (From the initial sender's MAIL FROM contact, this may
than the senders address as later reported in the log as "From" see sender_alias
sender's host name,
date and time mail connection made,
mail receiptents address, (From the inital RCPT TO record when connection was
established. This may be different than the "To" field recorded in the logfile
see the sendto_alias.
subject (if available),
mail status (accept, delete, reject),
mail size (in bytes),
spamassasin score,
spam trap result,
log file record number,
msg archive file name, (mail storage filter)
log file name
sender_alias
sendto_alias
Surpress the summary report normally sent to STDOUT. Use this if you only want the csv data file. If nosum is specified - then one dot for each e-mail connection recorded in the logfiles will be echoed to STDOUT.
Select entries that were recorded today and N days before. The default is todays entries only.
--days=0 = today only
--days=1 = today and yesterday
--days=7 = today and the last seven days
The range for days is 0-31.
Display the subject information without filtering to exclude unicode characters. Use this switch if you are expecting e-mails with unicode (extended character sets) included in the subject -- and spamlogs is converting them to " --unicode characters--".
Display ONLY the "spamlogs" program version number
This program reads the logfile created by NO SPAM TODAY! and writes to STDOUT a summary of the e-mails processed by NO SPAM TODAY. A requirement for using SPAMLOGS is that the advanced and extended logging information most be enabled in NO SPAM TODAY! If not enabled you can do this at the [FILE], then [Global Settings].
The summary includes the SpamAssassin score, accept/reject and delete status. Also included is an abbreviated subject line. A more detailed comma separated variable file can be produced with the 'csv' option.
Here is a sample of the summary:
Opening logfile: nospamtoday.log dennis@foo.net delete (unexpe capital consultant <iwvbeyoohn slash your payments 3.5 Mar 07, 2005, 00:41:38 dennis@foo.net 21.7 reject/delete johannesgama1@netscape.net **KINDLY ASSIST** Mar 07, 2005, 01:55:22 dennis@foo.net 44.4 reject/delete marcum@essex.com <marcum@essex Trend XPGZ Mar 07, 2005, 02:31:19 dennis@foo.net 18.9 reject/delete Attention-Ladies<Alexander@cal It is finally your turn Mar 07, 2005, 02:32:13
The first column is the receipents e-mail address. It is followed by the SpamAssassin score (if there is one), then the filter action, the senders's name, subject and the timestamp.
The summary output can be redirected to another file and sent as e-mail.
Example 1:
del amymail.txt
spamlogs.exe --file=nospamtoday.log --rcpt=amy --score=9 --remark=delete --days=0 > amymail.txt
if %ERRORLEVEL% GTR 0 mapisend.exe -u username -p password -r "Dennis Heidner"
-s "Mail rejected for Amy" -m "Logs included" -v -f "path to file\amymail.txt"
In this example, only mail sent to "amy" reported as "delete" with either a blank score OR a spam assassin score of 9 or higher AND only todays entries will be examined. The summary will be directed into a file called "amymail.txt". The shell variable ERRORLEVEL is checked, and a rejection e-mail is only sent IF there were qualifiying log entries found.
The Microsoft Resource Kit program MAPISEND is then used to forward the summary of deleted e-mail to the administrator "Dennis Heidner". NOTE!! The MAPISEND line above was split into two lines so that the documenation would be easer to read!
Example 2:
del mail.csv
spamlogs.exe --file=nospamtoday.log --nosum --csv=mail.csv --days=7
mapisend.exe -u username -p password -r "Dennis Heidner" -s "No SPAM Today mail summary" -m "CSV summary included"
-v -f "path to file\mail.csv"
In this example no summary is displayed to the screen, instead a comma separated variable (csv) file is created. The file will only include entries that have been received in the last seven days. The .csv file is sent to the mail administrator "Dennis Heidner"
Note MAPISEND is available from the Microsoft Back Office Resource Kit. There are similar open source programs that can be used to send e-mail with attachments to system adminstrators.
MIME filtering! No Spam Today! allows the administrator to install additional external filters or even install multiple instances of the an existing filter. Why would you want to do that? With the 2.0.4.2 version of No Spam Today! a new mail storage filter was added. Normally when the attachment filter is installed, you enable a MIME filtering policy. This works great to eliminate many items of spam -- even before being reveiwed by the SpamAssassin filter. Unfortunately some mail servers are not well behaved and appear to generate MIME violations. This means that the normal reject/delete option for the Attachment filter could result in lost "good" mail. Even worse - is that the mail is deleted without your chance to review the e-mail.
Here is a work around - but be aware this adds some additional overhead to your No Spam Today! scanning machine.
First, Make sure you have the normal attachment filter installed and near the top of the filters. On this attachment filter change the filtering action from "reject/delete" to "accept/deliver" and check the box for enforcing MIME violations.
Second, Install a Mail storage filter, configure it to hold several days e-mail.
Third, Install an additional attachment filter -- on the VERY FIRST screen of the attachment wizard -- you have the chance to name the filter. DO SO! Call this filter the "MIME Filter". Set all the same settings in it as in the very first attachment filter EXCEPT this time choose "reject/delete" for the action when MIME violations occur.
Now if you see an e-mail that looks like it was incorrectly deleted because of a MIME violation, you can use the timestamps from the SPAMLOGS summary, to find the corresponding file in the mail storage area. Use Notepad to text in and look at the e-mail.
Warning: Notice how easy it is to look at the e-mail -- just text it in with Notepad. E-mails stored in the mail storage area may be clear text. SO REMEMBER TO CHANGE THE PERMISSIONS ON THIS DIRECTORY TO RESTRICT THE EYES THAT MAY LOOK AT THE MAIL.