How to create custom spam database

Question

Hi,

I'm working on a project where I need to create a spam database and accept submissions from users. Accepting the submissions is easy, but I was trying to figure out how to weight these submissions.

Let's say the database consists of words, and i get the following submissions: * 137x "banana" * 22x "apple" * 1x "exploding mouse"

Now, there's a fairly good chance that "banana" is a spam word. "Apple" might be, but should probably be considered in a grey list, while "exploding mouse" is probably just a prank.

Anyone got any good ideas?

Cheers!

Answer 1

The standard method is "bayesian", where you compare the word frequencies in spam against the word frequencies in non-spam (aka "ham"). The problem with that is while people would be perfectly willing to forward you all their spam, they're unlikely to want to forward you their ham.

One program that does this already is called "bogofilter". There is a standard Debian package for it.