AnalyticBridge
Data Intelligence, Business Analytics
Why are spam detection algorithms so terrible?
It looks like most of them still rely on Naive Bayes applied to individual keywords, to flag messages. They fail to catch 90% of the spam, yet have a terrible "false positive" rate - as high as 5%.
Are there any companies working on customized (e.g. per email account) solutions? Are there any spam detector that
- use Botnet lists of (blacklisted) IP addresses for filtering as well as white lists,
- use lists of scammy URLs (embedded in an email message) as well as white lists
- use metrics other than individual keywords or combination of two keywords (e.g. positive / negative keywords) for spam detection, such as return address different from sender address, or return address looks spammy
- use algorithms that are much more modern than Naive Bayes, such as hidden decision trees?
Views: 357
Replies to This Discussion
-
Permalink Reply by Samik R on -
I thought the gmail's spam detection is pretty good. Haven't seen a spam in my gmail a/c for years. That is not to say that the problem is solved, but can you give a reference where you are seeing the terrible spam detection that you are mentioning?
-
Permalink Reply by Vincent Granville on -
I think spam detection algorithms to block outgoing mail are especially poor. The ones to block incoming mail are better.
-
Permalink Reply by Samik R on
-
Why should the algorithms be different for the two cases?
Also, who would employ spam detection algorithms to block outgoing mails? Not the spammers ...
-
Permalink Reply by Titus on -
Yahoo / Gmail / Hotmail certainly use spam detection for outgoing messages. Also, your mail client might be sending spam without your knowledge, if you've been infected by a Botnet. That's why blocking outgoing spam (which can be performed by your ISP) is as important as blocking incoming spam.
Top Content
Follow us
Big Data Jobs
© 2013 AnalyticBridge.com is a subsidiary and dedicated channel of Data Science Central LLC
