Help! What is best way to manage email against spam?

[bosveld]

I am managing my own email server (here at rimuhosting). Me and friend have about 20-30 email adresses on the server.

Over time I am receiving more and more spam (nothing to do with rimuhosting -- simply as our email addresses are becomming more well known). I also have an gmail account with almost zero spam -- so obviously some solutions do work.

What do I need to do to manage the spam?
Can I manage it myself?
Do I need to go via another service / company
(.. I am willing to pay for it.. not a lot though)
Does rimuhosting provide any services around this?Please help .. as we am getting drowned in spam

Any suggestions will be appreciated

Thanks
Liam

[timharig]

Help! What is best way to manage email against spam?

Don't get any. Never, ever, under any circumstances give away your main e-mail address. Always give away aliases. I suggest creating a separate alias for every website where you enter an e-mail address. First, you will aways know who is giving your e-mail address away to spammers and second you can burn any aliases that have started to receive spam. This also has the advantage of making it easy to automatically sort your incoming mail based on the incoming address.

If that doesn't work for you then I suggest using ASK (http://a-s-k.sourceforge.net/) or a similar proactive mail filtering system.

[bosveld]

tamharig --

I agree.
Problem is other users have not been so good at it. So now they battle. I am the administrator -- hence the question.

Feedback from someone has been to use SpamAssassin.
So I am also looking into that.

Thanks
Any other comments?

[timharig]

I had assumed when you mentioned that these addresses were for you and a friend that this was involving a personal system.

I cannot recommend any filtering technology as I find their fundamental design to be flawed by default. The spammer always has the upper hand as the system administrators and filter are always playing a catch up game. At least one message must get through and be marked as spam before the system can try to stop it. Then there are problems with false positives inherent in any filtering technology. Then end result is that filtering technologies "sort of work" but they place a heavy burden on system administrators. The advantage is always on the spammer's side.

Active systems such as ASK that I linked to above kill spam cold without generating any false positives. They send challenges to any addresses that are not contained in the users address book which require unknown users to prove that they are human before adding them to the address book and permitting their messages to pass. This stops 100% of spam in general and requires no work of any kind on the part of the users or yourself.

[ldryden]

I've had reasonbly good luck with greylisting in terms of:

- False positives vs. false negatives
- Maintenance hassle

It doesn't rely on another service being reachable to check reputations; it can do all its work on the existing mail pool. It doesn't need any special, human-guided "training" for the most part, and almost all RFC-complaint mail services manage to do the right thing. The main annoyance is that "new" e-mails or "new" MTAs do not manage to deliver e-mail all that promptly, at least not at first. As your system develops a trail of "familiar" senders, it gets faster and faster.

This is a specific measure to stop zombie-bot spam. It's no use against unsolicited e-mail sent from a for-real mail server, but I find that for myself I found some 3/4 of the garbage e-mail being logged as attempted just once, and never seen again.

[tetech]

Hi, I can make a few suggestions.

1. Be sure that you are using greylisting. In my experience, this alone filters 90% of the spam. It's not very resource-hungry and it shouldn't get rid of any wanted email. Only real negative is that email from senders not on your 'whitelist' is delayed slightly.

2. Route your email through a service like spamcop.net. $30/yr/person which I consider "not much".

Currently I get about one spam email per day. If these two services are disabled, I get over 350.

3. SpamAssassin is another useful tool. In my experience, spamcop.net and SpamAssassin are about equally effective at detecting spam, although you do get a small additive benefit (i.e. spamcop.net detects about 5% of the spam that SpamAssassin misses, and vice versa).

One issue with SpamAssassin is that it can take a bit of memory, which is a premium on your VPS. A second issue is what to do with the spam after it has been detected as such - this is one reason I'm happy to pay for spamcop.net. However, SpamAssassin is also free and has little delay, whereas spamcop.net becomes costly if you have many low-volume users.

[Lloyd]

I've had great success at stopping spam cold, with zero false positives and low memory use for my server. In fact, one of the main reasons I got a VPS was to be able to implement effective spam blocking.

I stop almost all spam at the SMTP level - that is, before the data portion of the email is ever accepted. Results: out of about 500-600 spams hitting my server per day (last I checked), only about one or 2 per day get through. (The procmail recipes mentioned below, which do use content filtering, are relatively insignificant)

I use postfix. I use 3 major techniques:

1) First the message is checked for SMTP protocol compliance, plus a few other related checks. In addition, unknown email addresses are rejected at this stage.

The names of some of the tests in postfix give you an idea of what this is about:
reject_invalid_hostname,
reject_non_fqdn_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
reject_unauth_pipelining,
permit_mynetworks,
reject_unauth_destination

2) Next the message sender IP is checked against several DNS blocklists. The most important are cbl.abuseat.org, dul.dnsbl.sorbs.net, and list.dsbl.org, in that order. This cuts out at least 70% of spam.

3) messages that get through #1 and #2 are then greylisted. Greylisting kills another approx. 90%+ of spam (I did actual measurements). To be honest, I could probably get by with only #1 and #2. I use sqlgrey with a mysql database.

With such a 3 pronged system, I find it completely unnecessary to do content filtering (spam assassin), and certainly unnecessary to do challenge-response. I used to use my own procmail+perl challenge-response system for years, before I got my own mail server.

Getting sqlgrey installed and working might be more of a challenge, but that's part of the fun of having your own server! I run Debian Etch - a great distro, and Rimuhosting servers are the best!

Here's an excellent starting place for configuring postfix for #1 and #2 above: http://jimsun.linxnet.com/misc/postfix-anti-UCE.txt

In addition, I use procmail recipes to get rid of spam that, for example, pretends to come from PayPal but does not carry my PayPal username. Those all go directly to /dev/null.

Tip: before you make a recipe to do that, test it first like this:

SUBJ_=`formail -xSubject: | expand | sed -e 's/^[ ]*//g' -e 's/[ ]*$//g'`

:0
* -1^0
* 1^0 ^Subject:.*naked
* 1^0 ^Subject:.*nude
* 1^0 $ B ?? naked
* 1^0 $ B ?? nude
{
:0 fiwh
| formail -I"Subject: (nude spam) ${SUBJ_}"
}

(more recipes)

:0:
$DEFAULT

This adds the comment "(nude spam)" to the subject of a suspect mail, instead of dumping it. When you're sure your recipe works right, you can sent it to /dev/null.

Good luck!

[bosveld]

At the end what I did was to root all my email via google apps. Are using it for about 20 users and all is working fine for about a month now.

Thanks for all the suggestions

[holisticgp]

Yes, have cut down spam from 200+ / day to < one / day but had a few implementation issues along the way

Step 1.
I followed the instructions re installing postgrey on:
http://bliki.rimuhosting.com/space/knowledgebase/linux/mail/greylisting+with+postgrey

This included
apt-get update
apt-get upgrade...all seemed to go well
then
apt-get install postgrey....not there
apt-cache search or apt-cache show...no sign of postgrey

Step 2
-->I contacted Paolo in support @ rimuhosting who did the following:
he first proceeded to add the additional repositories for CentOS to allow
the installation of postgrey via apt-get.

[root@myserver ~]# wget
http://dag.wieers.com/rpm/packages/rpmforge-release/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
--13:13:27--
[Saving to: `rpmforge-release-0.3.6-1.el5.rf.i386.rpm'

13:13:29 (90.9 KB/s) - `rpmforge-release-0.3.6-1.el5.rf.i386.rpm' saved
[16698/16698]

[root@myserver ~]# rpm -Uvh rpmforge-release-0.3.6-1.el5.rf.i386.rpm
warning: rpmforge-release-0.3.6-1.el5.rf.i386.rpm: Header V3 DSA
signature: NOKEY, key ID 6b8d79e6
Preparing... ###########################################
[100%]
1:rpmforge-release ###########################################
[100%]



Once this has been done, I refreshed the repository via apt-get update then installed postgrey.

[root@myserver ~]# apt-get install postgrey

Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
perl-BerkeleyDB perl-Digest-HMAC perl-Digest-SHA1 perl-IO-Multiplex
perl-Net-DNS perl-Net-Server
The following NEW packages will be installed:
perl-BerkeleyDB perl-Digest-HMAC perl-Digest-SHA1 perl-IO-Multiplex
perl-Net-DNS perl-Net-Server postgrey
0 upgraded, 7 newly installed, 0 removed and 15 not upgraded.
Need to get 818kB of archives.
After unpacking 1842kB of additional disk space will be used.
Do you want to continue? [Y/n]

Fetched 818kB in 1s (446kB/s)
Committing changes...
Preparing... ###########################################
[100%]
1:perl-BerkeleyDB ###########################################
[ 14%]
2:perl-IO-Multiplex ###########################################
[ 29%]
3:perl-Digest-SHA1 ###########################################
[ 43%]
4:perl-Digest-HMAC ###########################################
[ 57%]
5:perl-Net-DNS ###########################################
[ 71%]
6:perl-Net-Server ###########################################
[ 86%]
7:postgrey ###########################################
[100%]
Done.

Step 3.
Then I followed the advice in the following:
http://bliki.rimuhosting.com/space/knowledgebase/linux/mail/greylisting+with+postgrey
Next Step for CentOS/RHEL:
chkconfig postgrey on

Next step for Centos/rhel:
Tell it to start postgrey (any distro) :
/etc/init.d/postgrey start

Next Step:
Now, tell postfix to use it:

CentOS/RHEL: This package listens on a unix socket by default, so edit /etc/postfix/main.cf and add 'check_policy_service unix:/var/spool/postfix/postgrey/socket' to 'smtpd_recipient_restrictions. Again, make sure to add it after permit_sasl_authenticated so as not to greylist yourself.

Example:
smtpd_recipient_restrictions =
permit_mynetworks
permit_sasl_authenticated
reject_unauth_destination
check_policy_service unix:/var/spool/postfix/postgrey/socket


So the new entry at bottom of my /etc/postfix/main.cf, including the additional smtpd_recipient_restrictions, is:
smtpd_sasl_auth_enable=yes
smtpd_helo_required = yes
disable_vrfy_command = yes
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, reject_non_fqdn_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_invalid_hostname, reject_unauth_pipelining, reject_unknown_sender_domain, reject_rbl_client cbl.abuseat.org, reject_rbl_client dul.dnsbl.sorbs.net, reject_rbl_client list.dsbl.org, check_policy_service unix:/var/spool/postfix/postgrey/socket, permit
broken_sasl_auth_clients = yes

(nb care when doing this with my sftp program (winscp) which was adding a false return...postfix did not work...to ensure no false returns, just have the line span the whole screen)

Step 4
Final Step:
/etc/init.d/postfix restart

Step 5
Troubleshooting:
tail -30 /var/log/maillog -->shows what is happening (eg the last 30 lines in this example) in postfix

Good luck!
Holisticgp

[Lloyd]

Sqlgrey (http://sqlgrey.sourceforge.net/) also works well with Postfix. I have been using it for going on 2 years with a mysql database, with excellent results.

Here's a list of features from the project's site:

* SQLgrey can withstand a database crash (grey-listing is automatically switched off)
* 3 grey-listing algorithms to choose from
* Support for file-based IP and FQDN whitelists
* Support for fetching up-to-date whitelists from a repository
* Can mail the admin when the database is unavailable
* Auto-whitelists now understand SRS (SPF-aware mail forwarding)
* IPv6 support
* OPTIN / OPTOUT support
* Fine log controls
* Activity reports

[dblevitan]

My major problem with many of the solutions posted is that I don't like denying any mail that could be coming from misconfigured servers or people running mail servers on dynamic IPs (as I used to before I got a VPS). I have integrated greylisting into my spam setup, but in a much more restricted way which traps most spam but allows most normal mail to arrive instantly. I think my system cuts removes almost all the spam I get (I used to get 300+ messages/day) while still allowing all the benefits of fast e-mail. Here are the steps involved:

1. Minimal postfix filtering - primarily on invalid users and a couple of SMTP protocol violations
2. All mail is checked against several conditions using a policy server called apolicy:
- no reverse ptr
- anything but pass or neutral from SPF
- listed on several RBLs
Anything that hits one of those conditions is greylisted. This hits almost all spam.
3. Anything that goes through here is sent through SpamAssassin with Bayes filtering. This removes almost all the rest of the spam.

The first two steps has dropped me from roughly 300 messages/day to about 40 (this includes all non-spam mail as well). Of that number, almost all spam is identified as such by SpamAssassin and either discarded (for very high scores) or dumped in a Junk folder (for lower scores).

So far I'm very happy. I get almost all the advantages of no greylisting while still removing almost all spam. If anyone wants further details I can provide them.