Spam Filtering with Port Forwarding and Geo-Location

From MikroTik Wiki
Jump to: navigation, search

or how I defeated SPAM!

Background

The procedures outlined in this document should work with any mail server software.

For several years I have been administering a small ISP. We regularly receive approximately 20,000+ legit emails per day for approx. 5000 customers. We use several servers running postfix and amavisd-new. We have servers setup for our MX Records which forward to our mail storage server. We have been using greylisting (policyd v1) until recently when I concluded that it's effectiveness had diminished significantly. It was also becoming a problem with those who have blackberries and demand immediate mail delivery.

We recently moved all of our servers behind a trusty power-router (mikrotik x86) firewall and I was determined to find a way to use it effectively. With all of the servers having an internal address scheme, I now have control over which mail server handles incoming mail. To this end I added a couple of extra MX records to our primary domain to fool spammers.


Load Balancing & MX Failover

First off I wanted to load balance the inbound mail. My two inbound MX Records point to the router's IP's of XXX.XXX.237.4 and XXX.XXX.237.6 which forward to the respective inbound mail servers which are behind the firewall. So I created the following nat rules:

add action=dst-nat chain=dstnat comment="Port forward for incoming mail" disabled=no\
dst-address=XXX.XXX.237.4 dst-port=25 protocol=tcp src-address=0.0.0.0/1 to-addresses=10.100.100.4 to-ports=25
add action=dst-nat chain=dstnat comment="Port forward for incoming mail" disabled=no\
dst-address=XXX.XXX.237.4 dst-port=25 protocol=tcp src-address=128.0.0.0/1 to-addresses=10.100.100.5 to-ports=25
add action=dst-nat chain=dstnat comment="Port forward for incoming mail" disabled=no\
dst-address=XXX.XXX.237.6 dst-port=25 protocol=tcp src-address=0.0.0.0/1 to-addresses=10.100.100.5 to-ports=25
add action=dst-nat chain=dstnat comment="Port forward for incoming mail" disabled=no\
dst-address=XXX.XXX.237.6 dst-port=25 protocol=tcp src-address=128.0.0.0/1 to-addresses=10.100.100.4 to-ports=25

If you look at the rules you will see that I am directing mail based on the source addresses. If you are in the top half of IPV4 address space then you will hit one server, if you are in the bottom half you will hit the other mail server. This essentially makes both my primary and secondary MX records load balanced, since they are both receiving inbound mail based on the location of the sender. MX Failover continues to work properly in the event that one mail server goes down.


Geolocation

Now here is where it get's interesting. Being that I am in Canada, 97% of legit mail hitting our servers is most likely originating from English speaking countries. A few google searches of SPAM from different regions, reveals that large quantities of SPAM are originating in China, Korea, Brazil, and Russia etc.

So in the interest of fighting SPAM I was curious how I could filter by country. First off I setup a postfix server with some additional rules that I don't normally get away with. Things like reverse-dns helo checks and additional blacklists. I also setup policyd (Version 2) greylisting with a 35 minute window.

So I found the following website and generated a list of english speaking countries IP addresses. [1]

Now I ended up with a list of IP Blocks on the order of 28,000 long. I though that was a bit much and wondered if someone had written a CIDR Compressor and stumbled onto this program. [2] It's linux so you'll have to figure that bit out yourself. Once I had a working file I dumped it into OpenOffice spreadsheet and added the necessary tags to create a mikrotik.rsc file so that I could generate an address list. Due to mikrotik's file size limits that may not work. Instead open up an ssh session to your router and paste the contents into the router. It worked fine, took about five minutes and I had a complete list of all 20,000 english speaking IP Blocks.

Okay now for the fun part. One rule to rule them all!!!

add action=dst-nat chain=dstnat comment="Non-English speaking countries - go to third smtp machine"\
disabled=no dst-port=25 protocol=tcp src-address-list=!english to-addresses=10.100.100.3 to-ports=25

Notice the not (!) symbol in the src-address-list. Put this rule above the other rules. So now you have all inbound foreign mail going to a special server.

Here are some graphs of it's effectiveness. While I don't have before and after graphs it is definitely impressive. The amount of mail delivery attempts reaching my main inbound servers has plummeted. The foreign server now receives the majority of the mail delivery attempts. Normal greylisting systems suggest a five minute delay, with a thirty plus minute delay you greatly increase the chance that the spammer will have moved on. A delay also increases the chance that the sender will have been blocked by some sort of blacklist. Pay careful attention to the rejection levels in these graphs.

The received levels show all of the mail that was accepted for delivery, the sent level is the mail that was sent on to our customers. The difference between received and sent is the SPAM Level. Messages that have a Spam Assassin score of 13 or higher are automatically discarded. We have never had any false hits, that I'm aware of.

You will have a massive reduction in the level of attempted traffic hitting your primary mail servers, all of the incoming connection/rejection load is transferred off to the Third (Foreign) Spam Server. Your main maillogs are actually readable again.

You can use this method to target specific countries, depending on where you live. Targeting Asian and South American countries will make a big difference.

For those that are interested here is my main.cf from postfix, this is the machine with the additional filtering. The helo rules are quite restrictive. The "sleep 20" rule is also used to simulate a busy mail server and try to get spammers to disconnect.

content_filter = smtp-amavis:[XX.XX.XX.XX]:10024
smtpd_delay_reject = yes
smtpd_helo_required = yes
disable_vrfy_command = yes
smtpd_client_restrictions = sleep 20, permit
smtpd_sender_restrictions = permit_mynetworks, reject_unknown_sender_domain, permit
smtpd_helo_restrictions = permit_mynetworks, check_helo_access hash:/etc/postfix/helo_access,
reject_unknown_helo_hostname, reject_non_fqdn_hostname, reject_invalid_hostname, permit
smtpd_data_restrictions =
       reject_unauth_pipelining,
       reject_multi_recipient_bounce,
       permit
smtpd_end_of_data_restrictions =
       check_policy_service inet:XX.XX.XX.XX:10031,
       permit
smtpd_recipient_restrictions =
       check_recipient_access hash:/etc/postfix/bypass_policyd
       check_sender_access hash:/etc/postfix/sender_access,
       check_client_access hash:/etc/postfix/evil-clients,
       reject_unauth_pipelining,
       permit_mynetworks,
       reject_unknown_recipient_domain,
       reject_unauth_destination,
       reject_invalid_hostname,
       reject_non_fqdn_sender,
       reject_non_fqdn_recipient,
       reject_unknown_sender_domain,
       reject_unlisted_recipient,
       reject_rbl_client zen.spamhaus.org,
       reject_rbl_client bl.spamcop.net,
       reject_rbl_client hostkarma.junkemailfilter.com=127.0.0.2,
       reject_rbl_client psbl.surriel.com,
       check_policy_service inet:XX.XX.XX.XX:10031,
       permit
smtpd_recipient_limit = 35
smtpd_soft_error_limit = 7
smtpd_hard_error_limit = 15 
unknown_local_recipient_reject_code = 550
unknown_hostname_reject_code = 550

Fixing Helo/Greylisting Issues

Should you receive customer complaints, or have problems with hits on the above rules just train your techs to add/remove entries for that servers IP Block to the routers address lists.

Postscript

This method has been running for over a month now, with far fewer complaints then before. We haven't had any SPAM complaints from our customers either. Found a couple of IP Blocks for the Canadian Government that were not listed properly in the database, that was fixed up by adding them to the english address list.