Thursday, March 11, 2010

Five Critical Steps for Configuring an SMTP Mail Server

The most important thing you can do to protect the deliverability of your organization's email messages is to configure your mail server properly. More and more Internet Service Providers (ISPs) are rejecting messages based on improper mail server configuration, because an improper configuration is often an indication of a spammer at work. This article describes a few critical things you can do to prove that your messages are legitimate.

To demonstrate the following tips, I'll use a mythical mail server called smtp.mycompany.com. This mail server sends messages out through IP address 192.168.0.1.

Step 1: Assign an IP address and a dedicated domain name to the mail server.

Your SMTP mail server should have a dedicated domain name like smtp.mycompany.com. Avoid using the same domain name for your SMTP (outgoing) service and your POP3 (incoming) service, particularly if you think you might ever put them on separate IP addresses. Many organizations use something like mail.mycompany.com for the POP3 service and smtp.mycompany.com for the SMTP service. Just remember that an IP address can have multiple domain names associated with it, but a domain name can reference only one IP address.

In theory, it shouldn't matter if you dedicate an IP address to the SMTP service and nothing else, but it can't hurt to do so.

The action item for this step is to create the new subdomain host record (or an "A" record) in DNS and associate it with the IP address from which the service will send messages. In my case, I'd create an "smtp" subdomain and assign it to 192.168.0.1.

Step 2: Configure the mail server identity.

When you set up the SMTP service, put it on the IP address you selected and make sure the service identifies itself (sometimes called the "HELO" name) using the domain name you selected. In my example, the SMTP server at 192.168.0.1 identifies itself as "mail.mycompany.com."

Why are these first two steps important? Some receiving mail servers do a "HELO Lookup" to verify that the sending mail server is who it says it is. The receiver takes the domain name claimed by the sender and does a DNS lookup on that name. If the IP address that comes back from the lookup does not match the IP address the sender is using, the receiver rejects the message.

Step 3: Lock down message relay.

A "message relay" refers to when a computer connects to an SMTP server and hands it a message to send. For example, when you press the Send key in your mail client, you are "relaying" a message through your ISP's mail server.

Most SMTP services let you configure what IP addresses are allowed to connect and send messages. If you allow all IP addresses to connect, your server is considered an "open relay" and it will probably get found, abused, and then blacklisted very quickly.

Instead, limit the connections to known IP address ranges on your network. If the mail server will only be used to send messages from the local machine (web servers are sometimes set up this way), lock it down to allow only local connections.

Step 4: Enable reverse-DNS lookups.

DNS is normally thought of as a "forward lookup" service where the goal is to resolve an IP address from a domain name. Reverse DNS refers to the practice of resolving a domain name based on an IP address.

In order to configure a reverse lookup in DNS, you must be the registered owner of that IP address. Even if your ISP allows you to manage your own domain names, there's a good chance that you'll need their help in configuring a reverse lookup, because they "own" the IP addresses they are loaning you.

However you must manage it, the goal is to create a pointer (PTR) record in DNS. That record associates the IP address (192.168.0.1) with the domain name (smtp.mycompany.com).

The reason for this step is that some ISP's use a reverse DNS lookup to verify that the mail server's identity is agreed upon by both the owner of the IP address and the owner of the domain name. This filter is not widely used because it is slow and compliance is low.

Step 5: Configure an SPF record.

Sender Policy Framework (SPF) is a pseudo-standard that gives organizations a way to identify the domain names and IP addresses of authorized email servers. In other words, it gives the mythical "My Company" a way to identify mail servers that are allowed to send messages from the "mycompany.com" domain name.

SPF is a voluntary standard, and as such, a largely unreliable one. Many ISP's do not reject messages that fail to pass SPF testing, although they may pass along SPF test results in a custom message header (as of this writing, gmail does exactly that). That said, some ISP's DO reject messages that fail SPF, so it is a good idea to set up an SPF record for your organization.

Explaining how to set up SPF is an article in itself. The bottom line is that you need to set up an SPF and/or TXT record in DNS that contains SPF syntax for identifying valid mail senders. Just as an example, here's how the content of the MyCompany.com SFP record might look:

   v=spf1 a mx -all

That SPF instruction says to allow any sender that matches an existing A (host) or MX (mail exchange) record defined for the domain, and disallow all others. I set up a host record for smtp.MyCompany.com, so it should pass these rules.

Troubleshooting

If you run into difficulties implementing any of the above recommendations, you can use the following tools to troubleshoot:

Forward/Reverse DNS Lookup by ZoneEdit.com

SPF Record Testing Tools by Kitterman.com

Conclusion

I have to say that I really hate email. I love the idea of email, but the technical and political reality of email is enough to make you want to pull the plug on your computer forever.

The primary source of my loathing of all things email relates to the fact that I've had to help solve email delivery problems for large and small companies alike. I've also had to write a lot of program code relating to email automation and management. I say "had to" because this kind of work has never been my preference.

From a user and even a developer's viewpoint, email is pretty simple and easy to use. But from a system administrator's viewpoint, it is a nightmare. Spam is ultimately responsible for most of the angst. No government or technical intervention has been able to solve the real problem, which is that spammers are still able to exist. The reality is that we are all stuck with an antiquated and unsecure messaging network, and we have to resign ourselves to dealing with it.

I hope the suggestions I presented here will help you get your messages to their destinations and avoid getting blacklisted in the process.

2 comments:

Elsayed said...

Hi:
I am trying to set up james apache server and facing a problem in config.xml file to send mails to gmail or yahoo accounts whithout having a Domain name just from local machine.
Can I have your help in the configuration?

thanks in advance.

James H. Byrd said...

Elsayed:

The information in this article talks about subjects that are common to all mail servers, but my actual mail server configuration experience is limited to IIS and one other third-party service.

Sorry, I don't do Apache, but I'm sure you can find many experts out there who can help you.

Good luck with your project!

Post a Comment