Five Critical Steps for Configuring an SMTP Mail Server

The most important thing you can do to protect the deliverability of your organization's email messages is to configure your mail server properly. More and more Internet Service Providers (ISPs) are rejecting messages based on improper mail server configuration, because an improper configuration is often an indication of a spammer at work. This article describes a few critical things you can do to prove that your messages are legitimate.

To demonstrate the following tips, I'll use a mythical mail server called smtp.mycompany.com. This mail server sends messages out through IP address 192.168.0.1.

Step 1: Assign an IP address and a dedicated domain name to the mail server.

Your SMTP mail server should have a dedicated domain name like smtp.mycompany.com. Avoid using the same domain name for your SMTP (outgoing) service and your POP3 (incoming) service, particularly if you think you might ever put them on separate IP addresses. Many organizations use something like mail.mycompany.com for the POP3 service and smtp.mycompany.com for the SMTP service. Just remember that an IP address can have multiple domain names associated with it, but a domain name can reference only one IP address.

In theory, it shouldn't matter if you dedicate an IP address to the SMTP service and nothing else, but it can't hurt to do so.

The action item for this step is to create the new subdomain host record (or an "A" record) in DNS and associate it with the IP address from which the service will send messages. In my case, I'd create an "smtp" subdomain and assign it to 192.168.0.1.

Step 2: Configure the mail server identity.

When you set up the SMTP service, put it on the IP address you selected and make sure the service identifies itself (sometimes called the "HELO" name) using the domain name you selected. In my example, the SMTP server at 192.168.0.1 identifies itself as "mail.mycompany.com."

Why are these first two steps important? Some receiving mail servers do a "HELO Lookup" to verify that the sending mail server is who it says it is. The receiver takes the domain name claimed by the sender and does a DNS lookup on that name. If the IP address that comes back from the lookup does not match the IP address the sender is using, the receiver rejects the message.

Step 3: Lock down message relay.

A "message relay" refers to when a computer connects to an SMTP server and hands it a message to send. For example, when you press the Send key in your mail client, you are "relaying" a message through your ISP's mail server.

Most SMTP services let you configure what IP addresses are allowed to connect and send messages. If you allow all IP addresses to connect, your server is considered an "open relay" and it will probably get found, abused, and then blacklisted very quickly.

Instead, limit the connections to known IP address ranges on your network. If the mail server will only be used to send messages from the local machine (web servers are sometimes set up this way), lock it down to allow only local connections.

Step 4: Enable reverse-DNS lookups.

DNS is normally thought of as a "forward lookup" service where the goal is to resolve an IP address from a domain name. Reverse DNS refers to the practice of resolving a domain name based on an IP address.

In order to configure a reverse lookup in DNS, you must be the registered owner of that IP address. Even if your ISP allows you to manage your own domain names, there's a good chance that you'll need their help in configuring a reverse lookup, because they "own" the IP addresses they are loaning you.

However you must manage it, the goal is to create a pointer (PTR) record in DNS. That record associates the IP address (192.168.0.1) with the domain name (smtp.mycompany.com).

The reason for this step is that some ISP's use a reverse DNS lookup to verify that the mail server's identity is agreed upon by both the owner of the IP address and the owner of the domain name. This filter is not widely used because it is slow and compliance is low.

Step 5: Configure an SPF record.

Sender Policy Framework (SPF) is a pseudo-standard that gives organizations a way to identify the domain names and IP addresses of authorized email servers. In other words, it gives the mythical "My Company" a way to identify mail servers that are allowed to send messages from the "mycompany.com" domain name.

SPF is a voluntary standard, and as such, a largely unreliable one. Many ISP's do not reject messages that fail to pass SPF testing, although they may pass along SPF test results in a custom message header (as of this writing, gmail does exactly that). That said, some ISP's DO reject messages that fail SPF, so it is a good idea to set up an SPF record for your organization.

Explaining how to set up SPF is an article in itself. The bottom line is that you need to set up an SPF and/or TXT record in DNS that contains SPF syntax for identifying valid mail senders. Just as an example, here's how the content of the MyCompany.com SFP record might look:

   v=spf1 a mx -all

That SPF instruction says to allow any sender that matches an existing A (host) or MX (mail exchange) record defined for the domain, and disallow all others. I set up a host record for smtp.MyCompany.com, so it should pass these rules.

Troubleshooting

If you run into difficulties implementing any of the above recommendations, you can use the following tools to troubleshoot:

Forward/Reverse DNS Lookup by ZoneEdit.com

SPF Record Testing Tools by Kitterman.com

Conclusion

I have to say that I really hate email. I love the idea of email, but the technical and political reality of email is enough to make you want to pull the plug on your computer forever.

The primary source of my loathing of all things email relates to the fact that I've had to help solve email delivery problems for large and small companies alike. I've also had to write a lot of program code relating to email automation and management. I say "had to" because this kind of work has never been my preference.

From a user and even a developer's viewpoint, email is pretty simple and easy to use. But from a system administrator's viewpoint, it is a nightmare. Spam is ultimately responsible for most of the angst. No government or technical intervention has been able to solve the real problem, which is that spammers are still able to exist. The reality is that we are all stuck with an antiquated and unsecure messaging network, and we have to resign ourselves to dealing with it.

I hope the suggestions I presented here will help you get your messages to their destinations and avoid getting blacklisted in the process.

A Lot of Transact-SQL Still Out There

I had a question from a reader of my Nerdy Musings site today about how to check for a missing stored procedure parameter in Transact-SQL. I sent him a response and decided to write a quick article about the subject:

Using Optional Stored Procedure Parameters

While answering his question, I thought about how much Transact-SQL is still out there in the world. Even though SQL Server lets you create stored procedures with the more robust .NET languages like C# and Visual Basic, there are still millions of lines of Transact-SQL code executing every day, and developers create more all the time.

In fact, my article on Coding Transactions in Transact-SQL consistently ranks in the top five articles on Nerdy Musings, and it is usually in the top two.

It just goes to show you that new technology does not make old technology any less useful. I would not be surprised if Microsoft eventually drops support for Transact-SQL, but for now, I can still take advantage of the hard-won skills I have developed in it and share my expertise with other developers who need help.

The Agency Model for Book Pricing Doesn't Make Sense

The major publishers are pushing their heads into the sand once again. They seem to think that books are special products that deserve different treatment than any other retail product. The Agency Model that Macmillan and Hachette have adopted is just another way of clinging to obsolete business rules. Granted, Amazon's reaction to Macmillan's ultimatum was a little over the top, but I think they have a stronger leg to stand on.

If you've been watching the publishing news, you know that Amazon removed the buy buttons from all Macmillan books on Friday, January 29 after Macmillan informed Amazon of their intention to go to an Agency Model for book pricing. Although negotiations started immediately, the buy buttons were still gone on most Macmillan books well into this week, at least.

In a nutshell, the agency model means that the publisher sets the retail price of the book and gives the retailer a commission. This approach gives the publisher control over the price of the book and lets them stabilize the payments that get made throughout the channel along the way to the customer's hands, including author royalties and the publisher's profit margin.

But does this approach make any sense? Not at all.

With most retail products, the manufacturer (or publisher in this case) sets a wholesale price and retailers are free to sell the product at whatever price they can get. The manufacturer controls the wholesale price and the retailer controls the retail price. Each is in control of their own profit margin.

Yes, the book market has legacy business arrangements like returns and author royalties that complicate matters, but those elements should be simplified and incorporated into the cost of goods sold.

From Amazon's perspective, the Agency Model dictates what they can charge for their products and puts publishers in charge of Amazon's profit margin. No wonder Amazon balked.

The whole point of a retailer is to provide service to the consumer. They do that through marketing efforts, convenience factors, and customer services. On the other hand, a wholesaler/manufacturer/publisher has made a conscious decision to make their money on bulk sales to resellers (business-to-business) rather than deal with consumers directly (business-to-consumer). Wholesale is a very different business from retail, and wholesalers have no business telling retailers how to run theirs.

To be fair, it works both ways. Retailers have no business telling wholesalers how to run their business either. Amazon, Wal*Mart, and other superstores have shown how messed-up things can get when they do. In the name of "customer service," these retailers abuse their power in the marketplace to force wholesalers into a pricing model that is often not sustainable. This is a case where the retailer is trying to take control of the wholesaler's margin. The customer benefits in the short run with lower prices, but loses in the long run when their choices are reduced to the few high-volume manufacturers who were able to compromise product quality and improve production efficiency enough to satisfy the mega-retailer's demands.

In many ways, Amazon itself inspired publishers to implement agency pricing. The publishers were reacting in part to Amazon's strong-arm tactics of establishing a market price of $9.99 for an e-book because "that's what customers are willing to pay." What they really mean is "that is the price that will give us the sales volume we want," which is not the same thing. The problem is that even Amazon admits that they don't make any money at that price. And they are trying to force the publishers to accept even less?!

I believe both sides need to stop screwing with each other and get back to the basics. Publishers need to set a wholesale price that covers their costs and gives them the margin they want. Retailers need to resell the product at a marked-up price that covers their costs and gives them the margin they want.

It really is that simple.

Integrating AWeber with ASP.NET Web Sites

Ever since our company decided to use AWeber to manage our email lists, I've been in the business of integrating AWeber services with our ASP and ASP.NET web sites. I have to say that few tasks have been more frustrating. It's not so much that AWeber causes problems, it's just a Murphy's Law kind of thing exacerbated by the fact that lots of things CAN go wrong!

The difficulties in integrating AWeber with an ASP.NET Web site are particularly challenging, and I know I'm not alone in my frustration. Each time I've fallen into the AWeber time vortex, I've done Web searches and found many blog posts expressing similar angst. In the end, I get things working, but not without a lot of trial-and-error-and-trial-and-error-and-trial-and-error.

In an effort to save some of my fellow ASP.NET developers from a similar experience, I've written articles about various AWeber integration topics and I developed and published a free conversion tool for AWeber subscribe forms.

I recently published another AWeber-related article on my Nerdy Musings web site, which is where I publish most of my software development articles, and I thought I would take this opportunity to cross-reference all of the AWeber articles here in my blog.

Posting to an Off-Site Page in DotNetNuke

The "posting" article describes how to get an AWeber subscribe form to work in DotNetNuke. The same basic techniques can be applied to any ASP.NET Web site, as described in the next article.

Embedding Forms in an ASP.NET Web Page

This "embedding" article explains how to embed any kind of form-based widget (such as an AWeber subscribe form or a PayPal button) in an ASP.NET page.

Subscribing Customers to an AWeber List with an ASP.NET Application

The "subscribing" article is the latest in my AWeber series. It explains how to subscribe your site visitors to an AWeber list without making the visitor fill out a subscribe form. It also shows you how your application can detect and respond to successful subscribe confirmations.

AWeber Form Conversion for DotNetNuke

This free utility converts the form code generated by AWeber into something that can be pasted into an ASP.NET Web page.

If you are an ASP.NET developer tasked with integrating AWeber subscriptions into your applications, be sure to give these articles a read as they could save you boatloads of time.

Enjoy!

The Written Word Still Rules

The proliferation of high-speed Internet access and powerful computers with massive hard drives has made it possible for you to deliver more of your content in audio and video format. But just because you can, that doesn't mean you should. At least, not exclusively.

Granted, many people are visual or auditory learners, and a picture can definitely paint a thousand words, but A/V content seems to work best as an accompaniment to written material, not as a replacement for it.

The main problem with A/V content is that it is serial and linear. You can't "skim" it. You can't "speed read" it. You are stuck accepting the information at the pace it was originally recorded. Sure, you can fast forward or skip around, but that process is slow, cumbersome, and leaves you with no idea of what you might have skipped.

How many audio programs have you downloaded with the intent to "listen to it one of these days," but you never have? Conversely, how many times have you done an Internet search and quickly scanned dozens of pages to locate the information you need?

The book industry is seeing a rapid transition to electronic media with supplementary content in the form of audio, video, and Web page links. But that doesn't mean books are going away. It just means that digital books will provide extra resources that improve the learning or entertainment experience for readers who enjoy and benefit from the additional materials.

The key is to make sure you don't segregate any important part of your message into these alternative formats. When considering what content to provide in audio or video format, think "supplementary" or "alternative," not "essential." Your audience, or more to the point, your readers, will appreciate it.

Publishers Do Readers a Disservice by Withholding eBook Editions

It has long been true in the book industry that publishers hold back paperback editions for about a year after they release the hardback edition. Ebooks are now receiving the same treatment by many publishers. The theory is that paperback and ebook editions would cannibalize hardcover sales if they were made available at the same time. In other words, publishers are banking on the idea that you will buy the hardcover because you can't wait for the format you really want.

I've read many opinions on the subject, and most of them focus on the politics involved between the publishers, agents, authors, and the retail sales channels. I think they are all missing the point, or perhaps just I am. Almost no one focuses on the reader, except to talk about how the current strategy manipulates buyers into paying a premium for the hardcover edition by preying on their impatience. That strategy has never worked on me, so perhaps that is why my thoughts in this subject have a different focus.

Realistically, the only difference between the three book formats is the packaging and, artificially, the timing. There is no practical reason why all three could not be released at the same time. The content is the same in all of the editions.

Imagine what would happen if the publisher released all of the editions together? You would be able to buy the book in your preferred format along with everyone else. Good grief, that almost sounds like good customer service!

Instead, publishers create artificial scarcity by releasing the other editions long after the hardcover has come out. They want you to pay extra money for the edition you don't really want just so you can have the convenience of reading the book earlier. That kind of manipulation just annoys me.

Now, someone might point out that Logical Expressions has not made ebook editions of all our books available yet. That has nothing to do with manipulation and everything to do with me having the time to do the ebook conversions!

If you want to see the book industry change these manipulative practices, you have it in your power to do so. You can vote with your wallet. The first step is to refuse to buy a book in a format you don't want. The second is to specifically request the book in the format you DO want.

Publishers will only continue the current practices as long as those practices continue to work. Manipulation requires two participants: the manipulator and someone willing to be manipulated.

Software Applications Require an Ongoing Investment

The computer and software development industries are always focused on the Latest and Greatest as well as the Next Big Thing, but the businesses that utilize software are focused on trying to get a Return on Investment. These viewpoints are competing, if not mutually exclusive. As a result, the computing environment is always moving forward, and business applications are always falling farther behind.

Many businesses still don't understand that investing in software, particularly custom software, is an ongoing commitment. At the current pace of technology, software applications have a useful life of about 5 years. That lifespan can be stretched to 10 years, but you'll be jumping through hoops just to keep the application limping along there at the end.

It's not just about the money

In the meantime, the business has not only invested money, it has often developed a large business data asset that depends upon the application. Not to mention the investment in training to develop user expertise. Moving to another application generally means completely losing the training investment and facing a difficult or impossible data migration.

Upgrading an application can be just as challenging. It is rarely possible to import the old code base into the latest development tools, and even if it is possible, it usually produces poor results. Remember when developers tried to convert classic Visual Basic applications to Visual Basic .NET? Sure, Microsoft gave you a migration tool, but it was maybe an 80% solution that left you with a really crappy .NET application.

New development tools reflect new development practices that take advantage of new capabilities in the operating environment, so the old application code must often be completely replaced.

Unfortunately, most businesses look at the investment in any software as a one-time expense. But the truth is that even off-the-shelf software has to be upgraded fairly frequently. Even though nothing changes with the software itself (it's not like the code "wears out"), the operating environment changes. The business has to move to new computers and new operating systems because the old ones are no longer supported. Eventually, the system facilities that support the application simply don't function any more, as we are starting to see right now with 32-bit applications on 64-bit computers.

The business environment changes too

So far I've only described the changes on the technical side, but the business side changes as well. While many of the core business requirements may remain consistent, applications often need to adjust to new requirements that are brought on by changes in the business environment. I'm not just talking about usability enhancements here, I'm talking about complete changes in the way information is collected and analyzed.

Eventually, something has to give, and it's usually an emergency when it does. One day, the application can't be installed on that new computer you just bought. Or the auditor says you aren't keeping proper track of some critical piece of business information and could face fines or legal action. Management goes up in flames, and the IT staff works around the clock to come up with a solution fast.

Get a handle on the problem

How do you prevent an emergency like this from happening to you? I have a saying: you can't plan for the unexpected, but you can expect the unplanned.

The first step is to realize that maintaining your business data assets is a circular process, not a short, linear one. Whether you use off-the-shelf software or develop your own, the application requires a regular evaluation and maintenance schedule. It may not be necessary to take action every time through the cycle, but ignoring the need for the cycle eventually leads to a meltdown of some variety.

The second step is to recognize that the true value of a business application is in the data it stores and the interface it presents to your users. The data repository and the programming code may be completely replaced over time, but the integrity of the data and the familiarity of the user interface are assets that have tremendous long-term value to your organization and a big impact on your day-to-day operations.

Your best protection from the ever-changing technical and business environment is to keep an eye on things. Get out of the "we'll upgrade when we're forced to" rut and put upgrading into your schedule where it can be managed gracefully.