Any story about Internet email has to start back in the early 1970's when the first messages were sent across the Arpanet, the predecessor of today's Internet. In the olden days, email delivery was relatively simple, and generally consisted of moving mail files from one large host to another large host that served many users. As the Internet evolved and the network itself became more complex, more flexible tools were needed to move mail between different networks and different types of networks. The Sendmail package, released in the early 1980's was designed to deal with the many variations among mail systems. It quickly assumed a dominant role for mail delivery on the Internet.
Fast-forward to today where email continues to be the most widely used application on the Internet. However, nearly every Internet site now uses the SMTP protocol to deliver and receive mail messages. While the email landscape has become simpler in some ways, operating an email server can still be complex. This book takes you through setting up a Postfix server by showing you in partially narrative format how a fictional group of intrepid system administrators converted their own Sendmail server to Postfix. Based on their initial success, they build on their experience and eventually deploy Postfix throughout their company.
Our happy band works in the IT department at a medium-sized organization that recently merged with another company. Their manager, Dave, convened the group after receiving a security bulletin about Sendmail. The team has already patched their systems, but Dave wants to know if there are any alternatives that might be less vulnerable. As it happens, Jerry, the team lead, has recently been learning about and experimenting with Postfix.
"Postfix is one of the most popular alternatives to Sendmail," Jerry explains. "It's especially known for its reliability and security." Dave wants to hear more and asks Jerry to work on a brief presentation. Jerry puts together some background information for Dave and some other members of his team, including Maria and Will. Will, who has recently joined the team, has worked primarily with Windows and is coming up to speed on Unix concepts and administration.
Postfix Origins and Philosophy
Jerry begins his presentation by explaining that Postfix was developed by Wietse Venema, who is widely known for his security tools and papers. It was made available as open source software in December 1998. IBM Research sponsored the initial release and has continued to support its ongoing development. (IBM calls the package Secure Mailer.) Jerry then explains certain specific goals that first drove the design and development of Postfix:
First of all Postfix is written to be compatible with Sendmail.
It supports Sendmail conventions like
.forward files, so we can easily put Postfix
on a system without breaking any applications that depend on
Sendmail. The Sendmail executable program,
sendmail, is replaced with a Postfix version
that supports nearly all of the same command-line arguments but
runs in conjunction with the Postfix system.
"So, is Postfix like a clone of Sendmail?" Maria wonders out loud.
"No, it maintains a compatible external interface, but it doesn't necessarily implement things the same way. Our programs that depend on Sendmail will still work, but Postfix has been evolving independently of Sendmail, so we can't expect everything to behave exactly as it does with Sendmail."
"Which could be a good thing," someone in the group quips.
Postfix really shines as an MTA because of its rock-solid reliability. It's designed to behave sanely even under stressful conditions. Remember when we were attacked by the mail bomb?
"Right, conch was hosed," Maria remembers. "We ran out of disk space and memory, and most of the applications went berserk"
Postfix detects conditions like that and gives the system a chance to recover instead of making the problem worse. It takes every precaution to always function in a stable and reliable way.
- Reliability goes hand-in-hand with security. No matter what is thrown at it, reliable software stays in control. Postfix assumes that it is running in a hostile environment. It uses multiple layers of protection, and it maintains the security principle of least privilege, so that each of its processes has only the amount of privilege it needs to get its job done. The higher privileged processes do not trust the unprivileged ones in case they have been compromised. Also, since Postfix is made up of different modules for its different functions, we can always disable the modules we don't need. If a module is not running it obviously can't be attacked.
- Postfix was written with performance in mind. As a matter of fact, it takes steps to make sure that it doesn't overwhelm other systems that aren't responding quickly enough. Two of the techniques it uses to speed up performance are to minimize the number of times it has to read from the disk and to limit how many processes it launches when handling messages.
- Postfix's approach allows for great flexibility. It is highly tunable through several configuration options, and besides supporting email standards, it provides well-documented interfaces for working with other programs. Postfix provides hooks to plug in all kinds of content filters and policy servers.
- Postfix is one of the easier email packages to set up and administer. It uses straightforward configuration files and simple lookup tables for address translations and forwarding. The design follows the notion of least surprise, which means as much as possible, Postfix behaves the way most administrators would expect in a given situation.
With that, Jerry finishes his presentation of Postfix. Dave is definitely interested and offers to get back to the group with a decision after he confers with some other managers. Jerry asks if anyone in the group has questions about the presentation. Will speaks up, "You mentioned that Postfix is an MTA and that it works well with other programs, but isn't it a mail server? When I set up mail on a Windows platform, I install an email server and that does everything it needs to do. Isn't it the same with Postfix?"
"Sounds like we should have a review of Internet email and the various components," Jerry decides. "Actually, Maria is our Internet protocols expert. Maria, would you mind scheduling something in the conference room, and invite the rest of the group too. Some of the others might want a refresher."
"You got it," Maria replies.
Email and the Internet
As the group starts to gather in the conference room with cups of coffee and soda cans in hand, Maria has already started to draw on the board. She begins.
Like Will, many of you may have installed email systems in the past as a single package. Those proprietary email systems, mask the fact that Internet email is built from several standards and protocols that define how messages are composed and transferred from a sender to a recipient. There are many different pieces involved, each one handling a different step in delivering messages. The mail transfer agent (MTA) or Postfix in our case handles only a portion of the whole process. Most email users are familiar with the software they use to read, write, and send their messages. That software is called a mail user agent (MUA). You're all familiar with some of the common MUAs like mutt, elm, Pine, Netscape Communicator, and Outlook Express. MUAs are great for reading and composing messages, but when it comes to sending messages, they don't do much other than hand off a message to an MTA.
"And that's where Postfix fits in," Will says, starting to get the picture.
"Exactly." Maria says. She draws a diagram on the board like the one shown in Figure X and continues.
This figure shows the main components in a simple email transmission from a sender to a single recipient. The bulk of the work is handled by MTAs like Postfix. First the MTA gets the request to accept a message, usually from an MUA or another MTA. The MTA determines if it should take the message or not. Generally an MTA will accept the message if it's for one of its own local users; for another system it knows how to forward to; or if the message is being sent from a user, system, or network that it allows to relay messages to other destinations. Once the MTA accepts the message, it has to decide what to do with it next. It might deliver the message to a user on its system, or it might have to pass the message along to another MTA. If the MTA cannot deliver the message or pass it along, it bounces the message back to the original sender or notifies a system administrator.
"Seems like there could be complicated configuration and administration involved in that," somebody wonders.
"There can be," Jerry offers. "That's why MTA servers are usually managed by Internet Service Providers (ISP) for individuals or by corporate Information Systems departments for companies." On the other hand, for a traditional Unix email system, Postfix works out of the box after possibly setting the hostname and other local information.
Maria continues her presentation.
Ultimately, a message arrives at the MTA that is the final destination. If the message is destined for a user on the system, the MTA passes it to a message delivery agent (MDA) for the final delivery. The MDA might store the message as a plain file or pass it along to a specialized database for email. The term message store applies to persistent message storage regardless of how or where it is kept.
Once the message is placed in the message store, it stays there until the intended recipient is ready to pick it up. The recipient uses an MUA to retrieve the message and read it. The MUA contacts the server that provides access to the message store. This system is often a POP or an IMAP server and it is a separate package from the MTA that delivered the message. It's designed specifically to provide access for retrieving messages. It uses some mechanism to authenticate the requester and then transfers that user's messages to her MUA.
The beauty of Internet email standards is that they are open. There are lots of software packages available to handle the different parts of the delivery. We can replace other MTAs with Postfix without touching our IMAP servers. As long as the different packages implement the same protocols, they can work together regardless of who wrote them or even the type of system they are running on.
"But you could still have a single package that takes care of everything, couldn't you?" asks Will.
"You can," Maria responds, "but almost any Unix system will have a different package for handling SMTP versus POP/IMAP. There are so many choices for each aspect that most administrators choose each component to fit their needs."
Jerry asks Mary to explain a bit more about the underlying protocols that make the interoperability possible.
Major Email Protocols
Right, as I mentioned, the communications between each of these email system components are defined by Internet standards and protocols. The standards documents are maintained by the Internet Engineering Task Force (IETF) and are published as Request For Comments (RFC) documents, which are numbered documents that explain a particular technology or protocol.
The three big ones for Internet email are the Simple Mail Transport Protocol (SMTP) to send messages and then either the Post Office Protocol (POP) or Internet Mail Application Protocol (IMAP) for retrieving messages. SMTP, which is defined in RFC 2821, describes the conversation that takes place between two hosts across a network to exchange email messages. The IMAP (RFC 2060) and POP (RFC 1939) protocols describe how to retrieve messages from a mail store. The IMAP protocol was developed after POP and offers additional features. In both protocols email messages are kept on a central server for the recipients, and then the recipients usually pick them up across a network.
You've probably noticed that users' MUAs do not necessarily use the same system for POP/IMAP as they do for SMTP. ISPs usually provide different servers for each function, possibly to separate the load or administrative functions, and many of our users who are away from the office retrieve their messages from our POP/IMAP server, but they use the SMTP server from their mobile provider to send their messages.
"What if the user doesn't know the provider's SMTP server?"
"We also provide a way for them to use our SMTP servers," Maria answers. "Originally SMTP did not have any way to authenticate users, but more recent extensions to the protocol provide that capability. We'll talk more about that later (see Chapter 7)."
POP and IMAP, of course, have always used authentication to identify which users are retrieving their messages. The difference between POP and IMAP is that POP users generally take all of their messages from the server and manage their mail locally. IMAP provides features that make it easier to manage mail on the server itself. Most servers offer both protocols, so I usually say POP/IMAP. Keep in mind, though, that not everybody uses POP/IMAP to get access to the message store.
"I always have shell access to our systems, so my mail client just reads my messages directly from the mail files on our mail store," Jerry explains.
The Role of Postfix
Now that Maria has introduced the components of email generally, she moves on to explaining where Postfix fits in specifically. She sketches out a diagram on the board resembling Figure X.