Sender Policy Framework
|
|
In computing, Sender Policy Framework (SPF), formerly Sender Permitted From, is an extension to Simple Mail Transfer Protocol (SMTP), the standard Internet protocol for transmitting email. SPF makes it easier to counter most forged "From" addresses in email, and thus helps to counter e-mail spam. The combination is also called SMTP+SPF.
It was designed by a group led by Meng Weng Wong.
| Contents |
Benefits
Normal SMTP allows any computer to send email claiming to be from anyone. Thus, it's easy for spammers to send e-mail from forged addresses. This makes it much more difficult to trace back to where the spam truly comes from, and easy for spammers to appear to be senders the receiver would ordinarily trust. Many believe that the ability for anyone to forge "From" addresses is a security flaw in SMTP, and SPF is one of a variety of new methods being proposed which restricts this ability.
SPF allows the owner of an Internet domain to use special Domain Name System (DNS) records to specify which machines are authorized to transmit e-mail for that domain. For example, the owner of the wikipedia.org domain can designate which machines are authorized to send email whose email address ends with "@wikipedia.org". Receivers that implement SPF then treat as suspect any email that claims to come from that domain but fails to come from locations that domain authorizes.
SPF protects the use of the 'reverse-path', or the address that the mail claims to come from at the SMTP level; the address to which bounces would be sent if the mail is not delivered. It does not have any relation to the visible 'From:' header of the email, which can still be forged when SPF is used. Other schemes attempt to prevent forgery of the visible headers.
SPF makes it more difficult for spammers to send spam, because if they simply forge a reverse-path from a domain that implements SPF with a record containing -all, receivers that implement SPF will know to ignore the message. SPF only keeps spammers from forging the domain names given in the reverse-paths of an email. If a spammer legitimately has an account in that domain, or owns the domain, they can still send email; however, doing so makes the spam much easier to trace and prosecute, because they must reveal more about their location. It also makes it easier for service providers to drop support. The disclosure of the spammer's true domains makes it much easier to automatically "blacklist" domains that send spam. Spammers can also use computer attacks (such as viruses) to force authorized computers to send email, or use computer attacks against Internet infrastructure (such as corrupting DNS or attacking BGP to subvert entire address blocks) to take control over others' networks, however, such actions are illegal in most countries, are more likely to initiate serious government investigation, and the legal penalties for performing such attacks are often more severe than those for spamming alone.
Another benefit of SPF is to people whose email addresses are being forged as reverse-paths. People with these forged addresses typically receive a large mass of error messages (bounces), making it more difficult to use email normally. If these people use SPF to specify their legitimate senders, the number of error messages may be reduced because receivers implementing SPF will know that the message is forged. Note that SPF has advantages beyond helping identify unwanted email. In particular, if a sender provides SPF information, and the receiver uses it, the receiver has some justification in believing that the email at least came from the domain that it asserts it came from.
If a domain adopts SPF with a record containing -all, the domain thereby prevents anyone with an address in that domain from sending mail through mailing lists or forwarding schemes that don't change the reverse-path to suit SPF's requirements. Some believe that this breakage of SMTP will pressure these systems to be 'upgraded' to meet the new requirements: until they are, SPF will cause valid mail to be lost.
Implementation
Implementing SPF has two parts:
- Domains must identify the machines authorized to send email on their behalf. Domains do this by adding an additional record to their existing DNS information.
- Receivers must request and use SPF information. Receivers request this information using ordinary DNS requests (which are typically cached to enhance performance). Receivers must then correctly interpret the SPF information. SPF does not specify how the SPF results should be interpreted or acted upon.
Thus, the key issue in SPF is the specification for the new DNS information that domains set and receivers use. The exact specifications may change, but as of writing, the records are laid out like this (in typical DNS-syntax):
wikipedia.org IN TXT "v=spf1 mx ptr -all"
"v=" defines the version of SPF used, the following words provides the methods to use to determine if a domain is eligible to send mail. The "mx" and "ptr" specify the systems permitted to send messages for the given domain. The "-all" at the end specifies that, if the previous methods did not match, reject the message as a forgery.
The following methods are defined:
- A - If the domain name has an A record corresponding to the sender's address, it will match. (That is, the mail comes directly from the domain name.)
- MX - If the domain name has an MX record resolving to the sender's address, it will match. (That is, the mail comes from one of the domain's mail servers)
- PTR - If the sender reverse-resolves to a domain ending in the domain name, match.
- IP4 - If the sender is in a given IPv4 range, match.
- IP6 - If the sender is in a given IPv6 range, match.
- EXISTS - If the given domain resolves, match (no matter the address it resolves to). This is typically used along with the SPF macro language to perform more complex matches.
Limitations and controversies
SPF normally only validates the domain of the envelope sender (listed as "Return-Path: " in e-mail headers). Domains that share mail senders (e.g. with virtual hosting) can forge each others' domain, and mail forwarding systems may also allow other domains to forge domains. SPF does not validate that a given email actually comes from the claimed user, because it operates at the network level.
SPF breaks inter-system SMTP forwarding (where an agent forwards email to someone else without changing the "from" address). This is a problem for forwarding services (such as acm.org and pobox.com). One solution is for the SPF record to specifically specify the forwarder as an allowed sender. A more general solution is to switch from forwarding (where the envelope sender is preserved) to remailing (where the envelope sender is changed). Remailing is easy to implement in most systems, but until implemented, mail will be discarded.
Steve Bellovin has written a long email dated January 5, 2004, discussing some of his concerns with SPF. Some of these include:
- SPF uses of TXT records in DNS, which are supposed to be free-form text with no semantics attached. SPF proponents readily acknowledge that it would be better to have records specifically designated for SPF, but this choice was made to enable rapid implementation of SPF. It can take years to add a new record type. Once SPF is more established, a new record type could be added and transitioned to.
- The SPF specification provides ways to avoid processing that a spammer could forge. For example, a spam engine could send email with a local-seeming HELO, MAIL FROM, and From: entries, in which case SPF isn't to be used (however, public SMTP servers should reject anything that seems local). Spam from abuse@ or postmaster@ can also bypass checks. This may require additional restrictions and changes in the specification.
- Various other (fixable) problems exist in the current specification. It could be argued that this specification was rushed to make it rapidly available.
- As of the time he wrote his message, there was no consensus that this is the right way to go. The major ISPs (AOL, Yahoo, MSN, etc.) have not bought into this scheme. Unless and until they do, it doesn't help much, either for their customers (who make up a substantial proportion of the user population) or for everyone else (since their addresses could be forged). It's worth noting that since this concern was raised, AOL has embraced SPF (http://postmaster.aol.com/spf/). From Oct 1. (http://www.infoworld.com/article/04/07/22/HNmicrosoftid_1.html) Hotmail and MSN also published their own SPF records, and filter incoming mail using SPF.
Bellovin's strongest concerns involve the underlying assumptions of SPF (SPF's "semantic model"). When using SPF, the SPF DNS records determine how a sender is allowed to send. That means that the owner of the domain will control how senders are allowed to send. People who use "portable" email addresses (such as email addresses created by professional organizations) will be constrained to use the domain owner's SMTP sender, which may not currently even exist. Bellovin argues that many users using SPF will be tied more strongly to their ISPs and/or their employers. One counter to this concern by Bellovin is that those organizations could simply create SMTP servers, and use mechanisms to authenticate their users. Such mechanisms include secured VPNs, SMTP with authentication, and POP before SMTP.
Bellovin notes that it would be more flexible to permit delegation of individual user names to particular sending machines. However, he also grants that it would probably not be affordable, specifically noting that it would probably require too much public key cryptography. Although he does not note it, there are many other complications to this more flexible approach. Supporting individual user delegation would require services to maintain each entry, as well as requiring domains to manage each user account; an extraordinary undertaking for many large domains. In contrast, SPF requires adding only one line to a DNS entry, and only needs to be changed under relatively rare conditions. Also, SPF results can be cached for each domain, so SPF as specified is far more efficient in terms of network bandwidth.
In spite of its limitations, many people have decided that the pros of SPF outweigh its cons and have begun implementing SPF. Anti-spam products such as SpamAssassin version 3.0.0 implement SPF. Many mail transfer agents (MTAs) support SPF or have patches/plug-ins available that support SPF, including Postfix, Sendmail, Exim, and Qmail. Many prominent domains decided to post SPF data for their domains as of mid-2004, including AOL, EarthLink, Google, and Amazon.com. In early 2005, Amazon.com downgraded its SPF records to use '~all' instead of '-all' for the catch-all record, presumably because of the way that SPF causes valid mail to bounce.
Standardization Status
SPF has been part of the LMAP work done in the ASRG and was submitted as input along with other proposals to the IETF's MARID (MTA Authorization Records in DNS) workgroup. The latest version of SPF is named Marid Protocol (http://www.ietf.org/internet-drafts/draft-ietf-marid-protocol-03.txt).
With the subsequent disbanding of the MARID working group, caused in no small part by Intellectual Property problems caused by protocol implementors having to obtain a patent licence from Microsoft, it is now expected that SPF will be resubmitted for publication by the ASRG as an experimental RFC. Implementors are continuing to use the pre-MARID version of SPF (although Microsoft's patent applications, whose final long-awaited publication was one of the events that precipitated the disbanding of MARID, actually cover that, too, and Microsoft's patent licence still remains a concern). A newer alternative named CSV is believed not to be encumbered by Intellectual Property problems, but involves a different set of trade-offs.
As such, it is currently not an official Internet standard. It is suggested that readers check the current status of SPF.
See also
External links
SPF information
- SPF homepage (http://spf.pobox.com)
- Interview about SPF (http://www.circleid.com/article/634_0_1_0_C/)
- MIT Spam Conference Handout on SPF (2004) (http://spf.pobox.com/for-mit-spam-conference.html)
- Meng's Deployment White Paper (Nov 2004) (http://spf.pobox.com/whitepaper.pdf)
- Diagram of email protocol flow and SPF's impact (http://spf.pobox.com/mailflows.html)
- INFINITEPENGUINS SPF TOOLS - List of popular domains with SPF records (http://spftools.infinitepenguins.net/earlyadopters.php)
- Freeware support for SPF - the Sender Policy Framework (http://www.gfi.com/mes/mesfreeware.htm)
Limitations and controversy
- Steve Bellovin expresses doubts (http://www.interesting-people.org/archives/interesting-people/200401/msg00037.html)
- Jonathan de Boyne Pollard's list of the flaws in SPF (http://homepages.tesco.net./~J.deBoynePollard/FGA/smtp-spf-is-harmful.html)
- John R Levine's analysis of the Microsoft patent applications (http://www.taugh.com./weblog/patapp.html)
- David Woodhouse discusses the flaws in SPF, and summarises the alternative solutions (http://david.woodhou.se/why-not-spf.html)
MARID information
- MARID mailing list (http://www.imc.org/ietf-mxcomp/index.html)
- IETF announcement of the disbanding of MARID (http://ietf.org./mail-archive/web/ietf-announce/current/msg00505.html)de:Sender Policy Framework