IETF email infrastructure transition planned for 11 September

The web interface to the mailing list software (Mailman3) will also be unavailable during the transition; access to mailing list archives and IMAP access will be unaffected.

The new email infrastructure includes a complete refactoring of postconfirm, which is an open source mail filter (milter) and processing tool that acts as a mail gatekeeper—preventing nuisance traffic and handling first-time sender verifications via email-based challenge-response. 

The new software takes a modular approach to these functions, moving to new spam detection software (from spamassasin to rspamd), new address rewriting software, and improving certificate and TLS handling for DANE (DNS-based Authentication of Named Entities). 

The result will be better email hygiene: improved bounce handling, reduced spam opportunities, and removing a possibility to serve as an open relay.

Each discrete function is deployed in a separate container, scheduled via kubernetes in a dedicated cluster and communicates via the milter protocol providing the required functionality. This allows us to horizontally scale each component as required. The exception to containers is outbound mail, which is relayed to multiple virtual machines in known-good networks.

The postconfirm milter first examines the destination address, matching against a regular expression to determine whether a challenge is required. Next, it checks against a table of all previously approved addresses, including all current list subscribers to see if a challenge has already been answered. If a challenge is needed, it will store a copy of the original email while waiting for the sender to respond to the challenge. Once the challenge has been responded to appropriately the stored message will be released as originally sent and the user will be added to the allow list for future emails. This initial challenge is vital to ensure that all IETF’s mailing list participants have agreed to the Note Well statement of policies and procedures.

The address rewriting milter is more complex, checking the sender’s DMARC and SPF policies for each outbound mail. If the envelope From domain has an SPF policy set that does not include our ip addresses, we rewrite the envelope From from local-part@domain to $local-part=40$domain@$ietf-domain where $ietf-domain is based on the original destination domain prefaced with dmarc.; dmarc.ietf.org, dmarc.irtf.org, and so, on ensuring SPF alignment. If the sending domain has a p=reject or p=quarantine DMARC policy we will rewrite the header From in the same way, $local-part=40$domain@$ietf-domain. We then DKIM sign the message with the appropriate domain key, ensuring valid DKIM and DMARC alignment. For example, mail from dane@email.example sent to the tools-discuss@ietf.org list would get both envelope and header From rewritten as dane=40email.example@dmarc.ietf.org, and there would be a DKIM signature with d=dmarc.ietf.org when delivered to list subscribers. If the egress mail is a mailing list post, we also set the Return-Path header to a similar value enabling mailman list bounce processing as normal.

The mail rewriting service also parses ingress email, handling list bounce processing and relaying to previously rewritten addresses, checking against a database entry to ensure the rewritten address is fresh, discouraging open relaying through the rewrite service. 

If there are no SPF or DMARC policies on the originating From address, there will be no address rewrite effected.

Certificate management is done using a current + next 3 1 1 rollover scheme, rolling and refreshing the certificates as necessary, ensuring we always have a certificate that is valid for DANE, even during a rollover.

Rspamd provides DKIM signing, and spam assessments, with the capability to further tune after the initial deployment.

Longer term, the new infrastructure will be more robust and scalable, easier to maintain and upgrade in the future. The email processing software and infrastructure in use today has grown over many years to be extremely complicated and difficult to operate and update. Email is key to work in the IETF and this update provides a solid foundation for future improvements.

More generally, the email services transition is part of the multi-year effort to update the entire IT infrastructure supporting the IETF. This update to email services is a necessary step to enable IETF participants more integrated management of mailing list subscriptions and through the IETF Datatracker. All of the new software is available on the ietf-tools Github repository. Feedback and issues are welcome through Github, or directly via the tools-discuss@ietf.org mailing list.

NOTE: New Machine Futures is a contractor for the IETF Administration LLC working on various components of the IETF IT infrastructure transition.