As part of my public homelab setup, one of the services I configured was a mail server! I got started with around a month ago, and have been running a bare minimum setup for a week. Here are some notes about the configuration process.
Finding the right infrastructure
Most providers and ISPs block port 25 traffic due to it being an easy target for spam. It makes sense to do so, as even one machine sending spam would put the entire network on a blocklist, unable to send email.
Some providers have an option to request to unblock the port. Vultr is one such provider I heard of, so I started by hosting my website there, and submitted an unblock request within a few days that was eventually rejected.
That left me with two options:
- Switch to a provider that allows SMTP traffic
- Use an external service called a SMTP relay to handle sending and self-host receiving email.
The second option is the approach I saw recommended most often, as it delegates the responsbility of maintaining sender reputation while letting you store email on a server you control. I started with this option on the Vultr VPS, but eventually went with the first option.
In hindsight, I could have easily stuck with the Vultr VPS, and this was an important lesson about firewalls. Firewalls have separate rules for incoming and outgoing traffic, and port 25 is only blocked for outgoing traffic by most providers. Incoming traffic at port 25 is allowed, so receiving mail works fine. I didn’t test this and assumed that port 25 was blocked in either direction, which led to the switch to my current provider.
An unintentional advantage of this switch is that I’m now responsible for handling both sending and receiving, which has been a better learning experience. Whether this remains an advantage or not - time will tell.
Finding the right learning material
There isn’t a shortage of tutorials out there, but none of them went into details behind the “why”. While searching about one of the aspects, I came across a series of posts titled Setting up Postfix and Dovecot Slowly and Properly by brokker.net, which is exactly what I needed! It takes you through the setup in small increments, with each step ending with a working setup you can test, which I really liked. This series gave me the foundation I needed, and from here I could decide which aspects I wanted to implement.
Another setup guide I found useful was Install and Configure a Secure Mail Server on Debian with Postfix and Dovecot by std.rocks. It’s a newer, concise guide, and especially useful after having gone through the series. In particular, I followed its instructions for setting up OpenDKIM, SPF and DMARC much earlier than mentioned in the series, as these aspects seem to have gone from “nice to have” to “absolutely required” over time for sending to work, especially when sending emails to Gmail addresses.
Configuring a test server
I already had this domain hosted elsewhere and have been sending and receiving mails at this domain. I initially started by switching the primary mail server to the test server by changing the DNS record. However, the setup was taking me a while as I was taking time to understand and test the various aspects. That would mean I may or may not receive emails sent during that time, and needed a test setup.
The test setup that worked was a secondary mail server - mx.orgnizedmess.net - that receives emails at a subdomain instead of the root domain. I would send test emails to user@mx.orgnizedmess.net instead of user@orgnizedmess.net.
This was not as straightforward as I thought, as parts of the configuration varied for subdomains, that wasn’t as obvious to me:
-
TLS certificates: I issued a certificate with the test subdomain included during testing and re-issued them without it once the setup was complete. I was using Caddy to generate SSL certificates for my website earlier, so I switched to a dedicated client (I’m using dehyrdated) to be able to use the same certificates for both the website and the mail server.
-
DKIM: Similar to TLS, I had to generate the key pair for the subdomain during testing and then generate it for the root domain when I was ready to switch servers. The DNS record also had to be set to the subdomain during testing -
selector._domainkey.mx.orgnizedmess.netinstead ofselector._domainkey.orgnizedmess.net.
Where the lines between different services get blurred
There are two main servers - SMTP (Postfix) and IMAP (Dovecot). Postfix handles sending and receiving of emails (aka the MTA or Mail Transfer Agent), and Dovecot handles mailboxes for different users and lets me access mail in a mail client (aka the MUA or Mail User Agent). Any setup beyond is where the services started to rely on each other, and also where I got confused the most.
-
Postfix relies on Dovecot for authentication using SASL or the Simple Authentication and Secure Layer - a long name for something that’s meant to be simple.
-
Postfix runs a submission service at port 587 (aka MSA or Mail Submission Agent) that encrypts the email sent from a MUA before sending it to the MTA. Connections on this port start as plaintext, and then upgrade to TLS if supported (aka Opportunistic TLS). There is another service called SMTPS at port 465, which starts as with the TLS handshake directly (aka Implicit TLS). As per Section 3.3 of RFC 8314, it is recommended to run both services and eventually transition to SMTPS.
-
By default, Postfix delivers any received email to the user’s mailbox (aka MDA or Mail Delivery Agent). For any additional processing on the emails like filtering before being delivered, Postfix can be configured to use Dovecot as the MDA. The protocol used to do so is LMTP or the Local Mail Transfer Protocol.
Wow that’s a lot of acronyms.