So I've been running self-hosted email using Mailu for a couple of months (after migrating out of Google Workspace). Today it turned that although my server seems to be capable of sending and receiving emails, it also seems to be used by spammers. I've stumbled upon this accidentally by looking through logs. This seems to have been going on for all this time (first "unknown" access happened just a couple of hours after I've set everything up).
While browsing the logs there were just so many crazy things happening - the incoming connections were coming through some kind of proxy built-in to Mailu, so I couldn't even figure out what was their source IP. I have no idea why they could send emails without authorization - the server was not a relay. Every spammy email also got maximum spam score - which is great - but not very useful since SMTP agent ignored it and proceeded to send it out. Debugging was difficult because every service was running in a different container and they were all hooked up in a way that involved (in addition to the already mentioned proxy) bridges, virtual ethernet interfaces and a jungle of iptables-based NAT that was actually nft under the hood. Nothing in this architecture was actually documented anywhere, no network diagrams or anything - everything has to be inferred from netfilter rulesets. For some reason "docker compose" left some configuration mess during the "down" step and I couldn't "docker compose up" afterwards. This means that every change in configuration required a full OS reboot to be applied. Finally, the server kept retrying to send the spammy emails for hours so even after (hypothetically) fixing all the configuration issues, it would still be impossible to tell whether they really were fixed because the spammy emails that were submitted before the fix already got into the retry loop.
I have worked on obfuscation technologies and I'm honestly impressed by the state of email servers. I have temporarily moved back to Google Workspace but I'm still on the lookout for alternatives.
Do you know of any email server that could be described as simple? Ideally a single binary with sane defaults, similarly to what dnsmasq is for DNS+DHCP?
I agree that a static IP address is an absolute requirement for a mail server to send messages these days. You also need a host of checks in place like SPF, DKIM, and DMARC, along with a strong set of blocklists and spam filters. My own setup includes dual ISP connections from two different providers, and even with all that in place, Microsoft has always been a thorn. They will block me for no apparent reason, their own tools don't even show any detected spam activity, and sometimes they don't even block the same IP address (or provider) that my emails were sent from. Every other spam service on the planet behaves in a rational way, but of course Microsoft has made a point of locking in so many businesses to their own spam-ridden service that you simply can't run a mail server any more without being able to talk to them.
Overall, yeah it can be a pain to run your own mail server. I can't imagine trying to use a pre-built mail server and expect it to run, there's so much that you have to configure to each specific setup. It's not like a web server where you load up a docker container and it just works.
Are they often not ‘actually’ static? In what way?
Well... ok? I've only been running mine since around 2001, I guess I should give up?
Same, though I too switched to an aws instance and before that linode since 2008 or so.
Miss the old days of not having to worry about IPBL.
I've had similar experiences trying to send mail to Microsoft-hosted email addresses. My current "solution" is to send all outgoing mail directly from my VPS-hosted Mailu server... EXCEPT for Microsoft-destined mail. For those messages, they get transparently relayed from Postfix to a third-party email sending service that Microsoft apparently trusts.
The upshot is I can still use my own Postfix daemon for all mail sent to sane (non-Microsoft) providers.
I've never heard of anybody relaying just the Microsoft e-mails, but that's a really funny spiteful solution.
Lately I've been able to send to outlook just fine (maybe it's just dumb luck, who knows). I think I had troubles initially because they're really picky about rDNS matching the MX exactly. I also signed up for SNDS just in case, but I don't know if they factor that in...
Ooh, lucky you! Maybe it was both the rDNS and the SNDS. Part of the problem IMO is it's such a black box..
Definitely. It’d be nice if there was more reporting when a provider drops your messages.
In my experience (self hosting mail since 2005) signing up for SNDS does factor in. Although last time I had trouble with delivery to MS, my hosting provider Linode's support also helped out by contacting MS back channels on my behalf. The biggest problem I (rarely) have is when whole IP blocks end up on a ban list that MS seems to really trust. That said, fuck it, I will keep fighting the fight and self host my mail like a stubborn old git :p
Hell yeah :). I've heard people have had problems with Linode's IP blocks with MS... I'm glad to hear that Linode was actually able to help you resolve the problem. The biggest problem seems to be that you just might not realize if your e-mails are being dropped. Not sure if MS will notify you via DMARC if your IP block has been blacklisted, from the sounds of things they probably don't and just silently drop things, so I guess you have to monitor the blacklists yourself?