Self-Hosting — Running Your Own Infrastructure on Linux

Mail servers, DNS, backups, and the quiet satisfaction of running everything yourself.

Willem runs his own mail server, his own DNS, his own cloud. Not because it's easy, but because he believes in understanding and owning the infrastructure you depend on. These posts are the field notes from that ongoing project.

There's a particular kind of person who, when their email stops working, doesn't call support — they SSH into the server and read the logs. Willem is that kind of person, and he's been writing about it for years.

Mail, the Hard Way

The flagship is the Dovecot series. When Dovecot released version 2.4 with breaking changes, Willem wrote the definitive migration guide — the post that now ranks in the top results for anyone upgrading their mail server. But the mail infrastructure goes deeper: fighting backscatter spam at the server level, configuring Postfix, understanding the entire chain from DNS to inbox.

Breaking Changes

Breaking Changes

Upgrading Dovecot 2.3 to 2.4 in Debian Stable

Last week I ran into unexpected trouble during a routine maintenance procedure on my company's email infrastructure. Processing thousands of emails each day, we use Dovecot to give our clients access to their messages. That all came to a grinding halt when I updated to version 2.4 which features breaking changes... oh dear!

Fighting backscatter spam at server level

Fighting backscatter spam at server level

Configure Postfix to block spam before it enters the server

This month I had to deal with backscatter spam, affecting one of the mail servers I manage. As server engineer I make sure that servers don't send spam and that incoming email gets filtered. Despite all good efforts, this server kept being blacklisted for sending spam to iCloud, Office 365 and Google Gmail for Business (G Suite). Read along to find out how what caused this and how to fix this.

Backups and Data Sovereignty

If you run your own server, you need your own backup strategy. The rsync guide is a complete manual for backup rotation schemes, and the piece on running your own address book and calendar cloud is a quiet manifesto for digital independence.
How to use rsync to make backups

How to use rsync to make backups

Explaining a powerful file transfer tool

It is very important to make backups of your data, as you never know when disaster strikes! One powerful, cross platform, tool to help you achieve this is 'rsync'. In this post I'll explain why rsync is useful and how you can use it to set up your own backups.

Your own addressbook and calendar cloud

Your own addressbook and calendar cloud

Share contacts, agendas and tasks with CardDAV/CalDAV

If you use different devices and computers to get things done, you might want to synchronise contacts, agendas and tasks. You can use any of the 'big cloud' services for this, like Apple iCloud, Microsoft Office 365 and Google Gmail. But, if you prefer not to share your addressbook and calendar with big American companies, you can do it yourself.

Networking and Security

Traffic shaping with iptables and tc, DNS for beginners, the philosophy of offline-first computing — these are the building blocks posts. They assume you're willing to learn, and they teach you properly.

Working Offline First

Working Offline First

Learning from a 15-year old ThinkPad X200

As part of a greater plan to optimise my entire workflow, I am doing a little experiment this month: using a 15-year old ThinkPad X200 laptop as main computer. It’s from an era before “the cloud”, popular among free/libre software enthusiasts. What can we learn from this antithesis of modernity?

Traffic shaping using iptables and tc

Traffic shaping using iptables and tc

Limiting outbound network bandwidth per client IP-address

Last month I received an automated alert indicating excessive bandwidth usage, usually a sign of trouble. When this happens, you should follow a standard incident procedure, trying to isolate the source of the traffic before shutting it down. The cause of this incident was not what I expected however... requiring a different kind of mitigation than a simple blockade.

Dig for dummies

Dig for dummies

Explaining an highly useful network tool

When you're building websites, apps or email services you may run into domain names and their configurations. When everything is working as it should, most of this is invisible. But when troubleshooting a domain name configuration, it may be necessary to dig a little deeper... read along to learn how!

Also explore

built to last · retro computing · writing code · digital minimalism

All Self-Hosting Posts

The complete archive of posts about servers, Linux, networking, and running your own infrastructure.

Breaking Changes

Breaking Changes

Upgrading Dovecot 2.3 to 2.4 in Debian Stable

Last week I ran into unexpected trouble during a routine maintenance procedure on my company's email infrastructure. Processing thousands of emails each day, we use Dovecot to give our clients access to their messages. That all came to a grinding halt when I updated to version 2.4 which features breaking changes... oh dear!

Champions League of Webdesign

Champions League of Webdesign

Cutting-edge technology meets top-tier digital demands

Creating a high-performance website is an art form that blends cutting-edge technology with an eye for design and commerce. At escape.nl, home to one of Amsterdam’s most iconic venues, every detail reflects the precision and care required to deliver seamless, engaging experiences to large audiences. This post highlights often-unseen details with big impact.

Backup Rotation Scheme

Backup Rotation Scheme

Rotate your backups with 'rsync-backup-rotator'

In today's digital age, safeguarding your data is paramount. Simply creating a copy of your files may not be enough as they can get corrupted, overwritten or blocked by ransomware. Having multiple, time-rotated (and ideally, offsite) backups is a stronger defense. I created a new tool, rsync-backup-rotator, to help you with this.

Using AI to generate code

Using AI to generate code

Programming with super powers

As part of my software optimisation efforts to cut cloud costs, I needed to replace an existing piece of inefficient server software with something that uses more robust (yet fragmented) tooling available in Debian GNU/Linux. Could the GPT4 language model deliver me some AI magic? Read along!

Cutting Cloud Costs

Cutting Cloud Costs

The Significance of Software Optimisation

Like many, I faced soaring cloud service costs for my clients, prompting a wake-up call. To stay afloat, it's essential to incorporate software optimisation as a strategy for managing cloud expenses. Join me as I share how I managed to save up to 90% on cloud costs through optimisation!

CAPTCHA Alternative

CAPTCHA Alternative

Protect your forms in a user friendly way

Chances are that you've encountered a CAPTCHA in the wild: they protect web forms by asking you to type over weirdly rendered characters or by asking you to select photos containing a particular thing. Why are they used and is there a user friendly alternative?

Protecting VMware ESXi

Protecting VMware ESXi

Improve security using a firewall and proxy server

In response to an incident on a server, I detected a possible security breach. The affected machine ran VMware ESXi, a bare metal hypervisor used to run virtual private servers. In standalone mode, a web based management console offers full control over the infrastructure, posing a risk.

Migrating to the Cloud

Migrating to the Cloud

Clearing an on-premise server room

For a Dutch organisation that serves customers throughout Europe, I was asked to help dismantle a live server room. Instead of simply pulling the plugs, some of the services had to be migrated to the cloud. For fun and inspiration I share three different strategies to do so.

Realtime service uptime monitoring

Realtime service uptime monitoring

Discover problems before your customers do

Downtime happens and it is nothing to be ashamed of, just make sure that you set yourself up to discover problems as soon as possible! You can use realtime uptime monitoring to automatically keep an eye on your servers and services. Read along to find out how.

Designing a multi dimensional capacity queue

Designing a multi dimensional capacity queue

Managing kitchen-, delivery- and pickup capacity

This month I needed to create additional dimensions to a capacity queue mechanism. The food ordering app that I created needed to be able to restrict capacity based on the number of orders, the contents in individual orders and the dispatch type (takeaway/delivery). Read along to find out how I used a Lambda Architecture to do this.

How to use rsync to make backups

How to use rsync to make backups

Explaining a powerful file transfer tool

It is very important to make backups of your data, as you never know when disaster strikes! One powerful, cross platform, tool to help you achieve this is 'rsync'. In this post I'll explain why rsync is useful and how you can use it to set up your own backups.

Syncing files seamlessly between smartphone and tablet

Syncing files seamlessly between smartphone and tablet

Using unison to automatically sync between GNU/Linux and the iPhone

In order to optimise my workflow, I was looking for a way to seamlessly access the same files on both my computer and smartphone. This is useful to when you want to quickly send files from your computer using your smartphone through various messaging apps and vice versa. Read along to find out how I did it.

Linking Lemmid Store with kitchens

Linking Lemmid Store with kitchens

Integrating with external systems that you don't control

As part of the food ordering app I am building, I needed to design a reliable way to link the app to external systems. These external systems are beyond my direct control and include different checkout registers, kitchen management systems and ticket printers. Read along for more on designing for the unknown and unreliable.

Scalable application design without magic

Scalable application design without magic

Leveraging client computing power for high performance with many users

As part of the online food ordering app I'm building, I needed to design a scalable backend infrastructure that could handle lots of concurrent users. Scalability is considered a hard problem to tackle. Often it's presented like it's something magical, done by million dollar companies using secret tools. But, there is no such thing as magic, or is there?

Traffic shaping using iptables and tc

Traffic shaping using iptables and tc

Limiting outbound network bandwidth per client IP-address

Last month I received an automated alert indicating excessive bandwidth usage, usually a sign of trouble. When this happens, you should follow a standard incident procedure, trying to isolate the source of the traffic before shutting it down. The cause of this incident was not what I expected however... requiring a different kind of mitigation than a simple blockade.

Designing and implementing a (micro) payment system

Designing and implementing a (micro) payment system

Monetising my blog with coffee, Apple Pay and Mollie

Online payments are now more important than ever as businesses are disrupted by the COVID-19 virus. It drives my customers to seek new ways to make money online. I designed and implemented a (micro)payment system. This post is about achieving simplicity by solving complex challenges.

Monolithic vs Microservices software architecture

Monolithic vs Microservices software architecture

Choosing the right design for your app development

This week I flew to Gothenburg to meet people from a large international shipping company, talking about the development of enterprise level software. During the meeting there were various experts in the room, one of them asked me on choosing the right software architecture (for big, complex, enterprise level apps). A very good question, well worthy for a blog post.

Your own addressbook and calendar cloud

Your own addressbook and calendar cloud

Share contacts, agendas and tasks with CardDAV/CalDAV

If you use different devices and computers to get things done, you might want to synchronise contacts, agendas and tasks. You can use any of the 'big cloud' services for this, like Apple iCloud, Microsoft Office 365 and Google Gmail. But, if you prefer not to share your addressbook and calendar with big American companies, you can do it yourself.

Building a professional 72TB NAS

Building a professional 72TB NAS

Configuration and installation of an HPE ProLiant DL380 server

Over the past few years I have been moving my data and work from local computers (mostly laptops) into the cloud. Cloud computing is done by servers in a datacenter, powerful computers that do the hard work. As my company grew, I needed more capacity. It was time to add some power to my cloud!

Fighting backscatter spam at server level

Fighting backscatter spam at server level

Configure Postfix to block spam before it enters the server

This month I had to deal with backscatter spam, affecting one of the mail servers I manage. As server engineer I make sure that servers don't send spam and that incoming email gets filtered. Despite all good efforts, this server kept being blacklisted for sending spam to iCloud, Office 365 and Google Gmail for Business (G Suite). Read along to find out how what caused this and how to fix this.

Dig for dummies

Dig for dummies

Explaining an highly useful network tool

When you're building websites, apps or email services you may run into domain names and their configurations. When everything is working as it should, most of this is invisible. But when troubleshooting a domain name configuration, it may be necessary to dig a little deeper... read along to learn how!

WordPress: 10 tips to secure your website

WordPress: 10 tips to secure your website

Keep hackers out of the world's most popular content management system

A lot of people use WordPress to manage their website, therefore it's no surprise people ask me to have a look at their site's security. As ethical hacker, I encounter WordPress in different shapes, sizes and states. Some of them are really badly protected against hacks. Prevent your site from being hacked using these 10 practical tips.

Understanding the security concerns in shared hosting

Understanding the security concerns in shared hosting

Considering open ports and unused network facing services

People pay me to hack them, provided I'll explain how it was done, so future hacks can be prevented. As security consultant, I scan for weaknesses in my clients' apps, webshops and websites. Very often a hack starts by exploiting a security hole that is visible remotely. Read along to learn how hackers find security holes and what you can do to secure them.

Is the Apple Watch the modern tool watch?

Is the Apple Watch the modern tool watch?

Testing the Apple Watch's usefulness in real life

This week I went to Frankfurt for business. I had to perform maintenance to servers in a data centre. This seemed like a great opportunity to test the Apple Watch's usefulness in real life (other than health and fitness). I wondered, is the Apple Watch the modern tool watch?

Optimising images for the web and performance

Optimising images for the web and performance

Reduce filesize with jpegoptim, optipng, pngcrush and pngquant

One way to make your website faster is to make it smaller. Not with tiny fonts, but with less bytes! More than half the weight of an average website is because of images. Yet very few people optimise their images for the web and performance, time to find out how much bytes you can safe!

Google Analytics vs AWStats

Google Analytics vs AWStats

Understanding web traffic statistics software

This month I have been working on website statistics, tracking traffic using different technologies. Some of my customers use Google Analytics, others use AWStats, and some use both. Which is better is often debated, but few people really understand the differences. Time to shed some light on the magic of web statistics.

Cyber security: 5 easy tips to protect your server against hackers

Cyber security: 5 easy tips to protect your server against hackers

Server hardening best practices for Windows and Linux

This week one of my clients was hacked and asked me for emergency assistance to help secure their server infrastructure. It was a web server that ran WordPress websites on Apache (with PHP/MySQL), including a few webshops with customer data. This hack could easily have been prevented with the following best practices, is your server secure?

Updating Snake '97

Updating Snake '97

About the challenges of developing a wildly popular game

Few years ago my brother threw a beer on my iPhone in an Amsterdam bar. The poor thing didn't like the Dutch brew as much as I do: it died. While waiting for a new phone to arrive, I used an old one that couldn't do anything but texting, calling and... Snake! The idea for Snake '97 was born and this month it was time to update the wildly popular game.

Clouds below my floor

Clouds below my floor

Building a little datacenter in my basement

At my home I have this crazy fast optical internet connection. It is a 600MB up and down fibre connection which directly arrives in my home (no copper cables involved). It's like a private internet highway. Reason enough to find out if I could do something to make better use of all this speedy fiber galore...
Browse all posts