willem.com

Breaking Changes

Upgrading Dovecot 2.3 to 2.4 in Debian Stable

June 4, 2025 -

Last week I ran into unexpected trouble during a routine maintenance procedure on my company's email infrastructure. Processing thousands of emails each day, we use Dovecot to give our clients access to their messages. That all came to a grinding halt when I updated to version 2.4 which features breaking changes... oh dear!

⚠️ TL;DR: Upgrading Dovecot 2.3 → 2.4 on Debian

Walk in the park

Pending some security updates and to resolve some compatibility issues, I took the decision to update some packages on a Debian GNU/Linux server that is responsible for my company's email services. This particular distribution of GNU/Linux is known for its stability and reliability. There was no reason to expect any trouble and I started the upgrade expecting it to finish in mere minutes. Just "a walk in the park", what could possibly go wrong..?

Requiring 'dovecot_config_version'

Most of the server came back online as expected almost immediately, disrupting services so little that even the uptime monitor alarm didn't pick up any outage - except for Dovecot. This particular type of software is responsible for IMAP and POP3 access to mailboxes. It fits as a finely tuned cog in a larger setup where other programs like Postfix rely upon its working to accept incoming email. Dovecot's failure to start caused a series of problems.

The first thing you do as a server administrator is to check the server daemons ('systemctl status dovecot') followed by inspecting the logs, e.g. by running 'journalctl -u dovecot'. That quickly revealed that Dovecot refused to start because a config parameter was missing: 'dovecot_config_version'.

Simply add the parameter to the config at '/etc/dovecot/dovecot.conf' you might think? Well... NO. It turns out that Dovecot's creators put the error in for a good reason: the new 2.4 version of Dovecot is incompatible with previous 2.3 configuration files; meaning you'll need to rewrite many different parameters. It is not just the config variable names, it also introduces an entirely new syntax of settings variables expansion. An old variable value like '%d' now becomes '%{user | domain}'. The list of chances is long and there is no automated way of migrating your config. Imagine my face when I realised the sheer magnitude of my problem.

Integrated Complexity: cogs in a machine

The thing with email services is that it is hard. Not because of a single component being difficult, but because to run a complete email stack you need many different parts working together seamlessly. Here are some of the pieces that comprise my company's mail service:

A complete mail server stack comprises many smaller components finely tuned together
A complete mail server stack comprises many smaller components finely tuned together

These pieces are all finely tuned to each other, like cogs in a complex machine they fit precisely. Their configuration is tailored to their specific role, often interfacing with multiple other pieces. You can imagine that an unexpected change in one of these cogs (like Dovecot) can cause quite the disruption!

There was no quick fix available, no automated configuration upgrading. Given this update being relatively new, most large language models (AI) did not have an answer either. ChatGPT simply generated suggestions based on the old 2.3 documentation. To resolve the issue I needed to apply 'old school Linux server skills': manually creating a new tailored configuration for Dovecot 2.4.

Stop to Prevent Data Loss

Once I grasped the nature of the problem, I immediately stopped most of the email services (e.g. bringing the engine to a complete stop). This prevents data loss as emails are stopped being processed. This causes other mail servers to queue emails, sometimes issuing 4xx SMTP errors (like 421, 451, 454 or 455). The design of the world wide email protocol accommodates temporary outages and other servers will simply try again later.

Rewriting the config by RTFM

The error message in the logs pointed me to the official Dovecot docs, and there it was: a yellowishly highlighted WARNING describing the configuration changes. It's a lengthy document: as PDF it counts 18 pages. On a normal day it would take some effort to get through this - imagine having to do this while the proverbial house is on fire!

Dovecot's configuration consists of various files that all setup parts of the software's behaviour. Like how authentication is done, or how it integrates with a Sieve filter, or where it gets the username and password hashes for login. The changes introduced in Dovecot 2.4 nearly affect all the parameters as the creators have introduced a new syntax for server variables.

Variable notation syntax

Previously the configuration used variable notation like '%u' as a placeholder for a username. The new syntax allows variables to be piped to a modifier, enabling normalisation and things like sub string selection. For example: to get an email domain from a username you no longer use '%d' but use the 'user'-variable as base to pipe it to the domain modifier by using this syntax: '%{user | domain}'.

Once I understood this new language, I really appreciated the work of art that it entails. It is an improvement over the older abbreviated variable names as you no longer need a dictionary to look things up: instead of something vague like '%r' you now read '%{remove_ip}'. It is inherently more readable, something I really value.

Encryption: 'ssl_cert' becomes 'ssl_server_cert_file'

Other changes include the way the config points to external files used for encryption. Previously you would have a setting 'ssl_cert' that could contain the actual SSL/TLS certificate as string. In version 2.4 you'll have the variable 'ssl_server_cert_file' taking a path pointing to the certificate on the filesystem instead. It is not just renaming, the nature of the value is new, too! To fix the configuration, you'll have to walk each of these updated parameters and determine their impact on your particular setup. It is time consuming and error prone.

Splitting the 'mail_location' setting

New in 2.4 is how the 'mail_location' is split up into multiple smaller variables. Previously a single variable would define where email was saved on disk. You'll now need to split this in two variables 'mail_driver' and 'mail_path', AND use the new server variable syntax. If you get any of these wrong, it will cause all email to disappear (trust me!). Depending on your specific storage configuration, you'll want to double check the username part (e.g. do you include the entire email address or just the prefix sans domain name?).

Migrating mail_location into mail_driver and mail_path using the new server variable syntax
Migrating mail_location into mail_driver and mail_path using the new server variable syntax

Dovecot with SQL backend: passdb and userdb

If your Dovecot installation works with an external database (or store) to authenticate usernames and passwords, you'll have to rewrite the 'auth-sql.conf.ext' in '/etc/dovecot/conf.d/' which is called/included by the 10-auth.conf file. In Dovecot 2.4 there is a new way of segmenting passdb and userdb settings. In the Dovecot 2.3 setup I used an external file to provide the database queries and connection string. During authentication the multiple queries are used to 1) authenticate users, and 2) load user-specific settings like the path of the email store.

The userdb and passdb upgrade instructions are very helpful at first glance /s
The userdb and passdb upgrade instructions are very helpful at first glance /s
Dovecot 2.3 SQL backend setup (auth-sql.conf.ext and dovecot-sql.conf.ext)
Dovecot 2.3 SQL backend setup (auth-sql.conf.ext and dovecot-sql.conf.ext)

In the new 2.4 version you have to restructure the parameters to be enclosed in sections that include a backend type indicator right behind the 'userdb' and 'passdb' keywords in the config. So, if you use a MySQL database as backend, you'll need 'userdb' becomes 'userdb sql'. You can include the connection string parameters right in the 'auth-sql.conf.ext' file so all relevant parameters are in one configuration file. To reduce complexity I rewrote the userdb section to use a the 'static' database driver: generating configuration variables like the user's mail path based on the user's username. See the dedicated documents.

Dovecot 2.4 SQL backend setup (auth-sql.conf.ext) - now much more compact than its predecessor. There is now only one SQL query as the userdb-ection now uses the static backend
Dovecot 2.4 SQL backend setup (auth-sql.conf.ext) - now much more compact than its predecessor. There is now only one SQL query as the userdb-ection now uses the static backend

Sieve Plugin

If your setup uses Sieve to sort incoming emails, you'll need to adjust your setup with regard to the script's location, its defaults and behaviour. The new Dovecot 2.4 version introduces a concept called "Script Storage" which enables dynamically loaded scripts to be executed for different users at various events. Our setup involves a simple default sieve script that sorts email flagged as spam automatically to a folder called Spam. The new syntax enables a more compact config:

Sieve's config in Dovecot 2.3 (90-sieve.conf) with now obsolete variables sieve_default and sieve_dir
Sieve's config in Dovecot 2.3 (90-sieve.conf) with now obsolete variables sieve_default and sieve_dir
Sieve's config in Dovecot 2.4 (90-sieve.conf)
Sieve's config in Dovecot 2.4 (90-sieve.conf)

Testing the New Config

You might be tempted to simply start the server again after redefining the 2.4 configuration. But, you should be very careful as a wrong configuration could cause clients to loose access to all emails and/or force them to re-download their entire mailboxes. Instead I opted for a gradual approach where I changed the IMAP (and POP3) port numbers to something arbitrary. This way I could test the new setup all by myself while keeping the actual customers from connecting to the server. This enabled me to fix some type errors, refine some of the new server variable syntax and deal with some odd log messages. You can control the server's port numbers from the '10-master.conf' file.

Setup the server's IMAP and POP numbers to enable private/gradual debugging of the new configuration
Setup the server's IMAP and POP numbers to enable private/gradual debugging of the new configuration

Conclusion

Outages like this are rare, thanks to the stability of Debian GNU/Linux, but not impossible. Daily backups and a tested disaster recovery plan are in place for my company's Lemmid Online email service. While a full failover wasn’t required this time, prior drills proved crucial in swiftly adapting the configuration.

This incident was a reminder that even with automation and AI, deep system knowledge, preparation, and manual debugging remain essential tools when critical infrastructure components change unexpectedly and without automated migration paths.

Dovecot 2.4 Sample Config

If you find this post by looking for a solution for your own broken mail server, you might find these Dovecot 2.4 sample config files useful. I found them during my own repair work and they might save you some time figuring out the new syntax: https://source.willem.com/dovecot-2.4-sample-config/

Did you enjoy this post?

If you found this content useful,
consider showing your appreciation
by buying me a coffee ❤️😋:

Latest Stories

all DataFree SoftwareServerWork

Articles (165)