High Available (SoHo) dovecot setup using PVE+Ubuntu (+Docker)?
I currently run a HA postfix/dovecot setup that is failover behind a HAproxy. The two legs are a dovecot 2 on macOS (MacPorts) and a dovecot 2 on Ubuntu (PVE+Ubuntu+Docker), making the system somewhat antifragile. HA is realised via dovecot’s own syncing (which is not longer available in dovecot 3). This works OK. (In addition: there is rspamd as an important part of the setup, e.g. for DMARC). There are multiple virtual domains.
Given that I have lost APNS (Apple Push Notification) as Apple has retired support for former Mac OS X Server users, there is not longer a reason to keep the macOS version running (apart from that antifragility thing, but that was more an added bonus). I am looking at moving to a different setup where the entire mail setup may move to a full Linux foundation.
I am starting to look at future options (there is still time to actually create the setup somewhere the coming year or so.
If I want to be future-dovecot proof, I need to move to HA for storage as dovecot syncing is deprecated. I wonder what my best way forward its, based on a 2 physical machine, PVE+Ubuntu+Docker setup. Build a HA NFSv4 cluster? Use Ceph? ZFS?
Any tips, experiences, or ideas are welcome. Robustness/security/low-maintenance/own-your-own-data are my main objectives. I do not have high volumes, this is a setup for just a few users.
G
I currently run a HA postfix/dovecot setup that is failover behind a HAproxy. The two legs are a dovecot 2 on macOS (MacPorts) and a dovecot 2 on Ubuntu (PVE+Ubuntu+Docker), making the system somewhat antifragile. HA is realised via dovecot's own syncing (which is not longer available in dovecot 3). This works OK. (In addition: there is rspamd as an important part of the setup, e.g. for DMARC). There are multiple virtual domains. Given that I have lost APNS (Apple Push Notification) as Apple has retired support for former Mac OS X Server users, there is not longer a reason to keep the macOS version running (apart from that antifragility thing, but that was more an added bonus). I am looking at moving to a different setup where the entire mail setup may move to a full Linux foundation. I am starting to look at future options (there is still time to actually create the setup somewhere the coming year or so. If I want to be future-dovecot proof, I need to move to HA for storage as dovecot syncing is deprecated. I wonder what my best way forward its, based on a 2 physical machine, PVE+Ubuntu+Docker setup. Build a HA NFSv4 cluster? Use Ceph? ZFS? Any tips, experiences, or ideas are welcome. Robustness/security/low-maintenance/own-your-own-data are my main objectives. I do not have high volumes, this is a setup for just a few users. G
On 31/08/2025 14:39 EEST Gerben Wierda via dovecot
<[1]dovecot@dovecot.org> wrote:
I currently run a HA postfix/dovecot setup that is failover behind a
HAproxy. The two legs are a dovecot 2 on macOS (MacPorts) and a dovecot
2 on Ubuntu (PVE+Ubuntu+Docker), making the system somewhat antifragile.
HA is realised via dovecot's own syncing (which is not longer available
in dovecot 3). This works OK. (In addition: there is rspamd as an
important part of the setup, e.g. for DMARC). There are multiple virtual
domains.
Given that I have lost APNS (Apple Push Notification) as Apple has
retired support for former Mac OS X Server users, there is not longer a
reason to keep the macOS version running (apart from that antifragility
thing, but that was more an added bonus). I am looking at moving to a
different setup where the entire mail setup may move to a full Linux
foundation.
I am starting to look at future options (there is still time to actually
create the setup somewhere the coming year or so.
If I want to be future-dovecot proof, I need to move to HA for storage
as dovecot syncing is deprecated. I wonder what my best way forward its,
based on a 2 physical machine, PVE+Ubuntu+Docker setup. Build a HA NFSv4
cluster? Use Ceph? ZFS?Dovecot syncing is not deprecated, replicator is. You can run a PRI/BU setup with doveadm sync in cron.
Aki
References
Visible links
- mailto:dovecot@dovecot.org
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as ’syncing’ is deprecated, but HA-syncing (via replicator) is.
As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn’t match that.
So yes, syncing is not deprecated (I wrote that incorrectly). Doesn’t change my situation as replicator is, and I want a true HA solution.
G
On 31 Aug 2025, at 13:59, Aki Tuomi <aki.tuomi@open-xchange.com> wrote:
On 31/08/2025 14:39 EEST Gerben Wierda via dovecot <dovecot@dovecot.org <mailto:dovecot@dovecot.org>> wrote:
I currently run a HA postfix/dovecot setup that is failover behind a HAproxy. The two legs are a dovecot 2 on macOS (MacPorts) and a dovecot 2 on Ubuntu (PVE+Ubuntu+Docker), making the system somewhat antifragile. HA is realised via dovecot’s own syncing (which is not longer available in dovecot 3). This works OK. (In addition: there is rspamd as an important part of the setup, e.g. for DMARC). There are multiple virtual domains.
Given that I have lost APNS (Apple Push Notification) as Apple has retired support for former Mac OS X Server users, there is not longer a reason to keep the macOS version running (apart from that antifragility thing, but that was more an added bonus). I am looking at moving to a different setup where the entire mail setup may move to a full Linux foundation.
I am starting to look at future options (there is still time to actually create the setup somewhere the coming year or so.
If I want to be future-dovecot proof, I need to move to HA for storage as dovecot syncing is deprecated. I wonder what my best way forward its, based on a 2 physical machine, PVE+Ubuntu+Docker setup. Build a HA NFSv4 cluster? Use Ceph? ZFS?
Dovecot syncing is not deprecated, replicator is. You can run a PRI/BU setup with doveadm sync in cron.
Aki
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as 'syncing' is deprecated, but HA-syncing (via replicator) is. As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn't match that. So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't change my situation as replicator is, and I want a true HA solution. G
On 31 Aug 2025, at 13:59, Aki Tuomi <aki.tuomi@open-xchange.com> wrote:
On 31/08/2025 14:39 EEST Gerben Wierda via dovecot
<[1]dovecot@dovecot.org> wrote:
I currently run a HA postfix/dovecot setup that is failover behind a
HAproxy. The two legs are a dovecot 2 on macOS (MacPorts) and a
dovecot 2 on Ubuntu (PVE+Ubuntu+Docker), making the system somewhat
antifragile. HA is realised via dovecot's own syncing (which is not
longer available in dovecot 3). This works OK. (In addition: there is
rspamd as an important part of the setup, e.g. for DMARC). There are
multiple virtual domains.
Given that I have lost APNS (Apple Push Notification) as Apple has
retired support for former Mac OS X Server users, there is not longer
a reason to keep the macOS version running (apart from that
antifragility thing, but that was more an added bonus). I am looking
at moving to a different setup where the entire mail setup may move to
a full Linux foundation.
I am starting to look at future options (there is still time to
actually create the setup somewhere the coming year or so.
If I want to be future-dovecot proof, I need to move to HA for storage
as dovecot syncing is deprecated. I wonder what my best way forward
its, based on a 2 physical machine, PVE+Ubuntu+Docker setup. Build a
HA NFSv4 cluster? Use Ceph? ZFS?
Dovecot syncing is not deprecated, replicator is. You can run a PRI/BU
setup with doveadm sync in cron.
AkiReferences
Visible links
- mailto:dovecot@dovecot.org
On 31/08/2025 17:02, Gerben Wierda via dovecot wrote:
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as 'syncing' is deprecated, but HA-syncing (via replicator) is. As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn't match that. So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't change my situation as replicator is, and I want a true HA solution. G
Hi Gerben
it's clear that starting from Dovecot 2.4 the Community Edition is not meant to support a HA scenario. Dovecot Pro does, but it's not aimed at SoHo as far as I am aware.
Personally I have never had an HA solution (with dovecot Director etc) since for me it was overkill. I have always used Dovecot with replication as a Disaster Recovery solution - so a backup server that could be brought on line manually and that contained almost all the email history, with exception of limited amount of email that arrived while the disaster was happening. With replication I probably would have had a very small window of email that could be lost, but considering my volumes, expanding that window is not particularly risky.
I have abandoned replication (which if I remember had some unaddressed issues anyway), and am now doing periodic syncs. I do them every 10 minutes, but I guess it could be done more frequently too providing there is some locking logic in the cron job to avoid running multiple syncs (as well as the doveadm sync -l (lock) parameter that someone pointed out to me too).
I don't believe that you will be able to make a truly HA solution from Dovecot CE. If I'm wrong, I'd be glad to hear about it from anyone that has done it. I think you could aim for a Disaster Recovery solution that could be more automated the more effort is put into it.
John
Why can't you make HA out of Dovecot CE? Mounting disk using NAS to multiple servers and take it as a maildir storage, and have dovecot running in each with a POP loading balancing requests having HAProxy e.g. in frontend. Also using perhaps MySQL Cluster for virtual users and connect dovecot to it?
This sounds to me from experience point of view very doable, and not sure why you think it isnt? No need for syncing data, since NAS will have data already synced and the bigger it gets the more disks you need to extend the RAID rack and you can build another server and rack as backup and write small script to take backup everyday and its sorted?
Zak.
On 2025-08-31 17:30, John Fawcett via dovecot wrote:
On 31/08/2025 17:02, Gerben Wierda via dovecot wrote:
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as 'syncing' is deprecated, but HA-syncing (via replicator) is. As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn't match that. So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't change my situation as replicator is, and I want a true HA solution. G
Hi Gerben
it's clear that starting from Dovecot 2.4 the Community Edition is not meant to support a HA scenario. Dovecot Pro does, but it's not aimed at SoHo as far as I am aware.
Personally I have never had an HA solution (with dovecot Director etc) since for me it was overkill. I have always used Dovecot with replication as a Disaster Recovery solution - so a backup server that could be brought on line manually and that contained almost all the email history, with exception of limited amount of email that arrived while the disaster was happening. With replication I probably would have had a very small window of email that could be lost, but considering my volumes, expanding that window is not particularly risky.
I have abandoned replication (which if I remember had some unaddressed issues anyway), and am now doing periodic syncs. I do them every 10 minutes, but I guess it could be done more frequently too providing there is some locking logic in the cron job to avoid running multiple syncs (as well as the doveadm sync -l (lock) parameter that someone pointed out to me too).
I don't believe that you will be able to make a truly HA solution from Dovecot CE. If I'm wrong, I'd be glad to hear about it from anyone that has done it. I think you could aim for a Disaster Recovery solution that could be more automated the more effort is put into it.
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
Why can't you make HA out of Dovecot CE? Mounting disk using NAS to multiple servers and take it as a maildir storage, and have dovecot running in each with a POP loading balancing requests having HAProxy e.g. in frontend. Also using perhaps MySQL Cluster for virtual users and connect dovecot to it?
This sounds to me from experience point of view very doable, and not sure why you think it isnt? No need for syncing data, since NAS will have data already synced and the bigger it gets the more disks you need to extend the RAID rack and you can build another server and rack as backup and write small script to take backup everyday and its sorted?
Zak.
On 2025-08-31 17:30, John Fawcett via dovecot wrote:
On 31/08/2025 17:02, Gerben Wierda via dovecot wrote:
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as 'syncing' is deprecated, but HA-syncing (via replicator) is. As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn't match that. So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't change my situation as replicator is, and I want a true HA solution. G
Hi Gerben
it's clear that starting from Dovecot 2.4 the Community Edition is not meant to support a HA scenario. Dovecot Pro does, but it's not aimed at SoHo as far as I am aware.
Personally I have never had an HA solution (with dovecot Director etc) since for me it was overkill. I have always used Dovecot with replication as a Disaster Recovery solution - so a backup server that could be brought on line manually and that contained almost all the email history, with exception of limited amount of email that arrived while the disaster was happening. With replication I probably would have had a very small window of email that could be lost, but considering my volumes, expanding that window is not particularly risky.
I have abandoned replication (which if I remember had some unaddressed issues anyway), and am now doing periodic syncs. I do them every 10 minutes, but I guess it could be done more frequently too providing there is some locking logic in the cron job to avoid running multiple syncs (as well as the doveadm sync -l (lock) parameter that someone pointed out to me too).
I don't believe that you will be able to make a truly HA solution from Dovecot CE. If I'm wrong, I'd be glad to hear about it from anyone that has done it. I think you could aim for a Disaster Recovery solution that could be more automated the more effort is put into it.
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 31/08/2025 20:04, Zakaria via dovecot wrote:
Why can't you make HA out of Dovecot CE? Mounting disk using NAS to multiple servers and take it as a maildir storage, and have dovecot running in each with a POP loading balancing requests having HAProxy e.g. in frontend. Also using perhaps MySQL Cluster for virtual users and connect dovecot to it? This sounds to me from experience point of view very doable, and not sure why you think it isnt? No need for syncing data, since NAS will have data already synced and the bigger it gets the more disks you need to extend the RAID rack and you can build another server and rack as backup and write small script to take backup everyday and its sorted? Zak. On 2025-08-31 17:30, John Fawcett via dovecot wrote: > On 31/08/2025 17:02, Gerben Wierda via dovecot wrote: >> Will doveadm sync in a cron job give me something resembling HA? Don't >> think so. Replicator does. So, read my question not as 'syncing' is >> deprecated, but HA-syncing (via replicator) is. >> As far as I understand it, replicator sends the message immediately to the >> other side as it arrives. A, say, once every 15 minutes cron job running a >> new doveadm sync doesn't match that. >> So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't >> change my situation as replicator is, and I want a true HA solution. >> G >>
Hi Zak
I doubt that anyone has something like that working successfully. If they have, I'd be glad to hear of it. My reasoning is:
I would not assume that multiple Dovecot instances accessing and updating a common mail store is supported unless it is stated somewhere. My understanding is that you cannot just take a software that expects to handle a mail store being accessed only by itself and turn it into a multi instance unless the software supports that feature.
In the previous versions of Dovecot there was a specific architecture supporting multiple servers. Maybe it was there only for managing scaling, but I am assuming it also managed HA and that without it HA was not possible.
In the new version of Dovecot it has been stated that the features supporting multiple servers are available in Dovecot Pro and not in Dovecot CE. I recognize that this statement is not 100% correct since sync works across multiple servers, however not as HA.
I am not familiar with HAProxy, but my doubt about that is whether it would guarantee to have the same user always handled by the same Dovecot server, at least as a first approach to not having more than one Dovecot server accessing the same mailbox simultaneously.
John
Aha, I understood the points. I remember Dovecot team said they intend no longer maintain Director and plan to remove it in 2.4. it looks they went ahead with it. Well, then I hope you find a solution that fits in your stack but you might want to look at this as well. https://doc.dovecot.org/main/core/config/proxy/haproxy.html
Zak.
On 2025-08-31 20:05, John Fawcett via dovecot wrote:
On 31/08/2025 20:04, Zakaria via dovecot wrote:
Why can't you make HA out of Dovecot CE? Mounting disk using NAS to multiple servers and take it as a maildir storage, and have dovecot running in each with a POP loading balancing requests having HAProxy e.g. in frontend. Also using perhaps MySQL Cluster for virtual users and connect dovecot to it?
This sounds to me from experience point of view very doable, and not sure why you think it isnt? No need for syncing data, since NAS will have data already synced and the bigger it gets the more disks you need to extend the RAID rack and you can build another server and rack as backup and write small script to take backup everyday and its sorted?
Zak.
On 2025-08-31 17:30, John Fawcett via dovecot wrote:
On 31/08/2025 17:02, Gerben Wierda via dovecot wrote:
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as 'syncing' is deprecated, but HA-syncing (via replicator) is. As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn't match that. So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't change my situation as replicator is, and I want a true HA solution. G
Hi Zak
I doubt that anyone has something like that working successfully. If they have, I'd be glad to hear of it. My reasoning is:
I would not assume that multiple Dovecot instances accessing and updating a common mail store is supported unless it is stated somewhere. My understanding is that you cannot just take a software that expects to handle a mail store being accessed only by itself and turn it into a multi instance unless the software supports that feature.
In the previous versions of Dovecot there was a specific architecture supporting multiple servers. Maybe it was there only for managing scaling, but I am assuming it also managed HA and that without it HA was not possible.
In the new version of Dovecot it has been stated that the features supporting multiple servers are available in Dovecot Pro and not in Dovecot CE. I recognize that this statement is not 100% correct since sync works across multiple servers, however not as HA.
I am not familiar with HAProxy, but my doubt about that is whether it would guarantee to have the same user always handled by the same Dovecot server, at least as a first approach to not having more than one Dovecot server accessing the same mailbox simultaneously.
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
Aha, I understood the points. I remember Dovecot team said they intend no longer maintain Director and plan to remove it in 2.4. it looks they went ahead with it. Well, then I hope you find a solution that fits in your stack but you might want to look at this as well. https://doc.dovecot.org/main/core/config/proxy/haproxy.html
Zak.
On 2025-08-31 20:05, John Fawcett via dovecot wrote:
On 31/08/2025 20:04, Zakaria via dovecot wrote:
Why can't you make HA out of Dovecot CE? Mounting disk using NAS to multiple servers and take it as a maildir storage, and have dovecot running in each with a POP loading balancing requests having HAProxy e.g. in frontend. Also using perhaps MySQL Cluster for virtual users and connect dovecot to it?
This sounds to me from experience point of view very doable, and not sure why you think it isnt? No need for syncing data, since NAS will have data already synced and the bigger it gets the more disks you need to extend the RAID rack and you can build another server and rack as backup and write small script to take backup everyday and its sorted?
Zak.
On 2025-08-31 17:30, John Fawcett via dovecot wrote:
On 31/08/2025 17:02, Gerben Wierda via dovecot wrote:
Will doveadm sync in a cron job give me something resembling HA? Don't think so. Replicator does. So, read my question not as 'syncing' is deprecated, but HA-syncing (via replicator) is. As far as I understand it, replicator sends the message immediately to the other side as it arrives. A, say, once every 15 minutes cron job running a new doveadm sync doesn't match that. So yes, syncing is not deprecated (I wrote that incorrectly). Doesn't change my situation as replicator is, and I want a true HA solution. G
Hi Zak
I doubt that anyone has something like that working successfully. If they have, I'd be glad to hear of it. My reasoning is:
I would not assume that multiple Dovecot instances accessing and updating a common mail store is supported unless it is stated somewhere. My understanding is that you cannot just take a software that expects to handle a mail store being accessed only by itself and turn it into a multi instance unless the software supports that feature.
In the previous versions of Dovecot there was a specific architecture supporting multiple servers. Maybe it was there only for managing scaling, but I am assuming it also managed HA and that without it HA was not possible.
In the new version of Dovecot it has been stated that the features supporting multiple servers are available in Dovecot Pro and not in Dovecot CE. I recognize that this statement is not 100% correct since sync works across multiple servers, however not as HA.
I am not familiar with HAProxy, but my doubt about that is whether it would guarantee to have the same user always handled by the same Dovecot server, at least as a first approach to not having more than one Dovecot server accessing the same mailbox simultaneously.
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 31/08/2025 23:17, Zakaria via dovecot wrote:
Aha, I understood the points. I remember Dovecot team said they intend no longer maintain Director and plan to remove it in 2.4. it looks they went ahead with it. Well, then I hope you find a solution that fits in your stack but you might want to look at this as well. https://doc.dovecot.org/main/core/config/proxy/haproxy.html Zak.
Hi Zak
interesting that even in 2.4.1 "HAProxy is the recommended way to do software load balancing for Dovecot."
https://doc.dovecot.org/2.4.1/core/config/proxy/haproxy.html
In the configuration shown there I didn't identify anything that would enable the same user to always be sent to the same back end server, so in absence of HAProxy managing this, then the underlying Dovecot architecture would need to be able to manage simultaneous updates from multiple servers for the same user (for example if my mailbox is open from my PC and from my phone but those connections are being handled by different Dovecot backend servers). If I move a message to a folder using the PC I don't see how that same update would get to the phone, which is the kind of point the OP was making.
I am wondering whether this documentation on HA Proxy is still appropriate to 2.4.1 Dovecot CE.
John
On 31/08/2025 23:17, Zakaria via dovecot wrote:
Aha, I understood the points. I remember Dovecot team said they intend no
longer maintain Director and plan to remove it in 2.4. it looks they went
ahead with it. Well, then I hope you find a solution that fits in your
stack but you might want to look at this as well.
[1]https://doc.dovecot.org/main/core/config/proxy/haproxy.html
Zak.Hi Zak
interesting that even in 2.4.1 "HAProxy is the recommended way to do software load balancing for Dovecot."
[2]https://doc.dovecot.org/2.4.1/core/config/proxy/haproxy.html
In the configuration shown there I didn't identify anything that would enable the same user to always be sent to the same back end server, so in absence of HAProxy managing this, then the underlying Dovecot architecture would need to be able to manage simultaneous updates from multiple servers for the same user (for example if my mailbox is open from my PC and from my phone but those connections are being handled by different Dovecot backend servers). If I move a message to a folder using the PC I don't see how that same update would get to the phone, which is the kind of point the OP was making.
I am wondering whether this documentation on HA Proxy is still appropriate to 2.4.1 Dovecot CE.
John
References
Visible links
On 09/01/2025 6:28 AM MDT John Fawcett via dovecot <dovecot@dovecot.org> wrote:
interesting that even in 2.4.1 "HAProxy is the recommended way to do software load balancing for Dovecot."
https://doc.dovecot.org/2.4.1/core/config/proxy/haproxy.html
Yes, most of this page doesn't belong in CE documentation anymore.
https://github.com/dovecot/documentation/pull/1292
michael
participants (5)
-
Aki Tuomi
-
Gerben Wierda
-
John Fawcett
-
Michael Slusarz
-
Zakaria