Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
What are you trying to achieve with containerizing? If you want to deploy the container on a orchestrator so it is going to be (re)started on a random hosts. You have to take into account that your storage is persistant, you have to take into account getting something like proxy to redirect traffic etc etc. I managed to get this to work via alpine linux (nice small), but I think I am going back to vm.
-----Original Message----- From: dovecot dovecot-bounces@dovecot.org On Behalf Of MRob Sent: Tuesday, 24 August 2021 09:13 To: dovecot@dovecot.org Subject: Containerize dovecot?
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
Thank to other responses with links that i'll learning from! Thanks you very much
On 2021-08-24 00:35, Marc wrote:
What are you trying to achieve with containerizing?
hmm, easy deployment & scaling? also reslient against hardware crashes, etc.
You have to take into account that your storage is persistant,
Have you see what challenge this poses? Love to hear your opinion about it. Maybe this is where using object-storage backend becomes more powerful solution?
you have to take into account getting something like proxy to redirect traffic etc etc.
Yes I thought also proxy/director also become containerized. You said this for teaching a certain point?
If you want to deploy the container on a orchestrator so it is going to be (re)started on a random hosts.
you mention so because all hosts must be connected to same storage?
I managed to get this to work via alpine linux (nice small), but I think I am going back to vm.
what reasons? what kind of vm do you use?
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
I think the general concensus is that containerisation isn't always better than 'normal' VMs. 'Easy deployment & scaling' is also perfectly possible without containers.
MRob schreef op 2021-08-25 00:01:
Thank to other responses with links that i'll learning from! Thanks you very much
On 2021-08-24 00:35, Marc wrote:
What are you trying to achieve with containerizing?
hmm, easy deployment & scaling? also reslient against hardware crashes, etc.
You have to take into account that your storage is persistant,
Have you see what challenge this poses? Love to hear your opinion about it. Maybe this is where using object-storage backend becomes more powerful solution?
you have to take into account getting something like proxy to redirect traffic etc etc.
Yes I thought also proxy/director also become containerized. You said this for teaching a certain point?
If you want to deploy the container on a orchestrator so it is going to be (re)started on a random hosts.
you mention so because all hosts must be connected to same storage?
I managed to get this to work via alpine linux (nice small), but I think I am going back to vm.
what reasons? what kind of vm do you use?
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
-- With kind regards,
William Edwards
On 25/08/2021 06.46, William Edwards wrote:
I think the general concensus is that containerisation isn't always better than 'normal' VMs. 'Easy deployment & scaling' is also perfectly possible without containers.
I've had a few discussions with clients who are dead-set on running a docker this, or a snap that, or whatever container they just read about. But they want to run it on a VM in the cloud. To me it seems weird. In your cloud VM you already have an environment which you can configure exactly how you like it. You can resize it, clone it, snapshot it. Running a docker container inside this VM just seems to be adding an extra level of complexity to things, probably resulting in worse performance.
I can see docker containers have their uses, especially for development environments, but I don't think it makes sense in a lot of cases. Feel free to persuade me otherwise! I'm keen to hear counter arguments!
P.
I've had a few discussions with clients who are dead-set on running a docker this, or a snap that, or whatever container they just read about. But they want to run it on a VM in the cloud. To me it seems weird.
To anyone with a brain this is weird ;) If you have remote cloud, they are already arranged the failover. Maybe they just have multiple software developers that only give docker images???
your cloud VM you already have an environment which you can configure exactly how you like it. You can resize it, clone it, snapshot it. Running a docker container inside this VM just seems to be adding an extra level of complexity to things, probably resulting in worse performance.
Not really worse performance. Bad thing is, that all your processes are killed if dockerd crashes, this is still not fixed. That would be my main argument never to use dockerd.
I can see docker containers have their uses, especially for development environments, but I don't think it makes sense in a lot of cases. Feel free to persuade me otherwise! I'm keen to hear counter arguments!
I think it makes sense in most simple cases. Now I have multiple vm's for handling incoming mail. These all have a copy of the ldap,spamd,clamd,sendmail. Converting this to separate containers. I could scale up/down (automatically) the individual processes like clamd when necessary. If I want to start using a newer clamd because the lts distribution does not update it. I can just replace/run concurrently/revert this at any time. Having the orchestrator set your resource limits and monitoring your processes is also nice out of the box. I am not really checking in the vm if clamd hangs on 100% or OM's (ssshhhhhh ;)). In my container environment I do not need to.
Furthermore you can make a difference between stateless and stateful tasks. The ldap server can be stateless, because it syncs it's updates anyway, same as spamd and clamd. Which means if a hosts fails, I can just start these automatically on different hosts not needing to worry about fs problems. In my environment clamd / spamd are not even on a public facing environment these are now isolated, so by decreasing my remote attack surface my security goes up.
What I have also noticed, is that having such environment you are more like to quickly test with new applications. I also think that, now redhat is letting go of centos, the time of lts distributions is coming to an end.
However most applications are not really designed to be used as containers (micro services). But if when they are, all these tasks are communicating with each other on dynamic ip addresses and dynamic ports on dynamic hosts. I would recommend start playing with the containers, and I guarantee you will discover their use and beauty.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, August 24th, 2021 at 11:46 PM, William Edwards wedwards@cyberfusion.nl wrote:
I think the general concensus is that containerisation isn't always
better than 'normal' VMs. 'Easy deployment & scaling' is also perfectly
possible without containers.
Amen to that ! What's the old saying ? "Just because you can, doesn't mean you should !"
On 8/24/21 6:46 PM, William Edwards wrote:
I think the general concensus is that containerisation isn't always better than 'normal' VMs. 'Easy deployment & scaling' is also perfectly possible without containers.
Without a doubt VMs are a good solution. Containers use some of the same technology however and since they're based on the Linux kernel API they can do it with a bit more performance (as opposed to emulating hardware). While the performance hit for a VM is small, most of the folks who use containers are trying to squeeze as many apps in a "multi-tenant" host which means every CPU cycle matters.
Also, with recent CGROUPS 2 changes there's the ability to nest containers.
I think overall containers offer a "capability based" paradigm while VMs offer an emulation paradigm. In theory the capability based paradigm ought to be easier to use and more flexible so it's reasonable that folks want to use it for things like Dovecot.
MRob schreef op 2021-08-25 00:01:
Thank to other responses with links that i'll learning from! Thanks you very much
On 2021-08-24 00:35, Marc wrote:
What are you trying to achieve with containerizing?
hmm, easy deployment & scaling? also reslient against hardware crashes, etc.
You have to take into account that your storage is persistant,
Persistent storage is definitely something to think about and plan for but most containerization solutions offer easy ways to mount persistent storage into containers. This can be an advantage, your business logic can run in the container to which you ascribe a certain amount of CPU per your needs and your persistent data can live on the host file system.
Have you see what challenge this poses? Love to hear your opinion about it. Maybe this is where using object-storage backend becomes more powerful solution?
you have to take into account getting something like proxy to redirect traffic etc etc.
Indeed, ingress and egress as well as networking become more complex.
Yes I thought also proxy/director also become containerized. You said this for teaching a certain point?
If you want to deploy the container on a orchestrator so it is going to be (re)started on a random hosts.
you mention so because all hosts must be connected to same storage?
I managed to get this to work via alpine linux (nice small), but I think I am going back to vm.
what reasons? what kind of vm do you use?
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
Cheers,
Jeremiah
What are you trying to achieve with containerizing?
hmm, easy deployment & scaling? also reslient against hardware crashes, etc.
Forget about easy. You have a vm up and running more quickly. Scaling is a problem because somehow you have to add storage (which is easy) but what then, how do you manage what email is where and what accounts are where automatically? It is not so trivial.
You have to take into account that your storage is persistant,
Have you see what challenge this poses? Love to hear your opinion about it. Maybe this is where using object-storage backend becomes more powerful solution?
Yes indeed! In my test I was using the ceph object storage with the t-mobile software. This way you would be able to scale dovecot instances easily because the storage is remotely available anyway. However: currently it looks like there is some sort of bug or so, your email is stored in 2x disk usage. So with 3x replication in ceph (default), storing 1GB of emails, takes 6GB of raw harddisk usage, 1x3x2(this bug??) Another drawback of implementing this the dependency on cephfs for indexes. I do not know if I would like to have a dependency on the mds daemon of ceph currently, especially in my hyper converged environment. But looking at my old test results the cephfs was out performing the local rbd image. So I was thinking of maybe storing the emails in object storage and use an rbd ssd device mapping as local index storage (or maybe even a local persistent lvm on ssd/nvme). But if I would scale a 2nd instance, how would I synchronize indexes between these instances???
you have to take into account getting something like proxy to redirect traffic etc etc.
Yes I thought also proxy/director also become containerized. You said this for teaching a certain point?
Yes indeed if you have this already running, than it is more logical to pursue this path of containerizing. Otherwise it is acquiring more knowledge/experience etc etc.
If you want to deploy the container on a orchestrator so it is going to be (re)started on a random hosts.
you mention so because all hosts must be connected to same storage?
Yes indeed, but do not use dockerd.
I managed to get this to work via alpine linux (nice small), but I think I am going back to vm.
what reasons? what kind of vm do you use?
If you have a vm you have more and easier control over your environment. Using mdbox currently on ceph rbd images with 1000 - 2000 accounts is fine, no performance issues. So that drops already my scalability needs. If I would use the object storage, I would need to use special tools to access these emails which just complicates things. If I would allow dovecot to access the object storage directly, I would have better iops performance but I would also this bug needing twice the space. Furthermore I have no idea what influence this will have on the rest of the object storage clients, because in such case all dovecot instances could hammer the object storage, and maybe it is just better to have iops limits of using an rbd image. (I think my ceph version it is not easy throttling clients, but maybe you can solve this in haproxy also)
I also do not want to waste to much time on deciding, testing and figuring out what to do. I have been running for years without problems with a vm. If I want to get my new project online before the end of the year, I would need to stick with what I know works for sure.
Furthermore I am in a position where I would consider containerizing first vm's that are easier. Like incoming mailservers with sendmail, spamassassin, clamd and ldap.
But I have to admit having such container solution is cool. My container images were just between 27MB - 142MB and if I was killing it, it indeed was starting on different host, really really nice.
Hi Rob,
Am 24.08.21 um 09:13 schrieb MRob:
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
We (www.egroupware.org) run Dovecot only containerized:
as mail-server addon to our groupware server: - https://github.com/EGroupware/egroupware/wiki/EGroupwareMail - https://github.com/EGroupware/build.opensuse.org/tree/master/server:eGroupWa...
for our own SAAS hosting with multiple directors and backend-pairs
We use 2 different containers:
a) based on Dovecot CE repo including Lua for our push notifications b) based on Alpine for directors, SASL and backend-pairs without push
So far we're not autoscaling anything.
Ralf
-- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0
Am 2021-08-24 09:13, schrieb MRob:
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
Yes, Mailcow contains Dovecot. And Mailcow has been dockerized.
https://github.com/mailcow/mailcow-dockerized
-- Mit freundlichen Grüßen,
Marc Stürmer Ahornstraße 5 | 97506 Grafenrheinfeld Tel: 09723 9313969 | Mobil: 0174 3393862 | Fax: 09723 9314810 | mail@marc-stuermer.de
MRob wrote:
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
Fairly straight forward.
Use LXD running either Ubuntu or Alpine as the Distro.
You can also use Docker, which means you'll have to handroll your own LXD-ish filesystem management system (shudder...) using Docker volumes.
Good rule...
Use Docker for ephemeral data, where all data dies across container restarts.
Use LXD for persistent data... SQL databases, IMAP mailstores, etc... where data lives across container restarts.
Can you use Docker for this? Yes!
Should you? Well... depends on if you have a lot of free time to redevelop and entire set of LXD-ish tools.
If you're super lazy... er, I mean efficient, as I am... use LXD...
MRob wrote:
Hello, anyone here has containerized dovecot? Can I ask general advice and experience please? are there any recommended articles/tutorial for containerize deploymnt and auto-scaling? Thank you.
I made a start where the container would use distributed storage, so you do not have any issues with mailbox data being pinned to a specific dovecot instance. Then you only need to have some proxy/director to make sure users are pinned to the same dovecot instance. This situation of course is easy to autoscale. But then I thought I can easily host a few thousand accounts on a regular vm setup, so first get these accounts, then I will think again about auto scaling. Why would you need such a setup?
participants (9)
-
David Favor
-
Jeremiah C. Foster
-
Laura Smith
-
Marc
-
Marc Stürmer
-
MRob
-
Plutocrat
-
Ralf Becker
-
William Edwards