What are you trying to achieve with containerizing?
hmm, easy deployment & scaling? also reslient against hardware crashes, etc.
Forget about easy. You have a vm up and running more quickly. Scaling is a problem because somehow you have to add storage (which is easy) but what then, how do you manage what email is where and what accounts are where automatically? It is not so trivial.
You have to take into account that your storage is persistant,
Have you see what challenge this poses? Love to hear your opinion about it. Maybe this is where using object-storage backend becomes more powerful solution?
Yes indeed! In my test I was using the ceph object storage with the t-mobile software. This way you would be able to scale dovecot instances easily because the storage is remotely available anyway. However: currently it looks like there is some sort of bug or so, your email is stored in 2x disk usage. So with 3x replication in ceph (default), storing 1GB of emails, takes 6GB of raw harddisk usage, 1x3x2(this bug??) Another drawback of implementing this the dependency on cephfs for indexes. I do not know if I would like to have a dependency on the mds daemon of ceph currently, especially in my hyper converged environment. But looking at my old test results the cephfs was out performing the local rbd image. So I was thinking of maybe storing the emails in object storage and use an rbd ssd device mapping as local index storage (or maybe even a local persistent lvm on ssd/nvme). But if I would scale a 2nd instance, how would I synchronize indexes between these instances???
you have to take into account getting something like proxy to redirect traffic etc etc.
Yes I thought also proxy/director also become containerized. You said this for teaching a certain point?
Yes indeed if you have this already running, than it is more logical to pursue this path of containerizing. Otherwise it is acquiring more knowledge/experience etc etc.
If you want to deploy the container on a orchestrator so it is going to be (re)started on a random hosts.
you mention so because all hosts must be connected to same storage?
Yes indeed, but do not use dockerd.
I managed to get this to work via alpine linux (nice small), but I think I am going back to vm.
what reasons? what kind of vm do you use?
If you have a vm you have more and easier control over your environment. Using mdbox currently on ceph rbd images with 1000 - 2000 accounts is fine, no performance issues. So that drops already my scalability needs. If I would use the object storage, I would need to use special tools to access these emails which just complicates things. If I would allow dovecot to access the object storage directly, I would have better iops performance but I would also this bug needing twice the space. Furthermore I have no idea what influence this will have on the rest of the object storage clients, because in such case all dovecot instances could hammer the object storage, and maybe it is just better to have iops limits of using an rbd image. (I think my ceph version it is not easy throttling clients, but maybe you can solve this in haproxy also)
I also do not want to waste to much time on deciding, testing and figuring out what to do. I have been running for years without problems with a vm. If I want to get my new project online before the end of the year, I would need to stick with what I know works for sure.
Furthermore I am in a position where I would consider containerizing first vm's that are easier. Like incoming mailservers with sendmail, spamassassin, clamd and ldap.
But I have to admit having such container solution is cool. My container images were just between 27MB - 142MB and if I was killing it, it indeed was starting on different host, really really nice.