[Dovecot] mdbox: more then one altstorage
The mdbox-format with the altstorage-feature is quite nice and important.
But we're having much more then >> 30 TB of maildata and doing an fsck on huge partitions takes too much (down-) time for our mailsystem.
It would be much easier, if Dovecot would be able to use at least THREE storage-paths for his m-Files (and not just TWO like now).
Having an altstorage1 and altstorage2, we would be able to move away all mails from on altstorage-mountpoint, we could put this storage-moungpoint into "maintenance" and do an fsck without Downtime for Dovecot.
It would be nice having a doveadm-feature to put a altstorage-path in "maintenance-mode", so to move easily away all m-files to other alt-storage-paths and to let Dovecot know not to use this locked partition.
Having even more then 3 storage-paths would be nice for huge setups: In this case we could easily use different SMALLER partitions without having just two or three huge partitions > 15 TB.
The architecture of the altstorage-system is nice and simple. Wouldn't it be easily possible to look (at least) in three, then just in two places for a m-file?
What about a "hash_dir"-system for the storage? What about a configuration-parameter like
altstorage_dirs=4
which would mean, that there are 4 storage-mountpoints that could be addressed by "%m":
mail_location = mdbox:/vmail/%d/%n/mdbox:altstorage/alt/%m/%d/%n
In this case, Dovecot could easily use all existing partitions automatically "round-robin".
Or maybe it would be better to tell Dovecot the partitions by:
altstorage_dirs_existing = 1,2,3,4 altstorage_dirs_newmail = 1,4
so it would be very easy to tell Dovecot in which partitions he has to look for m-files and where he is allowed to save round-robin new e-mails. With this setup it's easy to put a partition in maintenance mode.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On 12/15/2012 1:44 AM, Peer Heinlein wrote:
But we're having much more then >> 30 TB of maildata and doing an fsck on huge partitions takes too much (down-) time for our mailsystem.
Peer are you using NFS or a cluster filesystem? Linux on the hosts or other?
-- Stan
Am 15.12.2012 09:23, schrieb Stan Hoeppner:> On 12/15/2012 1:44 AM, Peer Heinlein wrote:
But we're having much more then >> 30 TB of maildata and doing an fsck on huge partitions takes too much (down-) time for our mailsystem.
Peer are you using NFS or a cluster filesystem? Linux on the hosts or other?
I don't understand the reason for the question.
Even a NFS is based on a "real" filesystem that needs to be checked.
So: Where's the difference?!
But to answer your question: No, I'm using local Filesystems (and that's good!) and I'd absolutely avoid NFS or a ClusterFS for setups as big as this.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
El 15/12/12 08:44, Peer Heinlein escribió:
The mdbox-format with the altstorage-feature is quite nice and important.
But we're having much more then >> 30 TB of maildata and doing an fsck on huge partitions takes too much (down-) time for our mailsystem.
Wouldn't be easier just to split your users between different filesystems? I mean having different altstorages, but not as you propose but one for the half of your users and other for the other half.
-- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337
Am 17.12.2012 08:31, schrieb Angel L. Mateo:
But we're having much more then >> 30 TB of maildata and doing an fsck on huge partitions takes too much (down-) time for our mailsystem.
Wouldn't be easier just to split your users between differentfilesystems? I mean having different altstorages, but not as you propose but one for the half of your users and other for the other half.
For sure we DO have users split up into several 5 TB-partitions. This helps a little, but not a lot. And it creates some extra work and some extra sources of errors: You'll need Symlinks on the "real" fs or a user-dependend storage path in LDAP.
But anyway, those partitions are tooo big for short filesystem checks (or if you make them smaller, you'd have too many).
And, anyway: As I explained I'm interested in zero-downtime-checks, which wouldn't work just with several partitions.
Peer
-- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin
http://www.heinlein-support.de
Tel: 030 / 405051-42 Fax: 030 / 405051-19
Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin
On Sat, 2012-12-15 at 08:44 +0100, Peer Heinlein wrote:
The mdbox-format with the altstorage-feature is quite nice and important.
But we're having much more then >> 30 TB of maildata and doing an fsck on huge partitions takes too much (down-) time for our mailsystem.
It would be much easier, if Dovecot would be able to use at least THREE storage-paths for his m-Files (and not just TWO like now).
People have asked for this a few times, but it's not that easy to implement (because of how altstorage code is now implemented), and I'm not really convinced that it's worth the trouble to add.
Having an altstorage1 and altstorage2, we would be able to move away all mails from on altstorage-mountpoint, we could put this storage-moungpoint into "maintenance" and do an fsck without Downtime for Dovecot.
It would be nice having a doveadm-feature to put a altstorage-path in "maintenance-mode", so to move easily away all m-files to other alt-storage-paths and to let Dovecot know not to use this locked partition.
The new "doveadm mount" stuff can handle this. If you unmount a filesystem (and maybe mounting it to another mountpoint), Dovecot will just fail all commands that attempt to access the altstorage (instead of assuming all the mails are gone and recreating the index).
I think moving from one altstorage to another would be possible with rsync:
- Mount the new storage
- cp/rsync files to new storage [3. If it takes long enough that doveadm purge/altmove is already run, rsync again.]
- Move the new storage's mountpoint over the old one (not sure if there's a way to do this atomically?)
Having even more then 3 storage-paths would be nice for huge setups: In this case we could easily use different SMALLER partitions without having just two or three huge partitions > 15 TB.
Like others mentioned, you could do this also by dividing users to multiple different partitions.
participants (4)
-
Angel L. Mateo
-
Peer Heinlein
-
Stan Hoeppner
-
Timo Sirainen