readonly archive folders using squashfs
I have a dovecot server running under CentOS using maildir format. Due to the issue with minimum blocksize for files I would like to offer some kind of readonly archive using something like the compressed squashfs where I would move messages to be archived to a maildir folder and then convert "cur" directory into a squashfs and mount it in place of the original directory so my biggest users could have readonly access to older messages without it using so much disk space.
Has anyone tried anything like this before and is dovecot likely to complain about the readonly cur directory? If the complaints are minimal and didn't cause other problems it might be ok. Any better ideas to implement something like this?
Thank You,
Natu
For every user@domain.tld I created a user@backup.domain.tld where he could look up deleted messages (archive).
I then made user@backup.domain.tld's cur directory a shared directory to user@domain.tld but only with read privileges. So, anytime the user wants to read his old messages, he only needs to read his .archive folder, which is user@backup.domain.tld's cur. No need to make the filesystem read-only.
The difficulty is to make sure that every e-mail, sent or received, gets backed up properly before being deleted. Relying on cron jobs is not an option since the e-mail can be deleted and expunged before the script has a chance to get executed and do the backup. So what I did is to create a hidden sieve filter for every user's **main** mailbox (user@domain.tld) that automatically creates a copy of every incoming message to the user@backup.domain.tld mailbox. For outgoing e-mail, one can do a bcc map in postfix (or the equivalent in other SMTP software) that ensures that every sent mail is also sent to user+sent@backup.domain.tld, then you can create a filter in the user's **backup** mailbox (user@backup.domain.tld) that filters on the user+sent part of the e-mail and stores every e-mail sent to that e-mail address to the .Sent directory in the user@backup.domain.tld mailbox.
Finally, the backup.domain.tld doesn't even have to be declared in the DNS nor in /etc/hosts and can be entirely virtual to the MTA (for ex. in postfix that would only be added to virtual_mailbox_domains)
Yassine.
On 3/8/19 12:49 AM, Natu via dovecot wrote:
I have a dovecot server running under CentOS using maildir format. Due to the issue with minimum blocksize for files I would like to offer some kind of readonly archive using something like the compressed squashfs where I would move messages to be archived to a maildir folder and then convert "cur" directory into a squashfs and mount it in place of the original directory so my biggest users could have readonly access to older messages without it using so much disk space.
Has anyone tried anything like this before and is dovecot likely to complain about the readonly cur directory? If the complaints are minimal and didn't cause other problems it might be ok. Any better ideas to implement something like this?
Thank You,
Natu
On 3/9/19 11:49 PM, Yassine Chaouche via dovecot wrote:
For every user@domain.tld I created a user@backup.domain.tld where he could look up deleted messages (archive).
I then made user@backup.domain.tld's cur directory a shared directory to user@domain.tld but only with read privileges. So, anytime the user wants to read his old messages, he only needs to read his .archive folder, which is user@backup.domain.tld's cur. No need to make the filesystem read-only.
Thank you. This is a little bit different from what I am looking to do, but gives me ideas on at least one possibility of how to implement it. The reason that I am interested in the squashfs read-only filesystem is it has decent performance and from tests that I did it gave me between 40 and 78 % compression rates of maildir folders. The best compression rates are for tiny messages that have no attachments in them (because of the minimum block limits of files in most file systems. Squashfs has much less overhead here). I would probably just do an annual or bi annual archive run and move all messages older than a year into archive format.
The difficulty is to make sure that every e-mail, sent or received, gets backed up properly before being deleted. Relying on cron jobs is not an option since the e-mail can be deleted and expunged before the script has a chance to get executed and do the backup. So what I did is to create a hidden sieve filter for every user's **main** mailbox (user@domain.tld) that automatically creates a copy of every incoming message to the user@backup.domain.tld mailbox. For outgoing e-mail, one can do a bcc map in postfix (or the equivalent in other SMTP software) that ensures that every sent mail is also sent to user+sent@backup.domain.tld, then you can create a filter in the user's **backup** mailbox (user@backup.domain.tld) that filters on the user+sent part of the e-mail and stores every e-mail sent to that e-mail address to the .Sent directory in the user@backup.domain.tld mailbox.
Finally, the backup.domain.tld doesn't even have to be declared in the DNS nor in /etc/hosts and can be entirely virtual to the MTA (for ex. in postfix that would only be added to virtual_mailbox_domains)
Yassine.
On 3/8/19 12:49 AM, Natu via dovecot wrote:
I have a dovecot server running under CentOS using maildir format. Due to the issue with minimum blocksize for files I would like to offer some kind of readonly archive using something like the compressed squashfs where I would move messages to be archived to a maildir folder and then convert "cur" directory into a squashfs and mount it in place of the original directory so my biggest users could have readonly access to older messages without it using so much disk space.
Has anyone tried anything like this before and is dovecot likely to complain about the readonly cur directory? If the complaints are minimal and didn't cause other problems it might be ok. Any better ideas to implement something like this?
Thank You,
Natu
participants (2)
-
Natu
-
Yassine Chaouche