Shared folders using mbox format

[@lbutlr]

On 25 Sep 2020, at 16:27, Sven Hartge <sven at svenhartge.de> wrote:
> On 25.09.20 23:04, Alex wrote:
>> 
>>>> I'd like to create a shared folder to be used with an Outlook client
>>>> and a Thunderbird client. We're currently using mboxes.
>>>> 
>>>> I've read the documentation here about creating a shared folder:
>>>> https://wiki.dovecot.org/SharedMailboxes/Shared
>>>> 
>>>> It doesn't make reference to how to do this using mbox format. Is it
>>>> possible? Should we convert to Maildir? Perhaps there's a way to
>>>> provide both formats?
>>> 
>>> You should never use mbox for anything other than maybe read-only
>>> archival storage.
>>> 
>>> Use Maildir++ if you need to be compatible with anything accessing the
>>> mail other than Dovecot or better yet, use Dovecots own sdbox format.
>> 
>> I think the reason we're still using mbox is for historical reasons -
>> these mailboxes are decades old.
>> 
>> Should I investigate an mbox2maildir conversion? Can I be sure it will
>> do it properly?
> 
> Dovecot can do the conversion itself, there is documentation in the Wiki
> about this, for example https://wiki.dovecot.org/Migration/MailFormat
> 
> In the end, it boils down to something like this:
> 
> $ doveadm -D -v -o backup -u sven sdbox:~/sdbox

This is definitely the way to go. I used formal (part of procmail) to do this and it was very straightforward, but I did that when I was not running Dovecot.

In bash I did something like this:

for mail in **/cur/; do
formail -a "Status: RO" <"$mail" >>../mbox
done

Then I'd check if there were any messages in **/new/ and if so, run the same command without the "Status: RO".

Another easy way to do this with your own mail is to use a command-line mail program like mutt to save a mailbox in Maildir format.

> But I would not migrate to Maildir, if you can avoid it.

Maildir is 1,000,000 times better than mbox. The other choices are marginally better than maildir under certain circumstances and for certain use cases. Anything is a huge step up from mbox.

> While Maildir seems to be an obvious choice, it has some hidden
> performance impacts, namely it relies heavily on information encoded in
> the filename, which results in a big overhead in meta-operations,
> because Dovecot needs to stat() every file in a directory to gather
> those information.

This, if it's even true, is only relevant if you are dealing with huge masses of mail. My personal accounts have over 1,000,000 emails and Maildir is just fine. I've tried to find any sort of benchmarking od the various formats and other than theoretical numbers, I've not found anything. 

> (I see this in my NetApp NFS-Filer, 95% of NFS Ops are for "other" which
> cDOT classes any operation that is not "read" or "write".)

If your mail is stored over NFS that's something else. The bottleneck there is NFS and anything you can do to reduce packets is going to be a good thing.

> Also once a mail has been read or replied or marked for deletion, the
> file name changes which will trigger an additional backup of the mail.

Depends on your backup. Not all backups rely on filenames. It also depends on how much mail you feel the need to backup. In the first day, mail will change flags (and locations) as users read it, discard it, or archive it. After that, mail messages are pretty stable and rarely, if ever, change.

> This is why I advocate for the usage of "sdbox". It has a single file
> per mail, as has Maildir, but does not change the filename once the
> state of a mail changes, reducing unnecessary additional copies in your
> backup.

There are more tools to deal with maildir than sdbox, so you might have more problems if you deal with mail from the server's shell. I have no opinion on sdbox specifically, but I believe it is more involved to recover a single email and put it back into its directory, and when you do so you lose its state as sdbox sees it as a new email while Maildir will respect the flags in the filename.



-- 
"Are you pondering what I'm pondering?"
"I think so, Brain! But isn't a dreadlock hair extension awfully
	expensive?"