[Dovecot] merging the best of mdbox and sdbox

[Timo Sirainen]

In sdbox case there is always only one message per file. It's possible to use filesystem's hard linking feature to copy the file (=the message), because the message contents don't change.

In mdbox case there are multiple messages per file. It would again be possible to copy a message by hard linking the entire file, but it's pretty wasteful to link a 2 MB file to another mailbox for a 10 kB mail, especially if after a while 90% of that 2 MB might be deleted on the original mailbox already.

It would be also possible to create some kind of a hybrid solution here, where large messages were hard linked and small messages were copied. But even if this worked perfectly, it would be yet another dbox variant. For what reason would anyone choose that over sdbox or mdbox? Seems like a lot of work with little benefit.

On 6.1.2011, at 0.49, Joan Moreau wrote:

> 
> 
> Sorry to insist, but the process you describe (copy up to 1GB (why ?
> I still do not understand) ) is exactly the "sdbox" approach (one file
> per message in each IMAP subfolder) 
> 
> Let's please spend some more
> analysis on this. 
> 
> On Thu, 6 Jan 2011 00:45:29 +0200, Timo Sirainen
> wrote: 
> 
>> With maildir, single-dbox or multi-dbox when you copy a
> message, just about nothing is read from disk and very little is written
> to disk. For example in Maildir case a copy is the same as doing:
>> 
>> 
> ln ~/Maildir/.source/cur/1234 ~/Maildir/.dest/cur/1234
>> 
>> This only
> accesses the file's metadata. It would be just as fast for a file of 1
> kB or 1 GB. The file contents aren't read at all, only the inode is
> copied.
>> 
>> With your idea of multi-dbox-per-mailbox it would actually
> have to read the 1 kB or 1 GB of data and write the 1 kB or 1 GB of data
> to another file. Much slower.
>> 
>> On 6.1.2011, at 0.30, Joan Moreau
> wrote:
>> 
>>> I am not sure I understand why it would be "read entirely",
> as it is the same as reading a message when accessing a folder and
> writing a file when a new message in arriving.... which is the very most
> common activity of a mail server... so not sure about any "wasteful" I/O
> Can we spend some more thoughts on this ? On Thu, 6 Jan 2011 00:17:36
> +0200, Timo Sirainen wrote: 
>>> 
>>>> On 6.1.2011, at 0.11, Joan
> Moreau
>>> wrote: 
>>> 
>>>>> Is there a way to use the mdbox format (i.e.
> several
>>> message per files) for EACH IMAP folders (sdbox approach) ?
> ctual
>>> ritten entirely, rather than a) with sdbox it can use a hard
> link() call, b) with mdbox it can update 3 indexes. Copying is a rather
> common operation since clients often move messages to Trash, so the
> extra writing would be a bit wasteful.