On 03/17/2012 10:28 PM, Sven Hartge wrote:
Kaya Samankayasaman@gmail.com wrote:
<snip> > Question: do you need public or shared folders? I don't need anything apart from an IMAP storage solution. I don't intend to tie in Dovecot with an MTA either as I will simply be using this for storage. Long story but we don't have any control over our mail server which is handled by the parent company abroad and is on MS Exchange. To use an IMAP storage solution is the only way to get rid of pesky MS .pst files which have been causing everyone grief and havoc. So, how do you plan to get the mails into this storage? offlineimap? imapsync? mbsync? fetchmail?
Since everything is blocked at the Exchange end, users will have to manually transfer for now through MS Outlook.
Currently that's what they're doing to their PST's....
Hmm..... so FreeBSD coupled together with a ZFS repo for mail should take care of 'Snapshot' issues. Yes. Or using LVM on Linux. Yeah.... true but I specified ZFS as I'm a fan and also am quite comfortable with Solaris/*BSD too...... If you know ZFS and are familiar with it, then, by all means, go for it.
:-)
Depending on the amount of mail a user collects inside a folder, Maildir is not the best storage format. You may want to check into mdbox, if your users are kind of "mail hoarders" (like some of my users are).
In my opinion, Maildir has outlived its usefullnes. It was fine when users had 1,000 mails in some 10 folders, but today, users collect over 100,000 mails a year and Maildir is causing serious I/O trouble and the need to heavily fine tune your storage and filesystems to cope with those demands.
I cannot thank Timo enough for inventing mdbox, as this format breaks this viciuos cycle and, as someone else said "it ends the battle at the I/O front forever". So mdbox is a 'new' mailbox standard? ie. one can replace Maildir format with this and use mdbox instead. {Note to self: time to browse!} mdbox is a format invented by Timo for dovecot. But dovecot can use nearly all common mailbox formats (except MH, but no one uses that one today).
Ok so if you claim that mdbox is the 'best' mailbox storage solution then I'll look at implementing this.
Since where I'm implementing this is mainly an MS based environment they are concerned about /flat/ files.... which MS seems to typically do (although never used MS before so I wouldn't know). So there is some concern over performance, efficiency and manageability. Ye olde MBOX flat file format, as used in UW-imapd for ages, is a nightmare, no doubt about this.
But even with this crappy format, dovecot is able to deliver astounding performance by use of separete index files which allow it to access the storage in an efficient manner.
mbox has big problems with concurrent writes, the bigger the mbox is, the more problems you get. This is mainly caused by the meta-data of a message (meaning flags, status, etc.) which is stored inside the mbox file itself. Flagging a message as read or replied causes the whole mbox file to be rewritten.
mdbox solves this problem by a) storing all meta-data in the index and b) by only ever appending to a mdbox storage file, c) never truncating an existing mdbox storage file and d) using more than one mdbox storage file. Max size and TTL are configurable.
But this also means deleted mails are still inside a mdbox storage file and need to be finally removed by copying all remaining files into a new file. This process has to be manually run during low traffic hours, for example using a cronjob.
You can say, mdbox is like mbox on steroids. ;)
Flat files are not evil or bad or slow per se, but you have to use them the right way.
Thanks a lot for that info. I will research more into this but I maybe overridden at some point :-(
Need to make a strong case!
However, if like you say mdbox is the way to go then I will put a strong case together! You may want to start with something familiar and convert later, which is no problem with dovecot.
Maildir is what I'm familiar with currently and mbox format - though only use mbox as an unfortunate side product of /system mail/ accounts.
Works well with Alpine client though!
Grüße, Sven
Regards,
Kaya