On 11/9/2011 11:35 PM, Alexander Chekalin wrote:
Hello, Stan,
in fact the only thing I miss even with my current scheme is permanent ID assigned to the message so I can easily find it despite the IMAP mailbox it is now (so if someone moved the message from one mailbox/folder to another, the ID allows to retrieve it fast anyway).
You see, what I need is not only find message from|to someone on specified date, I also sometime need to restore that message back to user's original box. As far our mailserver and backup-mailserver are different machines, it is a bit tricky to copy messages between it fast enough. Say, if I need to find and restore all mails from user@domain.com within 2009 year, and search yields in some 1000's of messages, then use IMAP to copy it over to another server takes some time - and if you consider both search time and restore/copy time the whole process may take "ages".
Apparently I didn't fully understand all of your requirements.
Moving the archived mail to mbox/mdbox and/or getting a good indexing search engine installed will cut the search time down tremendously. Whether that would make up for the time consumed with an IMAP copy of many emails I don't know. If your servers aren't old and slow, and are not already overloaded, I would think the IMAP message copying over GbE would be pretty quick, even for the 1000 messages scenario.
There may be some Dovecot tweaks that might make this copy process faster. Timo would need to chime in on that. Do you perform the IMAP transfers with a GUI IMAP client on your management PC? Or are you using imapsync or some other util directly on the servers?
If the former you may be able to tweak your IMAP client to speed up the transfers as well. Try using IMAP and not IMAPS for the transfers. What is the network infrastructure between the servers and your management workstation? Is it all GbE with jumbo frames enabled?
With maildir I can rsync/scp needed files to another host and that's fast way - that's why I stick with maildir.
There is definitely some flexibility here.
FTS in my case can help (I can search for user@domain.com, for example), but it also return messages that contains such a string in message body (and that takes index space, too), so I'll need to filter it later, but surely it'll be faster than checking every message in the archive.
Sure. So you're concerned with your poor performance, but also with disk space. Unfortunately there's no free lunch to be had. You'll have to make sacrifices somewhere. You could go with mdbox and use compression, trading that saved space for search index files space.
-- Stan