On Wed, 2013-03-20 at 17:40 -0600, DormitionSkete@hotmail.com wrote:
On Mar 20, 2013, at 8:59 AM, Timo Sirainen wrote:
On Wed, 2013-03-20 at 08:15 -0600, DormitionSkete@hotmail.com wrote:
My experience with IMAP over the internet with a couple of servers outside our monastery (while I was in it, and we have considerably better download speeds than upload) has always been that IMAP has always been incredibly slow. So, I've always just allowed users to connect to the IMAP server via webmail. It's slow, but usable.
Another idea: Get some cheap server from outside, use dsync replication to keep it synced with your internal one, and set up DNS so that users get directed to the fastest server. http://wiki2.dovecot.org/Replication
I LIKE this idea, but I have a few questions about it to see if it would be appropriate for our situation. There are a few other things to consider that I didn't mention before because they did not seem relevant earlier.
First off, I'd just like to say that we have a web server set up at a location outside of our monastery that hosts all of our websites. I'm currently in the process of building new servers to replace both it and our current email server. So, assuming this is both plausible for our situation, and within my capabilities, I should be able to work on this at my leisure, and get the initial sync of our emails done while on the same LAN.
So, the additional info and questions are the following:
1.) Our download speeds are decent enough, but in addition to having poor upload speeds, we also have very strict limits on how much we are able to download. And we use almost every bit of it every day. We cannot get more, either. We have unlimited downloads for four hours at night, however.
If a delayed sync isn't a problem, you could do it only once at nights. You wouldn't need to use the replicator service at all, just run "doveadm sync -f -A -d" in a cronjob.
2.) We have very large message archives. We basically have 95% of the emails we've received for the past 16 years. So, the sync *must* only update items that have been changed. Is this how it it would work?
dsync can do full sync (= all messages' metadata is sent + new messages' contents), "changed sync" (= same as full sync, but only for changed folders) or incremental sync (= only new messages' metadata + contents are sent). The incremental sync is what replicator service does while it's running, but it's still currently doing a full sync at startup.
A nightly cronjob could do incremental syncing also, but it would have to run dsync separately for each user and store the sync state to some file.
The "changed sync" works well enough usually, but it has a problem if both replicas have had exactly the same amount of changes it doesn't realize that there may be differences between them and skip it.
3.) We are currently using uw-imap with mbox. If we switch to Dovecot, using Maildir format, will the sync only update the new messages and the header files for any folders that have been changed?
It works the same with all mailbox formats. Headers and bodies aren't synced separately, but metadata (= ~100 bytes/msg maybe) is.
4.) I thought I read somewhere in Dovecot's documentation last night that it has a 50 mb limit on folders. It can't write anything larger than that. Does this sound familiar? (Now I can't find it!) If so, is that for mbox? We currently have some mbox folders whose files are significantly larger than that. If we convert to Maildir format, where the individual messages are in their own files, could a folder contain messages totaling more than 50 MB using Dovecot?
Dovecot has no such limit. Postfix by default has set a file size limit for 50 MB, which effectively limits mbox sizes to 50 MB, but it can be removed with Postfix mailbox_size_limit setting.
4a. -- Oops. I just noticed this: "NOTE2: sdbox/mdbox mailboxformats are recommended for replication. Maildir still has some issues (although probably not noticeable in normal use)." Should I consider this a show-stopper for syncing like this?
With v2.2 I don't think there's much of a difference anymore.
5.) In the http://wiki2.dovecot.org/Replication page, would this be continuously synced each time a user sends, receives, deletes, or moves messages, etc.? Or would it be periodically synced?
With replicator it syncs immediately when something changes.
6.) Also, that page does not make it clear if one server is like the "master" and the other the "slave". Do I do the same changes to both servers?
Both servers are equal. Setup both servers exactly the same.
If, given the above additional information, it would not be an appropriate solution for us, this suggestion about syncing the two servers gave me another idea.
I was thinking, "Well, I wonder if I could just sync the Inboxes? We don't really need the folders synced. In the highly unlikely event a person would ever need something from one of his folders, he could always just log into the (slow) monastery server through web mail and get it that way."
If you're syncing via ssh, you can give "-m inbox" parameter to dsync_remote_cmd setting and it syncs only INBOX. But it's still unnecessarily running dsync whenever anything changes. With some hardcoding it would be easy to change that though.
(When we are on the road, we are generally working real hard, and we don't answer any more emails or do any other computer work than we absolutely have to.)
So, that led me to the idea to simply set up some message rules in procmail in our (slow) monastery server to copy any incoming messages to the server offsite in addition to delivering them locally. For the most part, that would be sufficient for us -- and considerably easier.
The only downsides to this are that when we reply to messages, they would not be marked as having been replied to, and we wouldn't have copies of our replies on our main server.
The not being marked as replied to is not a big deal. I know we could manually copy any sent messages from one server to the other when we returned to the monastery, if we really wanted to, but does anyone know of a better way to do it?
The users would then need to have two accounts I think, one for internal and one for outside mail. Otherwise whenever they switch between servers they need a full resync.