[Dovecot] Replication -- multiple users, three or more servers?
I'm running Dovecot 2.2.1 on an Ubuntu 12.04.2 server, with half a dozen accounts for various family members. I want to set up replication involving at least three Ubuntu servers, with different users replicated on different sets of servers.
For example, I might have mail for "user1" replicated on "server1", "server2", and "server3"...
while mail for "user2" would be on "server1" and "server2"...
and mail for "user3" would be on "server1" and "server3".
I've read the wiki page (http://wiki2.dovecot.org/Replication), but I'm still confused. I'd love to see an example that clearly shows how to set up specific individual mail users to be replicated on a different set of servers for each user, like what I described above.
Rich Wales richw@richw.org
On Mon, 2013-04-22 at 11:23 -0700, Rich Wales wrote:
I'm running Dovecot 2.2.1 on an Ubuntu 12.04.2 server, with half a dozen accounts for various family members. I want to set up replication involving at least three Ubuntu servers, with different users replicated on different sets of servers.
For example, I might have mail for "user1" replicated on "server1", "server2", and "server3"...
while mail for "user2" would be on "server1" and "server2"...
and mail for "user3" would be on "server1" and "server3".
I've read the wiki page (http://wiki2.dovecot.org/Replication), but I'm still confused. I'd love to see an example that clearly shows how to set up specific individual mail users to be replicated on a different set of servers for each user, like what I described above.
Everything is the same as in that wiki page, except you need to have userdb field override the mail_replica setting. Or I guess you wouldn't want to have a default mail_replica at all, so users won't accidentally get replicated to wrong place. See http://wiki2.dovecot.org/UserDatabase/ExtraFields
For example with SQL something like:
user_query = SELECT home, uid, gid,
concat('tcp:', replicahost) as mail_replica
FROM users WHERE userid = '%u'
Replying to Timo:
Everything is the same as in that wiki page, except you need to have userdb field override the mail_replica setting. Or I guess you wouldn't want to have a default mail_replica at all, so users won't accidentally get replicated to wrong place. See http://wiki2.dovecot.org/UserDatabase/ExtraFields
OK, thanks.
Is there a debugging option I can specify here to cause Dovecot to generate more verbose logging output, so I can see exactly what is happening (and what is not working) when I try to run replication?
In the "Replication" wiki page, you show one example using the string "remote:" at the start of the "mail_replica" value, and another example starting with "remoteprefix:". What is the difference between these? Or is there a typo here? I tried searching the wiki but couldn't find anything explaining this.
The example with a dsync wrapper script seems to be describing a situation where the first line of text sent to the remote host consists of the user name (which is read by the wrapper script and passed as a command-line argument to dsync-server). Is this what "remoteprefix:" does, in contrast to "remote:"?
In the dsync wrapper script example, is "vmail" in the mail_replica value an example of a user ID to be used on the remote host? What user ID is used on the local host? I think one reason why my tests so far haven't been working may be that I'm not sure which user ID is being used on each end, so my SSH keys aren't being used properly and the connection is failing.
Finally, the "Replication" wiki page mentions the "authorized_keys2" file, which (AFAIK) is deprecated in the current SSH -- all authorized keys should be in a single "authorized_keys" file on the target host, right?
Rich Wales richw@richw.org
I've ALMOST got this to work, but I'm still running into a problem.
First, here's what I'm doing so far. If my understanding of what's going on (described below) is off, I trust someone will correct me. Also, when/if I get these details straight, I'm willing to help update the "Replication" wiki page to make it clearer and more accurate.
I followed Obi-wan's admonition ("Use the source, Luke") and figured out that when the "mail_replica" value starts with "remoteprefix:", the behaviour is identical to "remote:" EXCEPT that the data stream sent to the remote server starts with a line containing the mail account name followed by a newline character.
Also, it appears that the "dsync_remote_cmd" is run by default as "root" on the local server. I'm assuming for the moment that %{user} and %{host} in "dsync_remote_cmd" are being replaced by the user and host information in the "mail_replica" value.
If the public key value included in "authorized_keys" for the target account on the remote server contains a command= parameter, the "ssh" documentation says that this command will override any command given on the "ssh" command line on the local server. Thus, it should not really be necessary to include a remote command on the tail end of the "ssh" specified in "dsync_remote_cmd".
Note, BTW, that the "authorized_keys2" file (mentioned in the current documentation) is deprecated now in SSH; all public keys on the remote server should be in "authorized_keys" now.
I also discovered that in order to get "ssh" to work properly in a non-interactive scenario -- without any prompting for typed input (which would break things) -- I needed to run the "ssh" command once by hand, to cache the remote server's host key information in the local "known_hosts" file.
So, with all the above in mind, I added the following to the Dovecot configuration on the local host. (My local host is named "richatwork", and my remote host is named "pigeon".)
mail_plugins = $mail_plugins notify replication service replicator { process_min_avail = 1 } dsync_remote_cmd = /usr/bin/ssh -i /root/.ssh/dsync_dsa %{user}@%{host} plugin { mail_replica = remoteprefix:root@pigeon.richw.org }
I added the public key value (from /root/.ssh/dsync_dsa.pub) to the /root/.ssh/authorized_keys file on the remote host (pigeon), with the command= parameter set to the name of a script on the remote host with the following content:
#! /bin/sh read username exec /usr/bin/doveadm dsync-server -u $username
With the above setup, replication ALMOST seems to work, EXCEPT that it's failing with the following error in the local host's mail.log:
richatwork dovecot: doveadm: Error: dsync-remote(richatwork): Error: dsync(local): Remote dsync doesn't use compatible protocol
Both servers are running the identical version of Dovecot (2.2.1), so I'm confused as to why I would be getting a protocol mismatch. Is there some other configuration option I need to check?
Rich Wales richw@richw.org
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
"dovecot -n" output on the local server (richatwork):
# 2.2.1 (e819374de157): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-40-generic-pae i686 Ubuntu 12.04.2 LTS auth_username_format = %Ln auth_verbose = yes dsync_remote_cmd = /usr/bin/ssh -i /root/.ssh/dsync_dsa %{user}@%{host} login_greeting = richatwork.richw.org (%{lip}) Dovecot ready; hello, %{rip} mail_location = maildir:~/Maildir mail_plugins = " notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = scheme=crypt username_format=%n /etc/dovecot/private/userdb driver = passwd-file } plugin { mail_replica = remoteprefix:root@pigeon.richw.org sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_actions = 1000 } protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = user = } } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 5 } service postlogin { executable = script-login -d rawlog -t } service replicator { process_min_avail = 1 } ssl_cert = </etc/apache2/ssl/richatwork.pem ssl_key = </etc/apache2/ssl/richatwork.pem userdb { args = username_format=%n /etc/dovecot/private/userdb driver = passwd-file } protocol lmtp { mail_plugins = " notify replication sieve" } protocol lda { mail_plugins = " notify replication sieve" } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags mail_plugins = " notify replication mail_log notify" }
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
"dovecot -n" output on the remote server (pigeon):
# 2.2.1 (e819374de157): /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-27-generic x86_64 Ubuntu 12.04.2 LTS auth_username_format = %Ln auth_verbose = yes login_greeting = pigeon.richw.org (%{lip}) Dovecot ready; hello, %{rip} mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = scheme=crypt username_format=%n /etc/dovecot/private/userdb driver = passwd-file } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_actions = 1000 } protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = user = } } service imap { executable = imap postlogin } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 5 } service postlogin { executable = script-login -d rawlog -t } ssl_cert = </etc/apache2/ssl/pigeon.pem ssl_key = </etc/apache2/ssl/pigeon.pem userdb { args = username_format=%n /etc/dovecot/private/userdb driver = passwd-file } protocol lmtp { mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags mail_plugins = " mail_log notify" }
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
participants (2)
-
Rich Wales
-
Timo Sirainen