Problem with replication ("Fatal: -N parameter requires syncing with remote host")

[Vanja]

I have been fighting, for 2 days, with a Dovecot setup and I now
officially need help :)

Basically, I am trying to setup replication between 2 identical Dovecot
servers, on different physical servers (different hostnames/etc, of
course), and although I've managed to solve many problems I can't
overcome the final hurdle.

When a mail comes in to one of the servers, replication over TCP should
kick in and mail should be replicated, but instead I get this in the
mail log:

-------------------------------------------------------------------------------------------------
dovecot2 dovecot: doveadm(user1 at test1.int): Fatal: -N parameter requires
syncing with remote host
-------------------------------------------------------------------------------------------------

Mail is delivered to dovecot2 server, Postfix delivers it via LMTP, and
everything seems to work ok. However, I can't figure out the cause for
the error that I am seeing. I have looked into source code and found
where this error is triggered (line 834 in
doveadm/dsync/doveadm-dsync.c, inside Dovecot 2.2.9 source code), but I
am afraid that I don't understand the context (ctx->run_type ==
DSYNC_RUN_TYPE_LOCAL) and it would take me a lot of time to debug the
whole thing from the scratch. I've never looked into Dovecot internals
before, so it's all a mystery to me at this point.

I've tried stracing the process but the flow does not make sense to me,
I can't understand what exactly happens before error is displayed. Also,
there is no traffic to port 12345 on other node, so execution fails
before syncing even starts (and error message is not related with remote
node config/setup/etc).

Triggering replication manually, from command line, works as expected:

# doveadm sync -A tcp:192.168.1.100

(at this point user1's mail folder is properly updated on remote server)

I've tried searching around, but it appears that noone else has this
problem(or isn't complaining about it :).

I am not sure if I am doing something wrong (apart from replication). I
will need to support multiple virtual domains and I am basically using
passwd-file for authentication, so I am using a single passwd file for
all virtual domain users which allows user lookup (doveadm user '*') to
work, which is required by replication.

This is configuration file from dovecot2 server (which I use for testing
and delivering mails):

--- config start ---

# 2.2.9: /etc/dovecot/dovecot.conf
# OS: Linux 3.16.0-30-generic x86_64 Ubuntu 14.04.2 LTS ext4
debug_log_path = /var/log/dovecot_debug.log
doveadm_password = 67890
doveadm_port = 12345
mail_debug = yes
mail_location = maildir:/vmail/mail/%d/%n/Maildir
mail_plugins = " notify replication"
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = username_format=%u /vmail/auth/passwd
  driver = passwd-file
}
plugin {
  mail_replica = 192.168.1.100
}
protocols = imap lmtp
service aggregator {
  fifo_listener replication-notify-fifo {
    user = vmail
  }
  unix_listener replication-notify {
    user = vmail
  }
}
service auth-worker {
  unix_listener auth-worker {
    user = dovecot-auth
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  user = dovecot-auth
}
service doveadm {
  inet_listener {
    port = 12345
  }
}
service imap-login {
  inet_listener imap {
    port = 0
  }
}
service lmtp {
  inet_listener lmtp {
    address = 127.0.0.1
    port = 10024
  }
  process_min_avail = 3
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0660
    user = postfix
  }
  user = vmail
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0666
  }
}
ssl_cert = </etc/dovecot/dovecot.pem
ssl_key = </etc/dovecot/private/dovecot.pem
userdb {
  args = username_format=%u /vmail/auth/passwd
  default_fields = uid=vmail gid=vmail home=/vmail/mail/%d/%n
  driver = passwd-file
}
protocol lmtp {
  postmaster_address = root at localhost
}


--- config end ---

Configuration file on other node is exactly the same, except different
IP address in mail_replica statement.

Any help is much appreciated.

Thank you.