Thank you for the reply.

I have done the sync manually with "doveadm sync" command. But, I have not configured the replication yet.

I am looking at the below webpage for the replication.

https://wiki.dovecot.org/Replication

I am using the dovecot version "2.2.36". I am confused with what needs to be done after reading that page.

1. They are talking about v2.3.1 and v2.2+. Which one do I need to follow? Could you please give me more details on this? Providing some sample settings will be more helpful for me, please.

2. Also, do I need to set the replication on both of my servers the same and as it is?

Please help me in understanding this?

Thanks, Kishore Potnuru

On Thu, Jul 16, 2020 at 6:20 PM Gerald Galster <list+dovecot@gcore.biz> wrote:

I have 2 test servers with the below configuration.

========================== Linux OS - Red Hat Enterprise Linux Server release 7.7 (Maipo) Dovecot version - 2.2.36 (1f10bfa63) Postfix version - 2.10.1

Trying to create High Availability.

I have added both of the above servers behind a F5 load balancer. I have got a Load Balancer FQDN "intl-dev-imaptest.testorg.com". I have enabled/opened the ports (25/110/143/993/995) on the above " intl-dev-imaptest.testorg.com".

When I send 10 emails to "intl-dev-imaptest.testorg.com", then those 10 emails are getting distributed between the above 2 backend servers (5 emails to each server). I see those 5 emails each in both the servers.

You should see 10 emails on each server if replication is working: 5 emails that were directly delivered via loadbalancer and 5 emails from the other server via replication.

From Outlook I have configured the email address using "POP and IMAP", when I gave the IMAP server as "intl-dev-imaptest.testorg.com" ,then it shows only 5 emails from server1 in outlook and after a few seconds/minutes, automatically it shows/refreshes the other 5 emails from server2. But I am not seeing all the 10 emails at the same time. why?

The loadbalancer does its job, sometimes the Outlook connection is forwarded to server A sometimes to server B. So you just see the mails on the respective server. This is very bad. Your Mailclient is probably syncing and deleting emails everytime the connection is moved to the other server. As I suggested in the other thread you should at least configure some kind of ip stickyness when using a loadbalancer, so that your mailclient reaches the same backend.

The purpose of replication is that two servers, operating independently, have the same dataset. Your servers seem to have completely distinct datasets, which indicates replication is not working. Did you configure replication?

So I tried the sync command. When I execute sync command like below from server1, it reflects the same emails in other server2 also. Then I see the same number of emails in both the servers. Is it not possible to access the both servers emails at one time with the "sync" command? Do we need to run this on all the email boxes on both servers? don't we miss/lose any emails during this sync process multiple times?

"doveadm sync -f -u kishore@test.testorg.com remote:vmail@bal3200dev002.testorg.com"

Is "replication" and "sync" are same?

Think of replication as a continous sync. This has to be done every time an email is delivered, which dovecot does automatically when replication is configured.

You don't lose any emails because the replication/sync is bidirectional, it copies from the respective other server what's missing. Of course this is not instant but usually happens within seconds.

Why are we not able to see all the emails at one time without the "sync" command?

Probably because you did not configure replication?

What is the best and easiest way to create High Availability with just 2 servers, like emails should travel to both servers equally and if one server goes down also, another server should take care of the emails/functionality. This is my requirement.

It seems you just have to configure replication.

My current real time environment: I have around 10 email domains and each domain is having 10 imap emails. In total around 100 email boxes/addresses. We receive around 50K emails in a day to those email addresses. We are using the "Maildir" format in our environment. Want to move to the High Availability option with 2 servers.

See my other mail, it may be better to use mdbox instead of maildir.

Best regards Gerald

Hi Gerald,

Thank you for the details. As per your suggestion, I have made the changes to dovecot.conf file. Still I don't see any replication is happening. Please see the dovecot.conf file.

I do not see "/etc/dovecot/conf.d/12-replication.conf" in my servers. So I had put everything in the dovecot.conf file only. Please see the complete data in it below. The below data is in server A. In other server (server B) Also I have the same configuration, except mail_replica line and it is pointing to the other server like, " mail_replica = remote:vmail@bal3200dev001.testorg.com ".

I have generated/configured the ssh keys also for vmail user in both servers. Now When i manually ssh to the server, it is not asking for a password.

=====================

disable_plaintext_auth = no

listen = * log_path = /var/log/dovecot.log #mail_location = maildir:/z1nfs/mail/virtual/%d/%n/Maildir/ mail_location = maildir:/z1devenv/mail/virtual/%d/%n/Maildir/ passdb { args = /etc/dovecot/passwd driver = passwd-file }

pop3_uidl_format = %g protocols = pop3 imap

#ssl = yes #ssl_cert = </etc/pki/dovecot/certs/dovecot.pem #ssl_key = </etc/pki/dovecot/private/dovecot.pem

userdb { # args = uid=vmail gid=vmail home=/z1nfs/mail/virtual/%d/%n args = uid=vmail gid=vmail home=/z1devenv/mail/virtual/%d/%n driver = static }

mail_debug = yes verbose_ssl = no

# Enable the replication plugin globally mail_plugins = $mail_plugins notify replication

service replicator { process_min_avail = 1 }

dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u plugin { mail_replica = remote:vmail@bal3200dev002.testorg.com }

service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } }

service replicator { unix_listener replicator-doveadm { mode = 0600 user = vmail } }

replication_max_conns = 10

=====================

Am i missing anything here? Please guide me.

Thanks, Kishore Potnuru

On Fri, Jul 17, 2020 at 12:36 AM Gerald Galster <list+dovecot@gcore.biz> wrote:

I have done the sync manually with "doveadm sync" command. But, I have not configured the replication yet.

If you don't tell dovecot where to replicate, nothing gets replicated.

I am looking at the below webpage for the replication.

https://wiki.dovecot.org/Replication

I am using the dovecot version "2.2.36". I am confused with what needs to be done after reading that page.

1. They are talking about v2.3.1 and v2.2+. Which one do I need to follow? Could you please give me more details on this? Providing some sample settings will be more helpful for me, please.

I don't understand your confusion. You are using 2.2.36, which is v2.2+ (meaning a version greater than 2.2). The documentation states you need at least 2.3.1 if you want to use the noreplicate feature. So you can't use that with 2.2.36, but as your goal is to replicate everything you don't need "noreplicate". Besides that I can't see any difference in configuring replication for 2.2/2.3.

If you want to replicate emails with ssh you just have to follow the first section, the sample settings are right on that page. It's basically copying everything from "mail_plugins = $mail_plugins notify replication" to "replication_max_conns = 10" into a config file like /etc/dovecot/conf.d/12-replication.conf

You only have to change the following line to match your server/ssh setup: mail_replica = remote:vmail@anotherhost.example.com

Then generate and configure ssh keys for user vmail (passwordless authentication) on both servers.

2. Also, do I need to set the replication on both of my servers the same and as it is?

On server A) you should configure mail_replica = remote:vmail@server_B and on server B) you should configure mail_replica = remote:vmail@server_A

If you skip B) and new mail arrives on B) it is not immediately synced to A) In that case you would have to wait until a mail gets synced from A) (you remember sync is bidirectional)

Best regards Gerald

On Fri, 17 Jul 2020, Gerald Galster wrote:

You could try https://serverfault.com/questions/939418/how-do-i-configure-doveadm-a-with-p...

passdb { args = scheme=sha512-crypt /etc/mail/passwd driver = passwd-file }

userdb { default_fields = uid=vmail gid=vmail home=/var/vmail/%d/%n args = /etc/mail/passwd driver = passwd-file }

I've never tested this as I have my users in a mysql database.

Just wanted to quickly butt in to confirm that the above does work (my config is very similar) and allows for the replication to work just fine.

For reference, I have:

userdb { driver = passwd-file args = username_format=%Lu /etc/dovecot/virtual_passwd default_fields = uid=vmail gid=vmail home=/var/mail/%d/%n }

passdb { driver = passwd-file args = scheme=SHA512-CRYPT username_format=%Lu /etc/dovecot/virtual_passwd }

Cheers, Bernardo