Duplicate messages if message is moved when using dsync

[Thom Pol]

Hi,

Thanks for your reply!

The dsync is triggered on IMAP/POP3/LDA/LMTP activity. See my config 
dump from the first email. I'm wondering if my config is not OK so it 
causes dsync to sync twice?

> you need / preferably use a database for your user auth
I already do this.

> run the command doveadm user '*' - it needs to return a list of users
> that will trigger replication (after doing config above)
This part works fine.

Thanks,

Thom Pol



On 18/05/2022 16:51, Paul Kudla (SCOM.CA Internet Services Inc.) wrote:
> ok - question
>
> when you say dsync are you running that manually or as a cronjob ??
>
> or using replication via dsync
>
> there is a difference
>
> replication is designed to handle mailbox changes by quing the changes
> on the fly and then update in the background.
>
> replication tracks all changes in a mailbox and then sends them one at a
> time to the target server.
>
> i run replication with tcp (not tcps or dsync)
>
> I find (other then the 300+folder issues) that it runs pretty good in
> real time.
>
> if you are running dsync manually then yes timing on its own will be a
> hit and miss and it will never sync properly if something appears during
> the dsync process.
>
> if you have the sive running as discussed that is the first step
>
> you should setup replication so things are handled in real time.
>
> pls advise if you have already done replication or just manually dsyncing.
>
> Below is basic replication setup but you need to make sure various
> pre-requirements are in place a head of time.
>
> 1. you need / preferably use a database for your user auth
>
>      you need to setup an iterate sql statement that returns the user
>      list of users to be replicated
>
> Example (i use pgsql) :
>
> # cat dovecot-pgsql.conf
> driver = pgsql
> connect = host=localhost port=5433 dbname=scom_billing user=pgsql password=
> default_pass_scheme = PLAIN
>
> password_query = SELECT username as user, password FROM email_users
> WHERE username = '%u' and password <> 'alias' and status = True and
> destination = '%u'
>
> user_query = SELECT home, uid, gid FROM email_users WHERE username =
> '%u' and password <> 'alias' and status = True and destination = '%u'
>
> iterate_query = SELECT "username" as user, domain FROM email_users WHERE
> status = True and alias_flag = False
>
> #iterate_query  above is what returns the user lists
>
> 2. run the command doveadm user '*' - it needs to return a list of users
> that will trigger replication (after doing config above)
>
> 3. in general here is my relative parts for tcp replication
>
> In dovecot.conf
>
> ---------------------------------------------------------------------------
>
> mail_plugins = " virtual notify replication fts fts_lucene "
>
> replication_dsync_parameters = -d -N -l 300 -U
>
> dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u
>
> plugin {
>
> .....
>     mail_replica = tcp:10.221.0.19:12345
>
> #(note this is the ip address of the other server)
> #it is the only config change ie server one above
> #server2 would be the other server for communicating.
>
>
>
> .....
>
> }
>
> protocol imap {
>     mail_max_userip_connections = 50
>     mail_plugins = $mail_plugins notify replication
> }
>
> protocol pop3 {
>     mail_max_userip_connections = 50
>     mail_plugins = $mail_plugins notify replication
> }
>
> protocol imaps {
>     mail_max_userip_connections = 25
>     mail_plugins = $mail_plugins notify replication
> }
>
> protocol pop3s {
>     mail_max_userip_connections = 25
>     mail_plugins = $mail_plugins notify replication
> }
>
> replication_max_conns = 100 (optional)
>
> replication_full_sync_interval = 1d
>
> service replicator {
>     client_limit = 0
>     drop_priv_before_exec = no
>     idle_kill = 4294967295s
>     process_limit = 1
>     process_min_avail = 0
>     service_count = 0
>     vsz_limit = 8g
>           unix_listener replicator-doveadm {
>       mode = 0600
>       user = vmail
>     }
>     vsz_limit = 8192M
> }
>
> service aggregator {
>     process_limit = 1000
>     #vsz_limit = 1g
>     fifo_listener replication-notify-fifo {
>       user = vmail
>       group = vmail
>       mode = 0666
>     }
>
> }
>
>
>
> ----------------------------------------------------------------------------
>
> that should do it?
>
>
>
>
>
>
>
>
>
>
>
>
> Happy Wednesday !!!
> Thanks - paul
>
> Paul Kudla
>
>
> Scom.ca Internet Services<http://www.scom.ca>
> 004-1009 Byron Street South
> Whitby, Ontario - Canada
> L1N 4S3
>
> Toronto 416.642.7266
> Main?1.866.411.7266
> Fax?1.888.892.7266
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20220525/2f601ffa/attachment.htm>