August 2020 - dovecot

Migration issue
by Kishore Potnuru 05 Aug '20

05 Aug '20

Hi All, I have tried the below command to migrate the data from the old server(dovecot version: 2.0.9) to new servers(dovecot version: 2.2.36 (1f10bfa63)). But it's throwing an error. old server name: production1 new server1 name: production2 (for HA/Replication) new server2 name: production3 (for HA/Replication) I have configured replication between production2 and production3, which is working fine. The issue is, the below command I am trying is from production2 server and trying to take the data from production1 server. ------------- [root@production2 log]# doveadm backup -a All -R -u kishore(a)test.testorg.com imapc: dsync(kishore(a)test.testorg.com): Info: imapc(production1.testorg.com:143): Connected to xxx.xxx.xxx.xxx:143 (local yyy.yyy.yyy.yyy:45992) dsync(kishore(a)test.testorg.com): Warning: imapc(production1.testorg.com:143): connect(xxx.xxx.xxx.xxx, 143) timed out after 30 seconds - reconnecting (delay 0 ms) dsync(kishore(a)test.testorg.com): Info: imapc(production1.testorg.com:143): Connected to xxx.xxx.xxx.xxx:143 (local yyy.yyy.yyy.yyy:46014) dsync(kishore(a)test.testorg.com): Error: imapc(production1.testorg.com:143): connect(xxx.xxx.xxx.xxx, 143) timed out after 30 seconds - disconnecting dsync(kishore(a)test.testorg.com): Error: User initialization failed: imapc: Login to production1.testorg.com failed: Disconnected from server [root@production2 log]# -------------- I looked into the production1 (old server) logs. I see the below error. =============== Jul 28 11:14:23 auth: Fatal: Master passdb can't have pass=yes if there are no passdbs Jul 28 11:14:23 master: Error: service(auth): command startup failed, throttling =============== after the above error, I have commented "pass=yes" in production1 (old server) server, then I see the below error. ================ Jul 28 11:17:10 auth: Fatal: No passdbs specified in configuration file. PLAIN mechanism needs one Jul 28 11:17:10 master: Error: service(auth): command startup failed, throttling =============== Please suggest me on fixing the issue. Adding the dovecot.conf configuration from old and new servers: ========================================= My old server dovecot.conf (production1): ========================================= disable_plaintext_auth = no listen = * log_path = /var/log/dovecot.log mail_location = maildir:/z1dev/mail/virtual/%d/%n/Maildir/ passdb { args = /etc/dovecot/passwd driver = passwd-file master = yes pass = yes } pop3_uidl_format = %g protocols = pop3 imap ssl = yes ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = uid=vmail gid=vmail home=/z1dev/mail/virtual/%d/%n driver = static } mail_debug = no verbose_ssl = no ========================================= My New server dovecot.conf(production2): ========================================= disable_plaintext_auth = no listen = * log_path = /var/log/dovecot.log mail_location = maildir:/z1devenv/mail/virtual/%d/%n/Maildir/ passdb { args = /etc/dovecot/passwd driver = passwd-file } pop3_uidl_format = %g protocols = pop3 imap ssl = yes ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = uid=vmail gid=vmail home=/z1devenv/mail/virtual/%d/%n driver = static } mail_debug = yes verbose_ssl = no # Enable the replication plugin globally mail_plugins = $mail_plugins notify replication service replicator { process_min_avail = 1 } dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u plugin { mail_replica = remote:vmail@production3.testorg.com } service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service replicator { unix_listener replicator-doveadm { mode = 0600 user = vmail } } replication_max_conns = 10 imapc_host = production1.testorg.com imapc_password = vmail imapc_master_user = vmail imapc_user = %u =================================== Please let me know if you need any other details. Thanks, Kishore Potnuru

3 8

How to access mailbox metadata in Lua push driver
by Ralf Becker 04 Aug '20

04 Aug '20

We're currently using the OX push driver, which is straight forward (simple web hook) and allows to store (multiple) push tokens of our webmailer direct in mailbox metadata. Only drawback is that it only supports new arriving mails in the INBOX, even mails moved via Sieve to other folders are NOT reported. Therefore we updated now to Dovecot 2.3(.10.1) to also get mails moved by user or Sieve scripts, deleted mails or flag changes. As far as I read the example Lua scripts and (a little) the Dovecot C code, the nice indirection of using mailbox metadata to a) enable push and b) store push tokens (modify reported user attribute with them) does NOT exist in the Lua driver by default. So my questions is: how can I access mailbox metadata from within the Lua script, to make eg. the example script behave like the OX driver with user_from_metadata set? I'm happy to update the Lua examples, if that's any help ... Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0

2 7

Question about migration 2.2 - 2.3 with replication
by Ralf Becker 03 Aug '20

03 Aug '20

Do both replication nodes need to be updated at the same time? Or can a 2.2(.36.4) node replicate with a 2.3(.10.1)? Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0

1 1

Trying to use solr
by Francis Augusto Medeiros-Logeay 02 Aug '20

02 Aug '20

Hi, I am finally trying to use an fts engine with dovecot. My version is 2.3.7.2 under Ubuntu 20.04. I installed solr 7.7.3 and then 8.6.0 to see if this was a version-related error. I copied the schema from 7.7.0 as many people said this was fine. I get the following error when trying to reindex a user's mailbox: doveadm(francis(a)francisaugusto.com): Error: fts_solr: Indexing failed: 400 Bad Request doveadm(francis(a)francisaugusto.com): Error: Mailbox INBOX: Transaction commit failed: FTS transaction commit failed: backend deinit doveadm(francis(a)francisaugusto.com): Debug: auth-master: conn unix:/var/run/dovecot/auth-userdb: Disconnected: Connection closed (fd=10) On Solr I get this error: org.apache.solr.common.SolrException: Exception writing document id 210/9fd7941e8297d25d91600000c3fdd3da/francis(a)francisaugusto.com to the index; possible analysis error: cannot change field "box" from index options=DOCS_AND_FREQS_AND_POSITIONS to inconsistent index options=DOCS at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:246) Parallel to this, I got some log messages on Solr before attempting to reindex the user (sorry for the garbadged text: TIME (LOCAL) LEVEL CORE LOGGER MESSAGE 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieLongField]. Please consult documentation how to replace it accordingly. 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.SynonymFilterFactory]. Please consult documentation how to replace it accordingly. 7/21/2020, 6:38:47 PM WARN false x:dovecot SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.WordDelimiterFilterFactory]. Please consult documentation how to replace it accordingly. 7/22/2020, 6:43:46 PM ERROR false x:dovecot RequestHandlerBase java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:46 PM ERROR false x:dovecot HttpSolrCall null:java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:49 PM ERROR false x:dovecot RequestHandlerBase java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:49 PM ERROR false x:dovecot HttpSolrCall null:java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:56 PM ERROR false x:dovecot RequestHandlerBase java.lang.IllegalStateException: Type mismatch: uid was indexed as SORTED_NUMERIC 7/22/2020, 6:43:56 PM ERROR false x:dovecot HttpSolrCall null:java.lang.IllegalSta Could anyone point me the right direction to make this work? Best,

3 5

Warning: Expunged message reappeared, giving a new UID with FTS enabled
by symphorien_dovecot＠xlumurb.eu 01 Aug '20

01 Aug '20

Hello, In some conditions with FTS enabled I get this warning: Warning: Expunged message reappeared, giving a new UID Here is how I can reproduce it: I have a test setup with a (postfix) virtual alias multi-alias(a)example.com which delivers emails to user1@ and user2@ If * I send an email to multi-alias@ * expunge user1@ mailbox with doveadm expunge -u user1(a)example.com mailbox Inbox ALL then the warning fires. If * FTS plugin is disabled (I tried with squat and xapian), or * multi-alias@ redirects to only user1@ * I send the email directly to user1@ then no warning. Can I just ignore the warning ? dovecot -n: # 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.10 (bf8ef1c2) # OS: Linux 5.4.51 x86_64 ext4 # Hostname: server auth_debug = yes auth_mechanisms = plain login base_dir = /run/dovecot2 default_internal_group = dovecot2 default_internal_user = dovecot2 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lmtp_save_to_detail_mailbox = yes mail_access_groups = virtualMail mail_debug = yes mail_gid = virtualMail mail_location = maildir:/var/vmail/%d/%n:INDEX=/var/mailindex mail_plugins = " fts fts_squat quota" mail_uid = virtualMail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded- character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = prefix = separator = . } passdb { args = /run/dovecot2/passwd driver = passwd-file } plugin { fts = squat fts_autoindex = yes fts_autoindex_exclude = Junk fts_enforced = yes fts_squat = partial=4 full=10 imapsieve_mailbox1_before = file:/var/lib/dovecot/imap_sieve/report- spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/var/lib/dovecot/imap_sieve/report- ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * plugin = fts fts_squat quota = maildir:User quota quota_grace = 10%% quota_rule = *:storage=100G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = file:/var/sieve/%u/scripts;active=/var/sieve/%u/active.sieve sieve_after = /var/lib/dovecot/sieve/after sieve_default = file:/var/sieve/%u/default.sieve sieve_default_name = default sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /nix/store/7cdslbbivi1b3yagvw18k8m4nfs2n4ym- pipe_bin/pipe/bin sieve_plugins = sieve_imapsieve sieve_extprograms } pop3_uidl_format = %08Xv%08Xu protocols = sieve imap lmtp sendmail_path = /run/wrappers/bin/sendmail service auth { unix_listener auth { group = postfix mode = 0660 user = postfix } user = root } service lmtp { unix_listener dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-status { client_limit = 1 executable = /nix/store/5b9lp6w8jzca86djawnk6aqc3n4p2mh0-dovecot- 2.3.10.1/libexec/dovecot/quota-status -p postfix inet_listener { port = 12340 } } ssl = required ssl_cert = </var/certs/cert-mail.example.com.pem ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = /run/dovecot2/passwd driver = passwd-file } verbose_ssl = yes protocol imap { mail_max_userip_connections = 100 mail_plugins = " fts fts_squat quota imap_quota imap_sieve" namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = no special_use = \Trash } prefix = } } protocol pop3 { mail_max_userip_connections = 100 } protocol lmtp { mail_plugins = " fts fts_squat quota sieve" }

1 0

Migrating from qmail server, invalid option "o" in dsync
by Justin Coleman 01 Aug '20

01 Aug '20

Hi folks, I have an old server running qmail with several years of mails in maildir format I want to migrate to a new server, without requiring fresh client download of everything. Old server is debian 6(?). Local IP is 192.168.0.1. New server is Debian 10, local IP is 192.168.0.25. I have dovecot 2.3.4.1 (latest from apt stable) installed and apparently working - if I manually copy the home directory for my account, I can connect (using thunderbird) and I get all the emails appearing as new, forcing redownload. Seems like the new config is at least functional. I'm trying to follow the migration guide at https://wiki2.dovecot.org/Migration/Dsync, but I am getting an error message: root@newserver:~# doveadm backup -R -u _user_ -o pop3c_user=_user_ -o pop3c_password=_password_ backup: invalid option -- 'o' doveadm backup [-u <user>|-A] [-S <socket_path>] [-fPRU] [-l <secs>] [-r <rawlog path>] [-m <mailbox>] [-g <mailbox_guid>] [-n <namespace> | -N] [-x <exclude>] [-s <state>] [-t <start date>] -d|<dest> Have tried this a couple different ways and according to everything i can find -o should be valid. Can anyone tell me what I'm doing wrong here? Thanks! # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-9-amd64 x86_64 Debian 10.4 # Hostname: newserver.domain.com disable_plaintext_auth = no mail_location = maildir:~/Maildir mail_privileged_group = mail namespace { hidden = yes list = no location = pop3c: prefix = POP3-MIGRATION-NS/ } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX } pop3_uidl_format = %f pop3c_host = 192.168.0.1 protocols = " imap pop3" ssl = no ssl_cert = </etc/dovecot/private/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { driver = passwd } protocol doveadm { mail_plugins = " pop3_migration" }

4 6