- dovecot - dovecot.org

sieve scripts not synching for 2.3.5.1 pre-built
by Jan-Pieter Cornet 29 Jul '19

29 Jul '19

Hi, We're synching mailboxes, changing format from maildir to mdbox, using doveadm backup/doveadm sync. When still running 2.2.36, 'doveadm backup' also synched the sieve scripts, without issues. After the upgrade to 2.3.5.1, the sieve sync stopped working. We're using the pre-built 2.3 packages from https://repo.dovecot.org/ce-2.3-latest/debian/stretch In fact, when I now remove the dovecot-2.3 packages, and use my own old 2.2.36 packages, using the exact same config, the sieve scripts are being copied again. In 2.2.36, when I migrate a mailbox, I do see Pigeonhole being initialized. Eg: userimap-dev1:~ # doveadm -D backup -u xtra30 "doveadm -D dsync-server -u xtra30+mdbox=.8d1&4&03&xtra30" (note: that strange destination user signals to the auth proxy that we want the new mdbox storage instead of the default for this user) Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so [...skipping a bunch of lines...] dsync-local(xtra30): Debug: brain M: Mailbox Spam: local=88b39d12998d9e5cbb700000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30): Debug: brain M: Mailbox Test: local=801bb932e98d9e5c3d6d0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30): Debug: brain M: Mailbox Trash: local=fc9b162b86479f5cc85a0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-remote(xtra30): Debug: brain S: Skipping unchanged mailbox e7c1961d84479f5c885a0000436edcce dsync-local(xtra30): Debug: doveadm-sieve: Iterating Sieve mailbox attributes dsync-local(xtra30): Debug: sieve: Pigeonhole version 0.4.24 () initializing dsync-local(xtra30): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. dsync-local(xtra30): Debug: sieve: file storage: Using active Sieve script path: /var/mail/.8d1/index/7/oldsieve/x/xt/xtra30/dovecot.sieve [... skipping a lot of sieve debug lines that seem fine ...] dsync-remote(xtra30): Debug: sieve: file script: Opened script `xtra30-testfilter' from `/var/mail/.8d1/index/4/03/xtra30/sieve/scripts/xtra30-testfilter.sieve' dsync-remote(xtra30): Debug: doveadm-sieve: Assigned value for key `vendor/vendor.dovecot/pvt/server/sieve/default' (last change: 2019-03-29 22:27:41) dsync-remote(xtra30): Debug: brain S: Import INBOX: Import attribute vendor/vendor.dovecot/pvt/server/sieve/default: Nonexistent locally dsync-remote(xtra30): Debug: brain S: Import INBOX: Last common UID=0. Delayed expunges= dsync-remote(xtra30): Debug: brain S: Import INBOX: Import change type=save GUID=1553894809.M254847P28859.userimap11.xs4all.net,S=1311,W=1342 UID=1 hdr_hash= result=Mail's UID is above local UIDNEXT - No more local mails found [...] Afterwards, the sieve scripts have been transferred, even including last-modified time of any scripts. However, with 2.3.5.1, it seems doveadm-sieve is never called from dsync. Output includes: userimap-dev1:~ # doveadm -D backup -u xtra30 "doveadm -D dsync-server -u xtra30+mdbox=.8d1&4&03&xtra30" Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so [... skipping some lines again ..] dsync-local(xtra30)<SCSfIc5Wo1zTKAAAsPW7qw>: Debug: brain M: Mailbox Spam: local=88b39d12998d9e5cbb700000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30)<SCSfIc5Wo1zTKAAAsPW7qw>: Debug: brain M: Mailbox Test: local=801bb932e98d9e5c3d6d0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-local(xtra30)<SCSfIc5Wo1zTKAAAsPW7qw>: Debug: brain M: Mailbox Trash: local=fc9b162b86479f5cc85a0000436edcce/0/1, remote=00000000000000000000000000000000/0/0: mailbox not selectable yet dsync-remote(xtra30)<h8hfIs5Wo1zWKAAAsPW7qw>: Debug: brain S: Skipping unchanged mailbox e7c1961d84479f5c885a0000436edcce dsync-remote(xtra30)<h8hfIs5Wo1zWKAAAsPW7qw>: Debug: brain S: Import INBOX: Last common UID=0. Delayed expunges= dsync-remote(xtra30)<h8hfIs5Wo1zWKAAAsPW7qw>: Debug: brain S: Import INBOX: Import change type=save GUID=1553894809.M254847P28859.userimap11.xs4all.net,S=1311,W=1342 UID=1 hdr_hash= result=Mail's UID is above local UIDNEXT - No more local mails found [...] In the above output from 2.3.5.1, all lines starting with "Debug: doveadm-sieve: Iterating Sieve mailbox attributes" are absent. I have no idea why. We can "work around" it by explicitly doing an rsync of the sieve storage after the migration, but that's a bit of a hack. Attached is a doveconf -n output for 2.3.5.1. -- Jan-Pieter Cornet <johnpc(a)xs4all.net> Systeembeheer XS4ALL Internet bv www.xs4all.nl

6 7

IMAP copy stopped copying flags
by Andrzej A. Filip 29 Jul '19

29 Jul '19

Is it intended behavior? It seems to be caused by upgrade to 1:2.3.2-2 on Debian/Testing. -- A. Filip

4 6

submission configuration issues
by Jean-Daniel Dupas 29 Jul '19

29 Jul '19

Hello, I'm having trouble configuring the submission proxy. I have configured the submission service as follow: submission_host = smtp.example.com submission_relay_host = localhost submission_relay_port = 8587 submission_relay_rawlog_dir = /var/log/dovecot/ submission_relay_trusted = yes My main issue is that until I login, dovecot-submission won't connect to the backend and query the capabilities and so won't report the right capabilities. That mean that the first EHLO message don't get the right capabilities list. " EHLO example.com 250-smtp.example.com 250-8BITMIME 250-AUTH PLAIN LOGIN 250-BURL imap 250-CHUNKING 250-ENHANCEDSTATUSCODES 250-SIZE 250 PIPELINING " This list don't contains VRFY, DNS, and SIZE is not specified (all of these is present in backend EHLO response). After login, if I send an new EHLO command, everything is properly reported. The raw log shows that unlike what the documentation says, dovecot don't try to connect to the backend until the user is properly logged. In my raw log I show that after I logged in dovecot-submission, the later open a connection to the backend and send a X-CLIENT command. Now, if I try to force the capabilities by using: submission_backend_capabilities = VRFY 8BITMIME DSN dovecot properly reports all SMTP capabilities in the first EHLO response, but it completely stops emitting X-CLIENT command to the backend and try to simply forward the command without authentication, which result in postfix rejecting the command with an unauthorized user error. What is wrong with my configuration ? Thanks. Jean-Daniel

4 7

Help with IMAP IDLE
by Jorge Bastos 29 Jul '19

29 Jul '19

Hi, I'm becaming crazy about this! I've asked before, not no matter what i do or conf, never works. How can i make IDLE work for real with MSOutlook, so that i can receive emails when they arrive, instead of changing folder/get out and in my account? Here's my conf, should it be reflected in postconf -n (is it the running conf?): ================== root@fastmail:/etc/dovecot# grep -ir idle * conf.d/20-imap.conf.ucf-dist:# If nothing happens for this long while client is IDLEing, move the connection conf.d/20-imap.conf.ucf-dist:# IDLEing. conf.d/20-imap.conf.ucf-dist:#imap_idle_notify_interval = 2 mins conf.d/20-imap.conf:imapc_max_idle_time = 1440 mins conf.d/20-imap.conf:# IDLEing. conf.d/20-imap.conf:#imap_idle_notify_interval = 2 mins conf.d/20-imap.conf:imap_idle_notify_interval = 1 min conf.d/20-imap.conf:#imap_client_workarounds = outlook-idle conf.d/10-mail.conf.ucf-dist:# When IDLE command is running, mailbox is checked once in a while to see if conf.d/10-mail.conf.ucf-dist:#mailbox_idle_check_interval = 30 secs conf.d/10-mail.conf:# When IDLE command is running, mailbox is checked once in a while to see if conf.d/10-mail.conf:#mailbox_idle_check_interval = 30 secs root@fastmail:/etc/dovecot# postconf -n|grep -i idle root@fastmail:/etc/dovecot#

7 16

Since multiple ,matches
by ＠lbutlr 27 Jul '19

27 Jul '19

I have a rule in sieve like the following: if anyof (header :contains ["to"] [ "box", "change", "cornell”, “twitter”]) { setflag "\\Seen"; fileinto :create "misc"; stop; } The messages that come in are using address extensions, so user+twitter(a)example.com, for example. The trouble is, this sieve recipe doesn’t get triggered and the message gets filed into the extension folder “twitter” or “cornell” instead of “misc”. I have a very similar recipe following that one: if header :contains "to" "root" { setflag "\\Seen"; addflag "$label5"; fileinto :create "root"; stop; } And that one works as I expect. Is that multiple match format valid for :contains (I’ve only seen it used for :regex)? I don’t get an error. Is there a way to run a debug log for the sieve recipe? Preferably for a single user. -- Vernon: Now this is the thought that wakes me up in the middle of the night. That when I get older, these kids are going to take care of me Carl: I wouldn't count on it.

1 1

Re: Dovecot, FreeBSD, and Solr?
by J. de Meijer 27 Jul '19

27 Jul '19

> Hello, > > Does anyone have this combination going? I'm running freebsd and dovecot, to which i'd like to add solr text search. I am stuck on xml schema integration, if anyone has any notes i'd appreciate them. I basically followed https://www.c0ffee.net/blog/mail-server-guide/ And dovecot with solr is running fine on freebsd for me.

3 3

corrupt mdbox index / zero mails showing in imap
by Mike 27 Jul '19

27 Jul '19

Hi, I have recently migrated (under emergency conditions) a dovecot imap/pop based server to a new instance. The mailboxes used mdbox format and due to various screwups I had corrupt indexes. I thought I'd cleaned this up but then I found that this new instance hadn't been set up correctly for nfs. Long story short, I still get users with new cases of corrupt indexes. The symptom is imap either showing NO mail in their inbox, or, not showing any recently delivered mail in the box, until I rm -f dovecot.map.index / doveadm force-resync -u user. It would be a huge help if there could be some method to detect if this is the case for any given user and to proactively do the force-resync process for them instead of waiting for that support call (or service cancellation...). I have looked around and have not found any tool capable of 'linting' an mdbox format inbox, and it seems like something like this should have been or would be an extremely useful debugging tool both during development as well as to troubleshoot stuff in the field. I would love to know if anyone either has such a tool, or any general suggestion how I could go about finding these cases and addressing them. I believe I have the nfs issue resolved and will not be creating new cases, so I just want to fix the ~3000 boxes I have now and move forward. Thank you. Mike-

2 2

doveadm migration to server with no root access
by jon rose 26 Jul '19

26 Jul '19

Hi I am trying to do mailbox migrations using doveadm to a server that I do not have root access on without success. Is this possible? What I have done and results: ssh to server with mailboxes to copy and then su cp /etc/dovecot/dovecot.conf /root/tmp/dovecot_new.conf append text below** to end of dovecot_new.conf try to copy mail 3 different ways (all fail see error messages) doveadm -D -v -c /root/tmp/dovecot_new.conf -o mail_fsync=never sync -1 -u test(a)propertyzone.co.za pop3c: fail with: Error: Mailbox INBOX.Archive sync: mailbox_set_subscribed failed: Subscriptions not supported doveadm -D -v -c /root/tmp/dovecot_new.conf -o mail_fsync=never sync -u test(a)propertyzone.co.za pop3c: fail with: Error: Can't create mailbox INBOX.Archive: POP3 mailbox creation isn't supported doveadm -D -v -c /root/tmp/dovecot_new.conf -o mail_fsync=never backup -u test(a)propertyzone.co.za pop3c: fail with: dsync(test(a)propertyzone.co.za): Panic: file mailbox-list-delete.c: line 473 (mailbox_list_delete_symlink_default): assertion failed: (ret > 0) ** append at end of dovecot_new.conf pop3c_host = dedi225.jnb3.host-h.net pop3c_user = test(a)propertyzone.co.za pop3c_password = ***** namespace inbox { inbox = yes } namespace { prefix = POP3-MIGRATION-NS/ location = pop3c: list = no hidden = yes } protocol doveadm { mail_plugins = $mail_plugins pop3_migration } plugin { pop3_migration_mailbox = POP3-MIGRATION-NS/INBOX -- Sent from: http://dovecot.2317879.n4.nabble.com/

2 4

Dovecot, FreeBSD, and Solr?
by David Mehler 26 Jul '19

26 Jul '19

Hello, Does anyone have this combination going? I'm running freebsd and dovecot, to which i'd like to add solr text search. I am stuck on xml schema integration, if anyone has any notes i'd appreciate them. Thanks. Dave.

2 1

dsync crash
by tono 25 Jul '19

25 Jul '19

Hello, I am getting a persistent seg fault while trying to sync our server to a new location. We are on 2.2.33.2, due for an upgrade but we have used dsync successfully in the past. In fact, we used it to populate this same instance. So I hope this is just a configuration issue. My logs are being filled with these errors:: 2019-07-10T08:29:53-04:00 localhost dovecot: [ID 583609 mail.crit] doveadm: Fatal: master: service(doveadm): child 44707 killed with signal 11 (core dumped) On the remote we see: 2019-07-10T10:17:24-04:00 localhost dovecot: [ID 583609 mail.error] doveadm: Error: read(111.111.111.111) failed: EOF (version not received) I tried doing sync manually: truss -rall -wall -f doveadm sync -u tpyro(a)metro-email.com tcp:123.123.123.123 55440: V E R S I O N\t d o v e a d m - s e r v e r\t 1\t 1\n 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: read(10, " -\n", 8192) = 2 55440: write(10, 0x08126598, 31) = 31 55440: P L A I N\t P A S s w o R d R E D A C t E D x W o e N X H h\n 55440: read(10, 0x0814980A, 8190) Err#11 EAGAIN 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: read(10, 0x0814980A, 8190) = 27 55440: V E R S I O N\t d o v e a d m - c l i e n t\t 1\t 1\n 55440: read(10, 0x08149825, 8163) Err#11 EAGAIN 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: read(10, " +\n", 8163) = 2 55440: write(10, 0x081738D0, 60) = 60 55440: \t u s e r 1 @ d o m a i n - n a m e . c o m\t d s y n c - s e r 55440: v e r\t - u u s e r 1 @ d o m a i n - n a m e . c o m\n 55440: pollsys(0x0816EB88, 1, 0x00000000, 0x00000000) = 1 55440: Incurred fault #6, FLTBOUNDS %pc = 0xFEC84490 55440: siginfo: SIGSEGV SEGV_MAPERR addr=0xD89CDB4E 55440: Received signal #11, SIGSEGV [default] 55440: siginfo: SIGSEGV SEGV_MAPERR addr=0xD89CDB4E I saw the note in the wiki about "dovecot --hostdomain" so I made sure they do not match. The new target (z-imap2) was built using a zfs clone from the source (z-imap1). I changed the 'hostname', 'instance_name", and 'mail_replica' fields but otherwise they are identical. Did I miss something? Thanks, Tony # 2.2.33.2 (d6601f4ec): /opt/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: SunOS 5.11 i86pc auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_default_realm = domain-name.com auth_mechanisms = plain login auth_verbose = yes default_process_limit = 500 default_vsz_limit = 512 M doveadm_password = # hidden, use -P to show it doveadm_port = 10993 hostname = z-imap1 instance_name = dovecot-2.2.13 # Note this was a reference to an earlier install, just an oversight. True version is 2.2.33.2. mail_location = mdbox:%h/mdbox mail_plugins = " mail_log notify replication" mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox "Deleted Messages" { special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = } prefix = separator = / } passdb { args = scheme=CRYPT username_format=%Lu /opt/local/etc/dovecot/users driver = passwd-file } plugin { imapsieve_mailbox1_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-spam.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_name = Spam imapsieve_mailbox3_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-ham.sieve imapsieve_mailbox3_causes = COPY imapsieve_mailbox3_from = Junk imapsieve_mailbox3_name = INBOX imapsieve_mailbox4_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-ham.sieve imapsieve_mailbox4_causes = COPY imapsieve_mailbox4_from = Spam imapsieve_mailbox4_name = INBOX imapsieve_mailbox5_before = file:/opt/local/etc/dovecot/sieve/spam-training/report-ham.sieve imapsieve_mailbox5_causes = COPY imapsieve_mailbox5_from = * imapsieve_mailbox5_name = NotJunk mail_replica = tcp:123.123.123.123 # True IP redacted sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /opt/local/etc/dovecot/sieve/spam-training/spam-to-folder.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve/spam-training sieve_plugins = sieve_imapsieve sieve_extprograms sieve_trace_dir = /opt/local/etc/dovecot/sieve/trace sieve_trace_level = matching } postmaster_address = postmaster(a)domain-name.com protocols = imap lmtp sieve replication_max_conns = 1 service aggregator { fifo_listener replication-notify-fifo { user = doveuser } unix_listener replication-notify { user = doveadm } } service anvil { client_limit = 1503 } service auth { client_limit = 2500 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0660 } } service doveadm { inet_listener { port = 10993 } vsz_limit = 4 G } service imap { executable = imap postlogin vsz_limit = 3 G } service indexer-worker { vsz_limit = 4 G } service lmtp { inet_listener lmtp { port = 51924 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service postlogin { executable = script-login rawlog -t } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl_cert = </opt/custom/etc/ssl/wildcard.domain-name.com/wildcard.domain-name.com.bundled.crt ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /opt/local/etc/dovecot/users driver = passwd-file } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 24 mail_plugins = mail_log notify replication imap_sieve } protocol lmtp { mail_plugins = mail_log notify replication sieve }

1 1