October 2019 - dovecot

Problem with converting mdbox to MailDir
by Adam Raszkiewicz 03 Oct '19

03 Oct '19

I have a problem to convert mdbox to MailDir. Log from that process looks like: /bin/dsync -Dv -u user(a)domian.com mirror maildir:/tmp/destination/mailboxes/user@domian.com Debug: Loading modules from directory: /lib/dovecot Debug: Module loaded: /lib/dovecot/lib01_acl_plugin.so Debug: Loading modules from directory: /lib/dovecot/doveadm Debug: Module loaded: /lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(user(a)domian.com): Debug: Effective uid=503, gid=503, home=/var/mailboxes/%2Mu/%2.2Mu/%u doveadm(user(a)domian.com): Debug: Home dir not found: /var/mailboxes/%2Mu/%2.2Mu/%u doveadm(user(a)domian.com): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/var/mailboxes/64/b1/user@domian.com doveadm(user(a)domian.com): Debug: fs: root=/var/mailboxes/64/b1/user(a)domian.com, index=, indexpvt=, control=, inbox=, alt= doveadm(user(a)domian.com): Debug: acl: initializing backend with data: vfile doveadm(user(a)domian.com): Debug: acl: acl username = user(a)domian.com doveadm(user(a)domian.com): Debug: acl: owner = 1 doveadm(user(a)domian.com): Debug: acl vfile: Global ACLs disabled doveadm(user(a)domian.com): Debug: Namespace : type=shared, prefix=shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=mdbox:/var/mailboxes/%2Mu/%2.2Mu/%u doveadm(user(a)domian.com): Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= doveadm(user(a)domian.com): Debug: acl: initializing backend with data: vfile doveadm(user(a)domian.com): Debug: acl: acl username = user(a)domian.com doveadm(user(a)domian.com): Debug: acl: owner = 0 doveadm(user(a)domian.com): Debug: acl vfile: Global ACLs disabled doveadm(user(a)domian.com): Debug: acl vfile: file /var/mailboxes/64/b1/user(a)domian.com/mailboxes/INBOX/dbox-Mails/dovecot-acl not found doveadm(user(a)domian.com): Debug: acl vfile: file /var/mailboxes/64/b1/user(a)domian.com/mailboxes/Drafts/dbox-Mails/dovecot-acl not found doveadm(user(a)domian.com): Debug: acl vfile: file /var/mailboxes/64/b1/user(a)domian.com/mailboxes/Sent/dbox-Mails/dovecot-acl not found doveadm(user(a)domian.com): Debug: acl vfile: file /var/mailboxes/64/b1/user(a)domian.com/mailboxes/Trash/dbox-Mails/dovecot-acl not found doveadm(user(a)domian.com): Debug: Namespace : Using permissions from /var/mailboxes/64/b1/user(a)domian.com: mode=0700 gid=default dsync(user(a)domian.com): Debug: Effective uid=503, gid=503, home=/var/mailboxes/%2Mu/%2.2Mu/%u dsync(user(a)domian.com): Debug: Home dir not found: /var/mailboxes/%2Mu/%2.2Mu/%u dsync(user(a)domian.com): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/tmp/destination/mailboxes/user@domian.com dsync(user(a)domian.com): Debug: maildir++: root=/tmp/destination/mailboxes/user(a)domian.com, index=, indexpvt=, control=, inbox=/tmp/destination/mailboxes/user(a)domian.com, alt= dsync(user(a)domian.com): Debug: Namespace : /tmp/destination/mailboxes/user(a)domian.com doesn't exist yet, using default permissions dsync(user(a)domian.com): Debug: Namespace : Using permissions from /tmp/destination/mailboxes/user(a)domian.com: mode=0700 gid=default dsync(user(a)domian.com): Debug: acl: initializing backend with data: vfile dsync(user(a)domian.com): Debug: acl: acl username = user(a)domian.com dsync(user(a)domian.com): Debug: acl: owner = 1 dsync(user(a)domian.com): Debug: acl vfile: Global ACLs disabled dsync(user(a)domian.com): Debug: Namespace : type=shared, prefix=shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=mdbox:/var/mailboxes/%2Mu/%2.2Mu/%u dsync(user(a)domian.com): Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= dsync(user(a)domian.com): Debug: acl: initializing backend with data: vfile dsync(user(a)domian.com): Debug: acl: acl username = user(a)domian.com dsync(user(a)domian.com): Debug: acl: owner = 0 dsync(user(a)domian.com): Debug: acl vfile: Global ACLs disabled dsync(user(a)domian.com): Error: User initialization failed: namespace configuration error: All list=yes namespaces must use the same separator but when I will specify namespace to the inbox like /bin/dsync -Dv -u user(a)domian.com mirror maildir:/tmp/destination/mailboxes/user@domian.com -n inbox it throws different error: doveadm(user(a)domian.com): Debug: Namespace : Using permissions from /var/mailboxes/64/b1/user(a)domian.com: mode=0700 gid=default doveadm(user(a)domian.com): Fatal: execvp(maildir:/tmp/destination/mailboxes/user@domian.com) failed: No such file or directory dsync-local(user(a)domian.com): Error: read(remote) failed: EOF (version not received when /tmp/destination/mailboxes/user(a)domian.com directory exists What I’m doing wrong in the process? -- Adam

1 2

lmtp and virtual users
by David Wells - Alfavinil S.A. 02 Oct '19

02 Oct '19

Good afternoon. I have dovecot setup to authenticate virtual users using either gssapi or doind a bind to an ldap server to achieve a single sign on capable imap server connected to a samba active directory DC. What I am also trying to achieve is to have dovecot's lmtp daemon handle the mails passed from postfix. However, the only way I've gotten this to work is setting allow_all_users = yes in the userdb but this causes lmtp to deliver mails to non existant accounts without rejection. I've been searching but haven't found a way to set this same thing up but having dovecots lmtp check the validity of the mails recipient against the same samba AD DC through ldap before delivering it and rejecting unknown email addresses. Could someone please provide some insight into how to achieve this? Thank you very much in advance. Best regards, David Wells.

3 7

Unable to connect/authenticate after Debian SSL update
by paul 02 Oct '19

02 Oct '19

Just applied Debian SSL updates, due to vulnerabilities reported: "Package : openssl CVE ID : CVE-2019-1547 CVE-2019-1549 CVE-2019-1563 Three security issues were discovered in OpenSSL: A timing attack against ECDSA, a padding oracle in PKCS7_dataDecode() and CMS_decrypt_set1_pkey() and it was discovered that a feature of the random number generator (RNG) intended to protect against shared RNG state between parent and child processes in the event of a fork() syscall was not used by default. For the oldstable distribution (stretch), these problems have been fixed in version 1.1.0l-1~deb9u1. For the stable distribution (buster), these problems have been fixed in version 1.1.1d-0+deb10u1." I am running Buster. Upon installing these updates, Dovecot immediately stopped allowing authentication: Oct 1 16:47:11 relay monit[539]: 'dovecot' failed protocol test [IMAP] at [localhost]:993 [TCP/IP TLS] -- SSL connection error: error:00000000:lib(0):func(0):reason(0) Oct 1 16:50:22 relay monit[539]: 'dovecot' failed protocol test [IMAP] at [localhost]:993 [TCP/IP TLS] -- SSL connection error: error:00000000:lib(0):func(0):reason(0) and Oct 01 16:51:18 imap-login: Info: Disconnected: TLS initialization failed. (no auth attempts in 0 secs): user=<>, rip=71.198.57.170, lip=10.0.0.81, session=<0EdTB+KT+/NHxjmq> Each time this occurs, dovecot restarts: Oct 01 16:50:23 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) Oct 01 16:50:42 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) Oct 01 16:51:10 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) Oct 01 16:53:35 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) Oct 01 16:56:46 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) Oct 01 16:59:57 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) Oct 01 17:03:08 master: Info: Dovecot v2.3.7.2 (3c910f64b) starting up for pop3, imap (core dumps disabled) I am at a loss. Dovecot had been working fine for ages. It's not clear to me why a routine SSL update would completely break it. I'm unclear how to proceed. This is a small personal server, but nevertheless I have a number of people using it who rely on it. Here is my local.conf ( I added the ssl_protocols and ssl_min_protocol entries just a few minutes ago while attempting to resolve) auth_cache_size = 100 k auth_cache_ttl = 12 hours auth_cache_negative_ttl = 12 hours auth_cache_verify_password_with_worker=yes auth_verbose = yes auth_failure_delay = 7 secs default_client_limit = 500 default_process_limit = 50 default_vsz_limit = 100 M verbose_proctitle = yes protocols = pop3 imap pop3_uidl_format = %08Xu%08Xv imap_idle_notify_interval = 244 secs info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot-error.log maildir_very_dirty_syncs=yes mail_location = maildir:~/Maildir mailbox_idle_check_interval = 244 secs mailbox_list_index = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = passwd-file args = /etc/dovecot/deny deny = yes } passdb { driver = pam } userdb { driver = passwd } service auth { unix_listener auth-client { mode = 0660 user = Debian-exim } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } protocol imap { mail_max_userip_connections = 20 } service imap { unix_listener imap-master { user = dovecot } } imap_hibernate_timeout = 1 mins service imap { extra_groups = dovenull } service imap-hibernate { unix_listener imap-hibernate { mode = 0660 group = dovenull } } ssl = required ssl_protocols = TLSv1 TLSv1.1 TLSv1.2 ssl_min_protocol=TLSv1 ssl_dh=</etc/dovecot/dh.pem ssl_cert = </etc/letsencrypt/live/relay.anastrophe.com/fullchain.pem ssl_key = </etc/letsencrypt/live/relay.anastrophe.com/privkey.pem

1 1

Lost mails with sieve "duplicate" extension in case of failure
by Hanno Stock 01 Oct '19

01 Oct '19

Hi! we just experienced the following issue: We have a global sieve_before script using the duplicate extension. require ["duplicate", "imap4flags"]; if duplicate { discard; } Now the storage of our mailbox store got full and lmtp deliveries started to fail. This resulted in logs like this: Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error: o_stream_send_istream(/var/mail/vhosts/mail.COMPANY.com/USERNAME/.dovecot/t… failed: No space left on device Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error: write(/var/mail/vhosts/mail.COMPANY.com/USERNAME/.dovecot/tmp/1569420896.M6… failed: No space left on device Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error: KT9HJGB2i128FgAATXMPig: sieve: msgid=<redmine.journal-441799.20190925141453.94a97fd570706f2f(a)mail.COMPANY.com>: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2019-09-25 16:14:56] Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error: KT9HJGB2i128FgAATXMPig: sieve: Execution of script /mnt/data/var/mail/vhosts/mail.COMPANY.com/USERNAME/.sieve/.dovecot.sieve was aborted due to temporary failure (user logfile /mnt/data/var/mail/vhosts/mail.COMPANY.com/USERNAME/.sieve/.dovecot.sieve.log may reveal additional details) Sep 25 16:14:56 mail-rc1 postfix/lmtp[5851]: 4D27690AC: to=<USERNAME(a)mail.COMPANY.com>, orig_to=<USER.NAME(a)COMPANY.com>, relay=mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp], delay=1.3, delays=1.3/0/0/0.01, dsn=4.2.0, status=deferred (host mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp] said: 451 4.2.0 <USERNAME(a)mail.COMPANY.com> Internal error occurred. Refer to server log for more information. [2019-09-25 16:14:56] (in reply to end of DATA command)) So far, so good... A 451 response is what we wanted. However on the next delivery attempt the following happens: Sep 25 16:24:00 mail-rc1 postfix/lmtp[7040]: 4D27690AC: to=<USERNAME(a)mail.COMPANY.com>, orig_to=<USER.NAME(a)COMPANY.com>, relay=mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp], delay=545, delays=545/0.09/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 <USERNAME(a)mail.COMPANY.com> DBcCEYB4i11qGwAATXMPig Saved) Sep 25 16:24:00 mail-rc1 dovecot: lmtp(USERNAME): DBcCEYB4i11qGwAATXMPig: sieve: msgid=<redmine.journal-441799.20190925141453.94a97fd570706f2f(a)mail.COMPANY.com>: marked message to be discarded if not explicitly delivered (discard action) Sep 25 16:24:00 mail-rc1 postfix/lmtp[7040]: 4D27690AC: to=<USERNAME(a)mail.COMPANY.com>, orig_to=<USER.NAME(a)COMPANY.com>, relay=mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp], delay=545, delays=545/0.09/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0 <USERNAME(a)mail.COMPANY.com> DBcCEYB4i11qGwAATXMPig Saved) As the duplicate extension has already seen the mail, it is now immediately discarded. RFC 7352 however explicitly states: Implementations MUST only update the internal duplicate-tracking list when the Sieve script execution finishes successfully. If failing script executions add the unique ID to the duplicate-tracking list, all "duplicate" tests in the Sieve script would erroneously yield "true" for the next delivery attempt of the same message. Dovecot seems not to honour this specification. Maybe this is because the "before_script" is not the script that fails, but the subsequent scripts fail? Am I missing some configuration option that would propagate the subsequent failure back to the before_script? Or is this a general limitation of the before_script mechanism? Any help appreciated how to alleviate this issue! Best regards Hanno --- Filesystem: ext4 Dovecot config: # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.0-11-amd64 x86_64 Debian 9.11 auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_cache_ttl = 2 hours auth_mechanisms = plain login auth_username_format = %Ln mail_gid = vmail mail_home = /var/mail/vhosts/mail.COMPANY.com/%n mail_location = maildir:~/.dovecot mail_plugins = " acl" mail_privileged_group = vmail mail_uid = vmail mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader imapflags notify namespace { hidden = yes list = no location = maildir:~/.dovecot-top-namespace prefix = separator = . subscriptions = yes type = private } namespace { location = maildir:/var/mail/vhosts/mail.COMPANY.com/%%n/.dovecot:INDEXPVT=/var/mail/v… prefix = shared.%%u. separator = . subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db sieve = file:/var/mail/vhosts/mail.COMPANY.com/%n/.sieve/sieve;active=/var/mail/vho… sieve_after = /var/mail/sieve/global-after.sieve sieve_before = /var/mail/sieve/global-before.sieve sieve_default = /var/mail/sieve/global.sieve sieve_extensions = +imapflags +notify +editheader sieve_global = /var/mail/sieve } protocols = imap lmtp sieve service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } process_limit = 9000 } service imap { process_limit = 9000 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } ssl = required ssl_cert = </etc/letsencrypt/live/mail.COMPANY.com/fullchain.pem ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } protocol imap { mail_max_userip_connections = 30 mail_plugins = " acl imap_acl" } protocol sieve { mail_debug = yes }

2 2

Mailbox INBOX sync: mailbox_delete failed
by Alex La 01 Oct '19

01 Oct '19

Hi. When I run the backup I always get this error. ``` [root@mail.DOMAIN.com:~]# doveadm -o mail_fsync=never backup -D -R -u office(a)DOMAIN.com imapc: dsync(office(a)DOMAIN.com): Info: imapc(imap.mailbox.org:993): Connected to 2001:67c:2050:106::143:199:993 (local 2a01:4f8:c2c:6d05::1:59928) dsync(office(a)DOMAIN.com): Info: Mailbox deleted: Drafts dsync(office(a)DOMAIN.com): Error: Mailbox INBOX sync: mailbox_delete failed: INBOX can't be deleted. ``` The suggested solution from this mail works, but is this still the best solution for the error above? https://dovecot.org/pipermail/dovecot/2016-January/102988.html My versions are # 2.2.36.4 (baf9232c1): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.24.2 (aaba65b7) # OS: Linux 3.10.0-1062.1.1.el7.x86_64 x86_64 CentOS Linux release 7.7.1908 (Core) Best regards Alex

2 2

Migration
by Hans Brage 01 Oct '19

01 Oct '19

Hi! I'm currently running an small imap-server on Dovecot 2.2.4 but will retire that server. I've set up an new server with Dovecot 2.3.4 and will migrate the mailboxes (maildir-format) from the old to the new server. I'm planning to restructure the accounts a bit when migrating so I need to move them one by one. Its only a few so thats not a big issue. Both servers are using virtual accounts (users file). I've understood that best method for migration is to run doveadm backup -R from the new server. But I really cant figure out what other parameters or settings that I need to perform that task and if it should be run as user vmail or root. Would be greatful for any tips, hints, links or similiar. Regards, Hans

6 9

Best way to move mail from one server to another
by James Brown 01 Oct '19

01 Oct '19

I’d like to migrate to a new server. I only need to transfer the mail store, have already done the users. I’ve seen different recommendations: IMAP-Sync Rsync Doveadmin backup and dsync What are the pros and cons of each? What would be best for me? I’m thinking that Doveadmin backup is the way to go. Both old and new servers are on same network. New server is running Dovecot 2.3.2.1, old server is 2.2.32. macOS X. Does anyone have any advice or examples? Thanks, James.

13 29

courier to dovecot migration
by tovises 01 Oct '19

01 Oct '19

I have a quite old home server based on Debian 6.x packages using courier imap server (exim4, fetchmail, courier, apache2, squirrelmail) to keep my and my wife emails (about 25G). I want renew my server to Debian 10 (buster) and got a recommendation to use dovecot instead of courier. For this purpose I'm using a different PC as a sandbox. Debian 10 currently offer dovecot version 2.3.4.1 Installation was "peaceful" but stuck with the migration. I have choose migration using dsync, from Wiki I was choose this command: For per-user user/passwords use: doveadm -o imapc_user=foo -o pop3c_user=foo -o imapc_password=bar -o pop3c_password=bar backup -R -u user@domain imapc: (foo and bar placed real username and password) But I get error: doveadm(tovis(a)nusi.tovis-lab.mydomain.org): Error: User doesn't exist Strongly I have no real domain, I'm using "freedns". "nusi" is the local name of my old server, "tovis-lab" is conjunction with "mydomain.org" choose from freedns. I was trying short as "nusi" which is in /etc/hosts file even ip address but the same result. 143 and 943 ports are working only inside the firewall (OpenWrt). I was checked using telnet courier on the "remote" box and dovecot on the localhost. I was able to login. What should I use as "user@domain" in this situation? tovis

4 10