- dovecot - dovecot.org

Doveadm / dsync: mirror?
by Larry Rosenman 01 Oct '17

01 Oct '17

On https://wiki.dovecot.org/Migration/MailFormat we see: mbox -> maildir migration. Set mail_location=maildir:~/Maildir and run dsync -u username mirror mbox:~/mail:INBOX=/var/mail/username but NOWHERE that I can find is the “mirror” command documented on the wiki. What are ALL the options here? (and it’s interesting to note that dsync is a symlink to doveadm). Thanks! -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: larryrtx(a)gmail.com US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106

1 0

imapc and masteruser
by Computerisms Corporation 30 Sep '17

30 Sep '17

Hi, My end goal is to set up shared mailboxes on a cluster as per: https://wiki.dovecot.org/SharedMailboxes/ClusterSetup I was having very little luck with it, so I had been trying to break down into pieces and get individual components working. So I have things setup on a single server, with a working dovecot instance. I have no director or any thing else running yet, and I am just trying to get the imapc portion of things working. Eventually I came across a post from Sven Hartge: https://www.dovecot.org/list/dovecot/2012-November/087046.html This is from several years ago, but I set up a separate dovecot instance on port 9993 specifically to run the shared folders as he describes, and copied his configuration suggestions pretty much verbatim. The only signficant difference is that I needed to not configure the imapc_user as he had done, and instead as the wiki suggests remove that config option entirely. With this, the imapc connection is being made from the primary instance on port 993 to the shared instance on port 9993. My problem is that it doesn't appear that the imapc_master_user is being fully acknowledged. The imapc connection must be made with the original user logging in as the master user to the shared user's account, else the ACLs can't do what they are supposed to do. It does appear the original user is being used to authenticate, maybe, at the very least it is being offered in the imapc connection, but not as a master user. To illustrate, I have two snippets of logs below. The first is from the shared folders instance on port 9993 after logging into the primary server on port 993, which triggers the imapc connection. This shows the user bob.test, who is the imapc_master_user as defined by %u, being acknowledged and then having the username changed to that of my shared user, in this case authapps. Further down, we can see that the auth_user is bob.test, but there is no master user, and the acl username at the bottom of the snipped remains that of my shared user. Below that, I have the same snippet of log, as obtained by connecting with openssl s_client to the shared instance on port 9993, with the string: a login authapps*bob.test XXXXXXX In order to get this working, I need to add to the shared instance's config file a static userdb like so: userdb { driver = static } this snippet shows the masteruser working as expected, and the acl username being that of the master user. Unfortunately, it has no apparent effect on the imapc login. So it seems master user logins work, just not through imapc. I note at the top of the first snippet, before the change in username, it appears the imapc connection is being directed to the static passdb, which I think is triggering the username change, where as the second snippet, the connection is starting with the ldap passdb. but no amount of playing "configuration file options" chairs has led me to a working solution. I am hoping someone can provide some insight as to what I am missing here. doveconf -n for both instances are pasted below the log snippets. If you are still reading, you have my gratitude for your attention this far... 2017-09-27 13:20:13.725943500 Sep 27 13:20:13 auth: Debug: static(bob.test,192.168.120.70,<y0vvhDFaYqDAqHhG>): lookup 2017-09-27 13:20:13.725960500 Sep 27 13:20:13 auth: Debug: static(bob.test,192.168.120.70,<y0vvhDFaYqDAqHhG>): username changed bob.test -> authapps 2017-09-27 13:20:13.726011500 Sep 27 13:20:13 auth: Debug: client passdb out: OK 1 user=authapps original_user=bob.test 2017-09-27 13:20:13.728299500 Sep 27 13:20:13 auth: Debug: master in: REQUEST 2523398145 13259 1 ba830ae1c199401e5dbee28e31025ac1 session_pid=13335 request_auth_token 2017-09-27 13:20:13.728344500 Sep 27 13:20:13 auth: Debug: master userdb out: USER 2523398145 authapps uid=509 gid=509 home=/CTFN/SharedMailboxes/CTFN/ auth_token=5e3d02714e82441cbbb3de8ff80f35bc3dc4b291 auth_user=bob.test 2017-09-27 13:20:13.728459500 Sep 27 13:20:13 imap-login: Info: Login: user=<authapps>, method=PLAIN, rip=192.168.120.70, lip=192.168.120.70, mpid=13335, TLS, session=<y0vvhDFaYqDAqHhG> 2017-09-27 13:20:13.729092500 Sep 27 13:20:13 imap(authapps): Debug: Loading modules from directory: /usr/local/lib/dovecot 2017-09-27 13:20:13.729781500 Sep 27 13:20:13 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so 2017-09-27 13:20:13.729841500 Sep 27 13:20:13 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib02_imap_acl_plugin.so 2017-09-27 13:20:13.730026500 Sep 27 13:20:13 imap(authapps): Debug: Effective uid=509, gid=509, home=/CTFN/SharedMailboxes/CTFN/ 2017-09-27 13:20:13.732014500 Sep 27 13:20:13 imap(authapps): Debug: Namespace CTFN: type=public, prefix=CTFN/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/CTFN/SharedMailboxes/ 2017-09-27 13:20:13.732050500 Sep 27 13:20:13 imap(authapps): Debug: maildir++: root=/CTFN/SharedMailboxes, index=, indexpvt=, control=, inbox=/CTFN/SharedMailboxes, alt= 2017-09-27 13:20:13.732053500 Sep 27 13:20:13 imap(authapps): Debug: acl: initializing backend with data: vfile 2017-09-27 13:20:13.732055500 Sep 27 13:20:13 imap(authapps): Debug: acl: acl username = authapps 2017-09-27 18:33:37.008636500 Sep 27 18:33:37 auth: Debug: ldap(bob.test,192.168.120.50,master,<y56y5TVaAYHAqHgy>): Master user lookup for login: authapps 2017-09-27 18:33:37.021400500 Sep 27 18:33:37 auth: Info: ldap(bob.test,192.168.120.50,master,<y56y5TVaAYHAqHgy>): Master user logging in as authapps 2017-09-27 18:33:37.021425500 Sep 27 18:33:37 auth: Debug: client passdb out: OK 1 user=authapps original_user=bob.test auth_user=bob.test 2017-09-27 18:33:37.023658500 Sep 27 18:33:37 auth: Debug: master in: REQUEST 2776498177 9162 1 2d247ac65c17769575edb6c9db86fdee session_pid=27552 request_auth_token 2017-09-27 18:33:37.023703500 Sep 27 18:33:37 auth: Debug: ldap(authapps,192.168.120.50,<y56y5TVaAYHAqHgy>): user search: base=CN=Users,dn=ctfn,dn=ca scope=onelevel filter=(&(sAMAccountName=authapps)) fields=homeDirectory,uidNumber,gidNumber 2017-09-27 18:33:37.026953500 Sep 27 18:33:37 auth: Error: ldap(authapps,192.168.120.50,<y56y5TVaAYHAqHgy>): ldap_search(base=CN=Users,dn=ctfn,dn=ca filter=(&(sAMAccountName=authapps))) failed: Operations error 2017-09-27 18:33:37.026984500 Sep 27 18:33:37 auth: Debug: master userdb out: USER 2776498177 authapps uid=509 gid=509 home=/CTFN/SharedMailboxes/CTFN/ master_user=bob.test auth_token=12961a562e9ef0698b363714d758d95cea9ff3f2 auth_user=bob.test 2017-09-27 18:33:37.027163500 Sep 27 18:33:37 imap-login: Info: Login: user=<authapps>, method=PLAIN, rip=192.168.120.50, lip=192.168.120.70, mpid=27552, TLS, session=<y56y5TVaAYHAqHgy> 2017-09-27 18:33:37.027826500 Sep 27 18:33:37 imap(authapps): Debug: Loading modules from directory: /usr/local/lib/dovecot 2017-09-27 18:33:37.028479500 Sep 27 18:33:37 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so 2017-09-27 18:33:37.028561500 Sep 27 18:33:37 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib02_imap_acl_plugin.so 2017-09-27 18:33:37.028607500 Sep 27 18:33:37 imap(authapps): Debug: Added userdb setting: plugin/master_user=bob.test 2017-09-27 18:33:37.028685500 Sep 27 18:33:37 imap(authapps): Debug: Effective uid=509, gid=509, home=/CTFN/SharedMailboxes/CTFN/ 2017-09-27 18:33:37.030993500 Sep 27 18:33:37 imap(authapps): Debug: Namespace CTFN: type=public, prefix=CTFN/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/CTFN/SharedMailboxes/CTFN 2017-09-27 18:33:37.031019500 Sep 27 18:33:37 imap(authapps): Debug: maildir++: root=/CTFN/SharedMailboxes/CTFN, index=, indexpvt=, control=, inbox=/CTFN/SharedMailboxes/CTFN, alt= 2017-09-27 18:33:37.031040500 Sep 27 18:33:37 imap(authapps): Debug: acl: initializing backend with data: vfile 2017-09-27 18:33:37.031047500 Sep 27 18:33:37 imap(authapps): Debug: acl: acl username = bob.test PRIMARY SERVER ON PORT 993: doveconf -i dovecotserver -n # 2.2.31 (65cde28): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.19 (e5c7051) # OS: Linux 4.9.0-3-amd64 x86_64 Debian 9.1 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_verbose_passwords = plain hostname = imap.ctfn.ca imapc_host = masterchieflian.ctfn.ca imapc_master_user = %u imapc_password = # hidden, use -P to show it imapc_port = 9993 imapc_ssl = imaps instance_name = dovecotserver lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 192.168.120.70 log_path = /dev/stderr login_greeting = CTFN IMAP server mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = stats quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds mmap_disable = yes namespace CTFN { list = yes location = imapc:/CTFN/mail/CTFN:INDEXPVT=/CTFN/mail/%n/Maildir/CTFN prefix = CTFN/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Archives { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox LearnAsSpam { auto = subscribe } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap master = yes } plugin { acl_shared_dict = fs:posix:prefix=/CTFN/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size vsize from subject flags sieve = ~/.dovecot.sieve sieve_default = /usr/local/lib/dovecot/default.sieve sieve_dir = ~/sieve sieve_extensions = +vacation +vacation-seconds sieve_max_actions = 92 sieve_max_redirects = 24 sieve_max_script_size = 10M sieve_quota_max_scripts = 20 sieve_quota_max_storage = 200000 } protocols = imap sieve service auth { client_limit = 1024 unix_listener auth-userdb { group = vmail mode = 0660 user = dovecot } } service doveadm { inet_listener { address = 192.168.120.70 port = 9092 } } service imap-login { inet_listener imap { address = 127.0.0.1 port = 143 ssl = no } inet_listener imaps { address = 192.168.120.70 port = 993 ssl = yes } process_min_avail = 5 service_count = 0 } service managesieve-login { inet_listener sieve-local { address = 127.0.0.1 port = 4190 } inet_listener sieve { address = 192.168.120.70 port = 4190 } process_min_avail = 5 service_count = 1 vsz_limit = 64 M } ssl = required ssl_ca = /var/CA/ctfn.ca/RapidSSLWildcard/2017/geotrust.intermediate.pem ssl_cert = <//var/CA/ctfn.ca/RapidSSLWildcard/2017/ctfn.ca.pem ssl_client_ca_dir = /etc/ssl/certs ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext default_fields = uid=vmail gid=vmail imapc_password=%w home=/CTFN/mail/%Ln driver = ldap } userdb { args = uid=vmail gid=vmail home=/CTFN/mail/%n driver = static } verbose_proctitle = yes protocol imap { imap_id_log = * } protocol lda { mail_plugins = stats quota sieve submission_host = 127.0.0.1:25 } protocol sieve { mail_max_userip_connections = 10 mail_plugins = managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 managesieve_notify_capability = managesieve_sieve_capability = } SHARED SERVER ON PORT 9993 doveconf -i dovshare -n # 2.2.31 (65cde28): /usr/local/etc/dovecot/dovshare.conf # Pigeonhole version 0.4.19 (e5c7051) # OS: Linux 4.9.0-3-amd64 x86_64 Debian 9.1 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_verbose_passwords = plain base_dir = /usr/local/var/run/dovshare/ hostname = imapshare.ctfn.ca instance_name = dovshare listen = 192.168.120.70 log_path = /dev/stderr login_greeting = CTFN IMAP Shared Mailboxes mail_debug = yes mail_fsync = always mail_gid = vmail mail_location = maildir:~ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = stats quota acl mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mmap_disable = yes namespace CTFN { inbox = yes location = maildir:/CTFN/SharedMailboxes/CTFN prefix = CTFN/ separator = / type = public } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap master = yes } passdb { args = user=authapps password=XXXXXXXXXX driver = static } plugin { acl = vfile acl_shared_dict = fs:posix:prefix=/CTFN/Config/dovecot/dovecot.acls/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size vsize from subject flags } protocols = imap service auth { client_limit = 1024 unix_listener auth-userdb { group = vmail mode = 0660 user = dovecot } } service imap-login { inet_listener imaps { address = 192.168.120.70 port = 9993 ssl = yes } process_min_avail = 5 service_count = 0 } ssl = required ssl_ca = </var/CA/ctfn.ca/RapidSSLWildcard/2017/geotrust.intermediate.pem ssl_cert = <//var/CA/ctfn.ca/RapidSSLWildcard/2017/ctfn.ca.pem ssl_client_ca_file = </var/CA/ctfn.ca/RapidSSLWildcard/2017/geotrust.intermediate.pem ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { driver = static } verbose_proctitle = yes protocol imap { imap_id_log = * mail_plugins = acl imap_acl } -- Bob Miller Cell: 867-334-7117 Office: 867-633-3760 www.computerisms.ca

2 4

2.2.32 'doveadm replicator replicate -f' segfault
by Robert Giles 29 Sep '17

29 Sep '17

Very minor bug; not specifying the user mask with 'doveadm replicator replicate -f' causes a segfault: server:~# doveadm replicator replicate -f Segmentation fault server:~# doveadm replicator replicate -f '*' 123 users updated server:~# gdb /usr/bin/doveadm core.2418 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-100.el7 This GDB was configured as "x86_64-redhat-linux-gnu". Reading symbols from /usr/bin/doveadm...Reading symbols from /usr/lib/debug/usr/bin/doveadm.debug...done. done. [New LWP 2418] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `doveadm replicator replicate -f'. Program terminated with signal 11, Segmentation fault. #0 0x00007fd2c059f921 in __strlen_sse2_pminub () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 glibc-2.17-196.el7.x86_64 nss-softokn-freebl-3.28.3-8.el7_4.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64 (gdb) bt full #0 0x00007fd2c059f921 in __strlen_sse2_pminub () from /lib64/libc.so.6 No symbol table info available. #1 0x00007fd2c08c68f4 in str_append_tabescaped (dest=dest@entry=0x5578cd69f668, src=0x0) at strescape.c:136 No locals. #2 0x00005578cb5a2731 in cmd_replicator_replicate (argc=1, argv=0x5578cd6a75f8) at doveadm-replicator.c:245 ctx = 0x5578cd69f608 str = 0x5578cd69f668 line = <optimized out> #3 0x00005578cb5999d8 in doveadm_try_run (argv=0x5578cd6a75f0, argc=2, cmd_name=0x5578cd6a7610 "replicator") at doveadm.c:223 cmd = <optimized out> #4 main (argc=3, argv=0x5578cd6a75e8) at doveadm.c:383 cctx = {cmd = 0x0, argc = 0, argv = 0x0, username = 0x5578cd6a77cf "root", cli = true, tcp_server = false, local_ip = { family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, conn = 0x0} cmd_name = 0x5578cd6a7610 "replicator" quick_init = false c = <optimized out> (gdb) frame 4 #4 main (argc=3, argv=0x5578cd6a75e8) at doveadm.c:383 383 !doveadm_try_run(cmd_name, argc, (const char **)argv) && dovecot -n: --------------------- # 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.20 (7cd71ba) # OS: Linux 3.10.0-693.2.2.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.4 (Maipo) auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 30000 imapc_features = rfc822.size fetch-headers imapc_host = some-imap-server imapc_user = %u last_valid_uid = 30000 mail_location = mdbox:~/mdbox mail_plugins = " zlib acl notify replication" mail_prefetch_count = 20 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus vnd.dovecot.duplicate mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /master-file driver = passwd-file master = yes } passdb { args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext driver = ldap } plugin { mail_replica = tcp:server2:1109 sieve = ~/.dovecot.sieve sieve_before = /some-path/global.sieve sieve_dir = ~/sieve sieve_extensions = +spamtest +spamtestplus +vnd.dovecot.duplicate sieve_spamtest_max_value = 100 sieve_spamtest_status_header = X-PerlMx-Spam: Gauge=[[:alnum:]]+, Probability=(-?[[:digit:]]+)%.* sieve_spamtest_status_type = score zlib_save = gz zlib_save_level = 6 } protocols = imap lmtp sieve replication_max_conns = 30 service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service doveadm { inet_listener { port = 1109 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { port = 24 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0660 user = vmail } } userdb { args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " zlib acl notify replication sieve" } protocol lda { mail_plugins = " zlib acl notify replication sieve" }

2 1

Securing postfix to dovecot (SASL) auth
by Raymond Sellars 29 Sep '17

29 Sep '17

Hi Is it possible to secure the Dovecot SASL auth provider for postfix? https://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL I'm currently using the inet option to provide SASL auth to postfix for dovecot. Both installs are on different hosts hence the use of inet rather than unix socket. I'm looking for the best way to secure this channel (rather than just assuming the VLAN is secure ) ,e.g. TLS. It would seem even the latest versions of dovecot/postfix3 don't support encrypting this channel? Has anyone managed to implement a secure internal approach they can share? I'm wondering if Postfix with Cyrus against IMAP using STARTTLS is my best alternative. Thanks Raymond

5 8

Conditionally disabling auth policy
by Mark Moseley 29 Sep '17

29 Sep '17

I've been digging into the auth policy stuff with weakforced lately. There are cases (IP ranges, so could be wrapped up in remote {} blocks) where it'd be nice to skip the auth policy (internal hosts that I can trust, but that are hitting the same servers as the outside world). Is there any way to disable auth policy, possibly inside a remote{}? auth_policy_server_url complains that it can't be used inside a remote block, so no dice there. Anything I'm missing?

3 7

doveadm backup panic, dsync-ibc.c, dsync_ibc_send_mailbox_tree_node, assertion failed: (*name != NULL)
by Robert Giles 28 Sep '17

28 Sep '17

Hi folks, Not sure how much extra information I'll be able to provide on this, but I'm experiencing a consistent, reproducible dsync backup error with a single, specific account I'm trying to migrate from UW-IMAP / Panda (mix mailbox) to a Dovecot 2.2.32 cluster. Didn't spot any core files laying around. Doveadm backup output, dovecot --version and dovecot -n below: server:~# doveadm -Dv -o imapc_user=username-dc -o imapc_password=password backup -R -u username -x .mailboxlist -x .procmailrc -x vacation.msg -x .bash_history -x .Xauthority -x remote_pinerc imapc: Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib20_zlib_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message) doveadm(username): Debug: Effective uid=35000, gid=35000, home=/storagedisk/username doveadm(username): Debug: acl: No acl setting - ACLs are disabled doveadm(username): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox doveadm(username): Debug: fs: root=/storagedisk/username/mdbox, index=, indexpvt=, control=, inbox=, alt= doveadm(username): Debug: Namespace : Using permissions from /storagedisk/username/mdbox: mode=0700 gid=default dsync(username): Debug: Effective uid=35000, gid=35000, home=/storagedisk/username dsync(username): Debug: acl: No acl setting - ACLs are disabled dsync(username): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=imapc: dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Created new connection dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Looking up IP address (reconnect_ok=true, last_connect=1506608870) dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Connecting to 192.168.0.100:143 dsync(username): Info: imapc(panda-imap-server.contoso.com:143): Connected to 192.168.0.100:143 (local 192.168.0.200:33630) dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Server capabilities: IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS STARTTLS dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Authenticating as username-dc dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Server capabilities: IMAP4REV1 I18NLEVEL=1 LITERAL+ IDLE UIDPLUS NAMESPACE CHILDREN MAILBOX-REFERRALS BINARY UNSELECT WITHIN SORT THREAD=REFERENCES THREAD=ORDEREDSUBJECT MULTIAPPEND SCAN dsync(username): Debug: imapc(panda-imap-server.contoso.com:143): Authenticated successfully dsync(username): Debug: imapc: root=, index=, indexpvt=, control=, inbox=, alt= dsync(username): Debug: brain S: Local mailbox tree: guid=00000000000000000000000000000000 uid_validity=0 uid_next=0 subs=no last_change=0 last_subs=0 dsync(username): Panic: file dsync-ibc.c: line 72 (dsync_ibc_send_mailbox_tree_node): assertion failed: (*name != NULL) dsync(username): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x9edee) [0x7f7d4fbfcdee] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f7d4fbfce5a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f7d4fb8e2cc] -> doveadm(+0x5332b) [0x563357a0032b] -> doveadm(dsync_brain_send_mailbox_tree+0x6e) [0x5633579f12be] -> doveadm(dsync_brain_run+0x49d) [0x5633579ee56d] -> doveadm(+0x290dc) [0x5633579d60dc] -> doveadm(+0x2b21c) [0x5633579d821c] -> doveadm(+0x2be1a) [0x5633579d8e1a] -> doveadm(doveadm_mail_try_run+0x215) [0x5633579d9775] -> doveadm(main+0x46a) [0x5633579c8a9a] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f7d4f7bcc05] -> doveadm(+0x1bc05) [0x5633579c8c05] Aborted dovecot --version 2.2.32 (dfbe293d4) dovecot -n: ----------------------------------------------- # 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.20 (7cd71ba) # OS: Linux 3.10.0-693.2.2.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.4 (Maipo) auth_master_user_separator = * auth_username_format = %Ln disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it first_valid_uid = 35000 imapc_features = rfc822.size fetch-headers imapc_host = panda-imap-server.contoso.com imapc_user = %u last_valid_uid = 35000 mail_location = mdbox:~/mdbox mail_plugins = " zlib acl notify replication" mail_prefetch_count = 20 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext spamtest spamtestplus vnd.dovecot.duplicate mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext driver = ldap } plugin { mail_replica = tcp:otherserver.contoso.com:1109 sieve = ~/.dovecot.sieve sieve_before = /etc/sieve.before sieve_dir = ~/sieve sieve_extensions = +spamtest +spamtestplus +vnd.dovecot.duplicate sieve_spamtest_max_value = 100 sieve_spamtest_status_header = X-PerlMx-Spam: Gauge=[[:alnum:]]+, Probability=(-?[[:digit:]]+)%.* sieve_spamtest_status_type = score zlib_save = gz zlib_save_level = 6 } protocols = imap lmtp sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service doveadm { inet_listener { port = 1109 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { port = 24 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0660 user = vmail } } userdb { args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " zlib acl notify replication sieve" } protocol lda { mail_plugins = " zlib acl notify replication sieve" }

1 1

questions about maildir to mdbox migration
by Dave McGuire 28 Sep '17

28 Sep '17

Hi folks! I converted some of the spools on my mail server from maildir to mdbox. It went smoothly and the performance has been great. However, I have two questions regarding the process. First, I failed to notice the fairly low default mdbox_rotate_size, and I changed it in dovecot.conf after the migration. Is it possible to somehow make that change retroactive, to coalesce the couple thousand "storage/m.*" files that are anywhere from a few hundred bytes up to a few megabytes, into a smaller number of larger files, without re-doing the migration? Second, I'd like to delete the old maildir files and directories to reclaim the space in the filesystem, but it's not clear to me which ones I can safely delete, i.e. ones upon which my new mdbox-based spools no longer depend. I know maildir uses new, tmp, and cur directories, but in the spool directory there are also files like dovecot-uidlist, dovecot-uidvalidity, dovecot.index.cache, dovecot.index.log, dovecot.mailbox.log, as well as separate directories for each of my mail folders containing their own new, tmp, and cur directories and their own dovecot* files. Can someone offer some guidance regarding which ones I can safely remove? Thanks, -Dave -- Dave McGuire, AK4HZ New Kensington, PA

2 2

Panic
by Matthijs Kooijman 28 Sep '17

28 Sep '17

(I'm not subscribed, please keep me CC'd) Hi, since a few months I'm experiencing repeated dovecot crashes. I mostly see them in the logs, I haven't had any users complain, so I haven't looked closely until now. I was originally using an older version fro mDebian and hoped that a newer future version would already fix this, but I just upgraded to version 1:2.2.32-2 from Debian testing, and the problem persists. This crash sometimes happens occasionally, but often happens repeatedly for some time (presumably a client keeps retrying) but can also be gone for hours or days (presumably the client no longer connects). I haven't been able to definitively figure out what client causes this, but based on the user ID I suspect it is Mail.app on OSX. I managed to get a core dump, which gave me the stacktrace below. I've also attached my config file. If any other details are relevant, just ask and I can squeeze them out of the core dump (but I'd rather not publicly share the file). I can probably set up rawlog to capture the actual IMAP session if needed, though I have no time for that right now. Gr. Matthijs 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007fc5eed9c3fa in __GI_abort () at abort.c:89 #2 0x00007fc5ef1a609a in default_fatal_finish (type=LOG_TYPE_PANIC, status=status@entry=0) at failures.c:201 #3 0x00007fc5ef1a619a in i_internal_fatal_handler (ctx=0x7ffc553b38d0, format=<optimized out>, args=<optimized out>) at failures.c:670 #4 0x00007fc5ef136cf8 in i_panic (format=format@entry=0x7fc5ef52e880 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 #5 0x00007fc5ef47f998 in mailbox_copy_int (_ctx=_ctx@entry=0x7ffc553b3ab0, mail=<optimized out>) at mail-storage.c:2473 #6 0x00007fc5ef47f9ff in mailbox_copy (_ctx=_ctx@entry=0x7ffc553b3ab0, mail=<optimized out>) at mail-storage.c:2489 #7 0x000055bae72a2f5f in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7ffc553b3a88, t=<optimized out>, move=<optimized out>, cmd=0x55bae816d5e8) at cmd-copy.c:72 #8 cmd_copy_full (cmd=0x55bae816d5e8, move=false) at cmd-copy.c:139 #9 0x000055bae72afd9c in command_exec (cmd=cmd@entry=0x55bae816d5e8) at imap-commands.c:200 #10 0x000055bae72ae322 in client_command_input (cmd=<optimized out>, cmd@entry=0x55bae816d5e8) at imap-client.c:1088 #11 0x000055bae72ae3bc in client_command_input (cmd=<optimized out>) at imap-client.c:1148 #12 0x000055bae72ae77d in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55bae816bcf8) at imap-client.c:1190 #13 client_handle_input (client=0x55bae816bcf8) at imap-client.c:1202 #14 0x000055bae72aeccc in client_input (client=0x55bae816bcf8) at imap-client.c:1249 #15 0x00007fc5ef1bb742 in io_loop_call_io (io=0x55bae816d4e0) at ioloop.c:599 #16 0x00007fc5ef1bcd8e in io_loop_handler_run_internal (ioloop=ioloop@entry=0x55bae8153990) at ioloop-epoll.c:223 #17 0x00007fc5ef1bb7d6 in io_loop_handler_run (ioloop=ioloop@entry=0x55bae8153990) at ioloop.c:648 #18 0x00007fc5ef1bb988 in io_loop_run (ioloop=0x55bae8153990) at ioloop.c:623 #19 0x00007fc5ef141353 in master_service_run (service=0x55bae8153830, callback=<optimized out>) at master-service.c:666 #20 0x000055bae72a1139 in main (argc=<optimized out>, argv=<optimized out>) at main.c:491 (gdb) up 9 #9 0x000055bae72afd9c in command_exec (cmd=cmd@entry=0x55bae816d5e8) at imap-commands.c:200 200 finished = cmd->func(cmd); (gdb) p *cmd $14 = {prev = 0x0, next = 0x0, client = 0x55bae816bcf8, pool = 0x55bae816d5c0, tag = 0x55bae816d750 "7.119309", name = 0x55bae816d760 "UID COPY", args = 0x55bae816d770 "20004 INBOX", human_args = 0x55bae816d780 "20004 INBOX", cmd_flags = COMMAND_FLAG_BREAKS_SEQS, tagline_reply = 0x0, func = 0x55bae72a33e0 <cmd_copy>, context = 0x0, module_contexts = {arr = {buffer = 0x55bae816d6e8, element_size = 8}, v = 0x55bae816d6e8, v_modifiable = 0x55bae816d6e8}, parser = 0x55bae8157aa0, state = CLIENT_COMMAND_STATE_WAIT_INPUT, stats = {start_time = {tv_sec = 1506507612, tv_usec = 777163}, last_run_timeval = {tv_sec = 1506507612, tv_usec = 777163}, start_ioloop_wait_usecs = 46340, running_usecs = 0, lock_wait_usecs = 0, bytes_in = 0, bytes_out = 0}, stats_start = {timeval = { tv_sec = 1506507612, tv_usec = 777241}, lock_wait_usecs = 0, bytes_in = 91, bytes_out = 852}, sync = 0x0, uid = 1, cancel = 0, param_error = 0, search_save_result = 0, search_save_result_used = 0, temp_executed = 0, tagline_sent = 0, executing = 1} (gdb) down #8 cmd_copy_full (cmd=0x55bae816d5e8, move=false) at cmd-copy.c:139 139 ret = fetch_and_copy(cmd, move, t, &src_trans, search_args, (gdb) #7 0x000055bae72a2f5f in fetch_and_copy (copy_count_r=<synthetic pointer>, src_uidset_r=<synthetic pointer>, search_args=<optimized out>, src_trans_r=0x7ffc553b3a88, t=<optimized out>, move=<optimized out>, cmd=0x55bae816d5e8) at cmd-copy.c:72 72 if (mailbox_copy(&save_ctx, mail) < 0) (gdb) #6 0x00007fc5ef47f9ff in mailbox_copy (_ctx=_ctx@entry=0x7ffc553b3ab0, mail=<optimized out>) at mail-storage.c:2489 2489 return mailbox_copy_int(_ctx, mail); (gdb) #5 0x00007fc5ef47f998 in mailbox_copy_int (_ctx=_ctx@entry=0x7ffc553b3ab0, mail=<optimized out>) at mail-storage.c:2473 2473 i_assert(!ctx->unfinished); (gdb) #4 0x00007fc5ef136cf8 in i_panic (format=format@entry=0x7fc5ef52e880 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 275 fatal_handler(&ctx, format, args);

3 7

Dovecot and Self-signed issue
by Michael A. Peters 28 Sep '17

28 Sep '17

I use dovecot on several servers. One of them uses a self-signed cert, it's just me. It worked fine until yesterday when I upgraded my desktop (NOT the server) to CentOS 7.4 Now thunderbird complains when it starts up, and won't let me confirm the security exception. On the server the following error occurs in the log: Sep 25 20:17:49 librelamp dovecot: imap-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=2600:1010:b064:f260:e83e:562d:2316:18df, lip=2600:3c01::f03c:91ff:fee4:310c, TLS handshaking: SSL_accept() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48, session=<u7agQAlasK8mABAQsGTyYOg+Vi0jFhjf> I believe this is a client issue, as it worked just fine in CentOS 7.3 client, but I am hoping this has been seen and fixed before.

3 6

dsync: assert panic on mbox // no INBOX sync on hybrid mbox/maildir
by David Gardner 27 Sep '17

27 Sep '17

Hi, This (hopefully!) might be a case of user error, as I'm new to dovecot... Alternatively it might be some confusion in an index, since I've been mucking about trying different things in my configuration files and had frequent crashes. I'm trying to sync 2 (LAN) hosts, ideally to have full automatic replication happening. They both have hybrid mbox INBOX and maildir++ folders, using namespaces, as suggested in the wiki. I might have deviated the config from the wiki version, in trying to get things working. Both hosts communicate perfectly with an external IMAP client (thunderbird). Replication is working, but only partially - it is syncing the folders but not INBOX. I've thus been trying various things to try to get the INBOX to sync. The crash is only happening one of the 2 hosts which I've named crashhost for this report. The process crashes no matter which host starts the sync. With a second user, the crash happens on the other host crashhost: Ubuntu 16.04, x86_64, laptop, Dovecot 2.2.22, and Dovecot 2.2.32 (from debian sid package, locally compiled) otherhost: Ubuntu 16.04, x86_64, desktop, Dovecot 2.2.22, and Dovecot 2.2.32 (from debian sid package, locally compiled) RESULTS: 1) doveadm sync -f -u david tcp:otherhost synchronises the folders, but does not touch INBOXes. 2) doveadm sync -f -m INBOX -u david tcp:otherhost Does not touch INBOXes 3) doveadm sync -n '#mbox/' -f -u david tcp:otherhost dsync-local(david): Panic: file mbox-lock.c: line 800 (mbox_lock): assertion failed: (lock_type == F_RDLCK || mbox->mbox_lock_type != F_RDLCK) I added some debug code, the assertion failure is being triggered by a call to mbox-lock() with: locktype=F_WRCLK mbox->mbox_lock_type=F_RDLCK, mbox->external_transactions=1 The config files on the 2 hosts only differ by the hostnames they're supposed to be syncing to. $ dovecot -n # 2.2.32 (dfbe293d4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.20 (7cd71ba) # OS: Linux 4.10.0-35-generic x86_64 Ubuntu 16.04.3 LTS doveadm_password = # hidden, use -P to show it doveadm_port = 12346 imapc_host = crashhost imapc_password = # hidden, use -P to show it imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = no imapc_user = %u mail_location = maildir:~/mail mail_plugins = " notify replication" mail_privileged_group = mail namespace { inbox = no list = yes location = prefix = separator = / subscriptions = yes } namespace inbox { hidden = yes inbox = yes list = no location = mbox:~/nomail:INBOX=/var/mail/%u:INDEX=/var/spool/mailindex/%u mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = "#mbox/" separator = / type = private } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcp:otherhost:12346 } protocols = " imap" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service config { unix_listener config { user = vmail } } service doveadm { inet_listener { port = 12346 } user = root } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl = required ssl_cert = </etc/dovecot/dovecot.pem ssl_client_ca_dir = /etc/ssl ssl_key = # hidden, use -P to show it userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } gdb) bt #0 0x00007f37f426e428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x00007f37f427002a in __GI_abort () at abort.c:89 #2 0x00007f37f469f7aa in ?? () from /usr/lib/dovecot/libdovecot.so.0 #3 0x00007f37f469f829 in default_fatal_handler () from /usr/lib/dovecot/libdovecot.so.0 #4 0x00007f37f46302a1 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 #5 0x00007f37f49ac235 in mbox_lock () from /usr/lib/dovecot/libdovecot-storage.so.0 #6 0x00007f37f49adb11 in mbox_save_begin () from /usr/lib/dovecot/libdovecot-storage.so.0 #7 0x00007f37f49769a1 in mailbox_save_begin () from /usr/lib/dovecot/libdovecot-storage.so.0 #8 0x00007f37f496a2e8 in mail_storage_copy () from /usr/lib/dovecot/libdovecot-storage.so.0 #9 0x00007f37f3e332a5 in ?? () from /usr/lib/dovecot/modules/lib15_notify_plugin.so #10 0x00007f37f4976b73 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #11 0x00007f37f4976d89 in mailbox_move () from /usr/lib/dovecot/libdovecot-storage.so.0 #12 0x000055b45f0da448 in ?? () #13 0x000055b45f0db35a in ?? () #14 0x000055b45f0ddbc0 in dsync_mailbox_import_changes_finish () #15 0x000055b45f0d8645 in dsync_brain_sync_mails () #16 0x000055b45f0d3df8 in dsync_brain_run () #17 0x000055b45f0d4140 in ?? () #18 0x000055b45f0ea3ff in ?? () #19 0x00007f37f46b4c82 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 #20 0x00007f37f46b6309 in io_loop_handler_run_internal () from /usr/lib/dovecot/libdovecot.so.0 #21 0x00007f37f46b4d1c in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 #22 0x00007f37f46b4ec8 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 #23 0x000055b45f0bb4c2 in ?? () #24 0x000055b45f0bd91c in ?? () #25 0x000055b45f0be5aa in ?? () #26 0x000055b45f0beeda in doveadm_mail_try_run () #27 0x000055b45f0ae47c in main ()

3 3