- dovecot - dovecot.org

director stuck in inifite loop on 2.2.35
by Jan-Pieter Cornet 26 Apr '18

26 Apr '18

Occasionally, all of our directors get stuck in an endless loop. Likely triggered by some network event, but I haven't been able to isolate exactly what triggers it, and am unable to reproduce at the moment. When this happens, it makes the dovecot/director process on all directors unresponsive, making login impossible. Restarting dovecot "fixes" it. Here's the info collected from the stuck process. I hope that a dovecot dev could use this to either fix a bug in the async IO code or point to a possible external source that could have triggered this. Would it help to compile dovecot with -g and run a version with debugging info, to be able to inspect more when the crash happens again? "strace" showed this endless loop: 09:27:52.837960 epoll_ctl(14, EPOLL_CTL_DEL, 213, 0x7ffe8e642cdc) = 0 09:27:52.837993 epoll_ctl(14, EPOLL_CTL_ADD, 213, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=3317538000, u64=94320799358160}}) = 0 09:27:52.838035 epoll_ctl(14, EPOLL_CTL_DEL, 213, 0x7ffe8e642cdc) = 0 09:27:52.838070 epoll_ctl(14, EPOLL_CTL_ADD, 213, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=3317538000, u64=94320799358160}}) = 0 "ltrace" showed the same loop as: 10:38:01.773867 timeout_add(0x7530, 0x55c8c351df75, 965, 0x55c8c3515a00) = 0x55c8c5c658b0 10:38:01.774060 timeout_remove(0x55c8c5acf7a8, 0, 0xffffffff, 0) = 0 10:38:01.774239 io_add(213, 1, 0x55c8c351df75, 1070) = 0x55c8c5c658b0 10:38:01.774429 io_set_pending(0x55c8c5c658b0, 1, 0x55c8c54b8aa0, 0x7fb4ab11b06a) = 0x55c8c54b8aa0 10:38:01.774598 p_strarray_dup(0x7fb4ab703660, 0x55c8c5786e00, 5, 0x55c8c3723740) = 0x55c8c56d7530 10:38:01.774771 io_remove(0x55c8c5acf788, 0x55c8c3516d60, 5, 48) = 0 10:38:01.774957 timeout_add(0x7530, 0x55c8c351df75, 965, 0x55c8c3515a00) = 0x55c8c5c658b0 10:38:01.775129 timeout_remove(0x55c8c5acf7a8, 0, 0xffffffff, 0) = 0 10:38:01.775301 io_add(213, 1, 0x55c8c351df75, 1070) = 0x55c8c5c658b0 10:38:01.775492 io_set_pending(0x55c8c5c658b0, 1, 0x55c8c54b8aa0, 0x7fb4ab11b06a) = 0x55c8c54b8aa0 10:38:01.775662 p_strarray_dup(0x7fb4ab703660, 0x55c8c56d7530, 5, 0x55c8c3723740) = 0x55c8c5786e00 10:38:01.775833 io_remove(0x55c8c5acf788, 0x55c8c3516d60, 5, 48) = 0 That fd 213 is the next-highest FD opened, and is bound to /var/run/dovecot/director-admin. In fact the last few FDs are all connections to director-admin. Last few lines of "lsof" say: director 16109 dovecot 207u unix 0xffff940d13c93800 0t0 1106341075 type=STREAM director 16109 dovecot 208u unix 0xffff9409f5280000 0t0 1106337217 /var/run/dovecot/director-userdb type=STREAM director 16109 dovecot 209u unix 0xffff940d580fa000 0t0 1106337218 type=STREAM director 16109 dovecot 210u unix 0xffff940ac0f61000 0t0 1106343439 /var/run/dovecot/director-userdb type=STREAM director 16109 dovecot 211u unix 0xffff940bed105400 0t0 1106343440 type=STREAM director 16109 dovecot 212u unix 0xffff940b76d31c00 0t0 1106342408 /var/run/dovecot/director-admin type=STREAM director 16109 dovecot 213u unix 0xffff940ab241d000 0t0 1106340510 /var/run/dovecot/director-admin type=STREAM director 16109 dovecot 214u unix 0xffff9409c317ec00 0t0 1106340511 /var/run/dovecot/director-admin type=STREAM I attached with "gdb" and did "bt" at two different moments, resulting in: (gdb) bt #0 0x00007fb4ab11b06a in epoll_ctl () from target:/lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fb4ab48964d in io_loop_handle_add () from target:/usr/lib/dovecot/libdovecot.so.0 #2 0x00007fb4ab4876b9 in ?? () from target:/usr/lib/dovecot/libdovecot.so.0 #3 0x00007fb4ab48776d in io_add () from target:/usr/lib/dovecot/libdovecot.so.0 #4 0x000055c8c35179d3 in doveadm_connections_ring_synced () #5 0x00007fb4ab488166 in io_loop_handle_timeouts () from target:/usr/lib/dovecot/libdovecot.so.0 #6 0x00007fb4ab489963 in io_loop_handler_run_internal () from target:/usr/lib/dovecot/libdovecot.so.0 #7 0x00007fb4ab48839c in io_loop_handler_run () from target:/usr/lib/dovecot/libdovecot.so.0 #8 0x00007fb4ab488548 in io_loop_run () from target:/usr/lib/dovecot/libdovecot.so.0 #9 0x00007fb4ab40c0a3 in master_service_run () from target:/usr/lib/dovecot/libdovecot.so.0 #10 0x000055c8c350b46d in main () (gdb) c Continuing. ^C Program received signal SIGINT, Interrupt. 0x00007fb4ab11b06a in epoll_ctl () from target:/lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x00007fb4ab11b06a in epoll_ctl () from target:/lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fb4ab489873 in io_loop_handle_remove () from target:/usr/lib/dovecot/libdovecot.so.0 #2 0x00007fb4ab487adf in ?? () from target:/usr/lib/dovecot/libdovecot.so.0 #3 0x000055c8c35158b7 in ?? () #4 0x000055c8c3516b4a in ?? () #5 0x000055c8c3516d78 in ?? () #6 0x000055c8c35179e4 in doveadm_connections_ring_synced () #7 0x00007fb4ab488166 in io_loop_handle_timeouts () from target:/usr/lib/dovecot/libdovecot.so.0 #8 0x00007fb4ab489963 in io_loop_handler_run_internal () from target:/usr/lib/dovecot/libdovecot.so.0 #9 0x00007fb4ab48839c in io_loop_handler_run () from target:/usr/lib/dovecot/libdovecot.so.0 #10 0x00007fb4ab488548 in io_loop_run () from target:/usr/lib/dovecot/libdovecot.so.0 #11 0x00007fb4ab40c0a3 in master_service_run () from target:/usr/lib/dovecot/libdovecot.so.0 #12 0x000055c8c350b46d in main () For completeness, doveconf -n output attached. -- Jan-Pieter Cornet <johnpc(a)xs4all.net> Systeembeheer XS4ALL Internet bv www.xs4all.nl

2 1

Alt Storage and dsync
by Matt Bryant 26 Apr '18

26 Apr '18

Stupid question but if dovecot mdbox is configured using alt path storage on a bi directional dysnc replicated pair of servers ... will the doveadm altmove need to be run on both halves of the cluster ??? rgds Matt -- <https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campai…>

1 0

replication and .dovecot.lda-dupes
by Patrick Cernko 25 Apr '18

25 Apr '18

Hi list, this question was already posted a few years ago (https://www.dovecot.org/list/dovecot/2014-November/098585.html) I already asked the original queriest and he told me, that he never got an solution or workaround but it was not important enough for him. When using replication in conjunction with sieve vacations, the .dovecot.lda-dupes file is not synced with the other server. So when delivering to both servers (round-robin or randomized), senders might get more vacation mails than configured as the other server does not know, that the first one already sent a vacation message. Is this a bug or intentional? If it is a bug, I hereby ask for a fix, please. We are using Dovecot version 2.2.27 on Debian/stretch. This is dovecot -n (hostnames anonymized): # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.9.76.1.amd64-smp x86_64 Debian 9.3 auth_verbose = yes default_vsz_limit = 2 G doveadm_password = # hidden, use -P to show it doveadm_port = 12345 listen = * login_log_format_elements = pid=%p user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_attachment_dir = /IMAP/mail/attachments mail_attachment_fs = sis-queue /IMAP/mail/attachments/queue:posix mail_home = /IMAP/mail/mailboxes/%u mail_location = mdbox:~/mdbox mail_log_prefix = "%s(%u[%p]): " mail_max_userip_connections = 0 mail_plugins = " notify replication zlib fts fts_squat" maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/ldap.conf driver = ldap } plugin { fts = squat fts_autoindex = yes fts_squat = partial=4 full=10 mail_replica = tcp:other-server sieve = file:~/sieve;active=~/.dovecot.sieve zlib_save = gz zlib_save_level = 3 } postmaster_address = <> protocols = " imap lmtp sieve" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0666 } } service anvil { client_limit = 2250 } service auth { client_limit = 2447 } service doveadm { inet_listener doveadm-server { port = 12345 } } service imap-login { inet_listener imap { port = 0 } process_limit = 2047 } service imap { process_limit = 2047 } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 0 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_cert = </etc/ssl/certificate.pem ssl_key = # hidden, use -P to show it userdb { args = /etc/dovecot/userdb.overrides driver = passwd-file } userdb { args = /etc/dovecot/ldap.conf driver = ldap } verbose_proctitle = yes protocol lmtp { auth_username_format = %n mail_plugins = " notify replication zlib fts fts_squat sieve" } protocol lda { mail_plugins = " notify replication zlib fts fts_squat sieve" } And this is /etc/dovecot/ldap.conf (hostnames anonymized): uris = ldaps://ldap1/ ldaps://ldap2/ base = dc=domain user_filter = (&(objectClass=posixAccount)(uid=%u)) user_attrs = \ =user=%{ldap:uid}, \ =uid=%{ldap:uidNumber}, \ =gid=%{ldap:gidNumber} pass_filter = (&(objectClass=posixAccount)(uid=%u)) iterate_filter = (&(objectClass=posixAccount)(istMailHomeServer=servername)) auth_bind = yes /etc/dovecot/userdb.override is currently an empty file. P.S.: Although I do not think, that this has something to do with our problem, I should mention that we run SIS with system uids. This needs some permission and posixacl tweaks on /IMAP/mail/attachments to work: # force all files created in this dir recursively) to have mode 0666 setfacl -d -m group::rwx -m o:rwx /IMAP/mail/attachments (you have to enable posixacls for the corresponding filesystem!) # trick dovecot to inherit parent dir's permissions (recursively) chmod 2777 /IMAP/mail/attachments Feel free to add these tweaks to the SIS documentation. ;-) Best Regards, -- Patrick Cernko <pcernko(a)mpi-klsb.mpg.de> +49 681 9325 5815 Joint Administration: Information Services and Technology Max-Planck-Institute fuer Informatik & Softwaresysteme

2 2

dovecot HA setup across large geographic distances
by Sophie Loewenthal 24 Apr '18

24 Apr '18

Hi, I have a working postfix/dovecot config on Debian located in Europe[EU]. I would like to provide either an HA/load balanced setup with a second site in Australia[AU] because of users located between EU/Asia/AU. The site to site traffic would be tunneled through either SSH or a VPN * Has this been done with Doevcot over such great distances? * How can the delay between mailstore synchonisation be handled? ( Currently looking for an opensource product like Netex's HyperIP) I'd be grateful for your thoughts. Best wishes, Sophie Sent from a phone, thus brief.

1 1

Pass UniqueID to sieve scripts
by Peter Blok 24 Apr '18

24 Apr '18

Hi, If a user moves a mail to a Junk folder, this e-mail is going thru imap_sieve and ends up in sa-learn. I wrote some tools to immediately propagate that to the other node of the cluster as well as locally. Everything works fine, but sometimes I see a storm of requests hogging the CPU and I would like to collect some debugging information, Preferably I would like the permanent unique id of the mail in the maildir, but am looking for some kind of unique id that will help me identify if an e-mail somehow passes the sieve filter multiple times. Basically I have changed the pipe to sa-learn to my proxy that will send it across to the other node as well as call sa-learn locally. So something I can get in a sieve script and pass along. Peter

2 1

imap-login segfaulting on 2.3.1
by Grant Keller 24 Apr '18

24 Apr '18

Hello, I have a new director ring I am setting up on centos 7 with dovecot 2.3.1. I haven't been able to replecate this in testing, but as soon as I start pushing production traffic to the new ring I see dozens of these in the logs: Apr 18 00:34:00 d.director.imapd.sonic.net kernel: imap-login[163107]: segfault at 10 ip 00007ff625698dd5sp 00007ffe4b77bb28 error 4 in libdovecot.so.0.0.0[7ff6255bf000+16e000] My config: # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.21.1.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) # Hostname: c.director.imapd.sonic.net auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 20000 director_mail_servers = 64.142.111.75 64.142.111.43 64.142.111.76 64.142.111.39 64.142.111.81 director_servers = c.director.imapd.sonic.net d.director.imapd.sonic.net disable_plaintext_auth = no doveadm_port = 1842 login_trusted_networks = 64.142.18.0/24 64.142.105.2 64.142.111.67 64.142.111.36 64.142.105.3 64.142.111.35 64.142.111.68 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 40103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_cert = </etc/dovecot/ssl/imap.sonic.net.pem ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it syslog_facility = local0 protocol imap { imap_max_line_length = 128 k ssl_cert = </etc/dovecot/ssl/imap.sonic.net.pem ssl_key = # hidden, use -P to show it } protocol pop3 { ssl_cert = </etc/dovecot/ssl/pop.sonic.net.pem ssl_key = # hidden, use -P to show it } backtrace: Stack trace of thread 56084: #0 0x00007fe529be2dd5 i_stream_get_root_io (libdovecot.so.0) #1 0x00007fe529be2e39 i_stream_set_input_pending (libdovecot.so.0) #2 0x00007fe527415a59 openssl_iostream_bio_sync (libssl_iostream_openssl.so) #3 0x00007fe527415c2a openssl_iostream_more (libssl_iostream_openssl.so) #4 0x00007fe527415c6b openssl_iostream_destroy (libssl_iostream_openssl.so) #5 0x00007fe529e8be72 client_unref (libdovecot-login.so.0) #6 0x00007fe529e90441 login_proxy_free_full (libdovecot-login.so.0) #7 0x00007fe529e90e07 login_proxy_finished (libdovecot-login.so.0) #8 0x00007fe529becfb5 io_loop_call_io (libdovecot.so.0) #9 0x00007fe529bee89f io_loop_handler_run_internal (libdovecot.so.0) #10 0x00007fe529bed0b2 io_loop_handler_run (libdovecot.so.0) #11 0x00007fe529bed2d8 io_loop_run (libdovecot.so.0) #12 0x00007fe529b69673 master_service_run (libdovecot.so.0) #13 0x00007fe529e93532 login_binary_run (libdovecot-login.so.0) #14 0x00007fe529767c05 __libc_start_main (libc.so.6) #15 0x00005567b256050a _start (imap-login) Grant Keller System Operations grant.keller(a)sonic.com

2 2

Re: Merging mailboxes with doveadm
by Carsten Schmitz 24 Apr '18

24 Apr '18

Hello Aki, yes, that's it - thank you. For reference this works: sudo doveadm import -u destination(a)company.net maildir:/var/vmail/company.net/some.name/Maildir some_name ALL Thanks again Carsten On 23.04.2018 15:06, Aki Tuomi wrote: > The command you are looking for is doveadm import > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Carsten Schmitz <carsten.schmitz(a)limesurvey.org> > Date: 23/04/2018 15:46 (GMT+02:00) > To: dovecot(a)dovecot.org > Subject: Merging mailboxes with doveadm > > Hello, > > I am trying to merge(=copy) all mails of all user mailboxes into one > mailbox for one-time archival purposes. > > The command I am using is > > sudo doveadm -v copy -A archive(a)somedomain.net ALL > > > The error I get for every mailbox is: > > doveadm(someuser1(a)somedomain.net): Error: Can't open mailbox > 'a(a)somedomain.net': Mailbox doesn't exist: archive(a)somedomain.net > > doveadm(someuser2(a)somedomain.net): Error: Can't open mailbox > 'a(a)somedomain.net': Mailbox doesn't exist: archive(a)somedomain.net > > [etc...] > > I don't exactly understand why. I think I am don't understand the > 'destination' parameter correctly, so somebody can enlighten me, please? > > Thank you > > > Carsten > > >

1 0

Sieve "redirect" changes envelope sender in 2.3. / pigeonhole 0.5
by Olaf Hopp 24 Apr '18

24 Apr '18

Hi (Stephan?), is it a new feature of dovecot 2.3 /pigeonhole 0.5 that a sieve "redirect" changes the envelope sender of a redirected mail or simply a bug ? A sends mail to B, B redirects to C C sees B (not A!) as envelope sender. It is not a problem if C gets the mail but if that mail bounces for various reasons it goes back to B and A will never know about this. I thick this is came with 2.3 / pigeonhole 0.5 ? # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.devel (61b47828) # OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final) Regards, Olaf -- Karlsruher Institut für Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Gebäude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp(a)kit.edu atis.informatik.kit.edu www.kit.edu KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.

4 10

dovecot vs. mutt: no full index sync on Maildir new/ mtime change
by Michael Büker 24 Apr '18

24 Apr '18

Hi, everyone! This is a follow-up to "Looks like a bug to me: Dovecot ignores Maildir/new timestamp" from Fredrik Roubert on 01.12.2015: https://www.dovecot.org/list/dovecot/2015-December/102585.html I've run into the same problem as Fredrik: When manipulating my Maildir locally with mutt, deleting a message from new/ doesn't cause a full update of the index. Therefore, IMAP clients still see the deleted message. I've read and understood Timo's reply saying that dovecot only performs a "partial sync" of the index when the mtime of new/, but not of cur/, changes. This makes perfect sense for performance reasons for most users: https://www.dovecot.org/list/dovecot/2015-December/102588.html I, however, would be willing to take the performance hit of a full index sync whenever the mtime of new/ changes. Therefore, I looked at the code and tried to implement a config option (maildir_fullsync_on_new_mtime_change) for this behavior. However, my understanding of src/lib-storage/index/maildir/maildir-sync.c was not good enough – I probably put the ctx->mbox->storage->set->maildir_fullsync_on_new_mtime_change check in the wrong place, and all my patch did was ruin the index ;) So, to summarize my question: I'd like dovecot to perform a full index sync when the mtime of a Maildir's new/ has changed. I'm willing to take the performance hit, because it would fix a problem I'm having with using mutt and dovecot together. Can this be done in principle by adding a config option check like ctx->mbox->storage->set->maildir_fullsync_on_new_mtime_change in the right place in src/lib-storage/index/maildir/maildir-sync.c? If so, where should it be put? Thanks for your time, Michael

2 1

smtp crashing with "out of memory" for ONE particular user
by Jacek Osiecki 24 Apr '18

24 Apr '18

Hi, I’m using dovecot 2.0.16 (I know, it’s oooold, but I have too many users to risk the upgrade while so many things have changed since) and virtual users based on MySQL. Today I had some troubles with user, who’s mailbox stopped working. No emails were delivered, with just following error line in EXIM mainlog: LMTP connection closed after end of data When I checked dovecot’s main log, and here’s what I found: Apr 23 14:06:17 lmtp(9963, jmk(a)silvercube.pl) Fatal: block_alloc(67108864): Out of memory Apr 23 14:06:17 lmtp(9963, jmk(a)silvercube.pl) Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x3dbda) [0x7f92f3193bda] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3dc1e) [0x7f92f3193c1e] -> /usr /lib64/dovecot/libdovecot.so.0(i_error+0) [0x7f92f316d8cd] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4cc45) [0x7f92f31a2c45] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4cc7f) [0x7f92f31a2c7f] -> /usr/lib64/ dovecot/libdovecot.so.0(p_strdup+0x3e) [0x7f92f31ae08e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_uidlist_sync_next_uid+0xf9) [0x7f92f3653e69] -> /usr/lib64/dovecot/libdovecot-storage.so.0(ma ildir_uidlist_sync_next+0x12) [0x7f92f3654232] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x6bd3a) [0x7f92f3651d3a] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x6c7d3) [0x7f92f36527d3] -> /usr/li b64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0x171) [0x7f92f3652d91] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x7f92f36173a1] -> /usr/lib64/dovecot/libdovec ot-storage.so.0(mailbox_sync+0x27) [0x7f92f36174b7] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(quota_count+0x132) [0x7f92f1f34fe2] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(+0x8430) [0 x7f92f1f36430] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(+0x863d) [0x7f92f1f3663d] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(quota_get_resource+0x72) [0x7f92f1f344c2] -> /usr/lib64/do vecot/plugins/lib10_quota_plugin.so(quota_test_alloc+0x175) [0x7f92f1f34d45] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(+0xb0ba) [0x7f92f1f390ba] -> /usr/lib64/dovecot/libdovecot-storage.so.0(ma ilbox_save_begin+0x46) [0x7f92f3617f56] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_storage_copy+0x7e) [0x7f92f361100e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_copy+0x51) [0x7f92f364 a8a1] -> /usr/lib64/dovecot/plugins/lib10_quota_plugin.so(+0xaefa) [0x7f92f1f38efa] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_copy+0x5a) [0x7f92f3617eba] -> /usr/lib64/dovecot/libdovecot-siev e.so.0(+0x36ed6) [0x7f92f1ae9ed6] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(+0x2d83c) [0x7f92f1ae083c] -> /usr/lib64/dovecot/libdovecot-sieve.so.0(sieve_result_execute+0x2fd) [0x7f92f1ae1a0d] -> /usr/l ib64/dovecot/libdovecot-sieve.so.0(sieve_execute+0x7d) [0x7f92f1af0a9d] Apr 23 14:06:17 master: Error: service(lmtp): child 9963 returned error 83 (Out of memory (vsz_limit=256 MB, you may need to increase it)) What was interesting - it happenned only with this particular mailbox. It does not has large sieve file (actually, other mailboxes have muuuch bigger ones, actually nothing strange with the account - but it was failing. I have increased vsz_limit to 384MB - didn’t help. Finally, 512MB helped. But it is quite strange to me… Why one particular mailbox, what could be causing it? Greetings, — Jacek Osiecki josiecki(a)silvercube.pl

2 1

question about using cram-md5 login passwords
by Fabian A. Santiago 23 Apr '18

23 Apr '18

hello dovecot community, question; if my user database and dovecot installation is currently setup to use plain login passwords, and i want to convert to cram-md5, after i configure dovecot accordingly and reset passwords into cram-md5, if anyone uses plain login method again in the future, will it still work? or must they always from this point on use encrypted passwords? Thanks. -- Thanks, Fabian S. OpenPGP: 0x643082042DC83E6D94B86C405E3DAA18A1C22D8F (new key) 0x3C3FA072ACCB7AC5DB0F723455502B0EEB9070FC (to be retired / revoked)

2 2

lost connection while sending end of data error
by Christos Chatzaras 23 Apr '18

23 Apr '18

I use dovecot-lmtp and postfix. When I send e-mail to 2 recipients (or more) at the same time and if one of them is over quota (or under quota and the message I send is bigger than his free space) mailq shows: -Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient------- 20B03336F22 26099 Thu Apr 19 18:02:47 support(a)example.com (lost connection with server25.example.org[private/dovecot-lmtp] while sending end of data -- message may be sent more than once) user1(a)example.com user2(a)example.com E-mails sent from the same domain on same server so it's a local delivery. If I send the e-mail to the over quota user ( only him on To: ) then I get a bounce that says that user is over quota which is the correct behaviour. I think the problem started after I upgrade dovecot-2.2.35 to dovecot-2.3.1. Is this a known bug?

2 3

Merging mailboxes with doveadm
by Carsten Schmitz 23 Apr '18

23 Apr '18

Hello, I am trying to merge(=copy) all mails of all user mailboxes into one mailbox for one-time archival purposes. The command I am using is sudo doveadm -v copy -A archive(a)somedomain.net ALL The error I get for every mailbox is: doveadm(someuser1(a)somedomain.net): Error: Can't open mailbox 'a(a)somedomain.net': Mailbox doesn't exist: archive(a)somedomain.net doveadm(someuser2(a)somedomain.net): Error: Can't open mailbox 'a(a)somedomain.net': Mailbox doesn't exist: archive(a)somedomain.net [etc...] I don't exactly understand why. I think I am don't understand the 'destination' parameter correctly, so somebody can enlighten me, please? Thank you Carsten

2 1

2.3.1 -> "doveadm backup -A" not working
by spamvoll＠googlemail.com 21 Apr '18

21 Apr '18

Hi, I'm running 2.3.1 and trying to do a backup. Wiki and manpage showes that -A will backup all mailboxes. It seems that "doveadm -Dv backup -A mdbox:/mnt/BACKUP/DOVECOT/" is saving each mailbox to " /mnt/BACKUP/DOVECOT/" but after finishing one, its overwriting the data from the previous user. So it ends with only the mails from the last user processed. I tried %u but thats not recognized and used as String in the path. Hans

1 0

RE: Re: Filesystem level backups?
by DurgaPrasad - DatasoftComnet 20 Apr '18

20 Apr '18

Hello all, Does doveadm backup - backup the attachments as well when using SIS ? Regards Durga Prasad +919849111010 -----Original Message----- From: dovecot [mailto:dovecot-bounces@dovecot.org] On Behalf Of Marc Stürmer Sent: 09 April 2018 18:05 To: Steve Atkins Cc: dovecot(a)dovecot.org Subject: {Spam?} Re: Filesystem level backups? Am 2018-04-09 00:55, schrieb Steve Atkins: > Will a simple backup of all the files of an multi-dbox give me a > consistent state when I recover, or do I need to do something (e.g. > lock writes to the mailbox) while I run a backup? If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names. If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely. But just doing rsync on the fly as with Maildir is just begging on your side to run into problems sooner or later... -- Greetings, Marc --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

3 4

massive mail move
by David Mehler 20 Apr '18

20 Apr '18

Hello, I've got dovecot 2.3 going and a mailbox with a lot of system messages in it. What I'm wanting to do is massively move them to another mailbox. Is this doable with sieve? Thanks. Dave.

2 2

maildir vs dbox?
by David Mehler 20 Apr '18

20 Apr '18

Hello, I've got a Dovecot virtual email setup using maildir as the mailbox format. I am wondering from others who have chosen dbox a comparison between maildir and dbox or those who have migrated from one format to another. I am wondering if changing to dbox would be beneficial? Thanks. Dave.

4 3

Replication between servers with different mail_home path
by Arie Peterson 19 Apr '18

19 Apr '18

Hi, I have two dovecot servers that are configured to be replicated using dsync over ssh. This has been working well for a long time. Recently I changed the mail_home path on one of the servers, because of mail data migration. The old file system is still available, but has been made read-only for now – I plan to remove it once everything runs smoothly again. Now it seems replication is failing: the mail.err log file on both servers has lots of error messages saying that it tried to open a file on the old location and failed because it is read-only. Is this setup, with different mail_home paths on the two servers being replicated, even supported? If so, do I need to specify this explicitly in the configuration? I couldn't find relevant settings in the documentation.

1 0

Comparing Indexing Speed + ZFS question
by Qmail 18 Apr '18

18 Apr '18

Hi, have two quick questions: I am re-indexing some large mailboxes and I wanted to get some opinions on these speeds. I ran a random sort command on an unindexed mailbox in order to get it to index. Results, for a mailbox with 10GB Inbox, spread in 65 000 files: 3 UID SORT (ARRIVAL) US-ASCII OR OR OR FROM "dfgfgfgf" SUBJECT "dfgfgfgf" TO "dfgfgfgf" CC "dfgfgfgf" a03 select INBOX * SORT 3 OK Sort completed (2455.549 + 0.122 + 1.477 secs). Index cache file at the end was 31MB in size: 30756376 Apr 18 02:14 dovecot.index.cache Is that normal, slow or fast ? Second question is, how can I speed this up ? I store messages using Director over NFS , on top of ZFS. From what I've read it would be fairly safe to disable mail_fsync but I don't think that would help much with the indexing speed. Any other tips ?

2 2

Postfix sendmail cannot be called from Sieve redirect
by Ulrich Zehl 17 Apr '18

17 Apr '18

Sending (in my case: forwarding) messages from Sieve via sendmail does not work with version 2.3.1. I have narrowed it down to this simple test case: # dovecot -n # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 4.4.0-119-generic x86_64 Ubuntu 16.04.4 LTS # Hostname: tuxi.topfen.net first_valid_gid = 200 first_valid_uid = 200 mail_location = mbox:~/mail passdb { args = password=secret driver = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = lmtp service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } userdb { args = uid=200 gid=200 home=/home/vmail driver = static } protocol lmtp { mail_plugins = sieve } # cat /home/vmail/.dovecot.sieve redirect "test(a)example.invalid"; stop; $ swaks --protocol lmtp -s 127.0.0.1 -f ulrich(a)topfen.net -t ulrich(a)topfen.net === Trying 127.0.0.1:24... === Connected to 127.0.0.1. <- 220 tuxi.topfen.net Dovecot ready. -> LHLO tuxi.topfen.net <- 250-tuxi.topfen.net <- 250-8BITMIME <- 250-CHUNKING <- 250-ENHANCEDSTATUSCODES <- 250-PIPELINING <- 250 VRFY -> MAIL FROM:<ulrich(a)topfen.net> <- 250 2.1.0 OK -> RCPT TO:<ulrich(a)topfen.net> <- 250 2.1.5 OK -> DATA <- 354 OK -> Date: Tue, 17 Apr 2018 18:44:42 +0200 -> To: ulrich(a)topfen.net -> From: ulrich(a)topfen.net -> Subject: test Tue, 17 Apr 2018 18:44:42 +0200 -> X-Mailer: swaks v20130209.0 jetmore.org/john/code/swaks/ -> -> This is a test mailing -> -> . <** 451 4.3.0 <ulrich(a)topfen.net> Temporary internal error -> QUIT <- 221 2.0.0 Bye === Connection closed with remote host. $ tail -n 10 /var/log/mail.log Apr 17 18:45:36 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/421882.22643: Permission denied Apr 17 18:45:36 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/421882.22643: Permission denied Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: program `/usr/sbin/sendmail' was forcibly terminated with signal 15 Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: sieve: msgid=unspecified: failed to redirect message to <test(a)test.at>: Failed to execute sendmail (temporary failure) Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: sieve: Execution of script /home/vmail/.dovecot.sieve was aborted due to temporary failure (user logfile /home/vmail/.dovecot.sieve.log may reveal additional details) Apr 17 18:45:36 tuxi dovecot: lmtp(22639): Disconnect from 127.0.0.1: Client has quit the connection (state = READY) Apr 17 18:45:46 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/422091.22643: Permission denied Apr 17 18:45:46 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/422091.22643: Permission denied Apr 17 18:45:56 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/422227.22643: Permission denied Apr 17 18:45:56 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/422227.22643: Permission denied As far as I can tell, this is because Dovecot somehow does not allow suid programs to run (Postfix' postdrop(1), which is invoked by sendmail(1), requires set-gid to write the queue file). What can I do to change this (besides using SMTP from Dovecot directly)? I am using Ubuntu 16.04 and the Dovecot packages from the community repository: ~$ dpkg -l dovecot* | grep ^ii ii dovecot-core 2:2.3.1-1 amd64 secure POP3/IMAP server - core files ii dovecot-lmtpd 2:2.3.1-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-sieve 2:2.3.1-1 amd64 secure POP3/IMAP server - Sieve filters support Ulrich

2 2

dovecot-lda "-f" argment optional?
by Sebastian Arcus 17 Apr '18

17 Apr '18

I use dovecot-lda to deliver incoming email from Exim to Dovecot. After having some problems with email bounces with empty envelope sender not being delivered because of an empty -f argument, I discovered that Exim can pass email to Dovecot using dovecot-lda without using the -f argument at all. Would this have any adverse effects anywhere else? The docs don't even seem to specify if -f is optional or not - but I guess it is - as it seems to be working without it. Does anybody know what is its role? I'd like to understand things a bit before I just start removing it permanently from my setup.

1 0

[bug] Auth cache, proxy and ITERINDEX
by azurit＠pobox.sk 17 Apr '18

17 Apr '18

Hi, today we have exceprienced really major difficulties with our proxy backend servers. Everything started after enabling auth cache: auth_cache_size = 100M auth_cache_verify_password_with_worker = yes auth_cache_ttl = 1 hour auth_cache_negative_ttl = 1 hour Soon after we start receiving lots of calls from our customers about 'missing all e-mail messages'. This was in errors logs on master server: Apr 16 14:37:10 server00 dovecot: imap(<censored_email_address>): Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/var/mail/vhosts/<censored_domain>/<censored_name>/home/dovecot.autoexpunge.lock) failed: safe_mkstemp(/var/mail/vhosts/<censored_domain>/<censored_name>/home/dovecot.autoexpunge.lock) failed: No such file or directory Looks like that all users, who are placed on proxy backends, were logged correctly BUT not proxied to the right server so they saw empty mailboxes (Dovecot also created some directories on master server). What is worse, their email software deleted all local emails so they are now downloading everything from scratch (100Gs of mails so it will take some time). After auth cache was disabled, things started to go to normal EXCEPT users were reporting they don't see all folders - which was true, only few on them were visible (INBOX, trash, sent etc.). We tried everything to make them visible again, for example deleting indexes, but nothing helped. Finally, removing option ITERINDEX from mail_location helped. Can anyone explain what happened? Thank you azurIt

2 2

spamc scripts in IMAPSieve docs.
by Reio Remma 17 Apr '18

17 Apr '18

Hello! I noticed SpamAssassin *spamc* usage has entered the documentation at https://wiki2.dovecot.org/HowTo/AntispamWithSieve I'm wondering if the -C (report) option in sa-learn-ham.sh should use revoke instead of report for --ham messages? I started using imapsieve with spamc myself just a few weeks ago, but I haven't used the reporting ability yet. Reio

2 4

File system permissions - setgid bit and Netapp NFS volumes
by Shaun Johnson 17 Apr '18

17 Apr '18

Greetings Dovecot List, I have a bit of an edge case I am trying to resolve. I am currently using dovecot on Ubuntu 14.04 - Ubuntu package version: 1:2.2.9-1ubuntu2.3 I have attached the output of doveconf -n to this email - but to describe the configuration in a nutshell: my server is configured to use Maildir storage I do not use dovecot delivery service (there is a separate program in my system delivering messages) I am using a 'checkpassword' mechanism for authentication My mail storage is mounted via NFSv3 from a Netapp Filer FAS3050 configured to set anonuid=89, read write access, and root access. Dovecot has been configured with: mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mmap_disable = yes lock_method = fcntl In order to function with the other components of my system, the Mail Storage directories are expected to have permissions: drwx--S--- uid 89: gid 89 The oddity I am encountering came up recently when I set up a job to monitor and correct directory permissions. I found after observing the system in action that overtime, the ~/Maildir directories for all active mailboxes would mysteriously lose the setgid bit. Now, while the system still functioned, it was something unexpected so I investigated further by turning on auditd to monitor what was changing the permissions on the path. After tracing through the results, I found an entry indicating that the 'imap' process of dovecot was performing a 'chown' call on the directory, with an end result of the previous mode 2700 being changed to 0700. Now I understand that on some filesystems, a 'chown' call can end up losing any setuid or setgid bits on an inode - and apparently Netapp's WAFL file system is one of them. Looking at the dovecot source code, I *believe* the source of the chown call is in src/lib/nfs-workarounds.c function nfs_flush_chown_uid - where a 'chown' call is being used to flush NFS attribute caches? for what I believe is the purposes of ascertaining file lock status? My question for you folks is if anyone else has encountered this on other file systems (or filers) - and if there is a native method in dovecot - either the version I am currently running or an updated newer version - that allows one to either retain the permissions assigned to a directory or specify the permissions I expect on the Maildir directory structures. Any feedback would be welcome, thanks! -- Shaun Johnson LinuxMagic Inc. 604-682-0300 Beautiful British Columbia, Canada

2 2

LDAP Homedir location: Needs dovecot restart after change it
by Andre Luiz Paiz 16 Apr '18

16 Apr '18

Dear group members. I work with Dovecot and Openldap authentication. Sometimes users change departments and we need to alter their homedir location. Every time this process is needed, I perform this steps: 1 - Change homedir location in openldap 2 - Move homedir folder to the new location 3 - Re-apply permissions 4 - Remove user index folder After I do that, users cannot authenticate unless I restart dovecot, process that I would like to avoid. Can you guys give a tip on what I need to change to avoid this last problematic step? After the restart, everything works. Does the auth_cache feature also store the homedir location? System: CentOS 7 Versions: dovecot-pigeonhole-2.2.10-8.el7.x86_64 dovecot-2.2.10-8.el7.x86_64 Doveconf -n: # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.5.2.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_mechanisms = plain login auth_username_format = %Ln default_client_limit = 1024 default_process_limit = 1024 default_vsz_limit = 1 G listen = * mail_location = maildir:~/Maildir:INDEX=/var/indexes/dovecot/%u mail_plugins = " mail_log notify quota" mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir quota_exceeded_message = Cota de armazenamento de e-mails excedida. Quota exceeded for mail store. quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/default.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags } protocols = imap pop3 lmtp sieve service anvil { client_limit = 14100 process_limit = 1 } service auth-worker { client_limit = 1 process_limit = 50 service_count = 1 } service auth { client_limit = 15100 process_limit = 1 } service dict { client_limit = 1 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 6000 process_min_avail = 10 service_count = 1 vsz_limit = 256 M } service imap { process_limit = 3000 vsz_limit = 1 G } service lmtp { client_limit = 1 inet_listener lmtp { port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 6000 process_min_avail = 10 service_count = 1 } service pop3 { process_limit = 3000 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = dovecot } user = root } ssl = required ssl_cert = ********** ssl_key = *********** userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " mail_log notify quota sieve quota" } protocol imap { mail_plugins = " mail_log notify quota imap_quota" } Thanks Andre

2 1