- dovecot - dovecot.org

dsync not replicating ".dovecot.sieve -> .sieve/managesieve.sieve"
by Tony 06 Aug '18

06 Aug '18

Hi, Currently using dovecot 2.3.2.1 and pigeonhole 0.5.2 on two identical hosts. I recently noticed ".dovecot.sieve" and ".sieve/managesieve.sieve" are not replicating from one host to the other in either direction. Mail is having no trouble replicating on both hosts with dsync, it is only sieve files that are not replicating for unclear reasons. # doveconf -n host 1: https://ptpb.pw/njs3 host 2: https://ptpb.pw/KsfK If I enable sieve debug logs I don't see anything that would cause alarm. The error logs are also not reporting any trouble. I have tested creating filters in Roundcube which is hosted from host1. This creates the sieve files as expected, but the same sieve files do not replicate to host2. When using Sieve with Thunderbird if the sieve files are created on host1 or host2, these are not replicating with dsync. So it does not seem to matter if the sieve files are created on either host and with any mail client, they just do not replicate at all. # host1 stat /home/vmail/example.email/user\(a)example.email/.dovecot.sieve File: /home/vmail/example.email/user(a)example.email/.dovecot.sieve -> .sieve/managesieve.sieve Size: 24 Blocks: 0 IO Block: 4096 symbolic link Device: 800h/2048d Inode: 596605 Links: 1 Access: (0777/lrwxrwxrwx) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2018-08-05 15:24:41.351827516 -0700 Modify: 2018-08-05 15:24:41.351827516 -0700 Change: 2018-08-05 15:24:41.351827516 -0700 Birth: - stat /home/vmail/example.email/user\(a)example.email/.sieve/managesieve.sieve File: /home/vmail/example.email/user(a)example.email/.sieve/managesieve.sieve Size: 65 Blocks: 8 IO Block: 4096 regular file Device: 800h/2048d Inode: 1657978 Links: 1 Access: (0600/-rw-------) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2018-08-05 15:27:30.034984088 -0700 Modify: 2018-08-05 15:27:30.034984088 -0700 Change: 2018-08-05 15:27:30.034984088 -0700 Birth: - # host2 stat /home/vmail/example.email/user\(a)example.email/.dovecot.sieve File: /home/vmail/example.email/user(a)example.email/.dovecot.sieve -> .sieve/managesieve.sieve Size: 24 Blocks: 0 IO Block: 4096 symbolic link Device: fe03h/65027d Inode: 787482 Links: 1 Access: (0777/lrwxrwxrwx) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2018-08-05 16:10:54.832151815 -0700 Modify: 2018-08-05 16:10:54.815484905 -0700 Change: 2018-08-05 16:10:54.815484905 -0700 Birth: - stat /home/vmail/example.email/user\(a)example.email/.sieve/managesieve.sieve File: /home/vmail/example.email/user(a)example.email/.sieve/managesieve.sieve Size: 18 Blocks: 8 IO Block: 4096 regular file Device: fe03h/65027d Inode: 786938 Links: 1 Access: (0600/-rw-------) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail) Access: 2018-08-05 16:10:54.815484905 -0700 Modify: 2018-08-05 16:10:49.818745528 -0700 Change: 2018-08-05 16:10:49.822078910 -0700 Birth: - Here is the Maildir tree view. tree -a /home/vmail/example.email/user\(a)example.email/ /home/vmail/example.email/user(a)example.email/ ├── .dovecot.sieve -> .sieve/managesieve.sieve ├── Maildir │ ├── .Archive │ │ ├── cur │ │ ├── dovecot.index.log │ │ ├── dovecot-uidlist │ │ ├── maildirfolder │ │ ├── new │ │ └── tmp │ ├── cur │ │ ├── 1531611877.M860311P418538.mail,S=5999,W=6167:2,RS │ │ ├── 1531624146.M201381P535494.mail,S=2133,W=2184:2,S │ │ ├── 1533194577.M483202P864896.mail,S=6117,W=6393:2,S │ │ └── 1533195582.M123163P877084.mail,S=1739,W=1786:2,S │ ├── dovecot.index │ ├── dovecot.index.cache │ ├── dovecot.index.log │ ├── dovecot-keywords │ ├── dovecot.list.index │ ├── dovecot.list.index.log │ ├── dovecot.mailbox.log │ ├── dovecot-uidlist │ ├── dovecot-uidvalidity │ ├── dovecot-uidvalidity.5b4a79b5 │ ├── .Drafts │ │ ├── cur │ │ ├── dovecot.index.log │ │ ├── dovecot-uidlist │ │ ├── maildirfolder │ │ ├── new │ │ └── tmp │ ├── maildirsize │ ├── new │ ├── .Sent │ │ ├── cur │ │ │ ├── 1531611855.M403833P418688.mail,S=3510,W=3623:2,S │ │ │ └── 1531624037.M238133P535494.mail,S=436,W=451:2,S │ │ ├── dovecot.index.cache │ │ ├── dovecot.index.log │ │ ├── dovecot-uidlist │ │ ├── maildirfolder │ │ ├── new │ │ └── tmp │ ├── .Sent Messages │ │ ├── cur │ │ ├── dovecot.index.log │ │ ├── dovecot-uidlist │ │ ├── maildirfolder │ │ ├── new │ │ └── tmp │ ├── .Spam │ │ ├── cur │ │ ├── dovecot.index.cache │ │ ├── dovecot.index.log │ │ ├── dovecot-uidlist │ │ ├── maildirfolder │ │ ├── new │ │ └── tmp │ ├── subscriptions │ ├── tmp │ └── .Trash │ ├── cur │ ├── dovecot.index.cache │ ├── dovecot.index.log │ ├── dovecot-uidlist │ ├── maildirfolder │ ├── new │ └── tmp └── .sieve ├── managesieve.sieve └── tmp 30 directories, 41 files I know at one point sieve files must have been replicating with dsync, it's just not clear to me at which version of dovecot and pigeonhole this began with the exception of mail files which are replicating. Cheers, Tony

2 1

Re: replication fails and corrupts index with zlib enabled
by fauno 05 Aug '18

05 Aug '18

El 04/08/18 a las 13:13, Aki Tuomi escribió: > There has definetly been fixes in this area since 2.2.13 and most likely your issue is fixed. Thanks! I built 2.2.36, I'll test it and come back with results :)

1 0

Assertion failure with IMAP NOTIFY
by Florian Pritz 05 Aug '18

05 Aug '18

Hi, For a few weeks I've been running a custom notification script[1] that uses IMAP NOTIFY. The script uses a single connection, sets up a NOTIFY watch on a couple of mailboxes + automatic FETCH for the selected mailbox. If a status update arrives for a different mailbox, it selects that box and fetches the mail. Today I got the following error in the log: > Aug 5 14:56:12 karif dovecot[1881]: imap(xxxx)<22699><TBjUXqtyZI5R2c0k>: Panic: file mail.c: line 242 (mail_get_header_stream): assertion failed: (headers->box == mail->box) > Aug 5 14:56:12 karif dovecot[1881]: imap(xxxx)<22699><TBjUXqtyZI5R2c0k>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0xcfbff) [0x7fda11806bff] -> /usr/lib/dovecot/libdovecot.so.0(+0xcfcba) [0x7fda11806cba] -> /usr/lib/dovecot/libdovecot.so.0(+0x414bb) [0x7fda117784bb] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x3566e) [0x7fda11aec66e] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x43b) [0x7fda11bb3a1b] -> dovecot/imap(+0x211ed) [0x55e00d2a21ed] -> dovecot/imap(+0x1f69c) [0x55e00d2a069c] -> dovecot/imap(imap_fetch_more_no_lock_update+0xf) [0x55e00d2a17ff] -> dovecot/imap(+0x23891) [0x55e00d2a4891] -> dovecot/imap(imap_client_notify_newmails+0xfa) [0x55e00d2a4a6a] -> dovecot/imap(imap_sync_more+0x755) [0x55e00d2a9f75] -> dovecot/imap(+0x29006) [0x55e00d2aa006] -> dovecot/imap(cmd_sync_delayed+0x27b) [0x55e00d2aa50b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0x10d) [0x7fda1181c82d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd2) [0x7fda1181df52] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x4d) [0x7fda1181caad] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x41) [0x7fda1181cc11] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x14) [0x7fda117a0074] -> dovecot/imap(main+0x342) [0x55e00d28fb82] -> /usr/lib/libc.so.6(__libc_start_main+0xeb) [0x7fda1139e06b] -> dovecot/imap(_start+0x2a) [0x55e00d28fd1a] > Aug 5 14:56:12 karif dovecot[1881]: imap(xxxx)<22699><TBjUXqtyZI5R2c0k>: Fatal: master: service(imap): child 22699 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2) I'm not sure what happened exactly, but I can tell you that a new mail, that went to the selected mailbox (INBOX), was arriving at the time of the crash. The script logs the IMAP traffic (sadly without timestamps so far). Here's a slightly censored (3 dots in a few places) and stripped log in the hope that it helps. Lines prefixed with >> are sent to the server, those with << are received from the server. > [login stuff] > server-speed >> CMD-2 notify set (selected (MessageExpunge MessageNew (uid body.peek[header.fields (from to subject)]))) (mailboxes (INBOX INBOX.Postmaster ... ... ... ...) (MessageNew MessageExpunge MailboxName)) > server-speed << CMD-2 OK NOTIFY completed (0.002 + 0.000 + 0.001 secs). > [various untagged fetches/selects + fetch/noops] > [script was in a different mailbox, status update for INBOX arrives thus script select INBOX] > server-speed >> CMD-51 select INBOX > server-speed << * OK [CLOSED] Previous mailbox closed. > server-speed << * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent Junk redirected $label2 $label4 $label5 NonJunk $Forwarded $label3 $label1 done Old) > server-speed << * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $MDNSent Junk redirected $label2 $label4 $label5 NonJunk $Forwarded $label3 $label1 done Old \*)] Flags permitted. > server-speed << * 20 EXISTS > server-speed << * 1 RECENT > server-speed << * OK [UNSEEN 14] First unseen. > server-speed << * OK [UIDVALIDITY 1336752926] UIDs valid > server-speed << * OK [UIDNEXT 41885] Predicted next UID > server-speed << * OK [HIGHESTMODSEQ 138681] Highest > server-speed << CMD-51 OK [READ-WRITE] Select completed (0.001 + 0.000 secs). > server-speed >> CMD-52 uid fetch 41884 (body.peek[header.fields (from to subject)]) > server-speed << * 20 FETCH (UID 41884 BODY[HEADER.FIELDS (FROM TO SUBJECT)] {164} > server-speed << From: ... > server-speed << To: ... > server-speed << Subject: ... > server-speed << > server-speed << ) > server-speed << CMD-52 OK Fetch completed (0.001 + 0.000 secs). > Got data for notification: [...] > Sending notification with data: {...} > [the next few lines happen over a span of roughly 9 minutes] > server-speed << * 20 EXPUNGE > server-speed << * 0 RECENT > server-speed << * 8 EXPUNGE > server-speed << * 11 EXPUNGE > server-speed << * 18 EXISTS > server-speed << * 1 RECENT > server-speed << * 18 FETCH (UID 41885 > [this is apparnently when the server crashed] I'm afraid I can't reproduce the issue and I don't have the time to try right now, so I can't provide much more information than what is in here. Still, if you have questions, feel free to ask me. Just know that I won't be getting you a better backtrace/coredump any time soon. [1] https://git.server-speed.net/users/flo/App-ImapNotify/tree/ Florian

1 1

limit sharing ability to certain users
by simeon.ott＠onnet.ch 05 Aug '18

05 Aug '18

Hello Is it possible to limit the ability of sharing it’s own mailboxes to only a few users? We have a few sensitive mailboxes of users where the ability to share via IMAP SETACL should be prevented. I tried the following so far… doveadm acl remove -u test(a)onnet.ch <mailto:test@onnet.ch> INBOX user=test(a)onnet.ch <mailto:user=test@onnet.ch> admin but when doing this the admin rights are still there doveadm acl rights -u test(a)onnet.ch <mailto:test@onnet.ch> INBOX vmail@buserver:~$ doveadm acl rights -u test(a)onnet.ch <mailto:test@onnet.ch> INBOX Rights lookup read write write-seen write-deleted insert post expunge create delete admin Thanks in advance for your help Simeon

1 0

replication fails and corrupts index with zlib enabled
by fauno 04 Aug '18

04 Aug '18

Hi, I have two Debian Jessie servers with Dovecot 2.2.13 TCP replication on that have worked fine for years, but now one of them is running low on disk space, so I wanted to try enabling zlib. I crafted a script following the description given in https://wiki.dovecot.org/Plugins/Zlib and xz'ed some inboxes on the stand-by server, the one with low disk space. So every email in those inboxes is xz'ed but the file name hasn't changed and contains the original size. This server is on stand-by so most of the email is replicated unidirectionally to it. But administrative emails like cronjobs and monitoring are delivered locally, so it replicates those to the hot server. The issue appeared when this stand-by server receives such an email and tries to replicate them to the other server. I'm attaching the full snippet of the log from the hot server, because it throws a longer backtrace. The short version is like this: dovecot[8438]: imap(redacted(a)address.org): Error: Cached message size larger than expected (478 > 289) dovecot[8438]: imap(redacted(a)address.org): Error: Maildir filename has wrong S value, renamed the file from /srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=478:2, to /srv/email/address.org/redacted/Maildir/cur/1533393328.M502775P20341.standby_server,S=289:2, dovecot[8438]: imap(redacted(a)address.org): Error: Corrupted index cache file /srv/email/address.org/redacted/Maildir/dovecot.index.cache: Broken physical size for mail UID 45123 After this there's an error and the replication fails. The file is there, it's gzipped and can be zcat'ed but it appears as a blank email on clients. I've recovered a backup but the issue persists. I also changed from xz to gz because the Debian package docs only mention gz and bzip2, but the issue is the same. From what I understand and tested, the stand-by server is receiving the email and compressing it but maintaining the original size on the file name. So that's ok, but when the hot server receives the copy, it believes the size is wrong and changes it to the compressed size. Then for some reason the index gets corrupted. I'm attaching the doveconf for both servers. They're mostly the same, and the only changes introduced were the zlib plugin and its options. Also the script(s) that I used to compress the inboxes. Am I correct? Is it an issue of replicator not understanding the emails are compressed? I couldn't find anything related to zlib with replication. Maybe it's something fixed in newer versions and I should go that rabbit hole? Thanks! :)

2 1

Proxy lmtp to smtp server
by grupo correo 03 Aug '18

03 Aug '18

Hi, I need to use lmtp server proxy, to proxy a remote smtp server, the documentation says "It's possible to use Dovecot LMTP server as a proxy to remote LMTP or SMTP servers", but i do not find how to configure. Can anybody gel me to understand? Best regards

3 6

Re: ssl_dh required, even though DH is disabled.
by Aki Tuomi 03 Aug '18

03 Aug '18

Hi! This change has now been committed, please find it at https://github.com/dovecot/core/compare/cd08262%5E...dd6323.patch Aki On 16.07.2018 09:53, Aki Tuomi wrote: > This is a known issue, but thanks for reporting it. > > > > --- > Aki Tuomi > Dovecot oy > > -------- Original message -------- > From: Eric Toombs <ewtoombs(a)uwaterloo.ca> > Date: 16/07/2018 08:41 (GMT+02:00) > To: dovecot(a)dovecot.org > Subject: ssl_dh required, even though DH is disabled. > > Here's my config: > > # 2.3.2 (582970113): /etc/dovecot/dovecot.conf > # OS: Linux 4.17.5-1-ARCH x86_64 Arch Linux > # Hostname: vault > passdb { > driver = pam > } > protocols = imap > service imap-login { > inet_listener imap { > port = 0 > } > } > ssl = required > ssl_cert = </etc/letsencrypt/live/myhostname.com/fullchain.pem > ssl_cipher_list = > ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384 > ssl_key = # hidden, use -P to show it > ssl_min_protocol = TLSv1.2 > > My filesystem is ext4. > > Even though I use ssl_cipher_list to forbid DH, dovecot still doesn't > work unless I provide an ssl_dh, delivering the following error: > > > Jul 14 21:48:08 vault dovecot[8349]: imap-login: Error: Failed to > initialize SSL server context: Couldn't parse DH parameters: > error:0906D06C:PEM routines:PEM_read_bio:no start line: Expecting: DH > PARAMETERS: user=<>, rip=10.0.0.1, lip=10.0.0.2, > session=<4sGi5/9w3pwKAAAB> > > While providing an ssl_dh is only a minor annoyance, it would be nice if > I didn't have to.

1 0

Dovecot lifecycle announcement
by Aki Tuomi 03 Aug '18

03 Aug '18

Hi! We are giving the following end of life (EOL) statements: For any version before 2.2, we are not going to be providing any patches or fixes. This has been the case before, but we are now officially declaring them EOL. For 2.2 versions, we are going to provide security and critical bug fixes for 2.2.36 only. If you are using 2.2.x version prior to .36, we will not be providing anything other than critical security fixes. This means that if you are experiencing a problem with Dovecot, we need you to upgrade into latest 2.3 release and reproduce the problem there. If this happens, please file a bug report and we are more than happy to take a look! If you are using a 2.2.x version packaged from an Operating System Vendor, please open a bug against them for fixing it. We do not provide patches or bug fixes against these OS Vendor specific releases. 2.3.x is the current main release, which will receive feature, bug and security patches. We accept bug reports against latest 2.3 version or master version of Dovecot. If you need packages for 2.3, please see https://repo.dovecot.org. Thank you for using Dovecot! Note: Users with a commercial agreement with Open-Xchange (Dovecot Pro) have a separate EOL policy, so this announcement does not apply to those customers. Aki Tuomi Open-Xchange Oy

1 0

Debugging rules with Pigeonhole Sieve
by Pete Gonzalez 03 Aug '18

03 Aug '18

I have a lot of rules in my Sieve script. Sometimes an email message ends up in the wrong folder, due to a problem with one of the rules. However, this is very difficult to debug, because I have no idea which rule matched the email. A log file might be helpful, but that wouldn't be readable without shell access to the mail server. This is not a solution that would work for everyday users, who configure their rules via a web page UI. What would be great would be for the Sieve interpreter to append a custom header to each email message after it gets processed. This header would record which Sieve rules matched the email. Does this feature exist already? Or is there a way to accomplish it via the Dovecot server configuration? Thanks!

1 0

Managesieve stopped working - Undefined symbol "i_stream_read_memarea"
by Henrik Larsson 02 Aug '18

02 Aug '18

After upgrading Dovecot to 2.3.2.1 and Pigeonhole to 0.5.2, managesieve stopped working. I'm using FreeBSD ports tree, to build these. Only domains have been modified in below output. A FreeBSD bug report have also been created. But I'm not sure if this is FreeBSD ports related or Dovecot/Pigeonhole related: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228067 %uname -a FreeBSD 10.4-RELEASE-p9 FreeBSD 10.4-RELEASE-p9 #0: Sat Jul 14 22:38:37 CEST 2018 /usr/obj/usr/src/sys/GENERIC amd64 %pkg version -v dovecot-2.3.2.1 = up-to-date with index dovecot-pigeonhole-0.5.2_2 = up-to-date with index %dmesg -a /usr/local/libexec/dovecot/managesieve: Undefined symbol "i_stream_read_memarea" doveconf: Error: managesieve-login: dump-capability process returned 1 % dovecot -n # 2.3.2.1 (0719df592): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.2 (7704de5e) /usr/local/libexec/dovecot/managesieve: Undefined symbol "i_stream_read_memarea" doveconf: Error: managesieve-login: dump-capability process returned 1 # OS: FreeBSD 10.4-RELEASE-p9 amd64 # Hostname: mail.example.com auth_mechanisms = plain login first_valid_uid = 125 hostname = mail.example.com listen = * log_path = /var/log/dovecot mail_plugins = zlib mail_privileged_group = postfix mail_temp_dir = /var/db/dovecot namespace compat { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox "Sent Items" { special_use = \Sent } mailbox Spam { special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = separator = . type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = squat fts_squat = partial=4 full=10 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster(a)example.com protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { unix_listener /home/mail/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service lmtp { executable = lmtp -L unix_listener /home/mail/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/example.com-chained.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { log_path = /var/log/dovecot-deliver mail_plugins = zlib sieve } protocol lda { mail_plugins = zlib sieve } protocol imap { mail_max_userip_connections = 30 mail_plugins = zlib imap_zlib }

2 5