October 2020 - dovecot

backup of namespace, is still looking at (touching?) other namespace?
by Marc Roos 17 Oct '20

17 Oct '20

When I am doing this: doveadm backup -f -n inbox -F /root/backup-accounts.txt tcp:mailxx.local:542 I am getting an error on the distributed storage, which I exactly did not wanted to be touched. doveadm(testaccount): Error: remote(mailxx.local:542): User initialization failed: Namespace 'Archive/': stat(/home/mail-archive/testaccount/Archive/mailboxes) failed: Permission denied (euid=xxx(testaccount) egid=xx(xxxxx) missing +x perm: /home/mail-archive, dir owned by 0:0 mode=0700) doveadm(testaccount): Error: remote(mailxx.local:542): dsync-server: User init failed doveadm(testaccount): Error: Failed to start remote dsync-server command: Remote exit_code=75

1 0

how to set smtp-client -> submission_relay_host for IPv4 only?
by PGNet Dev 16 Oct '20

16 Oct '20

In dovecot conf, for submission relay, I've config'd submission_relay_host = lan.example.com submission_relay_port = 465 hostname -- not IP -- must be used, to inform relay for cert verification match. Here, host lan.example.com lan.example.com has address 10.0.1.47 lan.example.com has IPv6 address fd80:25:01::47 The receiving smtp relay listens only on IPv4 -- no IPv6 service, telnet 10.0.1.47 465 Trying 10.0.1.47... Connected to 10.0.1.47. Escape character is '^]'. telnet -6 fd80:25:01::47 465 Trying fd80:25:01::47... telnet: connect to address fd80:25:01::47: Connection refused On the lan, all boxes are set, in "/etc/gai.conf", for IPv6 to take precedence over IPv4. On submission to dovecot submission port, after login/auth, dovecot smtp-client subsequently connects to the relay lan.example.com:465 IPv6 1st, & fails 2020-10-15 12:51:45 submission(mua(a)example.com)<8OJP+rqxuvho7Z95>: Error: smtp-client: conn lan.example.com:465 ([fd80:25:01::47]:465) [1]: connect(lan.example.com:465) failed: Connection refused then falls back to IPv4, and continues OK 2020-10-15 12:51:45 submission(mua(a)example.com)<8OJP+rqxuvho7Z95>: Info: Successfully relayed message: from=<test(a)remote.example.com>, size=84, id=LMe...Aw, nrcpt=1, reply=`247 2.0.0 Ok: queued as 4CC0KY1wXNzWf93' not fatal, but wasted effort, and noise in the logs. how/where do I configure (just) the dovecot smtp-client -> submission_relay_host to only connect IPv4?

3 4

Dovecot won't accept IMAP TLS 1 connections from older devices [SOLVED]
by Brian Martin 16 Oct '20

16 Oct '20

I've spent days scouring the Internet and trying various solutions on a problem with my Dovecot installation, so I thought I'd share what I learned in hopes of saving other people a lot of time. The dedicated Dovecot hands will know all of the following already. This is for those of us that have to cover a lot of bases. I upgraded my mail server from Ubuntu 18.04.1 to Ubuntu 20.04.1, and found that older Mac-books and iPads (and probably other devices) could no longer establish IMAP connections to Dovecot. Dovecot logged: SSL routines:tls_early_post_process_client_hello:unsupported protocol and TCP/IP traces showed that it dropped the connection after the client's initial HELLO. I tested what kinds of connections Dovecot would accept with (for example): openssl s_client -tls1_1 -connect localhost:993 # Test whether TLSv1.1 is accepted - received "unsupported protocol" message. Searching showed that Dovecot has a parameter "ssl_min_protocol", which is documented as defaulting to TLSv1. Nevertheless I explicitly set it to TLSv1 with no effect. This was a red herring. I spent a long time looking to see if Dovecot had a bug in handling this newish parameter, etc. Eventually I came across one posting regarding a web server, that told me the OpenSSL libraries that Dovecot and lots of other packages use has a single configuration file for the entire system. In Ubuntu 20.04 it defaults to requiring TLSv1.2 or above. Changing the configuration for OpenSSL affects everything on the system using the library. I changed the file, restarted Dovecot, and it immediately accepted TLSv1 connections. Obviously I'd prefer to maintain the improved security of TLSv1.2, but in my case it was better to continue providing mail service at a lower security level than to deny service to some users until they upgraded their personal devices. You'll need to make your on decision on that score. The file to change is (on Ubuntu, at least) /usr/lib/ssl/openssl.cnf. The change consists of adding a line of code in the initial section that invokes several new sections later: In the initial section I added: openssl_conf = default_conf Then at the bottom of the file I added: [default_conf] ssl_conf = ssl_sect [ssl_sect] system_default = system_default_sect [system_default_sect] MinProtocol = TLSv1 CipherString = DEFAULT@SECLEVEL=1 There is an alternative approach that I have read of but not tested. Basically you can create a new file elsewhere with the customized content, and then set an environmental variable (OPENSSL_CONF) just before launching Dovecot that points to your new file. This way, only Dovecot is using downgraded security. Since my mail server is a dedicated system and I didn't want to muck with the Dovecot start-up environment, I didn't feel the need to go that route. So all my digging into why Dovecot wouldn't accept TLSv1 connections and how to change it were completely on the wrong path. It would be nice if Dovecot could log a message when its ssl_min_protocol is set lower than what OpenSSL will accept, but Dovecot may not be able to tell what OpenSSL is doing. In any case, those are the symptoms, the real problem, and how to fix it. Good luck, and thanks to Matt Caswell for posting the answer that I eventually found. Ref: https://askubuntu.com/questions/1233186/ubuntu-20-04-how-to-set-lower-ssl-s…

3 2

Auro expunge
by ＠lbutlr 15 Oct '20

15 Oct '20

When using autoexpunge = 14 days in, for example. The trash of junk folders, does that physically remove the messages from disk or simply mark them to be removed and some other action needs to be taken? I ask because my mail server crapped out today and I discovered a Junk folder with 490,000 messages in it, and the sa-spamd and mariadb processes died a horrible death. Haven't had time to track down what was going on, but there were definitely messages in that junk folder from 2019 (though, of course, they may have been added within the last 14 days, I can't verify that as my solution was to remove the folder and recreate a new empty maildir). -- I don't believe there's a power in the 'verse can stop Kaylee from bein' cheerful. Sometimes you just wanna duct-tape her mouth and dump her in the hold for a month.

5 5

RE: dovecot Digest, Vol 210, Issue 27
by webadmin＠exalt.com.au 15 Oct '20

15 Oct '20

I am investigating whether dovecot(https://github.com/dovecot/core/) handles case insensitive Message-ID headers as per RFC. I can't raise a support issue with cPanel team (https://tickets.cpanel.net/review/login.cgi) as that needs an paid account with them. There may be an issue with the CPANEL/WHM DKIM module. Is the source code online? Rob

3 2

How do I enable sieve trace debugging?
by Graham Leggett 14 Oct '20

14 Oct '20

Hi all, I want to enable trace debugging for sieve. Most specifically, I want to know what sieve scripts are running, and whether they’re doing anything. I cannot get anything other than dead silence from dovecot with respect to sieve. I have tried the same as asked in this question: https://unix.stackexchange.com/questions/550618/dovecot-sieve-trace-does-no… In my case, the config looks like this: ## TRACE DEBUGGING # Trace debugging provides detailed insight in the operations performed by # the Sieve script. These settings apply to both the LDA Sieve plugin and the # IMAPSIEVE plugin. # # WARNING: On a busy server, this functionality can quickly fill up the trace # directory with a lot of trace files. Enable this only temporarily and as # selective as possible. # The directory where trace files are written. Trace debugging is disabled if # this setting is not configured or if the directory does not exist. If the # path is relative or it starts with "~/" it is interpreted relative to the # current user's home directory. sieve_trace_dir = /tmp/sieve-trace # The verbosity level of the trace messages. Trace debugging is disabled if # this setting is not configured. Possible values are: # # "actions" - Only print executed action commands, like keep, # fileinto, reject and redirect. # "commands" - Print any executed command, excluding test commands. # "tests" - Print all executed commands and performed tests. # "matching" - Print all executed commands, performed tests and the # values matched in those tests. sieve_trace_level = actions,command,tests,matching # Enables highly verbose debugging messages that are usually only useful for # developers. sieve_trace_debug = yes This looks sensible, syslog would be a great place to log, but nothing related to sieve appears. [root@gatekeeper dovecot]# doveadm log find Looking for log files from /var/log Debug: /var/log/maillog Info: /var/log/maillog Warning: /var/log/maillog Error: /var/log/maillog Fatal: /var/log/maillog Regards, Graham —

3 3

Changing IMAP separator - does it break things?
by Victor Sudakov 14 Oct '20

14 Oct '20

Dear Colleagues, I am using Dovecot 2.3.4.1 with maildirs. Just found out that MacOS Mail clients cannot work with nested folders, presumably because the MacOS Mail does not understand "." as the IMAP separator. Other clients (ThunderBird, Sylpheed, mutt) work fine. Do you think I can change the hierarchy separator in the "inbox" namespace without breaking other clients and the mail layout on disk, losing mail etc? I don't want to break mail for 50 people just to please one Mac Mail user. The users are advanced developers actively using mail sorting rules, nested folders etc. My `doveconf -n` is here: https://termbin.com/w1wd -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/

5 20

Re: Fatal: setgid from userdb lookup fails with wrong gid
by J. de Meijer 14 Oct '20

14 Oct '20

Hi, Not sure if this is it, but I used to have the same error when I started with dovecot. Aki's response was the following (and solved my problem). ------------- Hi! You can't set service imap { service_count = 256 } if you are using multiple system UIDs. See https://wiki.dovecot.org/Services#imap.2C_pop3.2C_submission.2C_managesieve *service_count* can be changed from 1 if only a single UID is used for mail users. This is improves performance, but it's less secure, because bugs in code may leak email data from another user's earlier connection. Aki ----------- Regards, Jeroen > Hello all, > > I'm quite new as well to Dovecot, just installed it on a FreeBSD system with Postfix and Rspamd as side apps. Things are running semi-smoothly for all users but I do have quite a few errors in the logs : > > Oct 13 19:43:56 apollo dovecot[24478]: > imap(user1)<34412><zIeI9ZCxXDmsFhZG>: Fatal: setgid(1030(user1) from userdb lookup) failed with euid=1022(user4), gid=1022(user4), > egid=1022(user4): Operation not permitted (This binary should probably be called with process group set to 1030(user1) instead of 1022(user4)) Oct 13 19:43:59 apollo dovecot[24478]: > imap(user1)<37376><pPS79ZCx+kasFhZG>: Fatal: setgid(1030(user1) from userdb lookup) failed with euid=1124(user3), gid=1124(user3), > egid=1124(user3): Operation not permitted (This binary should probably be called with process group set to 1030(user1) instead of 1124(user3)) Oct 13 19:46:45 apollo dovecot[24478]: > imap(user2)<38858><3hOk/5CxVO1dBDTq>: Fatal: setgid(1136(user2) from userdb lookup) failed with euid=1038(user5), gid=1038(user5), > egid=1038(user5): Operation not permitted (This binary should probably be called with process group set to 1136(user2) instead of 1038(user5)) Oct 13 19:48:55 apollo dovecot[24478]: > imap(user3)<40607><jQtWB5GxHuwKAkQ2>: Fatal: setgid(1124(user3) from userdb lookup) failed with euid=1022(user4), gid=1022(user4), > egid=1022(user4): Operation not permitted (This binary should probably be called with process group set to 1124(user3) instead of 1022(user4)) > > There seems to be confusion. The logs are trying to be helpful but I can't quite process it. Could someone point me in the right direction ? > > system is used by about 60 users. > > Thanks, > > j. > > -- > > doveconf -n > # 2.3.11.3 (502c39af9): /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.5.11 (d71e0372) > # OS: FreeBSD 12.1-RELEASE-p10 amd64 > # Hostname: apollo.domain1.tld > auth_mechanisms = plain login cram-md5 > auth_username_format = %Ln > mail_location = maildir:~/Maildir > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > type = private > } > passdb { > args = scheme=cram-md5 /usr/local/etc/dovecot/cram-md5.pwd > driver = passwd-file > } > plugin { > imapsieve_mailbox1_before = > file:/var/vmail/sieve/global/learn-spam.sieve > imapsieve_mailbox1_causes = COPY > imapsieve_mailbox1_name = Spam > imapsieve_mailbox2_before = > file:/var/vmail/sieve/global/learn-ham.sieve > imapsieve_mailbox2_causes = COPY > imapsieve_mailbox2_from = Spam > imapsieve_mailbox2_name = * > quota = maildir:User quota > quota_exceeded_message = Benutzer %u hat das Speichervolumen > Ã¼berschritten. / User %u has exhausted allowed storage space. > sieve = file:~/sieve;active=~/.dovecot.sieve > sieve_before = /var/vmail/sieve/global/spam-global.sieve > sieve_global_extensions = +vnd.dovecot.pipe > sieve_pipe_bin_dir = /usr/local/bin > sieve_plugins = sieve_imapsieve sieve_extprograms > } > postmaster_address = postmaster(a)apollo.domain1.tld > protocols = imap lmtp sieve > service auth { > client_limit = 3000 > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > } > service imap-login { > service_count = 0 > } > service imap { > process_min_avail = 4 > service_count = 512 > vsz_limit = 1 G > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > vsz_limit = 1 G > } > ssl_cert = > </usr/local/etc/letsencrypt/live/apollo.domain1.tld/fullchain.pem ssl_cipher_list = > EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA ssl_dh = # hidden, use -P to show it > ssl_key = # hidden, use -P to show it > ssl_prefer_server_ciphers = yes > syslog_facility = local5 > userdb { > driver = passwd > } > protocol lda { > mail_plugins = sieve > } > protocol lmtp { > mail_plugins = quota sieve > postmaster_address = postmaster(a)domain1.tld > } > protocol imap { > mail_max_userip_connections = 100 > mail_plugins = " quota imap_quota imap_sieve" > } > local_name imap.domain2.tld { > ssl_cert = > </usr/local/etc/letsencrypt/live/mail.domain2.tld/fullchain.pem > ssl_key = # hidden, use -P to show it > } > local_name mail.domain2.tld { > ssl_cert = > </usr/local/etc/letsencrypt/live/mail.domain2.tld/fullchain.pem > ssl_key = # hidden, use -P to show it > } >

2 1

Re: zlib errors after upgrading to 2.3.11.3
by Robert Nowotny 14 Oct '20

14 Oct '20

On 9. Sep 2020, at 11.14, Robert Nowotny <rnowotny1966(a)gmail.com> wrote: Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Also this way you can see if the broken mail is actually xz or zstd or zlib. It would be nice to know if there are any zstd or zlib compressed mails that have problems. We did a lot of stress testing with zstd and also with xz, but haven't been able to reproduce any problems. It's also strange that it says taht the error is "Broken pipe" - that doesn't indicate that the mail is corrupted but that there is something more strange going on. So perhaps you don't actually have any mails written as corrupted, but Dovecot is just somehow having trouble reading the mails. I managed to reproduce this. The files aren't corrupted, it's just that reading is failing. The attached patch should fix the xz code and should make your files readable again. Yes, it works ! I switched now to "zstd" compression and currently re-compressing all user imap folders (one by one) Actually, with further testing it looks like the mails were written truncated. My patch simply hides the truncation when reading the mail. But it's anyway best that can be done about them. zstd also has a bug in writing compressed output, but it should always crash instead of writing broken output. We'll try to get fixes to these merged to git soon. currently I am re-compressing the mailboxes and so far i can see only one (1) Error, after about 120GB of recomresses emails : Sep 10 11:32:24 lxc-imap dovecot: imap(ebay)<37070><1aDvPvKus+/AqKjk>: Panic: file ostream.c: line 287 (o_stream_sendv_int): assertion failed: (!stream->blocking) Sep 10 11:32:24 lxc-imap dovecot: imap(ebay)<37070><1aDvPvKus+/AqKjk>: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(backtrace_append+0x41) [0x7f5b00de6da1] -> /usr/local/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f5b00de6ec2] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xfe5eb) [0x7f5b00df05eb] -> /usr/local/lib/dovecot/libdovecot.so.0(+0xfe687) [0x7f5b00df0687] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x598e1) [0x7f5b00d4b8e1] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x5e2c1) [0x7f5b00d502c1] -> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_sendv+0x32) [0x7f5b00e14a92] -> /usr/local/lib/dovecot/libdovecot.so.0(io_stream_copy+0x5a) [0x7f5b00e1535a] -> /usr/local/lib/dovecot/libdovecot.so.0(o_stream_send_istream+0x50) [0x7f5b00e15060] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_storage_save_continue+0x2d) [0x7f5b00f9659d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(dbox_save_continue+0x30) [0x7f5b00f3d3b0] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_save_continue+0x3e) [0x7f5b00f1a34e] -> dovecot/imap [ebay 192.168.168.228 APPEND](+0x13801) [0x55d6905bf801] -> dovecot/imap [ebay 192.168.168.228 APPEND](command_exec+0x6c) [0x55d6905ce3ec] -> dovecot/imap [ebay 192.168.168.228 APPEND](+0x12fdb) [0x55d6905befdb] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x7f5b00e06ced] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x145) [0x7f5b00e08305] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x7f5b00e06d94] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0x7f5b00e06f08] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x7f5b00d7f8d7] -> dovecot/imap [ebay 192.168.168.228 APPEND](main+0x459) [0x55d6905bec99] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f5b00b0e0b3] -> dovecot/imap [ebay 192.168.168.228 APPEND](_start+0x2e) [0x55d6905bed2e] Sep 10 11:32:24 lxc-imap dovecot: imap-login: Panic: epoll_ctl(del, 21) failed: Bad file descriptor Sep 10 11:32:24 lxc-imap dovecot: imap(ebay)<37070><1aDvPvKus+/AqKjk>: Fatal: master: service(imap): child 37070 killed with signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) until now I only have one broken mailbox, I need to repair (I will try that later, just finish all the other users before): Sep 10 12:13:13 lxc-imap dovecot: imap(mtrenner)<41872><KVTf0PKu7+7AqKi0>: Error: lzma.read(/home/vmail/virtualmailboxes/mtrenner/storage/m.349): corrupted data at 3763912 Sep 10 12:13:13 lxc-imap dovecot: imap(mtrenner)<41872><KVTf0PKu7+7AqKi0>: Error: Mailbox Sent: UID=1480: read(zlib(/home/vmail/virtualmailboxes/mtrenner/storage/m.349)) failed: read(/home/vmail/virtualmailboxes/mtrenner/storage/m.349) failed: lzma.read(/home/vmail/virtualmailboxes/mtrenner/storage/m.349): corrupted data at 3763912 (FETCH BODY[]) > zstd also has a bug in writing compressed output, but it should always crash instead of writing broken output. > We'll try to get fixes to these merged to git soon. Uh-Oh, that would be good ... pls notify ! how such thing can slip through the tests ? I just can not remember, the last error with compression was like 10 years ago (dont know if it was me who reported that) However, thank You for Your fast response and taking care !!! Robert

3 2

Spam learning for rspamd
by Dan Egli 14 Oct '20

14 Oct '20

I'm quite new to Dovecot, so forgive me if this is a simple question. I've got rspamd running, and it's rewriting the subject of many messages as spam even when they are not. I've moved things out of the spam folder, which I was under the impression would teach rspamd since I've connected a sieve script that is supposed to call rspamd's learning tool, but nothing is happening. I'm really at a loss as to where to even begin searching for an answer, so any help is appreciated! -- Dan Egli On my Test server

4 5