January 2018 - dovecot

Dovecot 2.3.0 imap-login using 100% CPU
by Jason Kiniry 11 Jan '18

11 Jan '18

When in the process of testing out the 2.3.0 version of Dovecot, we noticed that on a busy server, imap-login can sometimes take 100% CPU and remain there indefinitely. We grabbed a gdb trace while it was happening: dovenull 36053 31.2 0.0 60460 17316 ? R 04:38 11:55 dovecot/imap-login root@server [~]# strace -p 36053 Process 36053 attached ^CProcess 36053 detached root@server [~]# gdb atGNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. (gdb) attach 36053 Attaching to process 36053 Reading symbols from /usr/libexec/dovecot/imap-login...done. Reading symbols from /usr/lib64/dovecot/libdovecot-login.so.0...done. Loaded symbols for /usr/lib64/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib64/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib64/dovecot/libdovecot.so.0 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl.so.10 Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcrypto.so.10 Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libgssapi_krb5.so.2 Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5.so.3 Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libcom_err.so.2 Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libk5crypto.so.3 Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5support.so.0 Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libkeyutils.so.1 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libselinux.so.1 Reading symbols from /usr/lib64/dovecot/libssl_iostream_openssl.so...done. Loaded symbols for /usr/lib64/dovecot/libssl_iostream_openssl.so 0x00007f758fb8dd18 in client_notify_auth_ready@plt () from /usr/lib64/dovecot/libdovecot-login.so.0 Missing separate debuginfos, use: debuginfo-install dovecot-2.3.0-4.cp1162.x86_64 (gdb) back #0 0x00007f758fb8dd18 in client_notify_auth_ready@plt () from /usr/lib64/dovecot/libdovecot-login.so.0 #1 0x00007f758fb90af0 in clients_notify_auth_connected () at client-common-auth.c:837 #2 0x00007f758f8cfc04 in auth_server_input_done (conn=0x7f7591911838) at auth-server-connection.c:127 #3 auth_server_connection_input_line (conn=0x7f7591911838) at auth-server-connection.c:229 #4 auth_server_connection_input (conn=0x7f7591911838) at auth-server-connection.c:281 #5 0x00007f758f8f5e35 in io_loop_call_io (io=0x7f759277e2a0) at ioloop.c:614 #6 0x00007f758f8f7d3f in io_loop_handler_run_internal (ioloop=<value optimized out>) at ioloop-epoll.c:222 #7 0x00007f758f8f5f25 in io_loop_handler_run (ioloop=0x7f75918e4d00) at ioloop.c:666 #8 0x00007f758f8f6148 in io_loop_run (ioloop=0x7f75918e4d00) at ioloop.c:639 #9 0x00007f758f875ab3 in master_service_run (service=0x7f75918e4b90, callback=<value optimized out>) at master-service.c:767 #10 0x00007f758fb96156 in login_binary_run (binary=<value optimized out>, argc=1, argv=0x7f75918e4890) at main.c:549 #11 0x00007f758f4a2d1d in __libc_start_main () from /lib64/libc.so.6 #12 0x00007f758ffc4469 in _start () (gdb) run The program being debugged has been started already. Start it from the beginning? (y or n) n Program not restarted. (gdb) continue Continuing. ^C Program received signal SIGINT, Interrupt. client_notify_auth_ready (client=0x7f7592775140) at client-common.c:1031 1031 client-common.c: No such file or directory. in client-common.c (gdb) back #0 client_notify_auth_ready (client=0x7f7592775140) at client-common.c:1031 #1 0x00007f758fb90af0 in clients_notify_auth_connected () at client-common-auth.c:837 #2 0x00007f758f8cfc04 in auth_server_input_done (conn=0x7f7591911838) at auth-server-connection.c:127 #3 auth_server_connection_input_line (conn=0x7f7591911838) at auth-server-connection.c:229 #4 auth_server_connection_input (conn=0x7f7591911838) at auth-server-connection.c:281 #5 0x00007f758f8f5e35 in io_loop_call_io (io=0x7f759277e2a0) at ioloop.c:614 #6 0x00007f758f8f7d3f in io_loop_handler_run_internal (ioloop=<value optimized out>) at ioloop-epoll.c:222 #7 0x00007f758f8f5f25 in io_loop_handler_run (ioloop=0x7f75918e4d00) at ioloop.c:666 #8 0x00007f758f8f6148 in io_loop_run (ioloop=0x7f75918e4d00) at ioloop.c:639 #9 0x00007f758f875ab3 in master_service_run (service=0x7f75918e4b90, callback=<value optimized out>) at master-service.c:767 #10 0x00007f758fb96156 in login_binary_run (binary=<value optimized out>, argc=1, argv=0x7f75918e4890) at main.c:549 #11 0x00007f758f4a2d1d in __libc_start_main () from /lib64/libc.so.6 #12 0x00007f758ffc4469 in _start () (gdb) Anyone have any thoughts? Regards, Jason Kiniry

3 4

Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000)
by Stefan Neben 11 Jan '18

11 Jan '18

Hello, i have a problem with a very huge mail account. The INBOX of this account has ~1.1 - ~1.7 million mails inside. And from time to time the index cache make problems and i got the following in the logs: --- Snip --- Jan 10 20:43:04 XXXXX dovecot: imap(XXXXX): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) --- Snap --- I skip the raw backtrace to keep the post "small", but can hand it in later if needed. Got this for UID FETCH, UID SEARCH, UID SORT, ... The size of the file dovecot.index.cache of INBOX was in the last seen problem situations between 1.1GB to 2.3GB. To solve the problem the last times was to reduce the amount of mails to under 1.7 million (seems to be the limit to trigger the problem), delete the index cache and let him rebuild with a controled request. But this is a odd situation (because it confused 3rd party tooling, which access this mailbox) and i cannot find any helpful informations to solve this problem. Sharding it into several folders to keep the amount of mails in INBOX small is not an option currently. If further informations are needed, just let me know. Thanks for any help! Kind regards Stefan

3 5

Managesieve cannot access script store
by dovelist 11 Jan '18

11 Jan '18

Hi, I am trying to get sieve working on a new OpenSuse leap 42.2 install. On my 'old' OpenSuse 13.2 machine it worked fine. The problem is that Managesieve can't access the script store and won't let me create any script. It says permission denied on ~/sieve directory. See log below. I 've activated debug logging, but that doesn't give any clues to me. Also, I've set the directory accessible to all, but Managesieve still complains. > cd ~ > ls -l drwx------ 1 rogier users 8340 5 feb 16:54 Maildir drwxrwxrwx 1 rogier users 24 5 feb 18:38 sieve To rule out client issues (kmail) I tested also with Manual TLS Login as described in: http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Troubleshooting Same result. I am puzzled. I can't find anything wrong in the dovecot configuration. The output of dovecot -n is shown below. Hope someone has a solution. A lot of mail is waiting to get sorted... Best Regards, Rogier The log: feb 05 20:22:18 p150 dovecot[12120]: managesieve-login: Login: user=<rogier>, method=PLAIN, rip=192.168.0.18, lip=192.168.0.20, mpid=12135, TLS, session=<gmb0bs1H5q/AqAAS> feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Effective uid=1000, gid=100, home=/home/rogier feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: maildir++: root=/home/rogier/Maildir, index=, indexpvt=, control=, inbox=/home/rogier/Maildir, alt= feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: Pigeonhole version 0.4.15 (97b3da0) initializing feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using active Sieve script path: /home/rogier/.dovecot.sieve feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using script storage path: /home/rogier/sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using permissions from /home/rogier/sieve/: mode=0777 gid=-1 feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: sync: Synchronization active feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Error: sieve: file storage: Failed to list scripts: opendir(/home/rogier/sieve) failed: Permission denied Output of dovecot -n: # 2.2.25 (7be1766): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 4.4.36-8-default x86_64 openSUSE 42.2 (x86_64) auth_username_format = %Ln base_dir = /var/run/dovecot/ mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = file:~/sieve/;active=~/.dovecot.sieve sieve_trace_debug = yes } protocols = imap lmtp sieve service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/ssl/private/dovecot.crt ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/dovecot.pem ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } verbose_ssl = yes protocol lmtp { mail_plugins = sieve postmaster_address = postmaster@xxxxxxxxxxxxxx }

5 14

Doveadm field for message body without attachments
by Athanasios Viennas 11 Jan '18

11 Jan '18

Happy new year to all, what is the message field in doveadm data model representing the body without attachments? I need to dump the message body without the attachments so as to avoid unnecessary size of the exported files and while I am also not interested in this part of the message. with kind regards Athanasios

2 2

Ignored mount points forgotten on reboot
by Kenneth Porter 11 Jan '18

11 Jan '18

<https://wiki2.dovecot.org/Mountpoints> The list of ignored mount points are stored in a directory (/var/run) that's cleared on reboot. (On CentOS it's a RAM disk.) It seems this should be stored in /etc/dovecot to remain persistent across reboots. <https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard> I discovered this when I saw a dovecot error message in /var/log/maillog that some of my automounts (that don't contain mail directories) were missing. Latest dovecot from this distro is dovecot-2.2.10-8.el7.x86_64. Does the new release fix this? (I haven't seen any real problems from it, just log noise.)

1 0

Can Dovecot reject unencrypted mail?
by David Seaward 10 Jan '18

10 Jan '18

Hi, Is it possible to configure Dovecot to reject mail that is not encrypted. In other words: 1. If the user tries to send an unencrypted message from their MUA, the server rejects it. 2. If a third-party tries to send an unencrypted message to the user, the server rejects it. The end result would be that no mail stored on the server can be decrypted by the administrator. I am aware that: * "Encrypted" could mean a lot of things. I'm imagining GPG encryption, but am open to other supported methods. * This configuration would not suit everyone, e.g. someone posting to a public mailing list :) Regards, David

3 2

Re: Can Dovecot reject unencrypted mail?
by Ryan Beethe 10 Jan '18

10 Jan '18

Hi David, I don't know how to do what you want with dovecot, but what you are asking is easy and straightforward with Postfix. Postfix can easily be configured to feed mail through a milter ("mail filter") interface. You would just need to write a milter (there is a nice python library) that checks if the messages is "encrypted" to your specifications and tells Postfix to bounce the message if its not. This would be a nice backscatter-free solution. After you have the milter written, you specify it with the "smtpd_milters" option for Postfix. Ryan On Wed, Jan 10, 2018 at 02:08:38PM +0200, David Seaward wrote: > Hi, > > Is it possible to configure Dovecot to reject mail that is not > encrypted. In other words: > > 1. If the user tries to send an unencrypted message from their MUA, > the server rejects it. > > 2. If a third-party tries to send an unencrypted message to the user, > the server rejects it. > > The end result would be that no mail stored on the server can be > decrypted by the administrator. > > I am aware that: > > * "Encrypted" could mean a lot of things. I'm imagining GPG > encryption, but am open to other supported methods. > > * This configuration would not suit everyone, e.g. someone posting to > a public mailing list :) > > Regards, > David >

1 0

openssl question
by Selcuk Yazar 10 Jan '18

10 Jan '18

Hi we are using dovecot on secure port when i try to command openssl s_client -connect mail.mydomain:pop3s it works perfect. [image: Inline image 1] Also i check from https://www.sslshopper.com/ssl-checker.html web page i can see all correct ceritificate paths but i try to this command openssl s_client -connect mail.mydomain:pop3s -starttls imap it says CONNECTED and hang. second command is correct? also my thunderbird client doesnt work with 143 port and STARTTLS. is this general issue or have we config error? thanks in advance Selçuk YAZAR

2 1

Make fails with 2.3.0 on Mac OS X 10.7.5
by James Brown 10 Jan '18

10 Jan '18

Compile is failing for me. ./configure --with-mysql --with-ssl=openssl: ... checking for SSL_read in -lssl... yes checking openssl/ssl.h usability... yes checking openssl/ssl.h presence... yes checking for openssl/ssl.h... yes checking openssl/err.h usability... yes checking openssl/err.h presence... yes checking for openssl/err.h... yes checking if OpenSSL version is 1.0.1 or newer... true checking if OpenSSL version is 1.0.2 or better... true checking whether SSL_clear_options exists... no ... No problems. But then ‘make’ ends with: dcrypt-openssl.c:117: error: field 'ctx' has incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_ctx_hmac_destroy': dcrypt-openssl.c:451: warning: implicit declaration of function 'HMAC_cleanup' dcrypt-openssl.c: In function 'dcrypt_openssl_generate_ec_key': dcrypt-openssl.c:555: error: dereferencing pointer to incomplete type dcrypt-openssl.c:556: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_ecdh_derive_secret_local': dcrypt-openssl.c:588: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_ecdh_derive_secret_peer': dcrypt-openssl.c:668: error: dereferencing pointer to incomplete type dcrypt-openssl.c:697: error: dereferencing pointer to incomplete type dcrypt-openssl.c:698: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_store_private_key_dovecot': dcrypt-openssl.c:1461: error: dereferencing pointer to incomplete type dcrypt-openssl.c:1482: error: dereferencing pointer to incomplete type dcrypt-openssl.c:1489: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_load_private_key': dcrypt-openssl.c:1615: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_store_public_key': dcrypt-openssl.c:1765: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_private_to_public_key': dcrypt-openssl.c:1801: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_name2oid': dcrypt-openssl.c:2062: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_public_key_id_old': dcrypt-openssl.c:2112: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_private_key_id_old': dcrypt-openssl.c:2135: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_public_key_id_evp': dcrypt-openssl.c:2153: error: dereferencing pointer to incomplete type make[3]: *** [libdcrypt_openssl_la-dcrypt-openssl.lo] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Any suggestions? Thanks, James.

1 0

Re: openssl question
by Joseph Tam 10 Jan '18

10 Jan '18

> TLS handshaking: SSL_accept() failed: error:1408A0C1:SSL > routines:SSL3_GET_CLIENT_HELLO:no shared cipher > > our dovecot (2.0.9 on redhat) 10-ssl.conf file we have > > ssl_cipher_list = > kEECDH:+kEECDH+SHA:kEDH:+kEDH+SHA:+kEDH+CAMELLIA:kECDH:+kECDH+SHA:kRSA:+kRSA+SHA:+kRSA+CAMELLIA:!aNULL:!eNULL:!SSLv2:!RC4:!MD5:!DES:!EXP:!SEED:!IDEA:!3DES:!SSLv3 Offhand, I don't know of a fast way to match up client cipher specs and server cipher specs. The hard part is trying to figure out what the client is doing. Maybe you can turn on dovecot "verbose_ssl = yes" and that will dump SSL diagnostics logs to point out where server/client cipher negotiations fail. You can also try and run "openssl s_server -cipher 'kEECDH:+...'" on an alternate port/host, point your client at it, and let this utility dump out the SSL cipher negotions. Joseph Tam <jtam.home(a)gmail.com>

1 0