January 2018 - dovecot

Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000)
by Stefan Neben 11 Jan '18

11 Jan '18

Hello, i have a problem with a very huge mail account. The INBOX of this account has ~1.1 - ~1.7 million mails inside. And from time to time the index cache make problems and i got the following in the logs: --- Snip --- Jan 10 20:43:04 XXXXX dovecot: imap(XXXXX): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) --- Snap --- I skip the raw backtrace to keep the post "small", but can hand it in later if needed. Got this for UID FETCH, UID SEARCH, UID SORT, ... The size of the file dovecot.index.cache of INBOX was in the last seen problem situations between 1.1GB to 2.3GB. To solve the problem the last times was to reduce the amount of mails to under 1.7 million (seems to be the limit to trigger the problem), delete the index cache and let him rebuild with a controled request. But this is a odd situation (because it confused 3rd party tooling, which access this mailbox) and i cannot find any helpful informations to solve this problem. Sharding it into several folders to keep the amount of mails in INBOX small is not an option currently. If further informations are needed, just let me know. Thanks for any help! Kind regards Stefan

3 5

Managesieve cannot access script store
by dovelist 11 Jan '18

11 Jan '18

Hi, I am trying to get sieve working on a new OpenSuse leap 42.2 install. On my 'old' OpenSuse 13.2 machine it worked fine. The problem is that Managesieve can't access the script store and won't let me create any script. It says permission denied on ~/sieve directory. See log below. I 've activated debug logging, but that doesn't give any clues to me. Also, I've set the directory accessible to all, but Managesieve still complains. > cd ~ > ls -l drwx------ 1 rogier users 8340 5 feb 16:54 Maildir drwxrwxrwx 1 rogier users 24 5 feb 18:38 sieve To rule out client issues (kmail) I tested also with Manual TLS Login as described in: http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Troubleshooting Same result. I am puzzled. I can't find anything wrong in the dovecot configuration. The output of dovecot -n is shown below. Hope someone has a solution. A lot of mail is waiting to get sorted... Best Regards, Rogier The log: feb 05 20:22:18 p150 dovecot[12120]: managesieve-login: Login: user=<rogier>, method=PLAIN, rip=192.168.0.18, lip=192.168.0.20, mpid=12135, TLS, session=<gmb0bs1H5q/AqAAS> feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Effective uid=1000, gid=100, home=/home/rogier feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: maildir++: root=/home/rogier/Maildir, index=, indexpvt=, control=, inbox=/home/rogier/Maildir, alt= feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: Pigeonhole version 0.4.15 (97b3da0) initializing feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using active Sieve script path: /home/rogier/.dovecot.sieve feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using script storage path: /home/rogier/sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using permissions from /home/rogier/sieve/: mode=0777 gid=-1 feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: sync: Synchronization active feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Error: sieve: file storage: Failed to list scripts: opendir(/home/rogier/sieve) failed: Permission denied Output of dovecot -n: # 2.2.25 (7be1766): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 4.4.36-8-default x86_64 openSUSE 42.2 (x86_64) auth_username_format = %Ln base_dir = /var/run/dovecot/ mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = file:~/sieve/;active=~/.dovecot.sieve sieve_trace_debug = yes } protocols = imap lmtp sieve service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/ssl/private/dovecot.crt ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/dovecot.pem ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } verbose_ssl = yes protocol lmtp { mail_plugins = sieve postmaster_address = postmaster@xxxxxxxxxxxxxx }

5 14

Doveadm field for message body without attachments
by Athanasios Viennas 11 Jan '18

11 Jan '18

Happy new year to all, what is the message field in doveadm data model representing the body without attachments? I need to dump the message body without the attachments so as to avoid unnecessary size of the exported files and while I am also not interested in this part of the message. with kind regards Athanasios

2 2

Ignored mount points forgotten on reboot
by Kenneth Porter 11 Jan '18

11 Jan '18

<https://wiki2.dovecot.org/Mountpoints> The list of ignored mount points are stored in a directory (/var/run) that's cleared on reboot. (On CentOS it's a RAM disk.) It seems this should be stored in /etc/dovecot to remain persistent across reboots. <https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard> I discovered this when I saw a dovecot error message in /var/log/maillog that some of my automounts (that don't contain mail directories) were missing. Latest dovecot from this distro is dovecot-2.2.10-8.el7.x86_64. Does the new release fix this? (I haven't seen any real problems from it, just log noise.)

1 0

Can Dovecot reject unencrypted mail?
by David Seaward 10 Jan '18

10 Jan '18

Hi, Is it possible to configure Dovecot to reject mail that is not encrypted. In other words: 1. If the user tries to send an unencrypted message from their MUA, the server rejects it. 2. If a third-party tries to send an unencrypted message to the user, the server rejects it. The end result would be that no mail stored on the server can be decrypted by the administrator. I am aware that: * "Encrypted" could mean a lot of things. I'm imagining GPG encryption, but am open to other supported methods. * This configuration would not suit everyone, e.g. someone posting to a public mailing list :) Regards, David

3 2

Re: Can Dovecot reject unencrypted mail?
by Ryan Beethe 10 Jan '18

10 Jan '18

Hi David, I don't know how to do what you want with dovecot, but what you are asking is easy and straightforward with Postfix. Postfix can easily be configured to feed mail through a milter ("mail filter") interface. You would just need to write a milter (there is a nice python library) that checks if the messages is "encrypted" to your specifications and tells Postfix to bounce the message if its not. This would be a nice backscatter-free solution. After you have the milter written, you specify it with the "smtpd_milters" option for Postfix. Ryan On Wed, Jan 10, 2018 at 02:08:38PM +0200, David Seaward wrote: > Hi, > > Is it possible to configure Dovecot to reject mail that is not > encrypted. In other words: > > 1. If the user tries to send an unencrypted message from their MUA, > the server rejects it. > > 2. If a third-party tries to send an unencrypted message to the user, > the server rejects it. > > The end result would be that no mail stored on the server can be > decrypted by the administrator. > > I am aware that: > > * "Encrypted" could mean a lot of things. I'm imagining GPG > encryption, but am open to other supported methods. > > * This configuration would not suit everyone, e.g. someone posting to > a public mailing list :) > > Regards, > David >

1 0

openssl question
by Selcuk Yazar 10 Jan '18

10 Jan '18

Hi we are using dovecot on secure port when i try to command openssl s_client -connect mail.mydomain:pop3s it works perfect. [image: Inline image 1] Also i check from https://www.sslshopper.com/ssl-checker.html web page i can see all correct ceritificate paths but i try to this command openssl s_client -connect mail.mydomain:pop3s -starttls imap it says CONNECTED and hang. second command is correct? also my thunderbird client doesnt work with 143 port and STARTTLS. is this general issue or have we config error? thanks in advance Selçuk YAZAR

2 1

Make fails with 2.3.0 on Mac OS X 10.7.5
by James Brown 10 Jan '18

10 Jan '18

Compile is failing for me. ./configure --with-mysql --with-ssl=openssl: ... checking for SSL_read in -lssl... yes checking openssl/ssl.h usability... yes checking openssl/ssl.h presence... yes checking for openssl/ssl.h... yes checking openssl/err.h usability... yes checking openssl/err.h presence... yes checking for openssl/err.h... yes checking if OpenSSL version is 1.0.1 or newer... true checking if OpenSSL version is 1.0.2 or better... true checking whether SSL_clear_options exists... no ... No problems. But then ‘make’ ends with: dcrypt-openssl.c:117: error: field 'ctx' has incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_ctx_hmac_destroy': dcrypt-openssl.c:451: warning: implicit declaration of function 'HMAC_cleanup' dcrypt-openssl.c: In function 'dcrypt_openssl_generate_ec_key': dcrypt-openssl.c:555: error: dereferencing pointer to incomplete type dcrypt-openssl.c:556: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_ecdh_derive_secret_local': dcrypt-openssl.c:588: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_ecdh_derive_secret_peer': dcrypt-openssl.c:668: error: dereferencing pointer to incomplete type dcrypt-openssl.c:697: error: dereferencing pointer to incomplete type dcrypt-openssl.c:698: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_store_private_key_dovecot': dcrypt-openssl.c:1461: error: dereferencing pointer to incomplete type dcrypt-openssl.c:1482: error: dereferencing pointer to incomplete type dcrypt-openssl.c:1489: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_load_private_key': dcrypt-openssl.c:1615: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_store_public_key': dcrypt-openssl.c:1765: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_private_to_public_key': dcrypt-openssl.c:1801: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_name2oid': dcrypt-openssl.c:2062: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_public_key_id_old': dcrypt-openssl.c:2112: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_private_key_id_old': dcrypt-openssl.c:2135: error: dereferencing pointer to incomplete type dcrypt-openssl.c: In function 'dcrypt_openssl_public_key_id_evp': dcrypt-openssl.c:2153: error: dereferencing pointer to incomplete type make[3]: *** [libdcrypt_openssl_la-dcrypt-openssl.lo] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Any suggestions? Thanks, James.

1 0

Re: openssl question
by Joseph Tam 10 Jan '18

10 Jan '18

> TLS handshaking: SSL_accept() failed: error:1408A0C1:SSL > routines:SSL3_GET_CLIENT_HELLO:no shared cipher > > our dovecot (2.0.9 on redhat) 10-ssl.conf file we have > > ssl_cipher_list = > kEECDH:+kEECDH+SHA:kEDH:+kEDH+SHA:+kEDH+CAMELLIA:kECDH:+kECDH+SHA:kRSA:+kRSA+SHA:+kRSA+CAMELLIA:!aNULL:!eNULL:!SSLv2:!RC4:!MD5:!DES:!EXP:!SEED:!IDEA:!3DES:!SSLv3 Offhand, I don't know of a fast way to match up client cipher specs and server cipher specs. The hard part is trying to figure out what the client is doing. Maybe you can turn on dovecot "verbose_ssl = yes" and that will dump SSL diagnostics logs to point out where server/client cipher negotiations fail. You can also try and run "openssl s_server -cipher 'kEECDH:+...'" on an alternate port/host, point your client at it, and let this utility dump out the SSL cipher negotions. Joseph Tam <jtam.home(a)gmail.com>

1 0

zlib plugin producing errors on 2.3.0
by Adam Weinberger 09 Jan '18

09 Jan '18

Hello, I use the zlib and imap_zlib plugins on FreeBSD. As of 2.3.0, my logs are producing these errors every so often, but AFAICT the messages themselves aren't getting corrupted. Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0) Fatal: master: service(imap): child 80128 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes }) Panic: file ostream-zlib.c: line 36 (o_stream_zlib_close): assertion failed: (zstream->ostream.finished || zstream->ostream.ostream.stream_errno != 0) Fatal: master: service(imap): child 80266 killed with signal 6 (core not dumped - set service imap { drop_priv_before_exec=yes }) They always come in pairs like that. Following is my doveconf. Let me know what else I can provide here. Thanks! # Adam # 2.3.0 (c8b89eb): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.0 (d68c23a1) # OS: FreeBSD 11.1-RELEASE-p6 amd64 nullfs auth_mechanisms = plain login first_valid_gid = 1021 first_valid_uid = 1021 last_valid_gid = 1022 last_valid_uid = 1022 listen = imap.jail.apnoea.adamw.org mail_location = mdbox:/mail/%u/mail mail_plugins = " zlib virtual fts fts_lucene" mail_prefetch_count = 5 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox FreeBSD { autoexpunge = 17 weeks } mailbox FreeBSD/TodaysCommits { autoexpunge = 2 days } mailbox FreeBSD/automation { autoexpunge = 1 days } mailbox FreeBSD/portmgr { autoexpunge = 26 weeks } mailbox FreeBSD/ports { autoexpunge = 12 weeks } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { autoexpunge = 30 days special_use = \Trash } mailbox spam/probably { autoexpunge = 30 days } mailbox spam/totally { autoexpunge = 5 days } prefix = separator = / } passdb { args = scheme=BLF-CRYPT username_format=%u /path/to/userdb.passwd driver = passwd-file } plugin { fts = lucene fts_autoindex = yes fts_autoindex_max_recent_msgs = 25 fts_lucene = whitespace_chars=@ sieve = file:/scripts/sieve/%u.sieve;bindir=/mail/%u/sieve/ sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.filter +editheader sieve_filter_bin_dir = /scripts/sieve/filter sieve_pipe_bin_dir = /scripts/sieve/pipe sieve_plugins = sieve_extprograms } postmaster_address = postmaster@... protocols = imap lmtp service imap-login { inet_listener imaps { port = 0 } } service lmtp { inet_listener lmtp { port = 24 } } ssl = required ssl_cert = </path/to/fullchain.pem ssl_cipher_list = TLSv1.2+AEAD:!aNULL ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = username_format=%u /path/to/userdb.passwd driver = passwd-file } protocol lmtp { mail_plugins = " zlib virtual fts fts_lucene sieve" } protocol imap { mail_max_userip_connections = 20 mail_plugins = " zlib virtual fts fts_lucene imap_zlib" } protocol lda { mail_plugins = " zlib virtual fts fts_lucene sieve" } -- Adam Weinberger adamw(a)adamw.org http://www.adamw.org

5 5