- dovecot - dovecot.org

Core Dumps with dovecot-2.2 -git as of 2.2.devel (eaf276b33)
by Reuben Farrelly 07 Oct '17

07 Oct '17

This appears to be breaking dsync at the moment and is caused by something committed in the last week or so. This is with the current -git of dovecot-2.2 (eaf276b33). The crash happens all the time - the logs are full of this message. Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Warning: I/O leak: 0x7f8bb6581b4c (ostream-file.c:350, fd 14) Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Panic: file ioloop.c: line 126 (io_remove_full): assertion failed: (io->callback != NULL) Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xd48a6) [0x7f8bb654a8a6] -> /usr/lib64/dovecot/li bdovecot.so.0(+0xd5e26) [0x7f8bb654be26] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8bb654ac39] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf5cb0) [0x7f8bb6 56bcb0] -> /usr/lib64/dovecot/libdovecot.so.0(io_remove+0x1d) [0x7f8bb656be31] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10b5f8) [0x7f8bb65815f8] -> /usr/lib64/dovec ot/libdovecot.so.0(o_stream_cork+0x5a) [0x7f8bb657e289] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0x6406) [0x7f8bb12d6406] -> /usr/lib64/dovecot/libssl_iost ream_openssl.so(openssl_iostream_bio_sync+0x18) [0x7f8bb12d6af6] -> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa52e) [0x7f8bb12da52e] -> / usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa6ef) [0x7f8bb12da6ef] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0xc7) [0x7f8bb657e765] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_send+0x61) [0x7f8bb657e688] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10dc2c) [0x7f8bb6583c2c] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10ddd4) [0x7f8bb6583dd4] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0xc7) [0x7f8bb657e765] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_nsendv+0x55) [0x7f8bb657e93f] -> /usr/lib64/dovecot/libdovecot.so.0(o_stream_nsend+0x61) [0x7f8bb657e8d3] -> dovecot/doveadm-server() [0x444a35] -> /usr/lib64/dovecot/libdovecot.so.0(i_warning+0xce) [0x7f8bb654afdc] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ca) [0x7f8bb656d8cd] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-server() [0x445dd6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0xd6) [0x7f8bb656d241] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x225) [0x7f8bb656fe58] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x3e) [0x7f8bb656d422] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0xbe) [0x7f8bb656d345] Oct 7 20:31:24 thunderstorm.reub.net dovecot: doveadm: Fatal: master: service(doveadm): child 27186 killed with signal 6 (core dumped) thunderstorm reuben # file core core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'dovecot/doveadm-server', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/usr/libexec/dovecot/doveadm-server', platform: 'x86_64' thunderstorm reuben # gdb /usr/libexec/dovecot/doveadm-server core GNU gdb (Gentoo 8.0.1 vanilla) 8.0.1 Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://bugs.gentoo.org/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/doveadm-server...done. [New LWP 27186] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/doveadm-server'. Program terminated with signal SIGABRT, Aborted. #0 0x00007f8bb60e7f70 in raise () from /lib64/libc.so.6 (gdb) bt full #0 0x00007f8bb60e7f70 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x00007f8bb60e9aa7 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x00007f8bb654a8f5 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 backtrace = 0x228e4d8 "/usr/lib64/dovecot/libdovecot.so.0(+0xd48a6) [0x7f8bb654a8a6] -> /usr/lib64/dovecot/libdovecot.so.0(+0xd5e26) [0x7f8bb654be26] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8bb654ac39] -> /usr"... #3 0x00007f8bb654be26 in i_internal_fatal_handler (ctx=0x7ffef81daa50, format=0x7f8bb65ba898 "file %s: line %d (%s): assertion failed: (%s)", args=0x7ffef81daa70) at failures.c:670 status = 0 #4 0x00007f8bb654ac39 in i_panic (format=0x7f8bb65ba898 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffef81dab50, reg_save_area = 0x7ffef81daa90}} #5 0x00007f8bb656bcb0 in io_remove_full (_io=0x220a100, closed=false) at ioloop.c:126 io = 0x22694a0 __FUNCTION__ = "io_remove_full" #6 0x00007f8bb656be31 in io_remove (io=0x220a100) at ioloop.c:169 No locals. #7 0x00007f8bb65815f8 in o_stream_file_cork (stream=0x220a010, set=true) at ostream-file.c:342 fstream = 0x220a010 ret = 32766 __FUNCTION__ = "o_stream_file_cork" #8 0x00007f8bb657e289 in o_stream_cork (stream=0x220a0a0) at ostream.c:146 _stream = 0x220a010 #9 0x00007f8bb12d6406 in openssl_iostream_bio_output (ssl_io=0x220aa00) at iostream-openssl.c:325 bytes = 140238036407941 max_bytes = 18446742978492891135 sent = 35692560 buffer = "\027\003\003\000\001\000\000\000\020\240 \002\000\000\000\000\200\254\035\370\376\177\000\000\022\343W\266\213\177\000\000`\214\033\002\000\000\000\000\240\240 \002\000\000\000\000`\214\033\002\000\000\000\000\020\240 \002\000\000\000\000\320\314\035\370\376\177\000\000\236f-\261\213\177\000\000\025\071\260fK\247\000\000\000\252 \002\000\000\000\000`\214\033", '\000' <repeats 14 times>, "\315\035\370\376\177\000\000\236f-\261\213\177\000\000\225\305^\001?\236,,\000\252 \002\000\000\000\000=$h\000\272Å«\033\000\000\000\000\000\000\000\000\221\371Í¦f\237\035\224\335-\365\214l\333\347|\005\310\345\035/\200r\242\222}\313I\226G\313*\223\364\312\310\313j\263\260/\245"... bytes_sent = false ret = 32766 __FUNCTION__ = "openssl_iostream_bio_output" #10 0x00007f8bb12d6af6 in openssl_iostream_bio_sync (ssl_io=0x220aa00) at iostream-openssl.c:459 ret = false #11 0x00007f8bb12da52e in o_stream_ssl_flush_buffer (sstream=0x220aca0) at ostream-openssl.c:110 pos = 6 ret = 6 #12 0x00007f8bb12da6ef in o_stream_ssl_sendv (stream=0x220aca0, iov=0x7ffef81dcd70, iov_count=1) at ostream-openssl.c:155 sstream = 0x220aca0 bytes_sent = 6 #13 0x00007f8bb657e765 in o_stream_sendv (stream=0x220ad30, iov=0x7ffef81dcd70, iov_count=1) at ostream.c:260 _stream = 0x220aca0 i = 1 total_size = 6 ret = 35771392 __FUNCTION__ = "o_stream_sendv" #14 0x00007f8bb657e688 in o_stream_send (stream=0x220ad30, data=0x2308850, size=6) at ostream.c:239 iov = {iov_base = 0x2308850, iov_len = 6} #15 0x00007f8bb6583c2c in o_stream_multiplex_sendv (mstream=0x221d270) at ostream-multiplex.c:90 channel = 0x0 ret = 0 #16 0x00007f8bb6583dd4 in o_stream_multiplex_ochannel_sendv (stream=0x221d720, iov=0x7ffef81dcef0, iov_count=1) at ostream-multiplex.c:124 channel = 0x221d720 ret = 7354428521 total = 1 #17 0x00007f8bb657e765 in o_stream_sendv (stream=0x221d7b0, iov=0x7ffef81dcef0, iov_count=1) at ostream.c:260 _stream = 0x221d720 i = 1 total_size = 1 ret = 36233296 __FUNCTION__ = "o_stream_sendv" #18 0x00007f8bb657e93f in o_stream_nsendv (stream=0x221d7b0, iov=0x7ffef81dcef0, iov_count=1) at ostream.c:295 No locals. #19 0x00007f8bb657e8d3 in o_stream_nsend (stream=0x221d7b0, data=0x7ffef81dcf42, size=1) at ostream.c:287 iov = {iov_base = 0x7ffef81dcf42, iov_len = 1} #20 0x0000000000444a35 in doveadm_server_log_handler (ctx=0x7f8bb6805bd0 <failure_ctx_warning>, format=0x7f8bb65bac69 "Raw backtrace for leaks: %s", args=0x7ffef81dcfb0) at client-connection.c:64 c = 3 '\003' corked = false str = 0x228e050 ptr = 0x0 start = 0x228e088 "Raw backtrace for leaks: /usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ae) [0x7f8bb656d8b1] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-ser"... va = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffef81dd090, reg_save_area = 0x7ffef81dcfd0}} _data_stack_cur_id = 4 #21 0x00007f8bb654afdc in i_warning (format=0x7f8bb65bac69 "Raw backtrace for leaks: %s") at failures.c:325 old_errno = 0 args = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffef81dd090, reg_save_area = 0x7ffef81dcfd0}} #22 0x00007f8bb656d8cd in io_loop_destroy (_ioloop=0x7ffef81dd138) at ioloop.c:766 backtrace = 0x228dc38 "/usr/lib64/dovecot/libdovecot.so.0(io_loop_destroy+0x2ae) [0x7f8bb656d8b1] -> dovecot/doveadm-server() [0x4454d1] -> dovecot/doveadm-server() [0x445793] -> dovecot/doveadm-server() [0x445dd6] -> /usr/"... ioloop = 0x221ddc0 to_idx = 0x221dfe0 item = 0x0 leaks = true __FUNCTION__ = "io_loop_destroy" #23 0x00000000004454d1 in doveadm_cmd_handle (conn=0x22059d8, cmd_name=0x21b8c20 "dsync-server", argc=3, argv=0x21b8c50, cctx=0x7ffef81dd1b0) at client-connection.c:337 ioloop = 0x0 prev_ioloop = 0x21bfc40 cmd = 0x0 mail_cmd = 0x21c65e8 mctx = 0x221d9f8 cmd_ver2 = 0x0 #24 0x0000000000445793 in client_handle_command (conn=0x22059d8, args=0x21b8c40) at client-connection.c:399 cctx = {cmd = 0x0, argc = 0, argv = 0x0, username = 0x21b8c19 "reuben", cli = false, tcp_server = true, local_ip = {family = 10, u = {ip6 = {__in6_u = { ---Type <return> to continue, or q <return> to quit--- Reuben

1 0

dovecot: master: Warning: Sent SIGKILL to 100 imap-login processes
by absolutely_free＠libero.it 07 Oct '17

07 Oct '17

Hi, I am using Dovecot 2.2.32 (dfbe293d4) I noticed lots of messages like: dovecot: master: Warning: Sent SIGKILL to 100 imap-login processes in /var/log/maillog I commented out "process_limit" service imap { # Most of the memory goes to mmap()ing files. You may need to increase this # limit if you have huge mailboxes. #vsz_limit = $default_vsz_limit # Max. number of IMAP processes (connections) process_limit = 1024 } but it seems that value doesn't change (still 100). Any suggestions? Thank you very much!

2 2

Re: moving from mysql to pgsql
by Joseph Tam 06 Oct '17

06 Oct '17

mws(a)alpenjodel.de writes: > 1) select digest('testing123','sha512'); > result: > ?\x4120117b3190ba5e24044732b0b09aa9ed50eb1567705abcbfa78431a4e0a96b1152ed7f4925966b1c82325e186a8100e692e6d2fcb6702572765820d25c7e9e > login fails I think 2 problems here: 1) the encoding should be some modified base64; and 2) this appears to compute the SHA512 hash, *not* the SHA512-Crypt hash, which is different. It involves adding a salt and doing many iterations of SHA512. A totally different algorithm. Perusing the PostgreSQL man pages, I think you need something like crypt(password,gen_salt('sha512')) I made the 'sha512' up -- I can't find PostgresSQL docs stating whether it supports this value. The docs I found support the older SHA1 ($5$) crypt hashes. If your version doesn't support creating SHA512 ($6$) salts, you can create your own by replacing gen_salt() with "$6$" + base64(long random value), and feed that to crypt(). Just as long as PostgreSQL uses the system crypt() and not its own implementation, it should produce a usable hash. Joseph Tam <jtam.home(a)gmail.com>

1 0

How to require client SSL certificate, except for local connections
by Evan Martin 06 Oct '17

06 Oct '17

Is there any way to make Dovecot 2.2.22 not require a client SSL certificate for a local IMAP connection, but require it for any remote IMAP connection? My server is configured to require client certificates: ssl = required ... auth_ssl_require_client_cert = yes I tried adding the following to create an exception for localhost: remote 127.0.0.1 { ssl = no auth_ssl_require_client_cert = no disable_plaintext_auth = no } But Dovecot fails to start with: doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 81: Auth settings not supported inside local/remote blocks: Is there any other way to do this? I don't need to override any other auth settings, just that one. I could probably use a Unix socket, if that would help. Thanks, Evan

1 0

Director DNS
by Travis Dolan 06 Oct '17

06 Oct '17

Hello, I am currently running 2.2.31. I have a cluster of dovecot instances running in AWS. - Two instances running Director/Proxy - Two instances running Dovecot. -- Storage for the Dovecot instances is EFS. My setup works, requests for 143 hit my Director cluster and are proxied to an appropriate dovecot backend. My question is how often if at all the Director service looks up the addresses for the following setting... *director_servers = mailproxy.internal* I have created a Lambda to continually update the values for the A records found in the *.internal* zone since EC2 instances can come and go for a variety of reasons. If those values changes will the Director service "notice" or will a "service dovecot restart" be needed? Thank in advance.

2 1

Re: ACL permission: give permission to move email into different folder
by Davide Marchi 06 Oct '17

06 Oct '17

> I've set ACL for two user (two primary email address, no alias), where > these users must not be able to delete email from imap server. > Now I would like to add (from the client) the ability to move any > incoming mail into different directory, if possible into an existing > directory chosen by me. Maybe it is not possible with Dovecot? Many Thanks Davide

2 1

seccomp filter for dovecot
by Phoenix Krypt 06 Oct '17

06 Oct '17

Hi, I would like to contribute to dovecot by adding seccomp system call filtering. Is this something you would like to merge into the dovecot codebase? If so, I can put up a PR on github once I complete it. Thanks, Archana

2 2

Dovecot Performance Issues on VMWARE Esxi
by Holger 06 Oct '17

06 Oct '17

Hello, I'm new to dovcot, but have serious issues with dovecot and IMAP. I'm struggling with low IMAP performance but have no glue, where to start my search. It is a small office setup. 10 Mailaccounts. Mine is the biggest with 4GB in size. But searching for mail takes several seconds... Zu switch to specific folder also takes some seconds. To long in my opinion. I use esxi 6.0 with local datastorage lenovo megaraid raid5 3x2TB SAS 7.2K. On the dovecot appliance i see high iowait. Some issues dovecot or the VM config? Any help i would appreciate... -- Sent from: http://dovecot.2317879.n4.nabble.com/

3 2

moving from mysql to pgsql
by Magnus 06 Oct '17

06 Oct '17

Hello, I hope that this mailing list is "alive", since I am looking for a solution for my problem for a long time. I would like to migrate my existing dovecot installation from mysql to pgsql. But I have problems with the passwords when using pgsql. The existing and working mysql-based installation looks like this: dovecot-sql.conf.ext: driver = mysql default_pass_scheme = SHA512-CRYPT Users are created like this: INSERT INTO mls_user (idx,domain,password,email) VALUES (1,99,ENCRYPT('Test'),'mws(a)alpenjodel.de'); This setup is working, which I can verify like this: $ telnet localhost 143 * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=CRAM-MD5] Dovecot ready. a login mws(a)alpenjodel.de Test OK Now let's take a look at the pgsql version of the setup: dovecot-sql.conf.ext: driver = pgsql default_pass_scheme = SHA512-CRYPT Users are created like this: INSERT INTO mls_user (idx,domain,password,email) VALUES (1,99,crypt('Test',gen_salt('des')),'mws(a)alpenjodel.de'); This setup is not working: $ telnet localhost 143 * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=DIGEST-MD5 AUTH=CRAM-MD5] Dovecot ready. a login mws(a)alpenjodel.de Test a NO [AUTHENTICATIONFAILED] Authentication failed. Assumptions: - I believe that the mysql encrypt function uses the crypt system call, which in turn uses the DES algorithm with a random salt. - I believe that the same is done with the pgsql function call crypt('Test',gen_salt('des')). But obviously some of these assumptions must be wrong. Besides that, the variable "default_pass_scheme" is set to "SHA512-CRYPT" in both cases. But obviously, not SHA but DES is used by the working mysql-based setup. I don't understand that. Could someone please explain the relationship between the default_pass_scheme variable and the encryption/hashing algorithm used to store the user passwords? And finally: What can I do to migrate to pgsql? Thank you Magnus

4 3

v2.2.33 release candidate released
by Timo Sirainen 05 Oct '17

05 Oct '17

https://dovecot.org/releases/2.2/rc/dovecot-2.2.33.rc1.tar.gz https://dovecot.org/releases/2.2/rc/dovecot-2.2.33.rc1.tar.gz.sig There are a couple more small changes still coming, but this should be very close to the final release. I'm especially interested in hearing if there are any problems with doveadm log proxying or with director. We've improved our automated director tests quite a lot now, and fixed some rarely occurring bugs. * doveadm director commands wait for the changes to be visible in the whole ring before they return. This is especially useful in testing. * Environments listed in import_environment setting are now set or preserved when executing standalone commands (e.g. doveadm) + doveadm proxy: Support proxying logs. Previously the logs were visible only in the backend's logs. + Added %{if}, see https://wiki2.dovecot.org/Variables#Conditionals + Added a new notify_status plugin, which can be used to update dict with current status of a mailbox when it changes. See https://wiki2.dovecot.org/Plugins/NotifyStatus + Mailbox list index can be disabled for a namespace by appending ":LISTINDEX=" to location setting. + dsync/imapc: Added dsync_hashed_headers setting to specify which headers are used to match emails. + pop3-migration: Add pop3_migration_ignore_extra_uidls=yes to ignore mails that are visible in POP3 but not IMAP. This could happen if new mails were delivered during the migration run. + pop3-migration: Further improvements to help with Zimbra + pop3-migration: Cache POP3 UIDLs in imapc's dovecot.index.cache if indexes are enabled. These are used to optimize incremental syncs. + cassandra, dict-sql: Use prepared statements if protocol version>3. - sdbox: Mails were always opened when expunging, unless mail_attachment_fs was explicitly set to empty. - lmtp/doveadm proxy: hostip passdb field was ignored, which caused unnecessary DNS lookups if host field wasn't an IP - lmtp proxy: Fix crash when receiving unexpected reply in RCPT TO - quota_clone: Update also when quota is unlimited (broken in v2.2.31) - mbox, zlib: Fix assert-crash when accessing compressed mbox - doveadm director kick -f parameter didn't work - doveadm director flush <host> resulted flushing all hosts, if <host> wasn't an IP address. - director: Various fixes to handling backend/director changes at abnormal times, especially while ring was unsynced. These could have resulted in crashes, non-optimal behavior or ignoring some of the changes. - director: Use less CPU in imap-login processes when moving/kicking many users. - lmtp: Session IDs were duplicated/confusing with multiple RCPT TOs when lmtp_rcpt_check_quota=yes - doveadm sync -1 fails when local mailboxes exist that do not exist remotely. This commonly happened when lazy_expunge mailbox was autocreated when incremental sync expunged mails.

1 0