- dovecot - dovecot.org

doveadm stats top crash
by Sven Roellig 31 Oct '16

31 Oct '16

Hi,when i start doveadm stats top it crash Panic: key not found from hash Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9560e) [0x7fe0e5e5c60e] -> /usr/lib/dovecot/libdovecot.so.0(+0x95688) [0x7fe0e5e5c688] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fe0e5df496e] -> doveadm(+0x25290) [0x7fe0e6abb290] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xfb) [0x7fe0e5e7104b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xc3) [0x7fe0e5e72643] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fe0e5e71265] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x30) [0x7fe0e5e71400] -> doveadm(+0x256a1) [0x7fe0e6abb6a1] -> doveadm(doveadm_cmd_ver2_to_cmd_wrapper+0x23a) [0x7fe0e6acf8aa] -> doveadm(doveadm_cmd_run_ver2+0x560) [0x7fe0e6ad0600] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x7fe0e6ad0657] -> doveadm(main+0x1e4) [0x7fe0e6ab07c4] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fe0e5a3db45] -> doveadm(+0x1aba8) [0x7fe0e6ab0ba8] Aborted Sven

1 0

Dovecot Proxy and Director
by Gandalf Corvotempesta 29 Oct '16

29 Oct '16

Hi, just a simple question: by using a directory and a proxy, I would be able to totally hide the pop3/imap server ip addresses from outside? I'm asking this because I would like to hide the real server IP for security reasosn (DDoS and so on). The proxy would be placed on servers with high bandwidth while the pop3/imap dovecot servers are placed in a small datacenter that would go down easily in case of attack

2 2

Panic: file dsync-brain-mailbox.c: line 358 (dsync_brain_sync_mailbox_deinit): assertion failed: (brain->failed || brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_CHANGED)
by Gilles Chauvin 28 Oct '16

28 Oct '16

Hello, Here is a Panic that happened while doing some testing with two servers both running Dovecot v2.2.26 on CentOS 7. These are test servers owning 32 accounts whose data were copied from our production server. What I've done is: server01# doveadm force-resync -A '*' server01# doveadm replicator replicate -f '*' For 5 accounts I obtained the following crash: 2016-10-28T14:09:43.236946+02:00 server01 dovecot: dsync-server(someuser): Panic: file dsync-brain-mailbox.c: line 358 (dsync_brain_sync_mailbox_deinit): assertion failed: (brain->failed || brain->sync_type == DSYNC_BRAIN_SYNC_TYPE_CHANGED) 2016-10-28T14:09:43.237441+02:00 server01 dovecot: dsync-server(someuser): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x8f7e0) [0x7f3d9318d7e0] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x8f8be) [0x7f3d9318d8be] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3d9312b9be] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](dsync_brain_sync_mailbox_deinit+0x163) [0x438243] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](dsync_brain_slave_recv_mailbox+0x277) [0x438da7] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](dsync_brain_run+0x5fe) [0x4368be] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x436c71] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x44becf] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f3d931a0c3c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f3d931a1fd7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f3d931a0cc5] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3d931a0e78] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x41fc7e] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x421256] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x433654] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f3d931a0c3c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xe7) [0x7f3d931a1fd7] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f3d931a0cc5] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3d931a0e78] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f3d93131a23] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox](main+0x197) [0x413c87] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f3d92d5db15] -> dovecot/doveadm-server [10.0.0.2 someuser slave_recv_mailbox]() [0x413d25] 2016-10-28T14:09:43.238013+02:00 server01 dovecot: dsync-server(someuser): Fatal: master: service(doveadm): child 96390 killed with signal 6 (core dumps disabled) 2016-10-28T14:09:43.505098+02:00 server01 dovecot: dsync-server(someuser): Error: read(server02.localdomain) failed: read(size=5807) failed: Connection reset by peer (last sent=mailbox_state, last recv=mailbox_state) Regards, Gilles.

2 2

use a second userdb that only returns extra fields
by Alexander 'Leo' Bergolth 28 Oct '16

28 Oct '16

Hi! Is it possible to get all basic userdb information from the passwd userdb and add a second userdb of type checkpassword that only sets some additional extra fields like namespaces? I tried the following setup: -------------------- 8< -------------------- userdb { driver = passwd result_success = continue-ok } userdb { driver = checkpassword args = /usr/local/sbin/dovecot-userdb.py skip = never } -------------------- 8< -------------------- ... but it seems that as soon as the second userdb is active, dovecot doesn't take settings like uid and gid from the first userdb anymore. (Even if I don't set userdb_uid and userdb_gid in checkpassword.) On the other hand there are no environment variables that pass the settings from the previous lookup to the checkpassword script. Cheers, --leo -- e-mail ::: Leo.Bergolth (at) wu.ac.at fax ::: +43-1-31336-906050 location ::: IT-Services | Vienna University of Economics | Austria

1 0

Dovecot POP3 - Enable POP for mail that arrives from now on
by Ricardo Machini Barbosa 28 Oct '16

28 Oct '16

Hello, Someone knows some method for implement feature like Gmail and Zimbra, about download POP3 messages only that "arrives from now" ? Regards, Ricardo Machini

1 0

v2.2.26 released
by Timo Sirainen 28 Oct '16

28 Oct '16

http://dovecot.org/releases/2.2/dovecot-2.2.26.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.26.tar.gz.sig There were some changes since rc1: https://github.com/dovecot/core/commit/54d654098032d96975b70749b505fae538e9… Mainly there are quite a lot of director fixes and improvements. Here's the full list of changes: * master: Removed hardcoded 511 backlog limit for listen(). The kernel should limit this as needed. * doveadm import: Source user is now initialized the same as target user. Added -U parameter to override the source user. * Mailbox names are no longer limited to 16 hierarchy levels. We'll check another way to make sure mailbox names can't grow larger than 4096 bytes. + Added a concept of "alternative usernames" by returning user_* extra field(s) in passdb. doveadm proxy list shows these alt usernames in "doveadm proxy list" output. "doveadm director&proxy kick" adds -f <passdb field> parameter. The alt usernames don't have to be unique, so this allows creation of user groups and kicking them in one command. + auth: passdb/userdb dict allows now %variables in key settings. + auth: If passdb returns noauthenticate=yes extra field, assume that it only set extra fields and authentication wasn't actually performed. + auth: passdb static now supports password={scheme} prefix. + auth, login_log_format_elements: Added %{local_name} variable, which expands to TLS SNI hostname if given. + imapc: Added imapc_max_line_length to limit maximum memory usage. + imap, pop3: Added rawlog_dir setting to store IMAP/POP3 traffic logs. This replaces at least partially the rawlog plugin. + dsync: Added dsync_features=empty-header-workaround setting. This makes incremental dsyncs work better for servers that randomly return empty headers for mails. When an empty header is seen for an existing mail, dsync assumes that it matches the local mail. + doveadm sync/backup: Added -I <max size> parameter to skip too large mails. + doveadm sync/backup: Fixed -t parameter and added -e for "end date". + doveadm mailbox metadata: Added -s parameter to allow accessing server metadata by using empty mailbox name. + Added "doveadm service status" and "doveadm process status" commands. + director: Added director_flush_socket. See http://wiki2.dovecot.org/Director#Flush_socket + doveadm director flush: Users are now moved only max 100 at a time to avoid load spikes. --max-parallel parameter overrides this. + Added FILE_LOCK_SLOW_WARNING_MSECS environment, which logs a warning if any lock is waited on or kept for this many milliseconds. - master process's listener socket was leaked to all child processes. This might have allowed untrusted processes to capture and prevent "doveadm service stop" comands from working. - login proxy: Fixed crash when outgoing SSL connections were hanging. - auth: userdb fields weren't passed to auth-workers, so %{userdb:*} from previous userdbs didn't work there. - auth: Each userdb lookup from cache reset its TTL. - auth: Fixed auth_bind=yes + sasl_bind=yes to work together - auth: Blocking userdb lookups reset extra fields set by previous userdbs. - auth: Cache keys didn't include %{passdb:*} and %{userdb:*} - auth-policy: Fixed crash due to using already-freed memory if policy lookup takes longer than auth request exists. - lib-auth: Unescape passdb/userdb extra fields. Mainly affected returning extra fields with LFs or TABs. - lmtp_user_concurrency_limit>0 setting was logging unnecessary anvil errors. - lmtp_user_concurrency_limit is now checked before quota check with lmtp_rcpt_check_quota=yes to avoid unnecessary quota work. - lmtp: %{userdb:*} variables didn't work in mail_log_prefix - autoexpunge settings for mailboxes with wildcards didn't work when namespace prefix was non-empty. - Fixed writing >2GB to iostream-temp files (used by fs-compress, fs-metawrap, doveadm-http) - director: Ignore duplicates in director_servers setting. - director: Many fixes related to connection handshaking, user moving and error handling. - director: Don't break with shutdown_clients=no - zlib, IMAP BINARY: Fixed internal caching when accessing multiple newly created mails. They all had UID=0 and the next mail could have wrongly used the previously cached mail. - doveadm stats reset wasn't reseting all the stats. - auth_stats=yes: Don't update num_logins, since it doubles them when using with mail stats. - quota count: Fixed deadlocks when updating vsize header. - dict-quota: Fixed crashes happening due to memory corruption. - dict proxy: Fixed various timeout-related bugs. - doveadm proxying: Fixed -A and -u wildcard handling. - doveadm proxying: Fixed hangs and bugs related to printing. - imap: Fixed wrongly triggering assert-crash in client_check_command_hangs. - imap proxy: Don't send ID command pipelined with nopipelining=yes - imap-hibernate: Don't execute quota_over_script or last_login after un-hibernation. - imap-hibernate: Don't un-hibernate if client sends DONE+IDLE in one IP packet. - imap-hibernate: Fixed various failures when un-hibernating. - fts: fts_autoindex=yes was broken in 2.2.25 unless fts_autoindex_exclude settings existed. - fts-solr: Fixed searching multiple mailboxes (patch by x16a0) - doveadm fetch body.snippet wasn't working in 2.2.25. Also fixed a crash with certain emails. - pop3-migration + dbox: Various fixes related to POP3 UIDL optimization in 2.2.25. - pop3-migration: Fixed "truncated email header" workaround.

5 10

Bugreport: managesieve-login won't start without a ssl-key
by Moritz Fago 28 Oct '16

28 Oct '16

Hello, If you don’t have a ssl_key and ssl_cert configured in your dovecot config managesieve-login will fail to start with the following error message: dovecot: managesieve-login: Fatal: Couldn't parse private ssl_key: error:0906D06C:PEM routines:PEM_read_bio:no start line: Expecting: ANY PRIVATE KEY, even if you haven’t enabled ssl for managesieve-login. Infos according to http://www.dovecot.org/bugreport.html: Filesystem: ext4 doveconf -n: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 auth_default_realm = toppoint.de auth_mechanisms = plain login auth_username_format = %Ln mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = dovecot driver = pam } plugin { sieve = ~/.sieve/dovecot.sieve sieve_dir = ~/.sieve } protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 ssl = yes } } ssl = required ssl_cert = </etc/ssl/private/imap.toppoint.de.crt ssl_cipher_list = HIGH::!aNULL:!eNULL:!kRSA:!kPSK:!kSRP:!aDSS:!kECDH:!kDH:!MD5:!SHA1:!RC2:!RC4:!SEED:!IDEA:!DES:!3DES ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/imap.toppoint.de.pem ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv3 !SSLv2 userdb { driver = passwd } protocol lmtp { mail_plugins = sieve } protocol imap { ssl_cert = </etc/ssl/private/imap.toppoint.de.crt ssl_key = </etc/ssl/private/imap.toppoint.de.pem } protocol pop3 { ssl_cert = </etc/ssl/private/pop3.toppoint.de.crt ssl_key = </etc/ssl/private/pop3.toppoint.de.pem } P.S I used doveconf -n to generate the config output, the website says you should use dovecot -n, is this an error or intentional?

3 2

Subscription not immediately reflected
by John van der Kamp 27 Oct '16

27 Oct '16

Hello, I noticed that somewhere between 2.2.22 and 2.2.25 the workings of subscriptions seem to have changed. In version 2.2.25, when a client subscribes to a folder, and then does an LSUB command, it will not see that subscribed folder. If you retry the LSUB command, the change is there. Same with unsubscribes. In version 2.2.22 I did not see this weird behavior. John

3 5

multiple SSL certificates story
by Arkadiusz Miśkiewicz 26 Oct '16

26 Oct '16

Hi. Little story :-) I'm playing with dovecot 2.2.25 and multiple SSL certificates. ~7000 certificates which are loaded twice, so my dovecot has ~14 000 certificate pairs (14k key + 14k cert) in config. 14 000 local_name entries. Like these: local_name imap.example.com { ssl_cert = </etc/certs/cert1.pem ssl_key = </etc/certs/cert1.pem } local_name pop3.example.com { ssl_cert = </etc/certs/cert1.pem ssl_key = </etc/certs/cert1.pem } (it needs to be "duplicated" like this because dovecot doesn't support globs/regexps for local_name, so can't use: local_name *.example.com { ssl_cert = </etc/certs/cert1.pem ssl_key = </etc/certs/cert1.pem } ) All certificates take 87MB on disk. Dovecot config for certificates itself is 3MB. Now fun begins. Trying to start dovecot fails Oct 26 12:49:59 ixion dovecot[19831]: master: Dovecot v2.2.25 (7be1766) starting up for imap, pop3, lmtp (core dumps disabled) Oct 26 12:50:01 ixion dovecot[19835]: config: Fatal: pool_system_malloc(1608): Out of memory Oct 26 12:50:01 ixion dovecot[19831]: master: Error: service(config): command startup failed, throttling for 2 secs Oct 26 12:50:01 ixion dovecot[19835]: config: Fatal: master: service(config): child 19837 returned error 83 (Out of memory (service config { vsz_limit=1024 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump) Oct 26 12:50:01 ixion dovecot[19835]: anvil: Fatal: Error reading configuration: read(/var/run/dovecot/config) failed: read(size=8192) failed: Connection reset by peer Oct 26 12:50:01 ixion dovecot[19835]: ssl-params: Fatal: Error reading configuration: read(/var/run/dovecot/config) failed: read(size=8192) failed: Connection reset by peer Oct 26 12:50:01 ixion dovecot[19831]: master: Error: service(anvil): command startup failed, throttling for 2 secs Oct 26 12:50:01 ixion dovecot[19831]: master: Error: service(ssl-params): command startup failed, throttling for 2 secs vsz_limit=1024 MB not enough. 2048 MB not enough. Raised to 3048 MB - now memory is enough but it still isn't able to start because it hits hardcoded config reading timeout (#define CONFIG_READ_TIMEOUT_SECS 10) Oct 26 13:00:24 ixion dovecot[20272]: master: Dovecot v2.2.25 (7be1766) starting up for imap, pop3, lmtp (core dumps disabled) Oct 26 13:00:34 ixion dovecot[20276]: anvil: Fatal: Error reading configuration: Timeout reading config from /var/run/dovecot/config Oct 26 13:00:34 ixion dovecot[20276]: ssl-params: Fatal: Error reading configuration: Timeout reading config from /var/run/dovecot/config Oct 26 13:00:34 ixion dovecot[20272]: master: Error: service(anvil): command startup failed, throttling for 2 secs Oct 26 13:00:34 ixion dovecot[20272]: master: Error: service(ssl-params): command startup failed, throttling for 2 secs On faster, SSD disk based machine with no load, it starts but then trying to do "doveadm reload" ends up with timeout Oct 26 12:22:12 master: Warning: SIGHUP received - reloading configuration Oct 26 12:22:22 master: Error: Error reading configuration: Timeout reading config from /var/run/dovecot/config dovecot also seem to stop handling clients when such reload happens. What can be done to make it work and how? ps. for comparison on how others handle SSL certificates. Exim MTA for example can easily handle milions of SSL certificates.That's because it loads certificates on demand, when these are actually needed. Core part of exim ssl config is two lines: tls_privatekey = ${if exists{/etc/certs/cert.${lc:${sha1:${lc:${tls_sni}}}}.pem}{/etc/certs//cert.${lc:${sha1:${lc:${tls_sni}}}}.pem}{/etc/certs/default-cert.key}} tls_certificate = ${if exists{/etc/certs/cert.${lc:${sha1:${lc:${tls_sni}}}}.pem}{/etc/certs/cert.${lc:${sha1:${lc:${tls_sni}}}}.pem}{/etc/certs/default-cert.pem}} which means check runtime if file on disk exists; if it exists then use it for tls_XYZ option; if it doesn't exist use default-cert.{pem/key} (sha1/lowercase game is to make sure that attacker cannot put own filename in TLS SNI servername). That's all. There is no memory impact and only little cpu impact with such solution. The other big advantage of exim solution is also no need for daemon to reload its config. You only change/add/remove certificate files on disk. ps2. thanks to letsencrypt project you can have thousands certificates for each domain/subdomain possible, so such usage should grow -- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

2 2

Replication with SSL
by deZillium 26 Oct '16

26 Oct '16

Hello, - Set up a pair of servers according to http://wiki2.dovecot.org/Replication -Enabled SSL for both servers - Dovecot version: 2.2.13 (Debian 8.6) I couldn't find an option to specify the certificate that doveadm should use when connecting to the other server. Both servers have hostnames that are different, as verified by dovecot --hostdomain(as per the instructions) but use a common certificate when emailclients connect to them (high availability setup). Yes, single server login works as expected, been working for the past few years :-). Setting up a custom ssl_client_ca_file doesn't work, since doveadm doesn't know which certificate it should send when connecting to the other doveadm. Setting the ssl_client_ca_dir tothe directory with the global CAsdoesn't work either, since doveadm doesn't use the hostname that dovecot actually uses. The custom self-signed CA works when used outside dovecot(mysql for example). Is there any configuration thatneeds to be changed in order for doveadm to use a custom self signed certificate? Thank you

1 0