June 2017 - dovecot - dovecot.org

Re: v2.2.30.1
by Joseph Tam 07 Jun '17

07 Jun '17

Aki Tuomi <aki.tuomi(a)dovecot.fi> wrote: > On 06.06.2017 14:11, Olaf Hopp wrote: >> On 06/05/2017 11:05 AM, Angel L. Mateo wrote: >>> I have updated my dovecot proxy servers from 2.2.28 to 2.2.30. >>> Since the upgrade I'm having the error: >>> >>> Jun 5 10:54:51 musio12 dovecot: auth: Fatal: master: service(auth): >>> child 63632 killed with signal 11 (core not dumped) +1. > We have identified a bug in auth process, and are working with a fix. This is the 1 line context around the same thing I am seeing, if it helps. It seems to happen with brute force scanning Jun 6 06:13:30 server dovecot: imap-login: Warning: Auth connection closed with 1 pending requests (max 7 secs, pid=20675 , EOF) Jun 6 06:13:30 server dovecot: auth: Fatal: master: service(auth): child 20676 killed with signal 11 (core not dumped - set service auth { drop_priv_before_exec=yes }) Jun 6 06:13:30 server dovecot: imap-login: Disconnected (auth failed, 1 attempts in 7 secs): user=<{dead@account}>, method=PLAIN, rip=5.188.11.11, lip={my-ip}, TLS, session=<EpJbYkpRInsFvAsL> Bots aren't known to exit cleanly, so it's probably related to prematurely dropped connections. Apart from that, I'm not seeing this with real users. Joseph Tam <jtam.home(a)gmail.com>

1 1

Bug with 2.2.29-1~auto+25 back to haunt me
by Ralf Hildebrandt 06 Jun '17

06 Jun '17

After upgrading from 2.2.28-1~auto+45 to 2.2.29-1~auto+25 I'm gettings this: May 31 16:44:31 mproxy dovecot: auth: Fatal: passdb imap: Cannot verify certificate without ssl_ca_dir or ssl_ca_file setting May 31 16:44:31 mproxy dovecot: master: Error: service(auth): command startup failed, throttling for 8 secs May 31 16:44:31 mproxy dovecot: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 2 secs): user=<>, rip=141.42.206.36, lip=141.42.206.11, TLS, session=<ze1A9dJQZ8yNKs4k> # doveconf -n # 2.2.devel (215fd61): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (403042e) # OS: Linux 4.4.0-71-generic x86_64 Ubuntu 16.04.2 LTS auth_mechanisms = plain login default_vsz_limit = 1 G imapc_host = exchange-imap.charite.de imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = no listen = *,:: mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_plugins = mail_log notify mail_uid = imapproxy passdb { args = host=exchange-imap.charite.de port=993 ssl=imaps default_fields = userdb_imapc_user=%u userdb_imapc_password=%w userdb_imapc_host=exchange-imap.charite.de userdb_imapc_ssl=imaps userdb_imapc_port=993 driver = imap } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap service auth { inet_listener { address = 127.0.0.1 port = 12345 } } ssl = required ssl_ca = </etc/ssl/certs/ca-certificates.crt ssl_cert = </etc/dovecot/dovecot.pem ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 userdb { driver = prefetch } verbose_proctitle = yes So I added ssl_ca_file = /etc/ssl/certs/ca-certificates.crt But alas: May 31 16:50:24 mproxy dovecot: config: Warning: Obsolete setting in /etc/dovecot/conf.d/10-ssl.conf:36: ssl_ca_file has been replaced by ssl_ca = <file Gnarf! As you can see I do HAVE ssl_ca in my doveconf -n output! ssl_ca = </etc/ssl/certs/ca-certificates.crt So what gives? -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 6

v2.2.30.2 released
by Timo Sirainen 06 Jun '17

06 Jun '17

https://dovecot.org/releases/2.2/dovecot-2.2.30.2.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.30.2.tar.gz.sig Hopefully the last 2.2.30.x.. - auth: Multiple failed authentications within short time caused crashes - push-notification: OX driver crashed at deinit

1 0

2.2.29.8: YACD (Yet Another Core Dump)
by Luciano Mannucci 06 Jun '17

06 Jun '17

I'm getting this in my logfile, should I worry?: Jun 06 12:28:47 imap(paco_mcs): Error: Next message unexpectedly corrupted in mbox file /var/spool/mail/paco_mcs at 639279728 Jun 06 12:28:47 imap(paco_mcs): Error: Unexpectedly lost From-line from mbox file /var/spool/mail/paco_mcs at 639279728 Jun 06 12:28:47 imap(paco_mcs): Panic: file mbox-sync.c: line 1338 (mbox_sync_handle_eof_updates): assertion failed: (trailer_size <= 2) Jun 06 12:28:47 imap(paco_mcs): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9cd16) [0xb75e7d16] -> /usr/lib/dovecot/libdovecot.so.0(+0x9cd9f) [0xb75e7d9f] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb757104e] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x81698) [0xb76d4698] -> /usr/lib/dovecot/libdovecot-storage.so.0(mbox_sync+0x4b) [0xb76d71cb] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x7728c) [0xb76ca28c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x778c6) [0xb76ca8c6] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_get_stream_because+0x5f) [0xb768112f] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x811) [0xb774bef1] -> dovecot/imap() [0x8064c76] -> dovecot/imap() [0x80629c6] -> dovecot/imap(imap_fetch_more+0x31) [0x8063ed1] -> dovecot/imap(cmd_fetch+0x4bb) [0x805574b] -> dovecot/imap(command_exec+0x4a) [0x80613aa] -> dovecot/imap() [0x805f8d6] -> dovecot/imap() [0x805f961] -> dovecot/imap(client_handle_input+0x185) [0x805fcc5] - > dovecot/imap(client_input+0x71) [0x80601b1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x67) [0xb75ff997] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xeb) [0xb76013ab] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x45) [0xb75ffa55] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0xb75ffc18] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2d) [0xb757c1ad] -> dovecot/imap(main+0x344) [0x806dbc4] -> /lib/libc.so.6(__libc_start_main+0xfe) [0xb73d6c2e] Jun 06 12:28:48 imap(paco_mcs): Fatal: master: service(imap): child 4989 killed with signal 6 (core dumped) Jun 06 12:28:52 imap(paco_mcs): Error: Next message unexpectedly lost from mbox file /var/spool/mail/paco_mcs at 1073259120 (noncached) Jun 06 12:28:52 imap(paco_mcs): Error: Sync failed for mbox file /var/spool/mail/paco_mcs: Couldn't get body size for seq=11213 My doveconf -n: # 2.2.29.1 (e0b76e3): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.17 (e179378) # OS: Linux 3.0.101-105-pae i686 openSUSE 11.4 (i586) ext3 auth_cache_size = 3 k auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes default_client_limit = 1249 default_vsz_limit = 712 M disable_plaintext_auth = no first_valid_gid = 0 first_valid_uid = 100 info_log_path = /var/log/dovecot/logfile.info listen = * log_path = /var/log/dovecot/logfile login_greeting = Dovecot at Baobab ready. login_trusted_networks = 127.0.0.0/8 212.45.144.0/24 192.168.134.0/24 mail_location = mbox:/var/spool/mailboxes/%u:INBOX=/var/spool/mail/%u:DIRNAME=mbox:INDEX=/var/dovecot_indexes/%u maildir_copy_with_hardlinks = no managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_lock_timeout = 443 secs namespace { inbox = yes location = prefix = separator = . type = private } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = pop3 imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { inet_listener imaps { address = * } process_limit = 512 } service pop3-login { chroot = executable = pop3-login -D inet_listener pop3s { address = * } process_limit = 512 } ssl_ca = </etc/ssl/ca.crt ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } version_ignore = yes protocol imap { imap_client_workarounds = delay-newmail mail_plugins = mail_log notify } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = yes } protocol lda { info_log_path = /var/log/dovecot/lda.log log_path = /var/log/dovecot/lda-errors.log postmaster_address = postmaster(a)mcs.it } Cheers, Luciano. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster(a)sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.lesassaie.IT/

2 2

Multiuser attachment storage (mail_attachment_dir)
by megaone＠yandex.ru 06 Jun '17

06 Jun '17

Was trying to set up attachment storage, but folders are receiving permission only for current user, so if another user is receiving a mail with same attachment he is getting permission denied error on reading hash file. Is it possible to setup global attachment storage to work with all users, at least in domain? mail_access_groups won't help, as folders are given 0 group permission. Umask and setfacl won't help either.

2 1

Two domains - same user names filter
by Sandbox 06 Jun '17

06 Jun '17

Hi, I have two LDAP domains, which has some equal users, eg: abc(a)domain1.com abc(a)domain2.com I sat up this config: domain1 users maildirs are stored in /home/vmail/username domain2 users maildirs are stored in /home/vmail/domain2/username This works fine except one thing: i cant set up the ldap query to choose the correct maildir if the user names are equal. Is it possible to use a user_filter which will choose the correct maildir and user/domain from the email address? My current ldap.conf for domain1: hosts = ldap.domain1.com base = ou=People,dc=domain1,dc=com ldap_version = 3 user_attrs = uid=user user_filter = (uid=%n) pass_attrs = uid=user,userPassword=password pass_filter = (uid=%n) default_pass_scheme = MD5 and for domain2: hosts = ldap.domain2.com base = ou=People,dc=domain2,dc=com ldap_version = 3 user_attrs = \ =mail=maildir:/home/vmail/%{ldap:departmentNumber)/%n/Maildir user_filter = (uid=%n) pass_attrs = uid=%n,userPassword=password pass_filter = (uid=%n) default_pass_scheme = MD5 Thanks, Robert

4 7

vnd.dovecot.pipe in sieve: what user....
by Larry Rosenman 06 Jun '17

06 Jun '17

I have a site-wide bayes database, that currently gets populated by scripts running as a user that is also the user that Exim uses to call spamd. When a script is running from the vnd.dovecot.pipe call, what user is it running as? Would a setuid script work? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: larryrtx(a)gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281

2 1

Re: corrupted indexes rebuilding over and over
by Joseph Tam 06 Jun '17

06 Jun '17

micah <micah(a)riseup.net> wrote: > Now, about 50 users have a problem where their indexes are > corrupted. Dovecot tries to fix them, but for some reason it is > failing. The users are seeing their mails duplicated and they cannot > delete mails. I can see in the logs dovecot detecting this and trying to > rebuild operations during delivery and imap login, but its repeating > over and over and over. For one user, I've seen more than 4k lines in > the logs about this. > > If i try to do doveadm force-resync -u username INBOX > > I get this: > > doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: Inconsistency in map index (3494,32332 != 3494,383536) I think mdbox is sensitive to index corruption. mbox/maildir indices can be rebuilt, but data loss can result if mdbox indices are corrupted: https://wiki2.dovecot.org/MailboxFormat/dbox "Grubert, Gordon" <gordon.grubert+lists(a)uni-greifswald.de> then writes: > we had the same problem after a system failure. Our problem could be > solved by > > rm /PATH/USER/mdbox/storage/dovecot.map.index* > doveadm -D -v force-resync -u USER INBOX Not sure given the above URL whether this will solve your problem of make it far worse. Better back up what you have before trying anything. Joseph Tam <jtam.home(a)gmail.com>

1 0

corrupted indexes rebuilding over and over
by micah 05 Jun '17

05 Jun '17

Hi, On saturday one of our dovecot machines had an OOPS, and we had to powercycle it. When it came up, it did a filesystem fsck (ext4) and fixed a couple things, but nothing in lost+found. Now, about 50 users have a problem where their indexes are corrupted. Dovecot tries to fix them, but for some reason it is failing. The users are seeing their mails duplicated and they cannot delete mails. I can see in the logs dovecot detecting this and trying to rebuild operations during delivery and imap login, but its repeating over and over and over. For one user, I've seen more than 4k lines in the logs about this. If i try to do doveadm force-resync -u username INBOX I get this: doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: Inconsistency in map index (3494,32332 != 3494,383536) doveadm(nuffsaid): Warning: fscking index file /srv/dovecot_indexes/n/nuffsaid/storage/dovecot.map.index doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: rebuilding indexes doveadm(nuffsaid): Warning: fscking index file /srv/dovecot_indexes/n/nuffsaid/storage/dovecot.map.index doveadm(nuffsaid): Warning: mdbox /n/nuffsaid/mdbox/storage: Inconsistency in map index (3494,32332 != 3494,384024) doveadm(nuffsaid): Warning: fscking index file /srv/dovecot_indexes/n/nuffsaid/storage/dovecot.map.index but then if i do it again, I get the same thing, over and over and over (the numbers change a little). This is dovecot 2.2.27 with mdbox format.... what can I do to fix this? Its driving users mad, and they keep going over quota as their space fills. thanks! micah

2 2

fts_solr: uid0?
by Larry Rosenman 05 Jun '17

05 Jun '17

I'm seeing the following: Jun 5 11:10:52 thebighonker dovecot: imap(ler): Error: fts_solr: received invalid uid '0' Is there something I can get for you to debug? doveconf -n: # 2.2.30.1 (eebd877): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.18 (29cc74d) # OS: FreeBSD 11.1-PRERELEASE amd64 auth_debug_passwords = yes auth_default_realm = lerctr.org auth_mechanisms = plain login auth_realms = lerctr.org thebighonker.lerctr.org tbh.lerctr.org auth_username_format = %Ln default_vsz_limit = 1 G deliver_log_format = msgid=%m: %$ (subject=%s from=%f size=%w) lda_mailbox_autocreate = yes listen = 192.147.25.65, :: lmtp_save_to_detail_mailbox = yes login_access_sockets = tcpwrap mail_attribute_dict = file:%h/mail/.imap/dovecot-mail-attributes mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mail/INBOX mail_plugins = " fts fts_solr notify stats virtual" mail_privileged_group = mail mail_server_admin = mailto:ler@lerctr.org mail_server_comment = LERCTR Mail Server mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds editheader mboxmetadata servermetadata imapsieve vnd.dovecot.imapsieve namespace archive { hidden = no list = no location = mbox:~/MAIL-ARCHIVE prefix = ARCHIVE/ separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { auto = create } mailbox Junk { special_use = \Junk } mailbox SA/FN { special_use = \Junk } mailbox SENT { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox virtual/Flagged { special_use = \Flagged } mailbox virtual/all { special_use = \All } prefix = } namespace virtual { hidden = no list = yes location = virtual:~/MAIL-VIRTUAL:INDEX=MEMORY prefix = Virtual/ separator = / } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/ fts_tika = http://localhost:9998/tika/ imapsieve_url = sieve://thebighonker.lerctr.org mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size from subject vsize flags recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +editheader +vacation-seconds +mboxmetadata +servermetadata sieve_plugins = sieve_imapsieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 5s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-client { mode = 0666 } unix_listener auth-master { mode = 0666 } } service indexer-worker { drop_priv_before_exec = yes } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0666 user = } fifo_listener stats-user { group = mode = 0666 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert = </home/ler/letsencrypt-home/lerctr.org/fullchain.cer ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+AESGCM:EECDH:EDH+AESGCM:EDH+aRSA:HIGH:!MEDIUM:!LOW:!aNULL:!eNULL:!LOW:!RC4:!MD5:!EXP:!PSK:!SRP:!DSS ssl_key = # hidden, use -P to show it ssl_protocols = !SSLv2 !SSLv3 userdb { driver = passwd } verbose_proctitle = yes protocol lmtp { mail_plugins = " fts fts_solr notify stats virtual sieve mail_log" } protocol lda { mail_plugins = " fts fts_solr notify stats virtual sieve mail_log" } protocol pop3 { mail_plugins = " fts fts_solr notify stats virtual mail_log" } protocol !doveadm { mail_plugins = " fts fts_solr notify stats virtual mail_log" } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags imap_logout_format = in=%i out=%o fhc=%{fetch_hdr_count} fhb=%{fetch_hdr_bytes} fbc=%{fetch_body_count} fbb=%{fetch_body_bytes} del=%{deleted} exp=%{expunged} trash=%{trashed} imap_metadata = yes mail_max_userip_connections = 50 mail_plugins = " fts fts_solr notify stats virtual mail_log imap_sieve imap_stats stats" } -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: larryrtx(a)gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281

1 0