January 2019 - dovecot

auth: Error - Request timed out
by Hajo Locke 01 Feb '19

01 Feb '19

Hello List, i use dovecot 2.2.22 and have the same problem described here: https://dovecot.org/pipermail/dovecot/2017-November/110020.html I can confirm that sometimes there is a problem with connection to mysql-db, but sometimes not. Reasons for failing are still under investigation by my mates. My current main problem is, that this fail seems to be a one way ticket for dovecot. Even if mysql is verifyable working again and waiting for connection dovecot stucks furthermore with errors like this: May 29 07:00:49 hostname dovecot: auth: Error: plain(mail(a)example.com,xxx.xxx.xx.xxx,<HGI4FVFt2Oy80j3P>): Request 999.7 timed out after 150 secs, state=1 When restarting dovecot all is immediately working again. Is there a way to tell dovecot to restart auth services or reinitialize mysql-connection after these hard fails? I could insert "idle_kill = 1 mins" into service auth and service auth-worker, but i dont know if this would work. Unfortunately i am not able to reproduce this error and there are always a couple of days between errors. Thanks, Hajo

4 6

Dovecot and FTS experiment
by Tomasz Nowak 30 Jan '19

30 Jan '19

Hello, I'm trying to experiment with Dovecot and Solr server. I have >30k email addresses that I want to index to speed up searching and save IOPS on mail servers. For now - I'm doing some experiments and I'm testing how it is working. I'm thinking about adding one additional server with Solr and configure all mail servers to use that server. I have some questions. 1. I have 15 mail servers. It will be good If I add new server with Solr and use it on all Dovecot servers? Or maybe I should install Solr on all mail servers? 2. I notice - I have mail account with 3GB of mail. Index files in mail dir has 5MB. After indexing mailbox in Solr - index files has 15MB. What changes in those files? FTS indexing adds something to that files - but what? 3. If I need to disconnect Dovecot from Solr server - simply Can I remove plugin configuration from dovecot.conf and everything would be working? Or maybe - I need to remove index files and recreate it? It is important for me because if some day I would decide to disconnect FTS server - I would have to recreate indexes for >30k mailboxes.. 4. What if some day I want to change Solr server to new server with empty Solr database? I can simply reindex mailboxes? Or I need to do something more? Thanks!

3 4

Re: "unknown user - trying the next userdb" Info in log
by Aki Tuomi 30 Jan '19

30 Jan '19

3 3

v2.3.4 released
by Timo Sirainen 30 Jan '19

30 Jan '19

https://dovecot.org/releases/2.3/dovecot-2.3.4.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.4.tar.gz.sig Binary packages in https://repo.dovecot.org/ * The default postmaster_address is now "postmaster@<user domain or server hostname>". If username contains the @domain part, that's used. If not, then the server's hostname is used. * "doveadm stats dump" now returns two decimals for the "avg" field. + Added push notification driver that uses a Lua script + Added new SQL, DNS and connection events. See https://wiki2.dovecot.org/Events + Added "doveadm mailbox cache purge" command. + Added events API support for Lua scripts + doveadm force-resync -f parameter performs "index fsck" while opening the index. This may be useful to fix some types of broken index files. This may become the default behavior in a later version. - director: Kicking a user crashes if login process is very slow - pop3_no_flag_updates=no: Don't expunge DELEted and RETRed messages unless QUIT is sent. - auth: Fix crypt() segfault with glibc-2.28+ - imap: Running UID FILTER script with errors assert-crashes - dsync, pop3-migration: POP3 UIDLs weren't added to dovecot.index.cache while mails were saved. - dict clients may have been using 100% CPU while waiting for dict server to finish commands. - doveadm user: Fixed user listing via HTTP API - All levels of Cassandra log messages were logged as Dovecot errors. - http/smtp client may have crashed after SSL handshake - Lua auth converted strings that looked like numbers into numbers.

18 145

"unknown user - trying the next userdb" Info in log
by James Brown 30 Jan '19

30 Jan '19

I’ve set up a new mail server. Auth-worker is giving me this error all the time: auth-worker(11160): Info: passwd(sales(a)bordo.com.au <mailto:sales@bordo.com.au>,10.0.0.54,<YSFoAYHAGoLAqAEo>): unknown user - trying the next userdb Next line is from imap-login with successful login. Using MySQL as user database. Auth-sql.comf.ext is: passdb { driver = sql # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # <doc/wiki/UserDatabase.Prefetch.txt> userdb { driver = prefetch } userdb { driver = sql args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } # If you don't have any user-specific settings, you can avoid the user_query # by using userdb static instead of userdb sql, for example: # <doc/wiki/UserDatabase.Static.txt> #userdb { #driver = static #args = uid=vmail gid=vmail home=/var/vmail/%u #} Any ideas why it is gives me this error before logging in? Where should I look? Thanks, James.

3 5

doveadm batch crash
by Jonathan Casiot 30 Jan '19

30 Jan '19

Hi I'm having an issue with doveadm batch - the following command always crashes: doveadm batch : mailbox status all inbox Whereas the following work as expected: doveadm mailbox status all inbox doveadm batch : mailbox list Any assistance gratefully received. Debug output from the crash: Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib10_mail_crypt_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so doveadm(jc): Debug: Effective uid=1001, gid=100, home=/home/jc doveadm(jc): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(jc): Debug: mail_crypt_plugin: mail_crypt_curve setting missing - generating EC keys disabled doveadm(jc): Debug: Namespace inbox: type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:ALT=~/Archive/tape/mdbox doveadm(jc): Debug: fs: root=/home/jc/mdbox, index=, indexpvt=, control=, inbox=, alt=/home/jc/Archive/tape/mdbox doveadm(jc): Debug: acl: initializing backend with data: vfile doveadm(jc): Debug: acl: acl username = jc doveadm(jc): Debug: acl: owner = 1 doveadm(jc): Debug: acl vfile: Global ACLs disabled doveadm(jc): Panic: file mail-storage.c: line 875 (mailbox_set_reason): assertion failed: (reason != NULL) doveadm(jc): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xa0e9e) [0x7fc993023e9e] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7fc993023f0a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc992fb3c97] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x49856) [0x7fc993301856] -> /usr/bin/doveadm(+0x348e2) [0x5631dbb5f8e2] -> /usr/bin/doveadm(+0x2e1ad) [0x5631dbb591ad] -> /usr/bin/doveadm(+0x2bd2c) [0x5631dbb56d2c] -> /usr/bin/doveadm(+0x2c92a) [0x5631dbb5792a] -> /usr/bin/doveadm(doveadm_mail_try_run+0x215) [0x5631dbb58285] -> /usr/bin/doveadm(main+0x46a) [0x5631dbb473fa] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc992bd83d5] -> /usr/bin/doveadm(+0x1c565) [0x5631dbb47565] Aborted (core dumped) And doveconf -n output: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.1.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: ### redacted ### first_valid_gid = 100 first_valid_uid = 1000 last_valid_gid = 100 last_valid_uid = 1999 listen = * login_greeting = IMAP ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e session=<%{session}> %k mail_attribute_dict = file:%h/mdbox/dovecot-attributes mail_location = mdbox:~/mdbox:ALT=~/Archive/tape/mdbox mail_plugins = acl stats mail_crypt mailbox_list_index = yes mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { acl = vfile mail_crypt_global_private_key = <### redacted ### mail_crypt_global_public_key = <### redacted ### mail_crypt_save_version = 2 stats_memory_limit = 16 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = ### redacted ### protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_limit = 200 process_min_avail = 4 service_count = 1 } service stats { fifo_listener stats-mail { group = users mode = 01224 } } ssl = required ssl_cert = <### redacted ### ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 !TLSv1 userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_idle_notify_interval = 10 mins imap_metadata = yes mail_max_userip_connections = 20 mail_plugins = acl stats mail_crypt imap_acl imap_stats } Regards, -- Jonathan

2 2

send and receive error message
by Chandran Manikandan 29 Jan '19

29 Jan '19

Hi Folks, I am running qmailtoaster on COS6 32 bit. Send and receive error message coming frequently and inbox emails are not downloading in outlook 2016 of office 365 email client. But if i restart the dovecot service then start downloading the emails in inbox then send and receive error message gone. Outlook 2010 doesn't have the same issue. Is this server issue? or email client version issues? or if i need to use standalone outlook product instead office 365? I do the dovecot service restart every one hour for this issue . if possible to do the cron job for dovecot service restart for every one hour daily or any other solution there. Kindly Any one help me. -- *Thanks,* *Manikandan.C* *System Administrator*

1 0

Re: Rsync to backup dbox with SIS (Daniel Miller)
by ash-dovecot＠comtek.co.uk 28 Jan '19

28 Jan '19

> Probably not the answer you want - but I would strongly suggest using > Dovecot replication.? Dovecot replication Just Works - so don't reinvent > the wheel when Timo provided such a polished tool already.? Okay, thanks, I'll look at replication for the warm spare. Although we do use shared folders, which your link below says don't work properly. > And based on > my previous SIS experience - while dbox is nice I would suggest avoiding > SIS until there are reports of more development.? Sdbox will be solid - > which is what you want for an archive - though maildir would be the > safest.? Archives don't need to be rapid-access - they need to be > dependable.? SIS is wonderful for space saving - but until there's more > safety checks built-in I'd suggest avoiding it for production backups.? > Drives are cheap - lost data, lost time, lost hair, lost sanity...is not. Our main motivation for dbox is SIS. We don't have a very high volume of mail, so maildir speed is unlikely to be an issue. What we do have is a lot of users who like to mail around large files. Drives are cheap, but there are advantages to a smaller mailstore. That said, data security wins. We will probably abandon dbox entirely and continue with maildir, thanks. > The backups don't have to be "visible" to other machines - don't even > have to be running IMAP/POP services (I think).? And the replication > command is run - via ssh (https://wiki.dovecot.org/Replication) - so > what's the problem? Well, our backup servers are supposed to be running a minimalist set of software. If we need to restore from backup after a security issue or application bug on a production machine, then we are trying to minimise the chance that the backup machine itself could be affected. Anyway, if we aren't using SIS, then we needn't use dbox, and therefore backup machines can continue to run a minimalist OS with only rsync. Thanks, Ash --- To opt-in to receiving communications from us or to update your email preferences please go to https://lists.comtek.co.uk/prefs. To be removed from receiving all communications from the Comtek Group, please unsubscribe at https://lists.comtek.co.uk/prefs and accept our apologies for any inconvenience caused. The Comtek Group respects your privacy. Please find our privacy policy at http://www.comtek.co.uk/privacy-policy This email and any attachments are confidential. If you are not the named recipient, please notify the sender immediately. The unauthorised use, disclosure, copying or alteration of this message or its attachments is strictly prohibited. Opinions expressed in this message are those of the individual and not Comtek Network Systems (UK) Ltd unless indicated by an authorised representative independent of this message. Comtek Network Systems does not accept responsibility for any changes to this message after it has been sent. Company Details: Comtek Network Systems (UK) Ltd. Registered Office: Comtek Network Systems (UK) Ltd, Unit 108, Tenth Avenue (Zone 3), Deeside Industrial Park, Deeside, Flintshire, CH5 2UA. Comtek Network Systems (UK) Ltd is a company registered in England and Wales with company number 05929974.

1 0

all of a sudden, slow resync
by Victor Sudakov 27 Jan '19

27 Jan '19

Dear Colleagues, All of a sudden, I've begun receiving this output when reindexing my mail boxes: # /usr/local/bin/doveadm force-resync -u vas "*" doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/sent/dovecot.index.log was locked for 55 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-30-06-2010/dovecot.index.log was locked for 56 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-2014/dovecot.index.log was locked for 95 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-20-06-2012/dovecot.index.log was locked for 72 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-18-05-2009/dovecot.index.log was locked for 31 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-07-12-2008/dovecot.index.log was locked for 56 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dns-operations/dovecot.index.log was locked for 95 seconds (Mailbox was synchronized) And it really takes forever to finish. What could cause this? FreeBSD 11.2-RELEASE-p8, dovecot-2.3.3_4 -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/

1 0

auth-worker: Error: double free or corruption (fasttop)
by Marcel Lorenz 27 Jan '19

27 Jan '19

Dear list, since the distri upgrade to glibc 2.28, i can't build working dovecot binary's. Older with glibc 2.27 are working. The distri is LFS based. MariaDB 10.2.18 and 10.3.11 makes no difference. Having this on different systems. Messages Log: Dec 19 17:10:38 test kernel: traps: auth[30189] general protection ip:7f4b96d6598c sp:7ffef1f31710 error:0 in libmariadb.so.3[7f4b96d4e000+67000] Dec 19 17:11:39 test kernel: auth[30299]: segfault at 560c57366c40 ip 0000560c57366c40 sp 00007fffe5b51f18 error 15 Dec 19 17:12:39 test kernel: traps: auth[30346] general protection ip:7f55601f298c sp:7ffe597929c0 error:0 in libmariadb.so.3[7f55601db000+67000] Dec 19 17:13:39 test kernel: traps: auth[30392] general protection ip:7fcd2317298c sp:7ffecd6f4cb0 error:0 in libmariadb.so.3[7fcd2315b000+67000] Dovecot log: 2018-12-19 17:07:37 auth-worker: Fatal: master: service(auth-worker): child 30053 killed with signal 11 (core dumped) 2018-12-19 17:09:38 auth-worker: Error: double free or corruption (fasttop) 2018-12-19 17:09:38 auth-worker: Fatal: master: service(auth-worker): child 30098 killed with signal 6 (core dumped) 2018-12-19 17:10:38 auth-worker: Fatal: master: service(auth-worker): child 30189 killed with signal 11 (core dumped) Output of dovecot -n: # 2.3.4 (0ecbaf23d): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 (60b0f48d) # OS: Linux 4.14.80 x86_64 # Hostname: test.testing.local auth_mechanisms = plain login auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@+*#" base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot/debug.log dict { expire = mysql:/etc/dovecot/dovecot-dict-sql-expire.conf sqldomainquota = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf sqluserquota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf } info_log_path = /var/log/dovecot/info.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot/main.log log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = dovecot ready login_log_format = %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_location = mdbox:/var/vmail/%d/%n/ mail_max_userip_connections = 4 mail_plugins = " zlib quota expire acl fts fts_lucene" mailbox_idle_check_interval = 1 mins maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = maildir:/var/vmail/%d/public:INDEXPVT=/var/vmail/%d/public prefix = Public. separator = . subscriptions = no type = public } namespace { list = children location = maildir:/var/vmail/%d/%%u:CONTROL=/var/vmail/%d/shared/%%u:INDEX=/var/vmail/$ prefix = shared.%%n. separator = . subscriptions = yes type = shared } namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes al = vfile expire = Trash expire2 = Trash/* expire3 = Spam expire_cache = yes expire_dict = proxy::expire fts = lucene fts_lucene = whitespace_chars=@.-+_*:/\"' last_login_key = # hidden, use -P to show it quota = dict:User Quota::proxy::sqluserquota quota_exceeded_message = Quota exceeded quota_grace = 10%% quota_rule2 = Trash:storage=+15%% quota_rule3 = Spam:ignore quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is over quota / Postfach ist voll quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/vmail/%d/%n/sieve/active-script.sieve sieve_before = /var/vmail/sieve/spam-global.sieve sieve_dir = /var/vmail/%d/%n/sieve/scripts/ sieve_global = /var/lib/dovecot/sieve/global/ sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M trash = /etc/dovecot/dovecot-trash.conf zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster(a)testing.local protocols = imap pop3 lmtp sieve service auth-worker { user = doveauth } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0660 user = doveauth } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 0 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 10022 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_alt_cert = </etc/ssl/certs/postfix-rsa.crt ssl_alt_key = # hidden, use -P to show it ssl_cert = </etc/ssl/certs/postfix-ecc.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_logout_format = in=%i out=%o imap_max_line_length = 64 k mail_max_userip_connections = 10 mail_plugins = " zlib quota expire acl fts fts_lucene acl imap_acl imap_quota mail_log last_login notify imap_zlib trash" } protocol pop3 { mail_max_userip_connections = 4 mail_plugins = " zlib quota expire acl fts fts_lucene last_login expire" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = no pop3_lock_session = no pop3_logout_format = retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = yes pop3_reuse_xuidl = no pop3_save_uidl = yes pop3_uidl_format = %08Xu%08Xv } protocol lmtp { mail_plugins = " zlib quota expire acl fts fts_lucene sieve" postmaster_address = postmaster(a)test.local } protocol lda { auth_socket_path = /var/run/dovecot/auth-master info_log_path = /var/log/dovecot/deliver.log log_path = /var/log/dovecot/deliver.log mail_fsync = optimized mail_plugins = " zlib quota expire acl fts fts_lucene trash autocreate sieve" postmaster_address = postmaster(a)testing.local quota_full_tempfail = yes rejection_reason = Your message to <%t> with subject of <%s> was automatically rejected:%n%r rejection_subject = Mailbox Full: %s } protocol sieve { mail_max_userip_connections = 10 managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 } Thanks in advance! Marcel

3 3